agora inbox for [email protected]
help / color / mirror / Atom feedRe: backup manifests
372+ messages / 12 participants
[nested] [flat]
* Re: backup manifests
@ 2020-02-27 15:52 Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-02-27 15:52 UTC (permalink / raw)
To: Suraj Kharage <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Jan 3, 2020 at 6:11 PM Suraj Kharage
<[email protected]> wrote:
> Thank you for review comments.
Here's a new patch set for this feature.
0001 adds checksum helper functions, similar to what Suraj had
incorporated into my original patch but separated out into a separate
patch and with some different aesthetic decisions. I also decided to
support all of the SHA variants that PG knows about as options and
added a function to parse a checksum algorithm name, along the lines I
suggested previously.
0002 teaches the server to generate a backup manifest using the format
I originally proposed. This is similar to the patch I posted
previously, but it spools the manifest to disk as it's being
generated, so that we don't run the server out of memory or fail when
hitting the 1GB allocation limit.
0003 adds a new utility, pg_validatebackup, to validate a backup
against a manifest. Suraj tried to incorporate this into
pg_basebackup, which I initially thought might be OK but eventually
decided wasn't good, partly because this really wants to take some
command-line options entirely unrelated to the options accepted by
pg_basebackup. I tried to improve the error checking and the order in
which various things are done, too. This is a basically a complete
rewrite as compared with Suraj's version.
0004 modifies the server to generate a backup manifest in JSON format
rather than my originally proposed format. This allows for some
comparison of the code doing it one way vs. the other. Assuming we
stick with JSON, I will squash this with 0002 at some point.
0005 is a very much work-in-progress and proof-of-concept to modify
the backup validator to understand the JSON format. It doesn't
validate the manifest checksum at this point; it just prints it out.
The error handling needs work. It has other problems, and bugs.
Although I'm still not very happy about the idea of using JSON here,
I'm pretty happy with the basic approach this patch takes. It
demonstrates that the JSON parser can be used for non-trivial things
in frontend code, and I'd say the code even looks reasonably clean -
with the exception of small details like being buggy and
under-commented.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v8-0001-Add-checksum-helper-functions.patch (9.8K, ../../CA+TgmoZRTBiPyvQEwV79PU1ePTtSEo2UeVncrkJMbn1sU1gnRA@mail.gmail.com/2-v8-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From 27a531dd00a2813bc340c4ecceb7aa3f12ba7863 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 5 Feb 2020 13:59:45 -0500
Subject: [PATCH v8 1/5] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
(In my version, I decided to support all of the SHA variants for
which we have code, added a function to translate values of the
enum type back to strings so that users of this code don't need
to do that, used different naming, and tried to be more careful
amount formatting and comments.)
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index ce01df68b9..e199ed7acb 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -47,6 +47,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 834c2c39d1..0550947b8c 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -119,8 +119,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
- f2s.c file_perm.c hashfn.c ip.c jsonapi.c
+ base64.c checksum_helper.c config_info.c controldata_utils.c d2s.c
+ encnames.c exec.c f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c username.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v8-0003-pg_validatebackup-Validate-a-backup-against-the-b.patch (34.2K, ../../CA+TgmoZRTBiPyvQEwV79PU1ePTtSEo2UeVncrkJMbn1sU1gnRA@mail.gmail.com/3-v8-0003-pg_validatebackup-Validate-a-backup-against-the-b.patch)
download | inline diff:
From 52b6e04e1e3a2535770c177ab1c0ee0baa2c35a5 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 7 Feb 2020 17:17:52 -0500
Subject: [PATCH v8 3/5] pg_validatebackup: Validate a backup against the
backup manifest.
Patch by me; some off-list review and testing from Mark Dilger,
Davinder Singh, Tushar Ahuja, Rajkumar Raghuwanshi, and Jeevan
Chalke.
(I chose here to make this a separate utility; Suraj wrote a previous
patch for this that made it part of pg_basebackup. Doing it this way
lets us have various command line options that are specific to backup
validation. I've added a few such options and we might want to add
more later. I also arranged things so that checksum failures are
reported last, as that is the most expensive part of validation. I
believe that my version also does better error checking and reporting.)
---
src/backend/replication/basebackup.c | 6 +-
src/bin/Makefile | 1 +
src/bin/pg_validatebackup/.gitignore | 1 +
src/bin/pg_validatebackup/Makefile | 32 +
src/bin/pg_validatebackup/pg_validatebackup.c | 1089 +++++++++++++++++
5 files changed, 1126 insertions(+), 3 deletions(-)
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 1729931597..99e102b2a7 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -63,7 +63,7 @@ struct manifest_info
pg_checksum_type checksum_type;
pg_sha256_ctx manifest_ctx;
uint64 manifest_size;
- int still_checksumming;
+ bool still_checksumming;
};
@@ -84,7 +84,7 @@ static void SendBackupHeader(List *tablespaces);
static void InitializeManifest(manifest_info *manifest, pg_checksum_type);
static void AppendStringToManifest(manifest_info *manifest, char *s);
static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
- const char *filename, size_t size, time_t mtime,
+ const char *pathname, size_t size, time_t mtime,
pg_checksum_context *checksum_ctx);
static void SendBackupManifest(manifest_info *manifest);
static char *escape_field_for_manifest(const char *s);
@@ -976,7 +976,7 @@ AppendStringToManifest(manifest_info *manifest, char *s)
*/
static void
AddFileToManifest(manifest_info *manifest, const char *spcoid,
- const char *filename, size_t size, time_t mtime,
+ const char *pathname, size_t size, time_t mtime,
pg_checksum_context *checksum_ctx)
{
char pathbuf[MAXPGPATH];
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..3ae1c1f03a
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1 @@
+/pg_validatebackup
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..aeb97d21d2
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,32 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..4f47b20855
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,1089 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/checksum_helper.h"
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * The first word of each line of the manifest file should be one of these
+ * key words. We define constants for the relevant lengths as well.
+ */
+#define KW_MANIFEST_VERSION "PostgreSQL-Backup-Manifest-Version"
+#define KW_MANIFEST_FILE "File"
+#define KW_MANIFEST_CHECKSUM "Manifest-Checksum"
+#define KWL_MANIFEST_VERSION (sizeof(KW_MANIFEST_VERSION)-1)
+#define KWL_MANIFEST_FILE (sizeof(KW_MANIFEST_FILE)-1)
+#define KWL_MANIFEST_CHECKSUM (sizeof(KW_MANIFEST_CHECKSUM)-1)
+
+/*
+ * How many fields are there for each "File" line in the manifest?
+ * Currently we have: file name, file size, timestamp, checksum.
+ */
+#define FIELDS_PER_FILE_LINE 4
+
+/*
+ * Each "File" line in the manifest file is parsed to produce an object
+ * like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash * parse_manifest_file(char *manifest_path);
+static void parse_file_line_from_manifest(manifestfile *f, char *rest,
+ int restlen);
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void pg_validator_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static char *extractstr(char *buffer, int length);
+static int findchar(char *buffer, int size, char c, int start_position);
+static int findfield(char *buffer, char *end, char **result);
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /* Always ignore backup_manifest file and pg_wal directory. */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ simple_string_list_append(&context.ignore_list, optarg);
+ break;
+ case 'm':
+ manifest_path = optarg;
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = argv[optind++];
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ pg_log_info("backup successfully verified");
+
+ exit(context.saw_any_error ? 1 : 0);
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ off_t bytes_read = 0;
+ off_t bytes_consumed = 0;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+ int buffer_position = 0;
+ int buffer_size = 0;
+ int buffer_maxsize = 2 * READ_CHUNK_SIZE;
+ int line_number = 0;
+ bool saw_manifest_checksum_line = false;
+ pg_sha256_ctx manifest_ctx;
+
+ /* Prepare to compute a checksum of the manifest itself. */
+ pg_sha256_init(&manifest_ctx);
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ pg_log_fatal("could not open file \"%s\": %m", manifest_path);
+ exit(1);
+ }
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ {
+ pg_log_fatal("could not stat file \"%s\": %m", manifest_path);
+ exit(1);
+ }
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /* Initialize our read buffer. */
+ buffer = pg_malloc(buffer_maxsize);
+
+ /*
+ * Loop until we've read it all.
+ *
+ * The file size shouldn't be changing, so it seems fine to just error out
+ * if the final length is different from what stat() told us.
+ */
+ while (bytes_consumed < statbuf.st_size)
+ {
+ int line_length;
+ int first_field_length;
+ char *rest;
+ int restlen;
+
+ /* Find next newline if any. */
+ line_length = findchar(buffer, buffer_size, '\n', buffer_position);
+
+ /* If no newline was found, we need to read more data and try again. */
+ if (line_length == -1)
+ {
+ size_t bytes_to_read;
+ int rc;
+
+ bytes_to_read = Min(statbuf.st_size - bytes_read, READ_CHUNK_SIZE);
+ if (bytes_to_read == 0)
+ {
+ pg_log_fatal("manifest file line not terminated by newline");
+ exit(1);
+ }
+ if (bytes_to_read + READ_CHUNK_SIZE > buffer_maxsize)
+ {
+ buffer_maxsize += READ_CHUNK_SIZE;
+ buffer = pg_realloc(buffer, buffer_maxsize);
+ Assert(bytes_to_read + READ_CHUNK_SIZE <= buffer_maxsize);
+ }
+ rc = read(fd, buffer + buffer_size, bytes_to_read);
+ if (rc != bytes_to_read)
+ {
+ if (rc < 0)
+ pg_log_fatal("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ pg_log_fatal("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, bytes_to_read);
+ exit(1);
+ }
+ buffer_size += rc;
+ bytes_read += rc;
+ continue;
+ }
+
+ /* Increment line number. */
+ ++line_number;
+
+ /* The manifest checksum should be the last thing in the file. */
+ if (saw_manifest_checksum_line)
+ {
+ pg_log_fatal("unexpected data follows manifest checksum");
+ exit(1);
+ }
+
+ /* Find first field on line, and remaining line contents. */
+ first_field_length =
+ findchar(buffer, buffer_size, '\t', buffer_position);
+ rest = buffer + buffer_position + first_field_length + 1;
+ restlen = line_length - (first_field_length + 1);
+
+ /*
+ * Check the first word of the line to see what kind of line it is.
+ */
+ if (first_field_length == KWL_MANIFEST_VERSION &&
+ memcmp(buffer + buffer_position, KW_MANIFEST_VERSION,
+ KWL_MANIFEST_VERSION) == 0)
+ {
+ if (line_number != 1)
+ {
+ pg_log_fatal("manifest file version should only be specified at line 1");
+ exit(1);
+ }
+ else
+ {
+ char *line = buffer + buffer_position;
+ char *version;
+
+ version = extractstr(line + first_field_length + 1,
+ line_length - (first_field_length + 1));
+ if (strcmp(version, "1") != 0)
+ {
+ pg_log_fatal("unrecognized manifest version: \"%s\"",
+ version);
+ exit(1);
+ }
+ }
+ }
+ else if (first_field_length == KWL_MANIFEST_FILE &&
+ memcmp(buffer + buffer_position, KW_MANIFEST_FILE,
+ KWL_MANIFEST_FILE) == 0)
+ {
+ manifestfile f;
+ manifestfile *tabent;
+ bool found;
+
+ /* Parse this line. */
+ parse_file_line_from_manifest(&f, rest, restlen);
+
+ /* Make a new entry in the hash table for it. */
+ tabent = manifestfiles_insert(ht, f.pathname, &found);
+ if (found)
+ {
+ pg_log_fatal("duplicate pathname in backup manifest: \"%s\"",
+ f.pathname);
+ exit(1);
+ }
+
+ /* Copy in all the relevant details. */
+ tabent->size = f.size;
+ tabent->checksum_type = f.checksum_type;
+ tabent->checksum_length = f.checksum_length;
+ tabent->checksum_payload = f.checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+ }
+ else if (first_field_length == KWL_MANIFEST_CHECKSUM &&
+ memcmp(buffer + buffer_position, KW_MANIFEST_CHECKSUM,
+ KWL_MANIFEST_CHECKSUM) == 0)
+ {
+ saw_manifest_checksum_line = true;
+ if (restlen != PG_SHA256_DIGEST_STRING_LENGTH - 1)
+ {
+ pg_log_fatal("manifest file checksum has unexpected length: %d",
+ restlen);
+ exit(1);
+ }
+ if (!hexdecode_string(manifest_checksum_expected, rest,
+ PG_SHA256_DIGEST_LENGTH))
+ {
+ pg_log_fatal("invalid manifest checksum: \"%s\"",
+ extractstr(rest, restlen));
+ exit(1);
+ }
+ }
+ else if (first_field_length == -1)
+ {
+ pg_log_fatal("manifest file keyword not terminated by tab");
+ exit(1);
+ }
+ else
+ {
+ char *kw;
+
+ kw = extractstr(buffer + buffer_position, first_field_length);
+ pg_log_fatal("unrecognized manifest file keyword: \"%s\"", kw);
+ exit(1);
+ }
+
+ /* Update manifest checksum, if needed. */
+ if (!saw_manifest_checksum_line)
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer + buffer_position,
+ line_length + 1);
+
+ /* Advance buffer position over the data we just read. */
+ buffer_position += line_length + 1;
+
+ /* Also mark these bytes as consumed so we know when to stop. */
+ bytes_consumed += line_length + 1;
+
+ /*
+ * We don't want to incur the expensive of using memmove() to discard
+ * data after every line, because the lines are short compared to the
+ * chunk size -- but we must do it at least now and then, or we'll
+ * have to keep growing the buffer.
+ */
+ if (buffer_position >= READ_CHUNK_SIZE)
+ {
+ int leftover_bytes = buffer_size - buffer_position;
+
+ if (leftover_bytes > 0)
+ memmove(buffer, buffer + buffer_position, leftover_bytes);
+ buffer_size -= buffer_position;
+ buffer_position = 0;
+ }
+ }
+
+ /* Checksum verification. */
+ if (!saw_manifest_checksum_line)
+ pg_log_fatal("manifest has no checksum");
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ {
+ pg_log_fatal("manifest checksum does not match");
+ exit(1);
+ }
+
+ /* OK, we're done with the manifest file. */
+ close(fd);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * The caller passes the remainder of the line, excluding the initial "File\t"
+ * portion.
+ */
+static void
+parse_file_line_from_manifest(manifestfile *f, char *rest, int restlen)
+{
+ char *end = rest + restlen;
+ char *field[FIELDS_PER_FILE_LINE];
+ unsigned long filesize;
+ char *ep;
+ pg_checksum_type checksum_type;
+ int raw_checksum_length = 0;
+ char *raw_checksum_payload = NULL;
+ int checksum_length;
+ uint8 *checksum_payload;
+ int i;
+ char *s;
+
+ /* Split the line into fields. */
+ for (i = 0; i < FIELDS_PER_FILE_LINE; ++i)
+ {
+ int toklen;
+
+ toklen = findfield(rest, end, &field[i]);
+ if (rest + toklen >= end && i + 1 < FIELDS_PER_FILE_LINE)
+ {
+ pg_log_fatal("manifest file line has too few fields");
+ exit(1);
+ }
+ rest += toklen + 1;
+ }
+
+ /* We expect to have used the entire line. */
+ if (rest < end)
+ {
+ pg_log_fatal("manifest file line has too many fields");
+ exit(1);
+ }
+
+ /* Parse the size. */
+ filesize = strtoul(field[1], &ep, 10);
+ if (*ep)
+ {
+ pg_log_fatal("manifest file size for file \"%s\" is not a number",
+ field[0]);
+ exit(1);
+ }
+
+ /* Parse the checksum type. */
+ for (s = field[3]; s[0] != '\0' && s[0] != ':'; ++s)
+ ;
+ if (*s)
+ {
+ raw_checksum_payload = s + 1;
+ raw_checksum_length = strlen(raw_checksum_payload);
+ *s = '\0';
+ }
+ if (!pg_checksum_parse_type(field[3], &checksum_type))
+ {
+ pg_log_fatal("unrecognized checksum algorithm for file \"%s\": \"%s\"",
+ field[0], field[3]);
+ exit(1);
+ }
+
+ /* Decode the checksum payload. */
+ checksum_length = raw_checksum_length / 2;
+ if (checksum_length == 0)
+ checksum_payload = NULL;
+ else
+ {
+ checksum_payload = palloc(checksum_length);
+ if (!hexdecode_string(checksum_payload, raw_checksum_payload,
+ checksum_length))
+ {
+ pg_log_fatal("invalid checksum for file \"%s\": \"%s\"",
+ field[0], raw_checksum_payload);
+ exit(1);
+ }
+ }
+
+ /* Fill the output struct. */
+ f->pathname = field[0];
+ f->size = filesize;
+ f->checksum_type = checksum_type;
+ f->checksum_length = checksum_length;
+ f->checksum_payload = checksum_payload;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ pg_validator_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Suppress further errors related to this path name and anything
+ * underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ pg_validator_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ pg_validator_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ pg_validator_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ pg_validator_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ pg_validator_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ pg_validator_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY, 0)) < 0)
+ {
+ pg_validator_error(context, "could not open file \"%s\": %m",
+ relpath);
+ pfree(fullpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ pg_validator_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ pg_validator_error(context, "could not close file \"%s\": %m",
+ relpath);
+ pfree(fullpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ pg_validator_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ pg_validator_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <[email protected]>.\n"));
+}
+
+/*
+ * Report an error. Update the context to indicate that we saw an error, and
+ * exit if the context says we should.
+ */
+static void
+pg_validator_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Extract a NUL-terminated string from a larger buffer.
+ */
+static char *
+extractstr(char *buffer, int length)
+{
+ char *s = palloc(length + 1);
+
+ memcpy(s, buffer, length);
+ s[length] = '\0';
+
+ return s;
+}
+
+/*
+ * Find the next instance of a given character within a buffer that
+ * occurs at or after start_position. If there is none, returns -1; else
+ * returns the difference between the position at which the character was
+ * found and the start position.
+ */
+static int
+findchar(char *buffer, int size, char c, int start_position)
+{
+ int i;
+
+ for (i = start_position; i < size; ++i)
+ if (buffer[i] == c)
+ return i - start_position;
+ return -1;
+}
+
+/*
+ * Extract the next field from a line of text read from the manifest file.
+ */
+static int
+findfield(char *buffer, char *end, char **result)
+{
+ int qoffset = 1;
+ int dqcount = 0;
+ int toklen;
+ int bufpos;
+ int resultpos;
+
+ /*
+ * If this field is unquoted, we just stop at the next tab; if there's
+ * none, we stop at the end of the line. Note that if buffer == end, it
+ * just means that the last field on the line is empty.
+ */
+ if (buffer == end || *buffer != '"')
+ {
+ toklen = findchar(buffer, end - buffer, '\t', 0);
+
+ if (toklen == -1)
+ toklen = end - buffer;
+ *result = extractstr(buffer, toklen);
+ return toklen;
+ }
+
+ /*
+ * Our escaping convention is that if the field contains a tab, it must be
+ * surrounded by double-quotes and any internal double-quotes must be
+ * doubled.
+ */
+ while (1)
+ {
+ /* Where's the next double quote? */
+ qoffset += findchar(buffer, end - buffer, '"', qoffset);
+ if (qoffset == -1)
+ {
+ pg_log_fatal("quoted field in backup manifest is not terminated");
+ exit(1);
+ }
+
+ /*
+ * If the double-quote we found is the last character on the line or
+ * if it's followed by a tab, we've reached the end of this field.
+ */
+ if (buffer + qoffset >= end || buffer[qoffset + 1] == '\t')
+ break;
+
+ /* Otherwise, the next character should be another double-quote. */
+ if (buffer[qoffset + 1] != '"')
+ {
+ pg_log_fatal("invalid quoted field in backup manifest");
+ exit(1);
+ }
+
+ /* Skip both double-quotes and go around again. */
+ qoffset += 2;
+ ++dqcount;
+ }
+
+ /*
+ * At this point, we know that qoffset is the offset, relative to buffer,
+ * of the closing double-quote, and that dqcount is the number of escaped
+ * double-quotes within the field, and that all of those escape sequences
+ * are proper. Extract and de-escape the data in the field.
+ *
+ * The amount of space needed for the result is equal to the raw token
+ * length, minus two for the double quotes at the start and end, minus one
+ * for each doubled double-quote within the token, plus one for the
+ * trailing zero byte.
+ */
+ toklen = qoffset + 1;
+ *result = palloc(toklen - dqcount - 1);
+ bufpos = 1;
+ resultpos = 0;
+ while (bufpos < qoffset)
+ {
+ (*result)[resultpos] = buffer[bufpos];
+ bufpos += (buffer[bufpos] == '"' ? 2 : 1);
+ ++resultpos;
+ }
+ (*result)[resultpos] = '\0';
+ Assert(resultpos == toklen - dqcount - 2);
+
+ return toklen;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
--
2.17.2 (Apple Git-113)
[application/octet-stream] v8-0004-Modify-server-code-to-generate-backup-manifest-in.patch (8.4K, ../../CA+TgmoZRTBiPyvQEwV79PU1ePTtSEo2UeVncrkJMbn1sU1gnRA@mail.gmail.com/4-v8-0004-Modify-server-code-to-generate-backup-manifest-in.patch)
download | inline diff:
From edbe7493547c7e2a83847acf4e34c72ea1fd7575 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Thu, 27 Feb 2020 17:46:43 +0530
Subject: [PATCH v8 4/5] Modify server code to generate backup manifest in JSON
format.
This will eventually get merged into the previous patch to add
backup manifest functionality to the server, but I'm keeping
it separate for now because I don't have the validator working
with this format yet.
---
src/backend/replication/basebackup.c | 156 +++++++++++++--------------
1 file changed, 78 insertions(+), 78 deletions(-)
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 99e102b2a7..56a10ae259 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
#include "utils/resowner.h"
@@ -63,6 +64,7 @@ struct manifest_info
pg_checksum_type checksum_type;
pg_sha256_ctx manifest_ctx;
uint64 manifest_size;
+ bool first_file;
bool still_checksumming;
};
@@ -87,7 +89,6 @@ static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
const char *pathname, size_t size, time_t mtime,
pg_checksum_context *checksum_ctx);
static void SendBackupManifest(manifest_info *manifest);
-static char *escape_field_for_manifest(const char *s);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -947,9 +948,12 @@ InitializeManifest(manifest_info *manifest, pg_checksum_type checksum_type)
manifest->checksum_type = checksum_type;
pg_sha256_init(&manifest->manifest_ctx);
manifest->manifest_size = UINT64CONST(0);
+ manifest->first_file = true;
manifest->still_checksumming = true;
- AppendToManifest(manifest, "PostgreSQL-Backup-Manifest-Version\t1\n");
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
}
/*
@@ -980,8 +984,8 @@ AddFileToManifest(manifest_info *manifest, const char *spcoid,
pg_checksum_context *checksum_ctx)
{
char pathbuf[MAXPGPATH];
- char *escaped_filename;
- static char timebuf[128];
+ int pathlen;
+ StringInfoData buf;
/*
* If this file is part of a tablespace, the filename passed to this
@@ -991,44 +995,86 @@ AddFileToManifest(manifest_info *manifest, const char *spcoid,
*/
if (spcoid != NULL)
{
- snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid, filename);
- filename = pathbuf;
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
}
- /* Escape filename, if necessary. */
- escaped_filename = escape_field_for_manifest(filename);
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
/*
- * Convert time to a string. Since it's not clear what time zone to use
- * and since time zone definitions can change, possibly causing confusion,
- * use GMT always.
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
*/
- pg_strftime(timebuf, sizeof(timebuf), "%Y-%m-%d %H:%M:%S %Z",
- pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
- /* Add to manifest. */
- AppendToManifest(manifest, "File\t%s\t%zu\t%s\t%s",
- escaped_filename == NULL ? filename : escaped_filename,
- size, timebuf, pg_checksum_type_name(checksum_ctx->type));
+ /* Add checksum information. */
if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
{
uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
- char checksumstringbuf[PG_CHECKSUM_MAX_LENGTH * 2 + 1];
int checksumlen;
- int checksumstringlen;
- AppendStringToManifest(manifest, ":");
checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
- checksumstringlen = hex_encode((char *) checksumbuf, checksumlen,
- checksumstringbuf);
- checksumstringbuf[checksumstringlen] = '\0';
- AppendStringToManifest(manifest, checksumstringbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
}
- AppendStringToManifest(manifest, "\n");
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
/* Avoid leaking memory. */
- if (escaped_filename != NULL)
- pfree(escaped_filename);
+ pfree(buf.data);
}
/*
@@ -1042,6 +1088,9 @@ SendBackupManifest(manifest_info *manifest)
char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
size_t manifest_bytes_done = 0;
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
/*
* Append manifest checksum, so that the problems with the manifest itself
* can be detected.
@@ -1055,11 +1104,11 @@ SendBackupManifest(manifest_info *manifest)
*/
manifest->still_checksumming = false;
pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
- AppendStringToManifest(manifest, "Manifest-Checksum\t");
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
AppendStringToManifest(manifest, checksumstringbuf);
- AppendStringToManifest(manifest, "\n");
+ AppendStringToManifest(manifest, "\"}\n");
/*
* We've written all the data to the manifest file. Rewind the file so
@@ -1107,55 +1156,6 @@ SendBackupManifest(manifest_info *manifest)
BufFileClose(manifest->buffile);
}
-/*
- * Escape a field for inclusion in a manifest.
- *
- * We use the following escaping rule: If a field contains \t, \r, or \n,
- * the field must be surrounded by double-quotes, and any internal double
- * quotes must be doubled. Otherwise, no escaping is required.
- *
- * The return value is a new palloc'd string with escaping added, or NULL
- * if no escaping is required.
- */
-static char *
-escape_field_for_manifest(const char *s)
-{
- bool escaping_required = false;
- int escaped_length = 2;
- const char *t;
- char *result;
- char *r;
-
- for (t = s; *t != '\0'; ++t)
- {
- if (*t == '\t' || *t == '\r' || *t == '\n')
- escaping_required = true;
- if (*t == '"')
- ++escaped_length;
- ++escaped_length;
- }
-
- if (!escaping_required)
- return NULL;
-
- result = palloc(escaped_length + 1);
- result[0] = '"';
- result[escaped_length - 1] = '"';
- result[escaped_length] = '\0';
- r = result + 1;
-
- for (t = s; *t != '\0'; ++t)
- {
- *(r++) = *t;
- if (*t == '"')
- *(r++) = *t;
- }
-
- Assert(r == &result[escaped_length - 1]);
-
- return result;
-}
-
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
--
2.17.2 (Apple Git-113)
[application/octet-stream] v8-0005-WIP-Validate-JSON-format-manifest.patch (23.3K, ../../CA+TgmoZRTBiPyvQEwV79PU1ePTtSEo2UeVncrkJMbn1sU1gnRA@mail.gmail.com/5-v8-0005-WIP-Validate-JSON-format-manifest.patch)
download | inline diff:
From 527eaf616026b8132937c543e961204b7051145c Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Thu, 27 Feb 2020 21:05:02 +0530
Subject: [PATCH v8 5/5] WIP: Validate JSON-format manifest.
---
src/bin/pg_validatebackup/pg_validatebackup.c | 690 ++++++++----------
1 file changed, 315 insertions(+), 375 deletions(-)
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
index 4f47b20855..1b0a449470 100644
--- a/src/bin/pg_validatebackup/pg_validatebackup.c
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -19,9 +19,11 @@
#include "common/checksum_helper.h"
#include "common/hashfn.h"
+#include "common/jsonapi.h"
#include "common/logging.h"
#include "fe_utils/simple_list.h"
#include "getopt_long.h"
+#include "mb/pg_wchar.h"
/*
* For efficiency, we'd like our hash table containing information about the
@@ -60,8 +62,8 @@
#define FIELDS_PER_FILE_LINE 4
/*
- * Each "File" line in the manifest file is parsed to produce an object
- * like this.
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
*/
typedef struct manifestfile
{
@@ -92,6 +94,49 @@ static uint32 hash_string_pointer(char *s);
#define SH_DEFINE
#include "lib/simplehash.h"
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+typedef struct
+{
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ manifestfiles_hash *ht;
+ char *pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+} JsonManifestParseState;
+
/*
* All of the context information we need while checking a backup manifest.
*/
@@ -105,8 +150,15 @@ typedef struct validator_context
} validator_context;
static manifestfiles_hash * parse_manifest_file(char *manifest_path);
-static void parse_file_line_from_manifest(manifestfile *f, char *rest,
- int restlen);
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+
static void validate_backup_directory(validator_context *context,
char *relpath, char *fullpath);
static void validate_backup_file(validator_context *context,
@@ -121,9 +173,6 @@ static void pg_validator_error(validator_context *context,
pg_attribute_printf(2, 3);
static bool should_ignore_relpath(validator_context *context, char *relpath);
-static char *extractstr(char *buffer, int length);
-static int findchar(char *buffer, int size, char c, int start_position);
-static int findfield(char *buffer, char *end, char **result);
static int hexdecode_char(char c);
static bool hexdecode_string(uint8 *result, char *input, int nbytes);
static void usage(void);
@@ -275,19 +324,15 @@ parse_manifest_file(char *manifest_path)
int fd;
struct stat statbuf;
off_t estimate;
- off_t bytes_read = 0;
- off_t bytes_consumed = 0;
uint32 initial_size;
manifestfiles_hash *ht;
char *buffer;
- uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
- uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
- int buffer_position = 0;
- int buffer_size = 0;
- int buffer_maxsize = 2 * READ_CHUNK_SIZE;
- int line_number = 0;
- bool saw_manifest_checksum_line = false;
+ int rc;
pg_sha256_ctx manifest_ctx;
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
/* Prepare to compute a checksum of the manifest itself. */
pg_sha256_init(&manifest_ctx);
@@ -313,296 +358,310 @@ parse_manifest_file(char *manifest_path)
/* Create the hash table. */
ht = manifestfiles_create(initial_size, NULL);
- /* Initialize our read buffer. */
- buffer = pg_malloc(buffer_maxsize);
-
/*
- * Loop until we've read it all.
+ * Slurp in the whole file.
*
- * The file size shouldn't be changing, so it seems fine to just error out
- * if the final length is different from what stat() told us.
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
*/
- while (bytes_consumed < statbuf.st_size)
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
{
- int line_length;
- int first_field_length;
- char *rest;
- int restlen;
+ if (rc < 0)
+ pg_log_fatal("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ pg_log_fatal("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ exit(1);
+ }
- /* Find next newline if any. */
- line_length = findchar(buffer, buffer_size, '\n', buffer_position);
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, statbuf.st_size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.ht = ht;
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Parse JSON. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ {
+ pg_log_fatal("could not parse backup manifest: %s",
+ json_errdetail(json_error, lex));
+ exit(1);
+ }
+ if (parse.state != JM_EXPECT_EOF)
+ {
+ pg_log_fatal("could not parse backup manifest: %s",
+ "manifest ended unexpectedly");
+ }
- /* If no newline was found, we need to read more data and try again. */
- if (line_length == -1)
- {
- size_t bytes_to_read;
- int rc;
+ /* OK, we're done with the manifest file. */
+ close(fd);
- bytes_to_read = Min(statbuf.st_size - bytes_read, READ_CHUNK_SIZE);
- if (bytes_to_read == 0)
- {
- pg_log_fatal("manifest file line not terminated by newline");
- exit(1);
- }
- if (bytes_to_read + READ_CHUNK_SIZE > buffer_maxsize)
- {
- buffer_maxsize += READ_CHUNK_SIZE;
- buffer = pg_realloc(buffer, buffer_maxsize);
- Assert(bytes_to_read + READ_CHUNK_SIZE <= buffer_maxsize);
- }
- rc = read(fd, buffer + buffer_size, bytes_to_read);
- if (rc != bytes_to_read)
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+static void
+json_manifest_parse_failure(char *msg)
+{
+ pg_log_fatal("could not parse backup manifest: %s", msg);
+ exit(1);
+}
+
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected object start");
+ break;
+ }
+}
+
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+ manifestfile *tabent;
+ bool found;
+ int checksum_string_length;
+ char *ep;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL)
+ json_manifest_parse_failure("missing pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure("missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure("checksum without algorithm");
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(parse->ht, parse->pathname, &found);
+ if (found)
{
- if (rc < 0)
- pg_log_fatal("could not read file \"%s\": %m",
- manifest_path);
- else
- pg_log_fatal("could not read file \"%s\": read %d of %zu",
- manifest_path, rc, bytes_to_read);
+ pg_log_fatal("duplicate pathname in backup manifest: \"%s\"",
+ parse->pathname);
exit(1);
}
- buffer_size += rc;
- bytes_read += rc;
- continue;
- }
- /* Increment line number. */
- ++line_number;
-
- /* The manifest checksum should be the last thing in the file. */
- if (saw_manifest_checksum_line)
- {
- pg_log_fatal("unexpected data follows manifest checksum");
- exit(1);
- }
+ /* Initialize some fields. */
+ tabent->matched = false;
+ tabent->bad = false;
- /* Find first field on line, and remaining line contents. */
- first_field_length =
- findchar(buffer, buffer_size, '\t', buffer_position);
- rest = buffer + buffer_position + first_field_length + 1;
- restlen = line_length - (first_field_length + 1);
+ /* Parse size. */
+ tabent->size = strtoll(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure("file size is not an integer");
- /*
- * Check the first word of the line to see what kind of line it is.
- */
- if (first_field_length == KWL_MANIFEST_VERSION &&
- memcmp(buffer + buffer_position, KW_MANIFEST_VERSION,
- KWL_MANIFEST_VERSION) == 0)
- {
- if (line_number != 1)
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ tabent->checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm,
+ &tabent->checksum_type))
{
- pg_log_fatal("manifest file version should only be specified at line 1");
+ pg_log_fatal("unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
exit(1);
}
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ tabent->checksum_length = 0;
+ tabent->checksum_payload = NULL;
+ }
else
{
- char *line = buffer + buffer_position;
- char *version;
- version = extractstr(line + first_field_length + 1,
- line_length - (first_field_length + 1));
- if (strcmp(version, "1") != 0)
+ tabent->checksum_length = checksum_string_length / 2;
+ tabent->checksum_payload = palloc(tabent->checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(tabent->checksum_payload,
+ parse->checksum,
+ tabent->checksum_length))
{
- pg_log_fatal("unrecognized manifest version: \"%s\"",
- version);
+ pg_log_fatal("invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, tabent->checksum_payload);
exit(1);
}
}
- }
- else if (first_field_length == KWL_MANIFEST_FILE &&
- memcmp(buffer + buffer_position, KW_MANIFEST_FILE,
- KWL_MANIFEST_FILE) == 0)
- {
- manifestfile f;
- manifestfile *tabent;
- bool found;
- /* Parse this line. */
- parse_file_line_from_manifest(&f, rest, restlen);
-
- /* Make a new entry in the hash table for it. */
- tabent = manifestfiles_insert(ht, f.pathname, &found);
- if (found)
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
{
- pg_log_fatal("duplicate pathname in backup manifest: \"%s\"",
- f.pathname);
- exit(1);
+ pfree(parse->size);
+ parse->size = NULL;
}
-
- /* Copy in all the relevant details. */
- tabent->size = f.size;
- tabent->checksum_type = f.checksum_type;
- tabent->checksum_length = f.checksum_length;
- tabent->checksum_payload = f.checksum_payload;
- tabent->matched = false;
- tabent->bad = false;
- }
- else if (first_field_length == KWL_MANIFEST_CHECKSUM &&
- memcmp(buffer + buffer_position, KW_MANIFEST_CHECKSUM,
- KWL_MANIFEST_CHECKSUM) == 0)
- {
- saw_manifest_checksum_line = true;
- if (restlen != PG_SHA256_DIGEST_STRING_LENGTH - 1)
+ if (parse->algorithm != NULL)
{
- pg_log_fatal("manifest file checksum has unexpected length: %d",
- restlen);
- exit(1);
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
}
- if (!hexdecode_string(manifest_checksum_expected, rest,
- PG_SHA256_DIGEST_LENGTH))
+ if (parse->checksum != NULL)
{
- pg_log_fatal("invalid manifest checksum: \"%s\"",
- extractstr(rest, restlen));
- exit(1);
+ pfree(parse->checksum);
+ parse->checksum = NULL;
}
- }
- else if (first_field_length == -1)
- {
- pg_log_fatal("manifest file keyword not terminated by tab");
- exit(1);
- }
- else
- {
- char *kw;
-
- kw = extractstr(buffer + buffer_position, first_field_length);
- pg_log_fatal("unrecognized manifest file keyword: \"%s\"", kw);
- exit(1);
- }
-
- /* Update manifest checksum, if needed. */
- if (!saw_manifest_checksum_line)
- pg_sha256_update(&manifest_ctx, (uint8 *) buffer + buffer_position,
- line_length + 1);
- /* Advance buffer position over the data we just read. */
- buffer_position += line_length + 1;
-
- /* Also mark these bytes as consumed so we know when to stop. */
- bytes_consumed += line_length + 1;
-
- /*
- * We don't want to incur the expensive of using memmove() to discard
- * data after every line, because the lines are short compared to the
- * chunk size -- but we must do it at least now and then, or we'll
- * have to keep growing the buffer.
- */
- if (buffer_position >= READ_CHUNK_SIZE)
- {
- int leftover_bytes = buffer_size - buffer_position;
-
- if (leftover_bytes > 0)
- memmove(buffer, buffer + buffer_position, leftover_bytes);
- buffer_size -= buffer_position;
- buffer_position = 0;
- }
+ /* Expect next file (or end of list). */
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected object end");
+ break;
}
+}
+
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
- /* Checksum verification. */
- if (!saw_manifest_checksum_line)
- pg_log_fatal("manifest has no checksum");
- pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
- if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
- PG_SHA256_DIGEST_LENGTH) != 0)
+ switch (parse->state)
{
- pg_log_fatal("manifest checksum does not match");
- exit(1);
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected array start");
+ break;
}
-
- /* OK, we're done with the manifest file. */
- close(fd);
-
- /* Return the hash table we constructed. */
- return ht;
}
-/*
- * The caller passes the remainder of the line, excluding the initial "File\t"
- * portion.
- */
static void
-parse_file_line_from_manifest(manifestfile *f, char *rest, int restlen)
+json_manifest_array_end(void *state)
{
- char *end = rest + restlen;
- char *field[FIELDS_PER_FILE_LINE];
- unsigned long filesize;
- char *ep;
- pg_checksum_type checksum_type;
- int raw_checksum_length = 0;
- char *raw_checksum_payload = NULL;
- int checksum_length;
- uint8 *checksum_payload;
- int i;
- char *s;
+ JsonManifestParseState *parse = state;
- /* Split the line into fields. */
- for (i = 0; i < FIELDS_PER_FILE_LINE; ++i)
+ switch (parse->state)
{
- int toklen;
-
- toklen = findfield(rest, end, &field[i]);
- if (rest + toklen >= end && i + 1 < FIELDS_PER_FILE_LINE)
- {
- pg_log_fatal("manifest file line has too few fields");
- exit(1);
- }
- rest += toklen + 1;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected array end");
+ break;
}
+}
- /* We expect to have used the entire line. */
- if (rest < end)
- {
- pg_log_fatal("manifest file line has too many fields");
- exit(1);
- }
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
- /* Parse the size. */
- filesize = strtoul(field[1], &ep, 10);
- if (*ep)
+ switch (parse->state)
{
- pg_log_fatal("manifest file size for file \"%s\" is not a number",
- field[0]);
- exit(1);
+ case JM_EXPECT_VERSION_FIELD:
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure("expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure("expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure("unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure("expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected object field");
+ break;
}
+}
- /* Parse the checksum type. */
- for (s = field[3]; s[0] != '\0' && s[0] != ':'; ++s)
- ;
- if (*s)
- {
- raw_checksum_payload = s + 1;
- raw_checksum_length = strlen(raw_checksum_payload);
- *s = '\0';
- }
- if (!pg_checksum_parse_type(field[3], &checksum_type))
- {
- pg_log_fatal("unrecognized checksum algorithm for file \"%s\": \"%s\"",
- field[0], field[3]);
- exit(1);
- }
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
- /* Decode the checksum payload. */
- checksum_length = raw_checksum_length / 2;
- if (checksum_length == 0)
- checksum_payload = NULL;
- else
+ switch (parse->state)
{
- checksum_payload = palloc(checksum_length);
- if (!hexdecode_string(checksum_payload, raw_checksum_payload,
- checksum_length))
- {
- pg_log_fatal("invalid checksum for file \"%s\": \"%s\"",
- field[0], raw_checksum_payload);
- exit(1);
- }
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure("unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ pg_log_info("* manifest_checksum = %s", token);
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected scalar");
+ break;
}
-
- /* Fill the output struct. */
- f->pathname = field[0];
- f->size = filesize;
- f->checksum_type = checksum_type;
- f->checksum_length = checksum_length;
- f->checksum_payload = checksum_payload;
}
/*
@@ -917,125 +976,6 @@ should_ignore_relpath(validator_context *context, char *relpath)
return false;
}
-/*
- * Extract a NUL-terminated string from a larger buffer.
- */
-static char *
-extractstr(char *buffer, int length)
-{
- char *s = palloc(length + 1);
-
- memcpy(s, buffer, length);
- s[length] = '\0';
-
- return s;
-}
-
-/*
- * Find the next instance of a given character within a buffer that
- * occurs at or after start_position. If there is none, returns -1; else
- * returns the difference between the position at which the character was
- * found and the start position.
- */
-static int
-findchar(char *buffer, int size, char c, int start_position)
-{
- int i;
-
- for (i = start_position; i < size; ++i)
- if (buffer[i] == c)
- return i - start_position;
- return -1;
-}
-
-/*
- * Extract the next field from a line of text read from the manifest file.
- */
-static int
-findfield(char *buffer, char *end, char **result)
-{
- int qoffset = 1;
- int dqcount = 0;
- int toklen;
- int bufpos;
- int resultpos;
-
- /*
- * If this field is unquoted, we just stop at the next tab; if there's
- * none, we stop at the end of the line. Note that if buffer == end, it
- * just means that the last field on the line is empty.
- */
- if (buffer == end || *buffer != '"')
- {
- toklen = findchar(buffer, end - buffer, '\t', 0);
-
- if (toklen == -1)
- toklen = end - buffer;
- *result = extractstr(buffer, toklen);
- return toklen;
- }
-
- /*
- * Our escaping convention is that if the field contains a tab, it must be
- * surrounded by double-quotes and any internal double-quotes must be
- * doubled.
- */
- while (1)
- {
- /* Where's the next double quote? */
- qoffset += findchar(buffer, end - buffer, '"', qoffset);
- if (qoffset == -1)
- {
- pg_log_fatal("quoted field in backup manifest is not terminated");
- exit(1);
- }
-
- /*
- * If the double-quote we found is the last character on the line or
- * if it's followed by a tab, we've reached the end of this field.
- */
- if (buffer + qoffset >= end || buffer[qoffset + 1] == '\t')
- break;
-
- /* Otherwise, the next character should be another double-quote. */
- if (buffer[qoffset + 1] != '"')
- {
- pg_log_fatal("invalid quoted field in backup manifest");
- exit(1);
- }
-
- /* Skip both double-quotes and go around again. */
- qoffset += 2;
- ++dqcount;
- }
-
- /*
- * At this point, we know that qoffset is the offset, relative to buffer,
- * of the closing double-quote, and that dqcount is the number of escaped
- * double-quotes within the field, and that all of those escape sequences
- * are proper. Extract and de-escape the data in the field.
- *
- * The amount of space needed for the result is equal to the raw token
- * length, minus two for the double quotes at the start and end, minus one
- * for each doubled double-quote within the token, plus one for the
- * trailing zero byte.
- */
- toklen = qoffset + 1;
- *result = palloc(toklen - dqcount - 1);
- bufpos = 1;
- resultpos = 0;
- while (bufpos < qoffset)
- {
- (*result)[resultpos] = buffer[bufpos];
- bufpos += (buffer[bufpos] == '"' ? 2 : 1);
- ++resultpos;
- }
- (*result)[resultpos] = '\0';
- Assert(resultpos == toklen - dqcount - 2);
-
- return toklen;
-}
-
/*
* Helper function for manifestfiles hash table.
*/
--
2.17.2 (Apple Git-113)
[application/octet-stream] v8-0002-Generate-backup-manifests-for-base-backups.patch (33.1K, ../../CA+TgmoZRTBiPyvQEwV79PU1ePTtSEo2UeVncrkJMbn1sU1gnRA@mail.gmail.com/6-v8-0002-Generate-backup-manifests-for-base-backups.patch)
download | inline diff:
From d2eca8b7a795302ec113ff72eb80cfac6ab3f222 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 5 Feb 2020 12:34:27 -0500
Subject: [PATCH v8 2/5] Generate backup manifests for base backups.
The manifest includes the file name, size, last modification time, and
a checksum for each file backed up, as well as a checksum for the
manifest itself. By default, we use CRC-32C for the checksum
algorithm, because we are trying to detect corruption and user error,
not foil an adversary. However, pg_basebackup and the server-side
BASE_BACKUP command now have options to select the checksum algorithm,
so users wanting a cryptographic hash function can select SHA-224,
SHA-256, SHA-384, or SHA-512; and users not wanting any checksums at
all can disable them. Using a cryptographic hash function in place of
CRC-32C consumes significantly more CPU cycles, which may slow down
backups in some cases.
Robert Haas with help from Rushabh Lathia and Suraj Kharage.
(This version is improvement from previous versions in that it can
spool the manifest to disk as it is being generated; thus, even a
very large manifest should not run the server out of memory, though
it could conceivably run it out of disk space.)
---
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 362 +++++++++++++++++++++++--
src/backend/replication/repl_gram.y | 6 +
src/backend/replication/repl_scanner.l | 1 +
src/backend/replication/walsender.c | 30 ++
src/bin/pg_basebackup/pg_basebackup.c | 130 ++++++++-
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
8 files changed, 512 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index d19408b3be..544ba4b84d 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10534,7 +10534,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index ca8bebf432..1729931597 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "lib/stringinfo.h"
#include "libpq/libpq.h"
@@ -31,6 +32,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -40,6 +42,7 @@
#include "utils/builtins.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
typedef struct
@@ -51,20 +54,40 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ pg_checksum_type checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ int still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest, pg_checksum_type);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *filename, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
+static char *escape_field_for_manifest(const char *s);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -100,6 +123,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -243,15 +276,21 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt->checksum_type);
total_checksum_failures = 0;
@@ -288,7 +327,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/* Send tablespace header */
@@ -335,7 +377,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -343,11 +386,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -355,10 +401,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -577,7 +624,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -600,16 +647,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -623,6 +674,8 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
}
/*
@@ -653,8 +706,11 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -741,6 +797,21 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval, &opt->checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
@@ -862,6 +933,229 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, pg_checksum_type checksum_type)
+{
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->still_checksumming = true;
+
+ AppendToManifest(manifest, "PostgreSQL-Backup-Manifest-Version\t1\n");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *filename, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ char *escaped_filename;
+ static char timebuf[128];
+
+ /*
+ * If this file is part of a tablespace, the filename passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid, filename);
+ filename = pathbuf;
+ }
+
+ /* Escape filename, if necessary. */
+ escaped_filename = escape_field_for_manifest(filename);
+
+ /*
+ * Convert time to a string. Since it's not clear what time zone to use
+ * and since time zone definitions can change, possibly causing confusion,
+ * use GMT always.
+ */
+ pg_strftime(timebuf, sizeof(timebuf), "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+
+ /* Add to manifest. */
+ AppendToManifest(manifest, "File\t%s\t%zu\t%s\t%s",
+ escaped_filename == NULL ? filename : escaped_filename,
+ size, timebuf, pg_checksum_type_name(checksum_ctx->type));
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ char checksumstringbuf[PG_CHECKSUM_MAX_LENGTH * 2 + 1];
+ int checksumlen;
+ int checksumstringlen;
+
+ AppendStringToManifest(manifest, ":");
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+ checksumstringlen = hex_encode((char *) checksumbuf, checksumlen,
+ checksumstringbuf);
+ checksumstringbuf[checksumstringlen] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ }
+ AppendStringToManifest(manifest, "\n");
+
+ /* Avoid leaking memory. */
+ if (escaped_filename != NULL)
+ pfree(escaped_filename);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "Manifest-Checksum\t");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
+/*
+ * Escape a field for inclusion in a manifest.
+ *
+ * We use the following escaping rule: If a field contains \t, \r, or \n,
+ * the field must be surrounded by double-quotes, and any internal double
+ * quotes must be doubled. Otherwise, no escaping is required.
+ *
+ * The return value is a new palloc'd string with escaping added, or NULL
+ * if no escaping is required.
+ */
+static char *
+escape_field_for_manifest(const char *s)
+{
+ bool escaping_required = false;
+ int escaped_length = 2;
+ const char *t;
+ char *result;
+ char *r;
+
+ for (t = s; *t != '\0'; ++t)
+ {
+ if (*t == '\t' || *t == '\r' || *t == '\n')
+ escaping_required = true;
+ if (*t == '"')
+ ++escaped_length;
+ ++escaped_length;
+ }
+
+ if (!escaping_required)
+ return NULL;
+
+ result = palloc(escaped_length + 1);
+ result[0] = '"';
+ result[escaped_length - 1] = '"';
+ result[escaped_length] = '\0';
+ r = result + 1;
+
+ for (t = s; *t != '\0'; ++t)
+ {
+ *(r++) = *t;
+ if (*t == '"')
+ *(r++) = *t;
+ }
+
+ Assert(r == &result[escaped_length - 1]);
+
+ return result;
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -922,11 +1216,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -959,6 +1257,10 @@ sendFileWithContent(const char *filename, const char *content)
MemSet(buf, 0, pad);
pq_putmessage('d', buf, pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -969,7 +1271,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1002,7 +1305,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1021,7 +1325,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1301,7 +1605,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1309,7 +1614,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1379,8 +1685,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1397,6 +1704,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1566,6 +1876,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
ereport(ERROR,
(errmsg("base backup could not send data, aborting backup")));
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1590,6 +1903,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
len += cnt;
throttle(cnt);
}
@@ -1597,7 +1911,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1621,6 +1936,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..0621884ad8 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,7 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -214,6 +215,11 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..5653d233b5 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,7 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index abb533b9d0..ae51faf284 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index 2551cf38c9..ad46f38e7f 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -135,6 +141,7 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -180,6 +187,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -386,6 +399,8 @@ usage(void)
printf(_(" --no-slot prevent creation of temporary replication slot\n"));
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1183,6 +1198,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1654,6 +1694,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1664,6 +1762,7 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_checksums_clause = NULL;
int i;
char xlogstart[64];
char xlogend[64];
@@ -1671,6 +1770,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1724,6 +1824,9 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1738,7 +1841,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s",
escaped_label,
showprogress ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1746,7 +1849,8 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_checksums_clause ? manifest_checksums_clause : "");
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1834,7 +1938,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1863,6 +1968,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && serverMajor >= 1300)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2065,6 +2183,7 @@ main(int argc, char **argv)
{"waldir", required_argument, NULL, 1},
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
+ {"manifest-checksums", required_argument, NULL, 'm'},
{NULL, 0, NULL, 0}
};
int c;
@@ -2092,7 +2211,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2233,6 +2352,9 @@ main(int argc, char **argv)
case 3:
verify_checksums = false;
break;
+ case 'm':
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-03 10:34 ` tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-03 10:34 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Suraj Kharage <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 2/27/20 9:22 PM, Robert Haas wrote:
> Here's a new patch set for this feature.
Thanks Robert. After applying all the 5 patches (v8-00*) against PG v13
(commit id -afb5465e0cfce7637066eaaaeecab30b0f23fbe3) ,
There are few issues/observations
1)Getting segmentation fault error if we try pg_validatebackup against
a valid backup_manifest file but data directory path is WRONG
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D bk
--manifest-checksums=sha224
[centos@tushar-ldap-docker bin]$ cp bk/backup_manifest /tmp/.
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup -m
/tmp/backup_manifest random_directory/
pg_validatebackup: * manifest_checksum =
f0460cd6aa13cf0c5e35426a41af940a9231e6425cd65115a19778b7abfdaef9
pg_validatebackup: error: could not open directory "random_directory":
No such file or directory
Segmentation fault
2) when used '-R' option at the time of create base backup
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D bar -R
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup bar
pg_validatebackup: * manifest_checksum =
a195d3a3a82a41200c9ac92c12d764d23c810e7e91b31c44a7d04f67ce012edc
pg_validatebackup: error: "standby.signal" is present on disk but not in
the manifest
pg_validatebackup: error: "postgresql.auto.conf" has size 286 on disk
but size 88 in the manifest
[centos@tushar-ldap-docker bin]$
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-03 14:49 ` tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-03 14:49 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Suraj Kharage <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/3/20 4:04 PM, tushar wrote:
> Thanks Robert. After applying all the 5 patches (v8-00*) against PG
> v13 (commit id -afb5465e0cfce7637066eaaaeecab30b0f23fbe3) ,
There is a scenario where pg_validatebackup is not throwing an error if
some file deleted from pg_wal/ folder and but later at the time of
restoring - we are getting an error
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D test1
[centos@tushar-ldap-docker bin]$ ls test1/pg_wal/
000000010000000000000010 archive_status
[centos@tushar-ldap-docker bin]$ rm -rf test1/pg_wal/*
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup test1
pg_validatebackup: * manifest_checksum =
88f1ed995c83e86252466a2c88b3e660a69cfc76c169991134b101c4f16c9df7
pg_validatebackup: backup successfully verified
[centos@tushar-ldap-docker bin]$ ./pg_ctl -D test1 start -o '-p 3333'
waiting for server to start....2020-03-02 20:05:22.732 IST [21441] LOG:
starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by gcc
(GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv6 address
"::1", port 3333
2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv4 address
"127.0.0.1", port 3333
2020-03-02 20:05:22.736 IST [21441] LOG: listening on Unix socket
"/tmp/.s.PGSQL.3333"
2020-03-02 20:05:22.739 IST [21442] LOG: database system was
interrupted; last known up at 2020-03-02 20:04:35 IST
2020-03-02 20:05:22.739 IST [21442] LOG: creating missing WAL directory
"pg_wal/archive_status"
2020-03-02 20:05:22.886 IST [21442] LOG: invalid checkpoint record
2020-03-02 20:05:22.886 IST [21442] FATAL: could not locate required
checkpoint record
2020-03-02 20:05:22.886 IST [21442] HINT: If you are restoring from a
backup, touch
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/recovery.signal" and
add required recovery options.
If you are not restoring from a backup, try removing the file
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label".
Be careful: removing
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label" will
result in a corrupt cluster if restoring from a backup.
2020-03-02 20:05:22.886 IST [21441] LOG: startup process (PID 21442)
exited with exit code 1
2020-03-02 20:05:22.886 IST [21441] LOG: aborting startup due to
startup process failure
2020-03-02 20:05:22.889 IST [21441] LOG: database system is shut down
stopped waiting
pg_ctl: could not start server
Examine the log output.
[centos@tushar-ldap-docker bin]$
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-04 09:56 ` tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-04 09:56 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Suraj Kharage <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
Another observation , if i change the ownership of a file which is under
global/ directory
i.e
[root@tushar-ldap-docker global]# chown enterprisedb 2396
and run the pg_validatebackup command, i am getting this message -
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup gggg
pg_validatebackup: * manifest_checksum =
e8cb007bcc9c0deab6eff51cd8d9d9af6af35b86e02f3055e60e70e56737e877
pg_validatebackup: error: could not open file "global/2396": Permission
denied
*** Error in `./pg_validatebackup': double free or corruption (!prev):
0x0000000001850ba0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81679)[0x7fa2248e3679]
./pg_validatebackup[0x401f4c]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa224884505]
./pg_validatebackup[0x402049]
======= Memory map: ========
00400000-00415000 r-xp 00000000 fd:03 4044545
/home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
00614000-00615000 r--p 00014000 fd:03 4044545
/home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
00615000-00616000 rw-p 00015000 fd:03 4044545
/home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
017f3000-01878000 rw-p 00000000 00:00 0
[heap]
7fa218000000-7fa218021000 rw-p 00000000 00:00 0
7fa218021000-7fa21c000000 ---p 00000000 00:00 0
7fa21e122000-7fa21e137000 r-xp 00000000 fd:03 141697
/usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fa21e137000-7fa21e336000 ---p 00015000 fd:03 141697
/usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fa21e336000-7fa21e337000 r--p 00014000 fd:03 141697
/usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fa21e337000-7fa21e338000 rw-p 00015000 fd:03 141697
/usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fa21e338000-7fa224862000 r--p 00000000 fd:03 266442
/usr/lib/locale/locale-archive
7fa224862000-7fa224a25000 r-xp 00000000 fd:03 134456
/usr/lib64/libc-2.17.so
7fa224a25000-7fa224c25000 ---p 001c3000 fd:03 134456
/usr/lib64/libc-2.17.so
7fa224c25000-7fa224c29000 r--p 001c3000 fd:03 134456
/usr/lib64/libc-2.17.so
7fa224c29000-7fa224c2b000 rw-p 001c7000 fd:03 134456
/usr/lib64/libc-2.17.so
7fa224c2b000-7fa224c30000 rw-p 00000000 00:00 0
7fa224c30000-7fa224c47000 r-xp 00000000 fd:03 134485
/usr/lib64/libpthread-2.17.so
7fa224c47000-7fa224e46000 ---p 00017000 fd:03 134485
/usr/lib64/libpthread-2.17.so
7fa224e46000-7fa224e47000 r--p 00016000 fd:03 134485
/usr/lib64/libpthread-2.17.so
7fa224e47000-7fa224e48000 rw-p 00017000 fd:03 134485
/usr/lib64/libpthread-2.17.so
7fa224e48000-7fa224e4c000 rw-p 00000000 00:00 0
7fa224e4c000-7fa224e90000 r-xp 00000000 fd:03 4044478
/home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
7fa224e90000-7fa225090000 ---p 00044000 fd:03 4044478
/home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
7fa225090000-7fa225093000 r--p 00044000 fd:03 4044478
/home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
7fa225093000-7fa225094000 rw-p 00047000 fd:03 4044478
/home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
7fa225094000-7fa2250b6000 r-xp 00000000 fd:03 130333
/usr/lib64/ld-2.17.so
7fa22527d000-7fa2252a2000 rw-p 00000000 00:00 0
7fa2252b3000-7fa2252b5000 rw-p 00000000 00:00 0
7fa2252b5000-7fa2252b6000 r--p 00021000 fd:03 130333
/usr/lib64/ld-2.17.so
7fa2252b6000-7fa2252b7000 rw-p 00022000 fd:03 130333
/usr/lib64/ld-2.17.so
7fa2252b7000-7fa2252b8000 rw-p 00000000 00:00 0
7ffdf354f000-7ffdf3570000 rw-p 00000000 00:00 0
[stack]
7ffdf3572000-7ffdf3574000 r-xp 00000000 00:00 0
[vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
[vsyscall]
Aborted
[centos@tushar-ldap-docker bin]$
I am getting the error message but along with "*** Error in
`./pg_validatebackup': double free or corruption (!prev):
0x0000000001850ba0 ***" messages
Is this expected ?
regards,
On 3/3/20 8:19 PM, tushar wrote:
> On 3/3/20 4:04 PM, tushar wrote:
>> Thanks Robert. After applying all the 5 patches (v8-00*) against PG
>> v13 (commit id -afb5465e0cfce7637066eaaaeecab30b0f23fbe3) ,
>
> There is a scenario where pg_validatebackup is not throwing an error
> if some file deleted from pg_wal/ folder and but later at the time of
> restoring - we are getting an error
>
> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -D test1
>
> [centos@tushar-ldap-docker bin]$ ls test1/pg_wal/
> 000000010000000000000010 archive_status
>
> [centos@tushar-ldap-docker bin]$ rm -rf test1/pg_wal/*
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup test1
> pg_validatebackup: * manifest_checksum =
> 88f1ed995c83e86252466a2c88b3e660a69cfc76c169991134b101c4f16c9df7
> pg_validatebackup: backup successfully verified
>
> [centos@tushar-ldap-docker bin]$ ./pg_ctl -D test1 start -o '-p 3333'
> waiting for server to start....2020-03-02 20:05:22.732 IST [21441]
> LOG: starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by
> gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
> 2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv6 address
> "::1", port 3333
> 2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv4 address
> "127.0.0.1", port 3333
> 2020-03-02 20:05:22.736 IST [21441] LOG: listening on Unix socket
> "/tmp/.s.PGSQL.3333"
> 2020-03-02 20:05:22.739 IST [21442] LOG: database system was
> interrupted; last known up at 2020-03-02 20:04:35 IST
> 2020-03-02 20:05:22.739 IST [21442] LOG: creating missing WAL
> directory "pg_wal/archive_status"
> 2020-03-02 20:05:22.886 IST [21442] LOG: invalid checkpoint record
> 2020-03-02 20:05:22.886 IST [21442] FATAL: could not locate required
> checkpoint record
> 2020-03-02 20:05:22.886 IST [21442] HINT: If you are restoring from a
> backup, touch
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/recovery.signal" and
> add required recovery options.
> If you are not restoring from a backup, try removing the file
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label".
> Be careful: removing
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label" will
> result in a corrupt cluster if restoring from a backup.
> 2020-03-02 20:05:22.886 IST [21441] LOG: startup process (PID 21442)
> exited with exit code 1
> 2020-03-02 20:05:22.886 IST [21441] LOG: aborting startup due to
> startup process failure
> 2020-03-02 20:05:22.889 IST [21441] LOG: database system is shut down
> stopped waiting
> pg_ctl: could not start server
> Examine the log output.
> [centos@tushar-ldap-docker bin]$
>
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-04 10:21 ` tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 03:50 ` Re: backup manifests Suraj Kharage <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: tushar @ 2020-03-04 10:21 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Suraj Kharage <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Another scenario, in which if we modify Manifest-Checksum" value from
backup_manifest file , we are not getting an error
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data/
pg_validatebackup: * manifest_checksum =
28d082921650d0ae881de8ceb122c8d2af5f449f51ecfb446827f7f49f91f65d
pg_validatebackup: backup successfully verified
open backup_manifest file and replace
"Manifest-Checksum":
"8d082921650d0ae881de8ceb122c8d2af5f449f51ecfb446827f7f49f91f65d"}
with
"Manifest-Checksum": "Hello World"}
rerun the pg_validatebackup
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data/
pg_validatebackup: * manifest_checksum = Hello World
pg_validatebackup: backup successfully verified
regards,
On 3/4/20 3:26 PM, tushar wrote:
> Hi,
> Another observation , if i change the ownership of a file which is
> under global/ directory
> i.e
>
> [root@tushar-ldap-docker global]# chown enterprisedb 2396
>
> and run the pg_validatebackup command, i am getting this message -
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup gggg
> pg_validatebackup: * manifest_checksum =
> e8cb007bcc9c0deab6eff51cd8d9d9af6af35b86e02f3055e60e70e56737e877
> pg_validatebackup: error: could not open file "global/2396":
> Permission denied
> *** Error in `./pg_validatebackup': double free or corruption (!prev):
> 0x0000000001850ba0 ***
> ======= Backtrace: =========
> /lib64/libc.so.6(+0x81679)[0x7fa2248e3679]
> ./pg_validatebackup[0x401f4c]
> /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa224884505]
> ./pg_validatebackup[0x402049]
> ======= Memory map: ========
> 00400000-00415000 r-xp 00000000 fd:03 4044545
> /home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
> 00614000-00615000 r--p 00014000 fd:03 4044545
> /home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
> 00615000-00616000 rw-p 00015000 fd:03 4044545
> /home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
> 017f3000-01878000 rw-p 00000000 00:00
> 0 [heap]
> 7fa218000000-7fa218021000 rw-p 00000000 00:00 0
> 7fa218021000-7fa21c000000 ---p 00000000 00:00 0
> 7fa21e122000-7fa21e137000 r-xp 00000000 fd:03
> 141697 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
> 7fa21e137000-7fa21e336000 ---p 00015000 fd:03
> 141697 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
> 7fa21e336000-7fa21e337000 r--p 00014000 fd:03
> 141697 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
> 7fa21e337000-7fa21e338000 rw-p 00015000 fd:03
> 141697 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
> 7fa21e338000-7fa224862000 r--p 00000000 fd:03
> 266442 /usr/lib/locale/locale-archive
> 7fa224862000-7fa224a25000 r-xp 00000000 fd:03
> 134456 /usr/lib64/libc-2.17.so
> 7fa224a25000-7fa224c25000 ---p 001c3000 fd:03
> 134456 /usr/lib64/libc-2.17.so
> 7fa224c25000-7fa224c29000 r--p 001c3000 fd:03
> 134456 /usr/lib64/libc-2.17.so
> 7fa224c29000-7fa224c2b000 rw-p 001c7000 fd:03
> 134456 /usr/lib64/libc-2.17.so
> 7fa224c2b000-7fa224c30000 rw-p 00000000 00:00 0
> 7fa224c30000-7fa224c47000 r-xp 00000000 fd:03
> 134485 /usr/lib64/libpthread-2.17.so
> 7fa224c47000-7fa224e46000 ---p 00017000 fd:03
> 134485 /usr/lib64/libpthread-2.17.so
> 7fa224e46000-7fa224e47000 r--p 00016000 fd:03
> 134485 /usr/lib64/libpthread-2.17.so
> 7fa224e47000-7fa224e48000 rw-p 00017000 fd:03
> 134485 /usr/lib64/libpthread-2.17.so
> 7fa224e48000-7fa224e4c000 rw-p 00000000 00:00 0
> 7fa224e4c000-7fa224e90000 r-xp 00000000 fd:03 4044478
> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
> 7fa224e90000-7fa225090000 ---p 00044000 fd:03 4044478
> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
> 7fa225090000-7fa225093000 r--p 00044000 fd:03 4044478
> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
> 7fa225093000-7fa225094000 rw-p 00047000 fd:03 4044478
> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
> 7fa225094000-7fa2250b6000 r-xp 00000000 fd:03
> 130333 /usr/lib64/ld-2.17.so
> 7fa22527d000-7fa2252a2000 rw-p 00000000 00:00 0
> 7fa2252b3000-7fa2252b5000 rw-p 00000000 00:00 0
> 7fa2252b5000-7fa2252b6000 r--p 00021000 fd:03
> 130333 /usr/lib64/ld-2.17.so
> 7fa2252b6000-7fa2252b7000 rw-p 00022000 fd:03
> 130333 /usr/lib64/ld-2.17.so
> 7fa2252b7000-7fa2252b8000 rw-p 00000000 00:00 0
> 7ffdf354f000-7ffdf3570000 rw-p 00000000 00:00
> 0 [stack]
> 7ffdf3572000-7ffdf3574000 r-xp 00000000 00:00
> 0 [vdso]
> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00
> 0 [vsyscall]
> Aborted
> [centos@tushar-ldap-docker bin]$
>
>
> I am getting the error message but along with "*** Error in
> `./pg_validatebackup': double free or corruption (!prev):
> 0x0000000001850ba0 ***" messages
>
> Is this expected ?
>
> regards,
>
> On 3/3/20 8:19 PM, tushar wrote:
>> On 3/3/20 4:04 PM, tushar wrote:
>>> Thanks Robert. After applying all the 5 patches (v8-00*) against PG
>>> v13 (commit id -afb5465e0cfce7637066eaaaeecab30b0f23fbe3) ,
>>
>> There is a scenario where pg_validatebackup is not throwing an error
>> if some file deleted from pg_wal/ folder and but later at the time
>> of restoring - we are getting an error
>>
>> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -D test1
>>
>> [centos@tushar-ldap-docker bin]$ ls test1/pg_wal/
>> 000000010000000000000010 archive_status
>>
>> [centos@tushar-ldap-docker bin]$ rm -rf test1/pg_wal/*
>>
>> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup test1
>> pg_validatebackup: * manifest_checksum =
>> 88f1ed995c83e86252466a2c88b3e660a69cfc76c169991134b101c4f16c9df7
>> pg_validatebackup: backup successfully verified
>>
>> [centos@tushar-ldap-docker bin]$ ./pg_ctl -D test1 start -o '-p 3333'
>> waiting for server to start....2020-03-02 20:05:22.732 IST [21441]
>> LOG: starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by
>> gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
>> 2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv6 address
>> "::1", port 3333
>> 2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv4 address
>> "127.0.0.1", port 3333
>> 2020-03-02 20:05:22.736 IST [21441] LOG: listening on Unix socket
>> "/tmp/.s.PGSQL.3333"
>> 2020-03-02 20:05:22.739 IST [21442] LOG: database system was
>> interrupted; last known up at 2020-03-02 20:04:35 IST
>> 2020-03-02 20:05:22.739 IST [21442] LOG: creating missing WAL
>> directory "pg_wal/archive_status"
>> 2020-03-02 20:05:22.886 IST [21442] LOG: invalid checkpoint record
>> 2020-03-02 20:05:22.886 IST [21442] FATAL: could not locate required
>> checkpoint record
>> 2020-03-02 20:05:22.886 IST [21442] HINT: If you are restoring from
>> a backup, touch
>> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/recovery.signal" and
>> add required recovery options.
>> If you are not restoring from a backup, try removing the file
>> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label".
>> Be careful: removing
>> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label" will
>> result in a corrupt cluster if restoring from a backup.
>> 2020-03-02 20:05:22.886 IST [21441] LOG: startup process (PID 21442)
>> exited with exit code 1
>> 2020-03-02 20:05:22.886 IST [21441] LOG: aborting startup due to
>> startup process failure
>> 2020-03-02 20:05:22.889 IST [21441] LOG: database system is shut down
>> stopped waiting
>> pg_ctl: could not start server
>> Examine the log output.
>> [centos@tushar-ldap-docker bin]$
>>
>
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-04 13:51 ` tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-04 13:51 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Suraj Kharage <[email protected]>; +Cc: Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
There is a scenario in which i add something inside the pg_tablespace
directory , i am getting an error like-
pg_validatebackup: * manifest_checksum =
77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
pg_validatebackup: error: "pg_tblspc/16385/*PG_13_202002271*/test" is
present on disk but not in the manifest
but if i remove 'PG_13_202002271 ' directory then there is no error
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data
pg_validatebackup: * manifest_checksum =
77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
pg_validatebackup: backup successfully verified
Steps to reproduce -
--connect to psql terminal , create a tablespace
postgres=# \! mkdir /tmp/my_tblspc
postgres=# create tablespace tbs location '/tmp/my_tblspc';
CREATE TABLESPACE
postgres=# \q
--run pg_basebackup
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D data_dir -T
/tmp/my_tblspc/=/tmp/new_my_tblspc
[centos@tushar-ldap-docker bin]$
[centos@tushar-ldap-docker bin]$ ls /tmp/new_my_tblspc/
PG_13_202002271
--create a new file under PG_13_* folder
[centos@tushar-ldap-docker bin]$ touch
/tmp/new_my_tblspc/PG_13_202002271/test
[centos@tushar-ldap-docker bin]$
--run pg_validatebackup ,Getting an error which looks expected
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data_dir/
pg_validatebackup: * manifest_checksum =
3951308eab576906ebdb002ff00ca313b2c1862592168c1f5f7ecf051ac07907
pg_validatebackup: error: "pg_tblspc/16386/PG_13_202002271/test" is
present on disk but not in the manifest
[centos@tushar-ldap-docker bin]$
--remove the added file
[centos@tushar-ldap-docker bin]$ rm -rf
/tmp/new_my_tblspc/PG_13_202002271/test
--run pg_validatebackup , working fine
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data_dir/
pg_validatebackup: * manifest_checksum =
3951308eab576906ebdb002ff00ca313b2c1862592168c1f5f7ecf051ac07907
pg_validatebackup: backup successfully verified
[centos@tushar-ldap-docker bin]$
--remove the folder PG_13*
[centos@tushar-ldap-docker bin]$ rm -rf
/tmp/new_my_tblspc/PG_13_202002271/
[centos@tushar-ldap-docker bin]$
[centos@tushar-ldap-docker bin]$ ls /tmp/new_my_tblspc/
--run pg_validatebackup , No error reported ?
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data_dir/
pg_validatebackup: * manifest_checksum =
3951308eab576906ebdb002ff00ca313b2c1862592168c1f5f7ecf051ac07907
pg_validatebackup: backup successfully verified
[centos@tushar-ldap-docker bin]$
Start the server -
[centos@tushar-ldap-docker bin]$ ./pg_ctl -D data_dir/ start -o '-p 9033'
waiting for server to start....2020-03-04 19:18:54.839 IST [13097] LOG:
starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by gcc
(GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2020-03-04 19:18:54.840 IST [13097] LOG: listening on IPv6 address
"::1", port 9033
2020-03-04 19:18:54.840 IST [13097] LOG: listening on IPv4 address
"127.0.0.1", port 9033
2020-03-04 19:18:54.842 IST [13097] LOG: listening on Unix socket
"/tmp/.s.PGSQL.9033"
2020-03-04 19:18:54.843 IST [13097] LOG: could not open directory
"pg_tblspc/16386/PG_13_202002271": No such file or directory
2020-03-04 19:18:54.845 IST [13098] LOG: database system was
interrupted; last known up at 2020-03-04 19:14:50 IST
2020-03-04 19:18:54.937 IST [13098] LOG: could not open directory
"pg_tblspc/16386/PG_13_202002271": No such file or directory
2020-03-04 19:18:54.939 IST [13098] LOG: could not open directory
"pg_tblspc/16386/PG_13_202002271": No such file or directory
2020-03-04 19:18:54.939 IST [13098] LOG: redo starts at 0/18000028
2020-03-04 19:18:54.939 IST [13098] LOG: consistent recovery state
reached at 0/18000100
2020-03-04 19:18:54.939 IST [13098] LOG: redo done at 0/18000100
2020-03-04 19:18:54.941 IST [13098] LOG: could not open directory
"pg_tblspc/16386/PG_13_202002271": No such file or directory
2020-03-04 19:18:54.984 IST [13097] LOG: database system is ready to
accept connections
done
server started
[centos@tushar-ldap-docker bin]$
regards,
On 3/4/20 3:51 PM, tushar wrote:
> Another scenario, in which if we modify Manifest-Checksum" value from
> backup_manifest file , we are not getting an error
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data/
> pg_validatebackup: * manifest_checksum =
> 28d082921650d0ae881de8ceb122c8d2af5f449f51ecfb446827f7f49f91f65d
> pg_validatebackup: backup successfully verified
>
> open backup_manifest file and replace
>
> "Manifest-Checksum":
> "8d082921650d0ae881de8ceb122c8d2af5f449f51ecfb446827f7f49f91f65d"}
> with
> "Manifest-Checksum": "Hello World"}
>
> rerun the pg_validatebackup
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data/
> pg_validatebackup: * manifest_checksum = Hello World
> pg_validatebackup: backup successfully verified
>
> regards,
>
> On 3/4/20 3:26 PM, tushar wrote:
>> Hi,
>> Another observation , if i change the ownership of a file which is
>> under global/ directory
>> i.e
>>
>> [root@tushar-ldap-docker global]# chown enterprisedb 2396
>>
>> and run the pg_validatebackup command, i am getting this message -
>>
>> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup gggg
>> pg_validatebackup: * manifest_checksum =
>> e8cb007bcc9c0deab6eff51cd8d9d9af6af35b86e02f3055e60e70e56737e877
>> pg_validatebackup: error: could not open file "global/2396":
>> Permission denied
>> *** Error in `./pg_validatebackup': double free or corruption
>> (!prev): 0x0000000001850ba0 ***
>> ======= Backtrace: =========
>> /lib64/libc.so.6(+0x81679)[0x7fa2248e3679]
>> ./pg_validatebackup[0x401f4c]
>> /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa224884505]
>> ./pg_validatebackup[0x402049]
>> ======= Memory map: ========
>> 00400000-00415000 r-xp 00000000 fd:03 4044545
>> /home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
>> 00614000-00615000 r--p 00014000 fd:03 4044545
>> /home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
>> 00615000-00616000 rw-p 00015000 fd:03 4044545
>> /home/centos/pg13_bk_mani/edb/edbpsql/bin/pg_validatebackup
>> 017f3000-01878000 rw-p 00000000 00:00
>> 0 [heap]
>> 7fa218000000-7fa218021000 rw-p 00000000 00:00 0
>> 7fa218021000-7fa21c000000 ---p 00000000 00:00 0
>> 7fa21e122000-7fa21e137000 r-xp 00000000 fd:03 141697
>> /usr/lib64/libgcc_s-4.8.5-20150702.so.1
>> 7fa21e137000-7fa21e336000 ---p 00015000 fd:03 141697
>> /usr/lib64/libgcc_s-4.8.5-20150702.so.1
>> 7fa21e336000-7fa21e337000 r--p 00014000 fd:03 141697
>> /usr/lib64/libgcc_s-4.8.5-20150702.so.1
>> 7fa21e337000-7fa21e338000 rw-p 00015000 fd:03 141697
>> /usr/lib64/libgcc_s-4.8.5-20150702.so.1
>> 7fa21e338000-7fa224862000 r--p 00000000 fd:03
>> 266442 /usr/lib/locale/locale-archive
>> 7fa224862000-7fa224a25000 r-xp 00000000 fd:03
>> 134456 /usr/lib64/libc-2.17.so
>> 7fa224a25000-7fa224c25000 ---p 001c3000 fd:03
>> 134456 /usr/lib64/libc-2.17.so
>> 7fa224c25000-7fa224c29000 r--p 001c3000 fd:03
>> 134456 /usr/lib64/libc-2.17.so
>> 7fa224c29000-7fa224c2b000 rw-p 001c7000 fd:03
>> 134456 /usr/lib64/libc-2.17.so
>> 7fa224c2b000-7fa224c30000 rw-p 00000000 00:00 0
>> 7fa224c30000-7fa224c47000 r-xp 00000000 fd:03
>> 134485 /usr/lib64/libpthread-2.17.so
>> 7fa224c47000-7fa224e46000 ---p 00017000 fd:03
>> 134485 /usr/lib64/libpthread-2.17.so
>> 7fa224e46000-7fa224e47000 r--p 00016000 fd:03
>> 134485 /usr/lib64/libpthread-2.17.so
>> 7fa224e47000-7fa224e48000 rw-p 00017000 fd:03
>> 134485 /usr/lib64/libpthread-2.17.so
>> 7fa224e48000-7fa224e4c000 rw-p 00000000 00:00 0
>> 7fa224e4c000-7fa224e90000 r-xp 00000000 fd:03 4044478
>> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
>> 7fa224e90000-7fa225090000 ---p 00044000 fd:03 4044478
>> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
>> 7fa225090000-7fa225093000 r--p 00044000 fd:03 4044478
>> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
>> 7fa225093000-7fa225094000 rw-p 00047000 fd:03 4044478
>> /home/centos/pg13_bk_mani/edb/edbpsql/lib/libpq.so.5.13
>> 7fa225094000-7fa2250b6000 r-xp 00000000 fd:03
>> 130333 /usr/lib64/ld-2.17.so
>> 7fa22527d000-7fa2252a2000 rw-p 00000000 00:00 0
>> 7fa2252b3000-7fa2252b5000 rw-p 00000000 00:00 0
>> 7fa2252b5000-7fa2252b6000 r--p 00021000 fd:03
>> 130333 /usr/lib64/ld-2.17.so
>> 7fa2252b6000-7fa2252b7000 rw-p 00022000 fd:03
>> 130333 /usr/lib64/ld-2.17.so
>> 7fa2252b7000-7fa2252b8000 rw-p 00000000 00:00 0
>> 7ffdf354f000-7ffdf3570000 rw-p 00000000 00:00
>> 0 [stack]
>> 7ffdf3572000-7ffdf3574000 r-xp 00000000 00:00
>> 0 [vdso]
>> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00
>> 0 [vsyscall]
>> Aborted
>> [centos@tushar-ldap-docker bin]$
>>
>>
>> I am getting the error message but along with "*** Error in
>> `./pg_validatebackup': double free or corruption (!prev):
>> 0x0000000001850ba0 ***" messages
>>
>> Is this expected ?
>>
>> regards,
>>
>> On 3/3/20 8:19 PM, tushar wrote:
>>> On 3/3/20 4:04 PM, tushar wrote:
>>>> Thanks Robert. After applying all the 5 patches (v8-00*) against
>>>> PG v13 (commit id -afb5465e0cfce7637066eaaaeecab30b0f23fbe3) ,
>>>
>>> There is a scenario where pg_validatebackup is not throwing an error
>>> if some file deleted from pg_wal/ folder and but later at the time
>>> of restoring - we are getting an error
>>>
>>> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -D test1
>>>
>>> [centos@tushar-ldap-docker bin]$ ls test1/pg_wal/
>>> 000000010000000000000010 archive_status
>>>
>>> [centos@tushar-ldap-docker bin]$ rm -rf test1/pg_wal/*
>>>
>>> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup test1
>>> pg_validatebackup: * manifest_checksum =
>>> 88f1ed995c83e86252466a2c88b3e660a69cfc76c169991134b101c4f16c9df7
>>> pg_validatebackup: backup successfully verified
>>>
>>> [centos@tushar-ldap-docker bin]$ ./pg_ctl -D test1 start -o '-p 3333'
>>> waiting for server to start....2020-03-02 20:05:22.732 IST [21441]
>>> LOG: starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled
>>> by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
>>> 2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv6 address
>>> "::1", port 3333
>>> 2020-03-02 20:05:22.733 IST [21441] LOG: listening on IPv4 address
>>> "127.0.0.1", port 3333
>>> 2020-03-02 20:05:22.736 IST [21441] LOG: listening on Unix socket
>>> "/tmp/.s.PGSQL.3333"
>>> 2020-03-02 20:05:22.739 IST [21442] LOG: database system was
>>> interrupted; last known up at 2020-03-02 20:04:35 IST
>>> 2020-03-02 20:05:22.739 IST [21442] LOG: creating missing WAL
>>> directory "pg_wal/archive_status"
>>> 2020-03-02 20:05:22.886 IST [21442] LOG: invalid checkpoint record
>>> 2020-03-02 20:05:22.886 IST [21442] FATAL: could not locate
>>> required checkpoint record
>>> 2020-03-02 20:05:22.886 IST [21442] HINT: If you are restoring from
>>> a backup, touch
>>> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/recovery.signal"
>>> and add required recovery options.
>>> If you are not restoring from a backup, try removing the file
>>> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label".
>>> Be careful: removing
>>> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/test1/backup_label" will
>>> result in a corrupt cluster if restoring from a backup.
>>> 2020-03-02 20:05:22.886 IST [21441] LOG: startup process (PID
>>> 21442) exited with exit code 1
>>> 2020-03-02 20:05:22.886 IST [21441] LOG: aborting startup due to
>>> startup process failure
>>> 2020-03-02 20:05:22.889 IST [21441] LOG: database system is shut down
>>> stopped waiting
>>> pg_ctl: could not start server
>>> Examine the log output.
>>> [centos@tushar-ldap-docker bin]$
>>>
>>
>
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-05 04:07 ` Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Suraj Kharage @ 2020-03-05 04:07 UTC (permalink / raw)
To: tushar <[email protected]>; +Cc: Robert Haas <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Wed, Mar 4, 2020 at 7:21 PM tushar <[email protected]> wrote:
> Hi,
>
> There is a scenario in which i add something inside the pg_tablespace
> directory , i am getting an error like-
>
> pg_validatebackup: * manifest_checksum =
> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
> pg_validatebackup: error: "pg_tblspc/16385/*PG_13_202002271*/test" is
> present on disk but not in the manifest
>
> but if i remove 'PG_13_202002271 ' directory then there is no error
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data
> pg_validatebackup: * manifest_checksum =
> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
> pg_validatebackup: backup successfully verified
>
>
This seems expected considering current design as we don't log the
directory entries in backup_manifest. In your case, you have tablespace
with no objects (empty tablespace) then backup_manifest does not have any
entry for this hence when you remove this tablespace directory, validator
could not detect it.
We can either document it or add the entry for directories in the manifest.
Robert may have a better idea on this.
--
--
Thanks & Regards,
Suraj kharage,
EnterpriseDB Corporation,
The Postgres Database Company.
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
@ 2020-03-05 07:39 ` Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Rajkumar Raghuwanshi @ 2020-03-05 07:39 UTC (permalink / raw)
To: Suraj Kharage <[email protected]>; +Cc: tushar <[email protected]>; Robert Haas <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
In a negative test scenario, if I changed size to -1 in backup_manifest,
pg_validatebackup giving
error with a random size number.
[edb@localhost bin]$ ./pg_basebackup -p 5551 -D /tmp/bold
--manifest-checksum 'SHA256'
[edb@localhost bin]$ ./pg_validatebackup /tmp/bold
pg_validatebackup: backup successfully verified
--change a file size to -1 and generate new checksum.
[edb@localhost bin]$ vi /tmp/bold/backup_manifest
[edb@localhost bin]$ shasum -a256 /tmp/bold/backup_manifest
c3d7838cbbf991c6108f9c1ab78f673c20d8073114500f14da6ed07ede2dc44a
/tmp/bold/backup_manifest
[edb@localhost bin]$ vi /tmp/bold/backup_manifest
[edb@localhost bin]$ ./pg_validatebackup /tmp/bold
pg_validatebackup: error: "global/4183" has size 0 on disk but size
*18446744073709551615* in the manifest
Thanks & Regards,
Rajkumar Raghuwanshi
On Thu, Mar 5, 2020 at 9:37 AM Suraj Kharage <[email protected]>
wrote:
>
> On Wed, Mar 4, 2020 at 7:21 PM tushar <[email protected]>
> wrote:
>
>> Hi,
>>
>> There is a scenario in which i add something inside the pg_tablespace
>> directory , i am getting an error like-
>>
>> pg_validatebackup: * manifest_checksum =
>> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
>> pg_validatebackup: error: "pg_tblspc/16385/*PG_13_202002271*/test" is
>> present on disk but not in the manifest
>>
>> but if i remove 'PG_13_202002271 ' directory then there is no error
>>
>> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data
>> pg_validatebackup: * manifest_checksum =
>> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
>> pg_validatebackup: backup successfully verified
>>
>>
> This seems expected considering current design as we don't log the
> directory entries in backup_manifest. In your case, you have tablespace
> with no objects (empty tablespace) then backup_manifest does not have any
> entry for this hence when you remove this tablespace directory, validator
> could not detect it.
>
> We can either document it or add the entry for directories in the
> manifest. Robert may have a better idea on this.
>
> --
> --
>
> Thanks & Regards,
> Suraj kharage,
> EnterpriseDB Corporation,
> The Postgres Database Company.
>
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
@ 2020-03-05 10:10 ` tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-05 10:10 UTC (permalink / raw)
To: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; +Cc: Robert Haas <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
There is one scenario where i somehow able to run pg_validatebackup
successfully but when i tried to start the server , it is failing
Steps to reproduce -
--create 2 base backup directory
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D db1
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D db2
--run pg_validatebackup , use backup_manifest of db1 directory against
db2/ . Will get an error
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup -m
db1/backup_manifest db2/
pg_validatebackup: * manifest_checksum =
5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
pg_validatebackup: error: checksum mismatch for file "backup_label"
--copy the backup_level of db1 to db2 folder
[centos@tushar-ldap-docker bin]$ cp db1/backup_label db2/.
--run pg_validatebackup .. working fine
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup -m
db1/backup_manifest db2/
pg_validatebackup: * manifest_checksum =
5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
pg_validatebackup: backup successfully verified
[centos@tushar-ldap-docker bin]$
--try to start the server
[centos@tushar-ldap-docker bin]$ ./pg_ctl -D db2 start -o '-p 7777'
waiting for server to start....2020-03-05 15:33:53.471 IST [24049] LOG:
starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by gcc
(GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2020-03-05 15:33:53.471 IST [24049] LOG: listening on IPv6 address
"::1", port 7777
2020-03-05 15:33:53.471 IST [24049] LOG: listening on IPv4 address
"127.0.0.1", port 7777
2020-03-05 15:33:53.473 IST [24049] LOG: listening on Unix socket
"/tmp/.s.PGSQL.7777"
2020-03-05 15:33:53.476 IST [24050] LOG: database system was
interrupted; last known up at 2020-03-05 15:32:51 IST
2020-03-05 15:33:53.573 IST [24050] LOG: invalid checkpoint record
2020-03-05 15:33:53.573 IST [24050] FATAL: could not locate required
checkpoint record
2020-03-05 15:33:53.573 IST [24050] HINT: If you are restoring from a
backup, touch
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/recovery.signal" and add
required recovery options.
If you are not restoring from a backup, try removing the file
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label".
Be careful: removing
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label" will result
in a corrupt cluster if restoring from a backup.
2020-03-05 15:33:53.574 IST [24049] LOG: startup process (PID 24050)
exited with exit code 1
2020-03-05 15:33:53.574 IST [24049] LOG: aborting startup due to
startup process failure
2020-03-05 15:33:53.575 IST [24049] LOG: database system is shut down
stopped waiting
pg_ctl: could not start server
Examine the log output.
[centos@tushar-ldap-docker bin]$
regards,
On 3/5/20 1:09 PM, Rajkumar Raghuwanshi wrote:
> Hi,
>
> In a negative test scenario, if I changed size to -1 in
> backup_manifest, pg_validatebackup giving
> error with a random size number.
>
> [edb@localhost bin]$ ./pg_basebackup -p 5551 -D /tmp/bold
> --manifest-checksum 'SHA256'
> [edb@localhost bin]$ ./pg_validatebackup /tmp/bold
> pg_validatebackup: backup successfully verified
>
> --change a file size to -1 and generate new checksum.
> [edb@localhost bin]$ vi /tmp/bold/backup_manifest
> [edb@localhost bin]$ shasum -a256 /tmp/bold/backup_manifest
> c3d7838cbbf991c6108f9c1ab78f673c20d8073114500f14da6ed07ede2dc44a
> /tmp/bold/backup_manifest
> [edb@localhost bin]$ vi /tmp/bold/backup_manifest
>
> [edb@localhost bin]$ ./pg_validatebackup /tmp/bold
> pg_validatebackup: error: "global/4183" has size 0 on disk but size
> *18446744073709551615* in the manifest
>
> Thanks & Regards,
> Rajkumar Raghuwanshi
>
>
> On Thu, Mar 5, 2020 at 9:37 AM Suraj Kharage
> <[email protected]
> <mailto:[email protected]>> wrote:
>
>
> On Wed, Mar 4, 2020 at 7:21 PM tushar
> <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi,
>
> There is a scenario in which i add something inside the
> pg_tablespace directory , i am getting an error like-
>
> pg_validatebackup: * manifest_checksum =
> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
> pg_validatebackup: error:
> "pg_tblspc/16385/*PG_13_202002271*/test" is present on disk
> but not in the manifest
>
> but if i remove 'PG_13_202002271 ' directory then there is no
> error
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data
> pg_validatebackup: * manifest_checksum =
> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
> pg_validatebackup: backup successfully verified
>
>
> This seems expected considering current design as we don't log the
> directory entries in backup_manifest. In your case, you have
> tablespace with no objects (empty tablespace) then backup_manifest
> does not have any entry for this hence when you remove this
> tablespace directory, validator could not detect it.
>
> We can either document it or add the entry for directories in the
> manifest. Robert may have a better idea on this.
>
> --
> --
>
> Thanks & Regards,
> Suraj kharage,
> EnterpriseDB Corporation,
> The Postgres Database Company.
>
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-05 12:05 ` tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-05 12:05 UTC (permalink / raw)
To: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; +Cc: Robert Haas <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
There is one small observation if we use slash (/) with option -i then
not getting the desired result
Steps to reproduce -
==============
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D test
[centos@tushar-ldap-docker bin]$ touch test/*pg_notify*/dummy_file
--working
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup
--ignore=*pg_notify* test
pg_validatebackup: * manifest_checksum =
be9b72e1320c6c34c131533de19371a10dd5011940181724e43277f786026c7b
pg_validatebackup: backup successfully verified
--not working
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup
--ignore=*pg_notify/* test
pg_validatebackup: * manifest_checksum =
be9b72e1320c6c34c131533de19371a10dd5011940181724e43277f786026c7b
pg_validatebackup: error: "pg_notify/dummy_file" is present on disk but
not in the manifest
regards,
On 3/5/20 3:40 PM, tushar wrote:
> Hi,
>
> There is one scenario where i somehow able to run pg_validatebackup
> successfully but when i tried to start the server , it is failing
>
> Steps to reproduce -
> --create 2 base backup directory
> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -D db1
> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -D db2
>
> --run pg_validatebackup , use backup_manifest of db1 directory
> against db2/ . Will get an error
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup -m
> db1/backup_manifest db2/
> pg_validatebackup: * manifest_checksum =
> 5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
> pg_validatebackup: error: checksum mismatch for file "backup_label"
> --copy the backup_level of db1 to db2 folder
> [centos@tushar-ldap-docker bin]$ cp db1/backup_label db2/.
>
> --run pg_validatebackup .. working fine
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup -m
> db1/backup_manifest db2/
> pg_validatebackup: * manifest_checksum =
> 5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
> pg_validatebackup: backup successfully verified
> [centos@tushar-ldap-docker bin]$
>
> --try to start the server
> [centos@tushar-ldap-docker bin]$ ./pg_ctl -D db2 start -o '-p 7777'
> waiting for server to start....2020-03-05 15:33:53.471 IST [24049]
> LOG: starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by
> gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
> 2020-03-05 15:33:53.471 IST [24049] LOG: listening on IPv6 address
> "::1", port 7777
> 2020-03-05 15:33:53.471 IST [24049] LOG: listening on IPv4 address
> "127.0.0.1", port 7777
> 2020-03-05 15:33:53.473 IST [24049] LOG: listening on Unix socket
> "/tmp/.s.PGSQL.7777"
> 2020-03-05 15:33:53.476 IST [24050] LOG: database system was
> interrupted; last known up at 2020-03-05 15:32:51 IST
> 2020-03-05 15:33:53.573 IST [24050] LOG: invalid checkpoint record
> 2020-03-05 15:33:53.573 IST [24050] FATAL: could not locate required
> checkpoint record
> 2020-03-05 15:33:53.573 IST [24050] HINT: If you are restoring from a
> backup, touch
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/recovery.signal" and
> add required recovery options.
> If you are not restoring from a backup, try removing the file
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label".
> Be careful: removing
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label" will
> result in a corrupt cluster if restoring from a backup.
> 2020-03-05 15:33:53.574 IST [24049] LOG: startup process (PID 24050)
> exited with exit code 1
> 2020-03-05 15:33:53.574 IST [24049] LOG: aborting startup due to
> startup process failure
> 2020-03-05 15:33:53.575 IST [24049] LOG: database system is shut down
> stopped waiting
> pg_ctl: could not start server
> Examine the log output.
> [centos@tushar-ldap-docker bin]$
>
> regards,
>
>
> On 3/5/20 1:09 PM, Rajkumar Raghuwanshi wrote:
>> Hi,
>>
>> In a negative test scenario, if I changed size to -1 in
>> backup_manifest, pg_validatebackup giving
>> error with a random size number.
>>
>> [edb@localhost bin]$ ./pg_basebackup -p 5551 -D /tmp/bold
>> --manifest-checksum 'SHA256'
>> [edb@localhost bin]$ ./pg_validatebackup /tmp/bold
>> pg_validatebackup: backup successfully verified
>>
>> --change a file size to -1 and generate new checksum.
>> [edb@localhost bin]$ vi /tmp/bold/backup_manifest
>> [edb@localhost bin]$ shasum -a256 /tmp/bold/backup_manifest
>> c3d7838cbbf991c6108f9c1ab78f673c20d8073114500f14da6ed07ede2dc44a
>> /tmp/bold/backup_manifest
>> [edb@localhost bin]$ vi /tmp/bold/backup_manifest
>>
>> [edb@localhost bin]$ ./pg_validatebackup /tmp/bold
>> pg_validatebackup: error: "global/4183" has size 0 on disk but size
>> *18446744073709551615* in the manifest
>>
>> Thanks & Regards,
>> Rajkumar Raghuwanshi
>>
>>
>> On Thu, Mar 5, 2020 at 9:37 AM Suraj Kharage
>> <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>>
>> On Wed, Mar 4, 2020 at 7:21 PM tushar
>> <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Hi,
>>
>> There is a scenario in which i add something inside the
>> pg_tablespace directory , i am getting an error like-
>>
>> pg_validatebackup: * manifest_checksum =
>> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
>> pg_validatebackup: error:
>> "pg_tblspc/16385/*PG_13_202002271*/test" is present on disk
>> but not in the manifest
>>
>> but if i remove 'PG_13_202002271 ' directory then there is no
>> error
>>
>> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data
>> pg_validatebackup: * manifest_checksum =
>> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
>> pg_validatebackup: backup successfully verified
>>
>>
>> This seems expected considering current design as we don't log
>> the directory entries in backup_manifest. In your case, you have
>> tablespace with no objects (empty tablespace) then
>> backup_manifest does not have any entry for this hence when you
>> remove this tablespace directory, validator could not detect it.
>>
>> We can either document it or add the entry for directories in the
>> manifest. Robert may have a better idea on this.
>>
>> --
>> --
>>
>> Thanks & Regards,
>> Suraj kharage,
>> EnterpriseDB Corporation,
>> The Postgres Database Company.
>>
>
> --
> regards,tushar
> EnterpriseDBhttps://www.enterprisedb.com/
> The Enterprise PostgreSQL Company
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-05 16:55 ` Robert Haas <[email protected]>
2020-03-06 08:58 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-05 16:55 UTC (permalink / raw)
To: tushar <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Thu, Mar 5, 2020 at 7:05 AM tushar <[email protected]> wrote:
> There is one small observation if we use slash (/) with option -i then not getting the desired result
Here's an updated patch set responding to many of the comments
received thus far. Since there are quite a few emails, let me
consolidate my comments and responses here.
Report: Segmentation fault if -m is used to point to a valid manifest,
but actual backup directory is nonexistent.
Response: Fixed; thanks for the report.
Report: pg_validatebackup doesn't complain about problems within the
pg_wal directory.
Response: That's out of scope. The WAL files are fetched separately
and are therefore not part of the manifest.
Report: Inaccessible file in data directory being validated leads to a
double free.
Response: Fixed; thanks for the report.
Report: Patch 0005 doesn't validate the manifest checksum.
Response: I know. I mentioned that when posting the previous patch
set. Fixed in this version, though.
Report: Removing an empty directory doesn't make backup validation
fail, even though it might cause problems for the server.
Response: That's a little unfortunate, but I'm not sure it's really
worth complicating the patch to deal with it. It's something of a
corner case.
Report: Negative file sizes in the backup manifest are interpreted as
large integers.
Response: That's also a little unfortunate, but I doubt it's worth
adding code to catch it, since any such manifest is corrupt. Also,
it's not like we're ignoring it; the error just isn't ideal.
Report: If I take the backup label from backup #1 and stick it into
otherwise-identical backup #2, validation succeeds but the server
won't start.
Response: That's because we can't validate the pg_wal directory. As
noted above, that's out of scope.
Report: Using --ignore with a slash-terminated pathname doesn't work
as expected.
Response: Fixed, thanks for the report.
Off-List Report: You forgot a PG_BINARY flag.
Response: Fixed. I thought I'd done this before but there were two
places and I'd only fixed one of them.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v9-0001-Add-checksum-helper-functions.patch (9.8K, ../../CA+TgmoYJxdwPSd2as1z5+W40hxuhrnKdgiN+4YY0ppmTC-r36Q@mail.gmail.com/2-v9-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From c6a22d622aabb47f5a4153777271f2258a592c45 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 5 Feb 2020 13:59:45 -0500
Subject: [PATCH v9 1/5] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
(In my version, I decided to support all of the SHA variants for
which we have code, added a function to translate values of the
enum type back to strings so that users of this code don't need
to do that, used different naming, and tried to be more careful
amount formatting and comments.)
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index ce01df68b9..e199ed7acb 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -47,6 +47,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index ec25042933..358e923a28 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -119,8 +119,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
- f2s.c file_perm.c hashfn.c ip.c jsonapi.c
+ base64.c checksum_helper.c config_info.c controldata_utils.c d2s.c
+ encnames.c exec.c f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c username.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v9-0004-Modify-server-code-to-generate-backup-manifest-in.patch (8.2K, ../../CA+TgmoYJxdwPSd2as1z5+W40hxuhrnKdgiN+4YY0ppmTC-r36Q@mail.gmail.com/3-v9-0004-Modify-server-code-to-generate-backup-manifest-in.patch)
download | inline diff:
From cceeb92251b8cd3612740c545a33dae6afefbf41 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Thu, 27 Feb 2020 17:46:43 +0530
Subject: [PATCH v9 4/5] Modify server code to generate backup manifest in JSON
format.
This will eventually get merged into the previous patch to add
backup manifest functionality to the server, but I'm keeping
it separate for now because I don't have the validator working
with this format yet.
---
src/backend/replication/basebackup.c | 151 +++++++++++++--------------
1 file changed, 75 insertions(+), 76 deletions(-)
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 27fa67f321..77f15f6233 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -41,6 +41,7 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
#include "utils/resowner.h"
@@ -64,6 +65,7 @@ struct manifest_info
pg_checksum_type checksum_type;
pg_sha256_ctx manifest_ctx;
uint64 manifest_size;
+ bool first_file;
bool still_checksumming;
};
@@ -88,7 +90,6 @@ static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
const char *pathname, size_t size, time_t mtime,
pg_checksum_context *checksum_ctx);
static void SendBackupManifest(manifest_info *manifest);
-static char *escape_field_for_manifest(const char *s);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -1004,9 +1005,12 @@ InitializeManifest(manifest_info *manifest, pg_checksum_type checksum_type)
manifest->checksum_type = checksum_type;
pg_sha256_init(&manifest->manifest_ctx);
manifest->manifest_size = UINT64CONST(0);
+ manifest->first_file = true;
manifest->still_checksumming = true;
- AppendToManifest(manifest, "PostgreSQL-Backup-Manifest-Version\t1\n");
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
}
/*
@@ -1037,8 +1041,8 @@ AddFileToManifest(manifest_info *manifest, const char *spcoid,
pg_checksum_context *checksum_ctx)
{
char pathbuf[MAXPGPATH];
- char *escaped_pathname;
- static char timebuf[128];
+ int pathlen;
+ StringInfoData buf;
/*
* If this file is part of a tablespace, the pathname passed to this
@@ -1053,40 +1057,81 @@ AddFileToManifest(manifest_info *manifest, const char *spcoid,
pathname = pathbuf;
}
- /* Escape pathname, if necessary. */
- escaped_pathname = escape_field_for_manifest(pathname);
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
/*
- * Convert time to a string. Since it's not clear what time zone to use
- * and since time zone definitions can change, possibly causing confusion,
- * use GMT always.
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
*/
- pg_strftime(timebuf, sizeof(timebuf), "%Y-%m-%d %H:%M:%S %Z",
- pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
- /* Add to manifest. */
- AppendToManifest(manifest, "File\t%s\t%zu\t%s\t%s",
- escaped_pathname == NULL ? pathname : escaped_pathname,
- size, timebuf, pg_checksum_type_name(checksum_ctx->type));
+ /* Add checksum information. */
if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
{
uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
- char checksumstringbuf[PG_CHECKSUM_MAX_LENGTH * 2 + 1];
int checksumlen;
- int checksumstringlen;
- AppendStringToManifest(manifest, ":");
checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
- checksumstringlen = hex_encode((char *) checksumbuf, checksumlen,
- checksumstringbuf);
- checksumstringbuf[checksumstringlen] = '\0';
- AppendStringToManifest(manifest, checksumstringbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
}
- AppendStringToManifest(manifest, "\n");
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
/* Avoid leaking memory. */
- if (escaped_pathname != NULL)
- pfree(escaped_pathname);
+ pfree(buf.data);
}
/*
@@ -1100,6 +1145,9 @@ SendBackupManifest(manifest_info *manifest)
char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
size_t manifest_bytes_done = 0;
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
/*
* Append manifest checksum, so that the problems with the manifest itself
* can be detected.
@@ -1113,11 +1161,11 @@ SendBackupManifest(manifest_info *manifest)
*/
manifest->still_checksumming = false;
pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
- AppendStringToManifest(manifest, "Manifest-Checksum\t");
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
AppendStringToManifest(manifest, checksumstringbuf);
- AppendStringToManifest(manifest, "\n");
+ AppendStringToManifest(manifest, "\"}\n");
/*
* We've written all the data to the manifest file. Rewind the file so
@@ -1165,55 +1213,6 @@ SendBackupManifest(manifest_info *manifest)
BufFileClose(manifest->buffile);
}
-/*
- * Escape a field for inclusion in a manifest.
- *
- * We use the following escaping rule: If a field contains \t, \r, or \n,
- * the field must be surrounded by double-quotes, and any internal double
- * quotes must be doubled. Otherwise, no escaping is required.
- *
- * The return value is a new palloc'd string with escaping added, or NULL
- * if no escaping is required.
- */
-static char *
-escape_field_for_manifest(const char *s)
-{
- bool escaping_required = false;
- int escaped_length = 2;
- const char *t;
- char *result;
- char *r;
-
- for (t = s; *t != '\0'; ++t)
- {
- if (*t == '\t' || *t == '\r' || *t == '\n')
- escaping_required = true;
- if (*t == '"')
- ++escaped_length;
- ++escaped_length;
- }
-
- if (!escaping_required)
- return NULL;
-
- result = palloc(escaped_length + 1);
- result[0] = '"';
- result[escaped_length - 1] = '"';
- result[escaped_length] = '\0';
- r = result + 1;
-
- for (t = s; *t != '\0'; ++t)
- {
- *(r++) = *t;
- if (*t == '"')
- *(r++) = *t;
- }
-
- Assert(r == &result[escaped_length - 1]);
-
- return result;
-}
-
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
--
2.17.2 (Apple Git-113)
[application/octet-stream] v9-0003-pg_validatebackup-Validate-a-backup-against-the-b.patch (35.0K, ../../CA+TgmoYJxdwPSd2as1z5+W40hxuhrnKdgiN+4YY0ppmTC-r36Q@mail.gmail.com/4-v9-0003-pg_validatebackup-Validate-a-backup-against-the-b.patch)
download | inline diff:
From bc37c64ac691e97bc16292466dccad9889a4d521 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 7 Feb 2020 17:17:52 -0500
Subject: [PATCH v9 3/5] pg_validatebackup: Validate a backup against the
backup manifest.
Patch by me; review by Tushar Ahuja and Rajkumar Raghuwanshi, and also
off-list by Mark Dilger, Davinder Singh, and Jeevan Chalke.
---
src/backend/replication/basebackup.c | 6 +-
src/bin/Makefile | 1 +
src/bin/pg_validatebackup/.gitignore | 1 +
src/bin/pg_validatebackup/Makefile | 32 +
src/bin/pg_validatebackup/pg_validatebackup.c | 1081 +++++++++++++++++
5 files changed, 1118 insertions(+), 3 deletions(-)
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 914859aea8..27fa67f321 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -64,7 +64,7 @@ struct manifest_info
pg_checksum_type checksum_type;
pg_sha256_ctx manifest_ctx;
uint64 manifest_size;
- int still_checksumming;
+ bool still_checksumming;
};
@@ -85,7 +85,7 @@ static void SendBackupHeader(List *tablespaces);
static void InitializeManifest(manifest_info *manifest, pg_checksum_type);
static void AppendStringToManifest(manifest_info *manifest, char *s);
static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
- const char *filename, size_t size, time_t mtime,
+ const char *pathname, size_t size, time_t mtime,
pg_checksum_context *checksum_ctx);
static void SendBackupManifest(manifest_info *manifest);
static char *escape_field_for_manifest(const char *s);
@@ -1033,7 +1033,7 @@ AppendStringToManifest(manifest_info *manifest, char *s)
*/
static void
AddFileToManifest(manifest_info *manifest, const char *spcoid,
- const char *filename, size_t size, time_t mtime,
+ const char *pathname, size_t size, time_t mtime,
pg_checksum_context *checksum_ctx)
{
char pathbuf[MAXPGPATH];
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..3ae1c1f03a
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1 @@
+/pg_validatebackup
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..aeb97d21d2
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,32 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..d4d041ef7d
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,1081 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/checksum_helper.h"
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * The first word of each line of the manifest file should be one of these
+ * key words. We define constants for the relevant lengths as well.
+ */
+#define KW_MANIFEST_VERSION "PostgreSQL-Backup-Manifest-Version"
+#define KW_MANIFEST_FILE "File"
+#define KW_MANIFEST_CHECKSUM "Manifest-Checksum"
+#define KWL_MANIFEST_VERSION (sizeof(KW_MANIFEST_VERSION)-1)
+#define KWL_MANIFEST_FILE (sizeof(KW_MANIFEST_FILE)-1)
+#define KWL_MANIFEST_CHECKSUM (sizeof(KW_MANIFEST_CHECKSUM)-1)
+
+/*
+ * How many fields are there for each "File" line in the manifest?
+ * Currently we have: file name, file size, timestamp, checksum.
+ */
+#define FIELDS_PER_FILE_LINE 4
+
+/*
+ * Each "File" line in the manifest file is parsed to produce an object
+ * like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash * parse_manifest_file(char *manifest_path);
+static void parse_file_line_from_manifest(manifestfile *f, char *rest,
+ int restlen);
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void pg_validator_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void pg_validator_fatal(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static char *extractstr(char *buffer, int length);
+static int findchar(char *buffer, int size, char c, int start_position);
+static int findfield(char *buffer, char *end, char **result);
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ pg_log_info("backup successfully verified");
+
+ exit(context.saw_any_error ? 1 : 0);
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ off_t bytes_read = 0;
+ off_t bytes_consumed = 0;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+ int buffer_position = 0;
+ int buffer_size = 0;
+ int buffer_maxsize = 2 * READ_CHUNK_SIZE;
+ int line_number = 0;
+ bool saw_manifest_checksum_line = false;
+ pg_sha256_ctx manifest_ctx;
+
+ /* Prepare to compute a checksum of the manifest itself. */
+ pg_sha256_init(&manifest_ctx);
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ pg_validator_fatal("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ pg_validator_fatal("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /* Initialize our read buffer. */
+ buffer = pg_malloc(buffer_maxsize);
+
+ /*
+ * Loop until we've read it all.
+ *
+ * The file size shouldn't be changing, so it seems fine to just error out
+ * if the final length is different from what stat() told us.
+ */
+ while (bytes_consumed < statbuf.st_size)
+ {
+ int line_length;
+ int first_field_length;
+ char *rest;
+ int restlen;
+
+ /* Find next newline if any. */
+ line_length = findchar(buffer, buffer_size, '\n', buffer_position);
+
+ /* If no newline was found, we need to read more data and try again. */
+ if (line_length == -1)
+ {
+ size_t bytes_to_read;
+ int rc;
+
+ bytes_to_read = Min(statbuf.st_size - bytes_read, READ_CHUNK_SIZE);
+ if (bytes_to_read == 0)
+ pg_validator_fatal("manifest file line not terminated by newline");
+ if (bytes_to_read + READ_CHUNK_SIZE > buffer_maxsize)
+ {
+ buffer_maxsize += READ_CHUNK_SIZE;
+ buffer = pg_realloc(buffer, buffer_maxsize);
+ Assert(bytes_to_read + READ_CHUNK_SIZE <= buffer_maxsize);
+ }
+ rc = read(fd, buffer + buffer_size, bytes_to_read);
+ if (rc != bytes_to_read)
+ {
+ if (rc < 0)
+ pg_validator_fatal("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ pg_validator_fatal("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, bytes_to_read);
+ }
+ buffer_size += rc;
+ bytes_read += rc;
+ continue;
+ }
+
+ /* Increment line number. */
+ ++line_number;
+
+ /* The manifest checksum should be the last thing in the file. */
+ if (saw_manifest_checksum_line)
+ pg_validator_fatal("unexpected data follows manifest checksum");
+
+ /* Find first field on line, and remaining line contents. */
+ first_field_length =
+ findchar(buffer, buffer_size, '\t', buffer_position);
+ rest = buffer + buffer_position + first_field_length + 1;
+ restlen = line_length - (first_field_length + 1);
+
+ /*
+ * Check the first word of the line to see what kind of line it is.
+ */
+ if (first_field_length == KWL_MANIFEST_VERSION &&
+ memcmp(buffer + buffer_position, KW_MANIFEST_VERSION,
+ KWL_MANIFEST_VERSION) == 0)
+ {
+ if (line_number != 1)
+ pg_validator_fatal("manifest file version should only be specified at line 1");
+ else
+ {
+ char *line = buffer + buffer_position;
+ char *version;
+
+ version = extractstr(line + first_field_length + 1,
+ line_length - (first_field_length + 1));
+ if (strcmp(version, "1") != 0)
+ pg_validator_fatal("unrecognized manifest version: \"%s\"",
+ version);
+ }
+ }
+ else if (first_field_length == KWL_MANIFEST_FILE &&
+ memcmp(buffer + buffer_position, KW_MANIFEST_FILE,
+ KWL_MANIFEST_FILE) == 0)
+ {
+ manifestfile f;
+ manifestfile *tabent;
+ bool found;
+
+ /* Parse this line. */
+ parse_file_line_from_manifest(&f, rest, restlen);
+
+ /* Make a new entry in the hash table for it. */
+ tabent = manifestfiles_insert(ht, f.pathname, &found);
+ if (found)
+ pg_validator_fatal("duplicate pathname in backup manifest: \"%s\"",
+ f.pathname);
+
+ /* Copy in all the relevant details. */
+ tabent->size = f.size;
+ tabent->checksum_type = f.checksum_type;
+ tabent->checksum_length = f.checksum_length;
+ tabent->checksum_payload = f.checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+ }
+ else if (first_field_length == KWL_MANIFEST_CHECKSUM &&
+ memcmp(buffer + buffer_position, KW_MANIFEST_CHECKSUM,
+ KWL_MANIFEST_CHECKSUM) == 0)
+ {
+ saw_manifest_checksum_line = true;
+ if (restlen != PG_SHA256_DIGEST_STRING_LENGTH - 1)
+ pg_validator_fatal("manifest file checksum has unexpected length: %d",
+ restlen);
+ if (!hexdecode_string(manifest_checksum_expected, rest,
+ PG_SHA256_DIGEST_LENGTH))
+ pg_validator_fatal("invalid manifest checksum: \"%s\"",
+ extractstr(rest, restlen));
+ }
+ else if (first_field_length == -1)
+ pg_validator_fatal("manifest file keyword not terminated by tab");
+ else
+ {
+ char *kw;
+
+ kw = extractstr(buffer + buffer_position, first_field_length);
+ pg_validator_fatal("unrecognized manifest file keyword: \"%s\"", kw);
+ }
+
+ /* Update manifest checksum, if needed. */
+ if (!saw_manifest_checksum_line)
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer + buffer_position,
+ line_length + 1);
+
+ /* Advance buffer position over the data we just read. */
+ buffer_position += line_length + 1;
+
+ /* Also mark these bytes as consumed so we know when to stop. */
+ bytes_consumed += line_length + 1;
+
+ /*
+ * We don't want to incur the expensive of using memmove() to discard
+ * data after every line, because the lines are short compared to the
+ * chunk size -- but we must do it at least now and then, or we'll
+ * have to keep growing the buffer.
+ */
+ if (buffer_position >= READ_CHUNK_SIZE)
+ {
+ int leftover_bytes = buffer_size - buffer_position;
+
+ if (leftover_bytes > 0)
+ memmove(buffer, buffer + buffer_position, leftover_bytes);
+ buffer_size -= buffer_position;
+ buffer_position = 0;
+ }
+ }
+
+ /* Checksum verification. */
+ if (!saw_manifest_checksum_line)
+ pg_validator_fatal("manifest has no checksum");
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ pg_validator_fatal("manifest checksum does not match");
+
+ /* OK, we're done with the manifest file. */
+ close(fd);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * The caller passes the remainder of the line, excluding the initial "File\t"
+ * portion.
+ */
+static void
+parse_file_line_from_manifest(manifestfile *f, char *rest, int restlen)
+{
+ char *end = rest + restlen;
+ char *field[FIELDS_PER_FILE_LINE];
+ unsigned long filesize;
+ char *ep;
+ pg_checksum_type checksum_type;
+ int raw_checksum_length = 0;
+ char *raw_checksum_payload = NULL;
+ int checksum_length;
+ uint8 *checksum_payload;
+ int i;
+ char *s;
+
+ /* Split the line into fields. */
+ for (i = 0; i < FIELDS_PER_FILE_LINE; ++i)
+ {
+ int toklen;
+
+ toklen = findfield(rest, end, &field[i]);
+ if (rest + toklen >= end && i + 1 < FIELDS_PER_FILE_LINE)
+ pg_validator_fatal("manifest file line has too few fields");
+ rest += toklen + 1;
+ }
+
+ /* We expect to have used the entire line. */
+ if (rest < end)
+ pg_validator_fatal("manifest file line has too many fields");
+
+ /* Parse the size. */
+ filesize = strtoul(field[1], &ep, 10);
+ if (*ep)
+ pg_validator_fatal("manifest file size for file \"%s\" is not a number",
+ field[0]);
+
+ /* Parse the checksum type. */
+ for (s = field[3]; s[0] != '\0' && s[0] != ':'; ++s)
+ ;
+ if (*s)
+ {
+ raw_checksum_payload = s + 1;
+ raw_checksum_length = strlen(raw_checksum_payload);
+ *s = '\0';
+ }
+ if (!pg_checksum_parse_type(field[3], &checksum_type))
+ pg_validator_fatal("unrecognized checksum algorithm for file \"%s\": \"%s\"",
+ field[0], field[3]);
+
+ /* Decode the checksum payload. */
+ checksum_length = raw_checksum_length / 2;
+ if (checksum_length == 0)
+ checksum_payload = NULL;
+ else
+ {
+ checksum_payload = palloc(checksum_length);
+ if (!hexdecode_string(checksum_payload, raw_checksum_payload,
+ checksum_length))
+ pg_validator_fatal("invalid checksum for file \"%s\": \"%s\"",
+ field[0], raw_checksum_payload);
+ }
+
+ /* Fill the output struct. */
+ f->pathname = field[0];
+ f->size = filesize;
+ f->checksum_type = checksum_type;
+ f->checksum_length = checksum_length;
+ f->checksum_payload = checksum_payload;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ pg_validator_fatal("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ pg_validator_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ pg_validator_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ pg_validator_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ pg_validator_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ pg_validator_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ pg_validator_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ pg_validator_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ pg_validator_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ pg_validator_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ pg_validator_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ pg_validator_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ pg_validator_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <[email protected]>.\n"));
+}
+
+/*
+ * Report an error. Update the context to indicate that we saw an error, and
+ * exit if the context says we should.
+ */
+static void
+pg_validator_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+pg_validator_fatal(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Extract a NUL-terminated string from a larger buffer.
+ */
+static char *
+extractstr(char *buffer, int length)
+{
+ char *s = palloc(length + 1);
+
+ memcpy(s, buffer, length);
+ s[length] = '\0';
+
+ return s;
+}
+
+/*
+ * Find the next instance of a given character within a buffer that
+ * occurs at or after start_position. If there is none, returns -1; else
+ * returns the difference between the position at which the character was
+ * found and the start position.
+ */
+static int
+findchar(char *buffer, int size, char c, int start_position)
+{
+ int i;
+
+ for (i = start_position; i < size; ++i)
+ if (buffer[i] == c)
+ return i - start_position;
+ return -1;
+}
+
+/*
+ * Extract the next field from a line of text read from the manifest file.
+ */
+static int
+findfield(char *buffer, char *end, char **result)
+{
+ int qoffset = 1;
+ int dqcount = 0;
+ int toklen;
+ int bufpos;
+ int resultpos;
+
+ /*
+ * If this field is unquoted, we just stop at the next tab; if there's
+ * none, we stop at the end of the line. Note that if buffer == end, it
+ * just means that the last field on the line is empty.
+ */
+ if (buffer == end || *buffer != '"')
+ {
+ toklen = findchar(buffer, end - buffer, '\t', 0);
+
+ if (toklen == -1)
+ toklen = end - buffer;
+ *result = extractstr(buffer, toklen);
+ return toklen;
+ }
+
+ /*
+ * Our escaping convention is that if the field contains a tab, it must be
+ * surrounded by double-quotes and any internal double-quotes must be
+ * doubled.
+ */
+ while (1)
+ {
+ /* Where's the next double quote? */
+ qoffset += findchar(buffer, end - buffer, '"', qoffset);
+ if (qoffset == -1)
+ pg_validator_fatal("quoted field in backup manifest is not terminated");
+
+ /*
+ * If the double-quote we found is the last character on the line or
+ * if it's followed by a tab, we've reached the end of this field.
+ */
+ if (buffer + qoffset >= end || buffer[qoffset + 1] == '\t')
+ break;
+
+ /* Otherwise, the next character should be another double-quote. */
+ if (buffer[qoffset + 1] != '"')
+ {
+ pg_log_fatal("invalid quoted field in backup manifest");
+ exit(1);
+ }
+
+ /* Skip both double-quotes and go around again. */
+ qoffset += 2;
+ ++dqcount;
+ }
+
+ /*
+ * At this point, we know that qoffset is the offset, relative to buffer,
+ * of the closing double-quote, and that dqcount is the number of escaped
+ * double-quotes within the field, and that all of those escape sequences
+ * are proper. Extract and de-escape the data in the field.
+ *
+ * The amount of space needed for the result is equal to the raw token
+ * length, minus two for the double quotes at the start and end, minus one
+ * for each doubled double-quote within the token, plus one for the
+ * trailing zero byte.
+ */
+ toklen = qoffset + 1;
+ *result = palloc(toklen - dqcount - 1);
+ bufpos = 1;
+ resultpos = 0;
+ while (bufpos < qoffset)
+ {
+ (*result)[resultpos] = buffer[bufpos];
+ bufpos += (buffer[bufpos] == '"' ? 2 : 1);
+ ++resultpos;
+ }
+ (*result)[resultpos] = '\0';
+ Assert(resultpos == toklen - dqcount - 2);
+
+ return toklen;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
--
2.17.2 (Apple Git-113)
[application/octet-stream] v9-0002-Generate-backup-manifests-for-base-backups.patch (33.4K, ../../CA+TgmoYJxdwPSd2as1z5+W40hxuhrnKdgiN+4YY0ppmTC-r36Q@mail.gmail.com/5-v9-0002-Generate-backup-manifests-for-base-backups.patch)
download | inline diff:
From cfbca4bda75e04c7162f11347b6197e7418268f9 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 5 Feb 2020 12:34:27 -0500
Subject: [PATCH v9 2/5] Generate backup manifests for base backups.
The manifest includes the file name, size, last modification time, and
a checksum for each file backed up, as well as a checksum for the
manifest itself. By default, we use CRC-32C for the checksum
algorithm, because we are trying to detect corruption and user error,
not foil an adversary. However, pg_basebackup and the server-side
BASE_BACKUP command now have options to select the checksum algorithm,
so users wanting a cryptographic hash function can select SHA-224,
SHA-256, SHA-384, or SHA-512; and users not wanting any checksums at
all can disable them. Using a cryptographic hash function in place of
CRC-32C consumes significantly more CPU cycles, which may slow down
backups in some cases.
Robert Haas with help from Rushabh Lathia and Suraj Kharage.
(This version is improvement from previous versions in that it can
spool the manifest to disk as it is being generated; thus, even a
very large manifest should not run the server out of memory, though
it could conceivably run it out of disk space.)
---
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 364 +++++++++++++++++++++++--
src/backend/replication/repl_gram.y | 6 +
src/backend/replication/repl_scanner.l | 1 +
src/backend/replication/walsender.c | 30 ++
src/bin/pg_basebackup/pg_basebackup.c | 130 ++++++++-
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
8 files changed, 514 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 4361568882..2a38ef90df 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10547,7 +10547,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index f66cbc2428..914859aea8 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -41,6 +43,7 @@
#include "utils/builtins.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
typedef struct
@@ -52,20 +55,40 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ pg_checksum_type checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ int still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest, pg_checksum_type);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *filename, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
+static char *escape_field_for_manifest(const char *s);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +125,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -251,6 +284,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -258,12 +292,17 @@ perform_base_backup(basebackup_options *opt)
backup_streamed = 0;
pgstat_progress_start_command(PROGRESS_COMMAND_BASEBACKUP, InvalidOid);
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt->checksum_type);
total_checksum_failures = 0;
@@ -301,7 +340,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -380,7 +422,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -388,11 +431,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -400,10 +446,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -632,7 +679,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -655,16 +702,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -678,6 +729,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -709,8 +763,11 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -797,6 +854,21 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval, &opt->checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
@@ -918,6 +990,230 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, pg_checksum_type checksum_type)
+{
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->still_checksumming = true;
+
+ AppendToManifest(manifest, "PostgreSQL-Backup-Manifest-Version\t1\n");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *filename, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ char *escaped_pathname;
+ static char timebuf[128];
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /* Escape pathname, if necessary. */
+ escaped_pathname = escape_field_for_manifest(pathname);
+
+ /*
+ * Convert time to a string. Since it's not clear what time zone to use
+ * and since time zone definitions can change, possibly causing confusion,
+ * use GMT always.
+ */
+ pg_strftime(timebuf, sizeof(timebuf), "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+
+ /* Add to manifest. */
+ AppendToManifest(manifest, "File\t%s\t%zu\t%s\t%s",
+ escaped_pathname == NULL ? pathname : escaped_pathname,
+ size, timebuf, pg_checksum_type_name(checksum_ctx->type));
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ char checksumstringbuf[PG_CHECKSUM_MAX_LENGTH * 2 + 1];
+ int checksumlen;
+ int checksumstringlen;
+
+ AppendStringToManifest(manifest, ":");
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+ checksumstringlen = hex_encode((char *) checksumbuf, checksumlen,
+ checksumstringbuf);
+ checksumstringbuf[checksumstringlen] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ }
+ AppendStringToManifest(manifest, "\n");
+
+ /* Avoid leaking memory. */
+ if (escaped_pathname != NULL)
+ pfree(escaped_pathname);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "Manifest-Checksum\t");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
+/*
+ * Escape a field for inclusion in a manifest.
+ *
+ * We use the following escaping rule: If a field contains \t, \r, or \n,
+ * the field must be surrounded by double-quotes, and any internal double
+ * quotes must be doubled. Otherwise, no escaping is required.
+ *
+ * The return value is a new palloc'd string with escaping added, or NULL
+ * if no escaping is required.
+ */
+static char *
+escape_field_for_manifest(const char *s)
+{
+ bool escaping_required = false;
+ int escaped_length = 2;
+ const char *t;
+ char *result;
+ char *r;
+
+ for (t = s; *t != '\0'; ++t)
+ {
+ if (*t == '\t' || *t == '\r' || *t == '\n')
+ escaping_required = true;
+ if (*t == '"')
+ ++escaped_length;
+ ++escaped_length;
+ }
+
+ if (!escaping_required)
+ return NULL;
+
+ result = palloc(escaped_length + 1);
+ result[0] = '"';
+ result[escaped_length - 1] = '"';
+ result[escaped_length] = '\0';
+ r = result + 1;
+
+ for (t = s; *t != '\0'; ++t)
+ {
+ *(r++) = *t;
+ if (*t == '"')
+ *(r++) = *t;
+ }
+
+ Assert(r == &result[escaped_length - 1]);
+
+ return result;
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -978,11 +1274,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1017,6 +1317,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1027,7 +1331,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1060,7 +1365,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1079,7 +1385,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1359,7 +1665,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1367,7 +1674,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1437,8 +1745,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1455,6 +1764,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1625,6 +1937,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1649,6 +1964,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1657,7 +1973,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1682,6 +1999,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..0621884ad8 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,7 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -214,6 +215,11 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..5653d233b5 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,7 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index ae4a9cbe11..cc0b97627c 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index 48bd838803..235416a7c2 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -135,6 +141,7 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -180,6 +187,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -386,6 +399,8 @@ usage(void)
printf(_(" --no-slot prevent creation of temporary replication slot\n"));
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1184,6 +1199,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1655,6 +1695,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1665,6 +1763,7 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_checksums_clause = NULL;
int i;
char xlogstart[64];
char xlogend[64];
@@ -1672,6 +1771,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1725,6 +1825,9 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1739,7 +1842,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s",
escaped_label,
showprogress ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1747,7 +1850,8 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_checksums_clause ? manifest_checksums_clause : "");
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1835,7 +1939,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1864,6 +1969,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && serverMajor >= 1300)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2066,6 +2184,7 @@ main(int argc, char **argv)
{"waldir", required_argument, NULL, 1},
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
+ {"manifest-checksums", required_argument, NULL, 'm'},
{NULL, 0, NULL, 0}
};
int c;
@@ -2093,7 +2212,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2234,6 +2353,9 @@ main(int argc, char **argv)
case 3:
verify_checksums = false;
break;
+ case 'm':
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
[application/octet-stream] v9-0005-Adjust-pg_validate-to-validate-a-JSON-format-mani.patch (24.6K, ../../CA+TgmoYJxdwPSd2as1z5+W40hxuhrnKdgiN+4YY0ppmTC-r36Q@mail.gmail.com/6-v9-0005-Adjust-pg_validate-to-validate-a-JSON-format-mani.patch)
download | inline diff:
From fc6ad5b9812bd417ab3d3aee534219a75e64bb6a Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Thu, 5 Mar 2020 11:27:14 -0500
Subject: [PATCH v9 5/5] Adjust pg_validate to validate a JSON-format manifest.
This is still somewhat rough around the edges.
---
src/bin/pg_validatebackup/pg_validatebackup.c | 698 +++++++++---------
1 file changed, 358 insertions(+), 340 deletions(-)
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
index d4d041ef7d..1af83204b1 100644
--- a/src/bin/pg_validatebackup/pg_validatebackup.c
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -19,9 +19,11 @@
#include "common/checksum_helper.h"
#include "common/hashfn.h"
+#include "common/jsonapi.h"
#include "common/logging.h"
#include "fe_utils/simple_list.h"
#include "getopt_long.h"
+#include "mb/pg_wchar.h"
/*
* For efficiency, we'd like our hash table containing information about the
@@ -60,8 +62,8 @@
#define FIELDS_PER_FILE_LINE 4
/*
- * Each "File" line in the manifest file is parsed to produce an object
- * like this.
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
*/
typedef struct manifestfile
{
@@ -92,6 +94,50 @@ static uint32 hash_string_pointer(char *s);
#define SH_DEFINE
#include "lib/simplehash.h"
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+typedef struct
+{
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ manifestfiles_hash *ht;
+ char *pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
/*
* All of the context information we need while checking a backup manifest.
*/
@@ -105,8 +151,15 @@ typedef struct validator_context
} validator_context;
static manifestfiles_hash * parse_manifest_file(char *manifest_path);
-static void parse_file_line_from_manifest(manifestfile *f, char *rest,
- int restlen);
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+
static void validate_backup_directory(validator_context *context,
char *relpath, char *fullpath);
static void validate_backup_file(validator_context *context,
@@ -123,9 +176,6 @@ static void pg_validator_fatal(const char *pg_restrict fmt,...)
pg_attribute_printf(1, 2) pg_attribute_noreturn();
static bool should_ignore_relpath(validator_context *context, char *relpath);
-static char *extractstr(char *buffer, int length);
-static int findchar(char *buffer, int size, char c, int start_position);
-static int findfield(char *buffer, char *end, char **result);
static int hexdecode_char(char c);
static bool hexdecode_string(uint8 *result, char *input, int nbytes);
static void usage(void);
@@ -301,19 +351,20 @@ parse_manifest_file(char *manifest_path)
int fd;
struct stat statbuf;
off_t estimate;
- off_t bytes_read = 0;
- off_t bytes_consumed = 0;
uint32 initial_size;
manifestfiles_hash *ht;
char *buffer;
+ int rc;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
- int buffer_position = 0;
- int buffer_size = 0;
- int buffer_maxsize = 2 * READ_CHUNK_SIZE;
- int line_number = 0;
- bool saw_manifest_checksum_line = false;
- pg_sha256_ctx manifest_ctx;
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
/* Prepare to compute a checksum of the manifest itself. */
pg_sha256_init(&manifest_ctx);
@@ -333,252 +384,335 @@ parse_manifest_file(char *manifest_path)
/* Create the hash table. */
ht = manifestfiles_create(initial_size, NULL);
- /* Initialize our read buffer. */
- buffer = pg_malloc(buffer_maxsize);
-
/*
- * Loop until we've read it all.
+ * Slurp in the whole file.
*
- * The file size shouldn't be changing, so it seems fine to just error out
- * if the final length is different from what stat() told us.
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
*/
- while (bytes_consumed < statbuf.st_size)
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
{
- int line_length;
- int first_field_length;
- char *rest;
- int restlen;
-
- /* Find next newline if any. */
- line_length = findchar(buffer, buffer_size, '\n', buffer_position);
+ if (rc < 0)
+ pg_validator_fatal("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ pg_validator_fatal("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
- /* If no newline was found, we need to read more data and try again. */
- if (line_length == -1)
+ /* Find the last two newlines in the file. */
+ for (size_t i = 0; i < statbuf.st_size; ++i)
+ {
+ if (buffer[i] == '\n')
{
- size_t bytes_to_read;
- int rc;
-
- bytes_to_read = Min(statbuf.st_size - bytes_read, READ_CHUNK_SIZE);
- if (bytes_to_read == 0)
- pg_validator_fatal("manifest file line not terminated by newline");
- if (bytes_to_read + READ_CHUNK_SIZE > buffer_maxsize)
- {
- buffer_maxsize += READ_CHUNK_SIZE;
- buffer = pg_realloc(buffer, buffer_maxsize);
- Assert(bytes_to_read + READ_CHUNK_SIZE <= buffer_maxsize);
- }
- rc = read(fd, buffer + buffer_size, bytes_to_read);
- if (rc != bytes_to_read)
- {
- if (rc < 0)
- pg_validator_fatal("could not read file \"%s\": %m",
- manifest_path);
- else
- pg_validator_fatal("could not read file \"%s\": read %d of %zu",
- manifest_path, rc, bytes_to_read);
- }
- buffer_size += rc;
- bytes_read += rc;
- continue;
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
}
+ }
- /* Increment line number. */
- ++line_number;
+ /*
+ * Make sure that the last newline is right at the end, and that there
+ * are at least two lines total.
+ */
+ if (number_of_newlines < 2)
+ pg_validator_fatal("could not parse backup manifest: %s",
+ "expected at least 2 lines");
+ if (ultimate_newline != statbuf.st_size - 1)
+ pg_validator_fatal("could not parse backup manifest: %s",
+ "last line not newline-terminated");
- /* The manifest checksum should be the last thing in the file. */
- if (saw_manifest_checksum_line)
- pg_validator_fatal("unexpected data follows manifest checksum");
+ /*
+ * The manifest checksum covers everything in the file except for the
+ * very last line.
+ */
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
- /* Find first field on line, and remaining line contents. */
- first_field_length =
- findchar(buffer, buffer_size, '\t', buffer_position);
- rest = buffer + buffer_position + first_field_length + 1;
- restlen = line_length - (first_field_length + 1);
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, statbuf.st_size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.ht = ht;
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Parse JSON. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ pg_validator_fatal("could not parse backup manifest: %s",
+ json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ pg_validator_fatal("could not parse backup manifest: %s",
+ "manifest ended unexpectedly");
+
+ /* Verify manifest checksum. */
+ if (parse.manifest_checksum == NULL)
+ pg_validator_fatal("backup manifest checksum is missing");
+ if (strlen(parse.manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse.manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ pg_validator_fatal("invalid manifest checksum: \"%s\"",
+ parse.manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ pg_validator_fatal("manifest checksum mismatch");
- /*
- * Check the first word of the line to see what kind of line it is.
- */
- if (first_field_length == KWL_MANIFEST_VERSION &&
- memcmp(buffer + buffer_position, KW_MANIFEST_VERSION,
- KWL_MANIFEST_VERSION) == 0)
- {
- if (line_number != 1)
- pg_validator_fatal("manifest file version should only be specified at line 1");
- else
- {
- char *line = buffer + buffer_position;
- char *version;
-
- version = extractstr(line + first_field_length + 1,
- line_length - (first_field_length + 1));
- if (strcmp(version, "1") != 0)
- pg_validator_fatal("unrecognized manifest version: \"%s\"",
- version);
- }
- }
- else if (first_field_length == KWL_MANIFEST_FILE &&
- memcmp(buffer + buffer_position, KW_MANIFEST_FILE,
- KWL_MANIFEST_FILE) == 0)
- {
- manifestfile f;
- manifestfile *tabent;
- bool found;
+ /* OK, we're done with the manifest file. */
+ close(fd);
- /* Parse this line. */
- parse_file_line_from_manifest(&f, rest, restlen);
+ /* Return the hash table we constructed. */
+ return ht;
+}
- /* Make a new entry in the hash table for it. */
- tabent = manifestfiles_insert(ht, f.pathname, &found);
+static void
+json_manifest_parse_failure(char *msg)
+{
+ pg_validator_fatal("could not parse backup manifest: %s", msg);
+}
+
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected object start");
+ break;
+ }
+}
+
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+ manifestfile *tabent;
+ bool found;
+ int checksum_string_length;
+ char *ep;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL)
+ json_manifest_parse_failure("missing pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure("missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure("checksum without algorithm");
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(parse->ht, parse->pathname, &found);
if (found)
pg_validator_fatal("duplicate pathname in backup manifest: \"%s\"",
- f.pathname);
+ parse->pathname);
- /* Copy in all the relevant details. */
- tabent->size = f.size;
- tabent->checksum_type = f.checksum_type;
- tabent->checksum_length = f.checksum_length;
- tabent->checksum_payload = f.checksum_payload;
+ /* Initialize some fields. */
tabent->matched = false;
tabent->bad = false;
- }
- else if (first_field_length == KWL_MANIFEST_CHECKSUM &&
- memcmp(buffer + buffer_position, KW_MANIFEST_CHECKSUM,
- KWL_MANIFEST_CHECKSUM) == 0)
- {
- saw_manifest_checksum_line = true;
- if (restlen != PG_SHA256_DIGEST_STRING_LENGTH - 1)
- pg_validator_fatal("manifest file checksum has unexpected length: %d",
- restlen);
- if (!hexdecode_string(manifest_checksum_expected, rest,
- PG_SHA256_DIGEST_LENGTH))
- pg_validator_fatal("invalid manifest checksum: \"%s\"",
- extractstr(rest, restlen));
- }
- else if (first_field_length == -1)
- pg_validator_fatal("manifest file keyword not terminated by tab");
- else
- {
- char *kw;
- kw = extractstr(buffer + buffer_position, first_field_length);
- pg_validator_fatal("unrecognized manifest file keyword: \"%s\"", kw);
- }
-
- /* Update manifest checksum, if needed. */
- if (!saw_manifest_checksum_line)
- pg_sha256_update(&manifest_ctx, (uint8 *) buffer + buffer_position,
- line_length + 1);
-
- /* Advance buffer position over the data we just read. */
- buffer_position += line_length + 1;
-
- /* Also mark these bytes as consumed so we know when to stop. */
- bytes_consumed += line_length + 1;
+ /* Parse size. */
+ tabent->size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure("file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ tabent->checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm,
+ &tabent->checksum_type))
+ pg_validator_fatal("unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ tabent->checksum_length = 0;
+ tabent->checksum_payload = NULL;
+ }
+ else
+ {
+ tabent->checksum_length = checksum_string_length / 2;
+ tabent->checksum_payload = palloc(tabent->checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(tabent->checksum_payload,
+ parse->checksum,
+ tabent->checksum_length))
+ pg_validator_fatal("invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname,
+ tabent->checksum_payload);
+ }
- /*
- * We don't want to incur the expensive of using memmove() to discard
- * data after every line, because the lines are short compared to the
- * chunk size -- but we must do it at least now and then, or we'll
- * have to keep growing the buffer.
- */
- if (buffer_position >= READ_CHUNK_SIZE)
- {
- int leftover_bytes = buffer_size - buffer_position;
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
- if (leftover_bytes > 0)
- memmove(buffer, buffer + buffer_position, leftover_bytes);
- buffer_size -= buffer_position;
- buffer_position = 0;
- }
+ /* Expect next file (or end of list). */
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected object end");
+ break;
}
+}
- /* Checksum verification. */
- if (!saw_manifest_checksum_line)
- pg_validator_fatal("manifest has no checksum");
- pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
- if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
- PG_SHA256_DIGEST_LENGTH) != 0)
- pg_validator_fatal("manifest checksum does not match");
-
- /* OK, we're done with the manifest file. */
- close(fd);
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
- /* Return the hash table we constructed. */
- return ht;
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected array start");
+ break;
+ }
}
-/*
- * The caller passes the remainder of the line, excluding the initial "File\t"
- * portion.
- */
static void
-parse_file_line_from_manifest(manifestfile *f, char *rest, int restlen)
+json_manifest_array_end(void *state)
{
- char *end = rest + restlen;
- char *field[FIELDS_PER_FILE_LINE];
- unsigned long filesize;
- char *ep;
- pg_checksum_type checksum_type;
- int raw_checksum_length = 0;
- char *raw_checksum_payload = NULL;
- int checksum_length;
- uint8 *checksum_payload;
- int i;
- char *s;
+ JsonManifestParseState *parse = state;
- /* Split the line into fields. */
- for (i = 0; i < FIELDS_PER_FILE_LINE; ++i)
+ switch (parse->state)
{
- int toklen;
-
- toklen = findfield(rest, end, &field[i]);
- if (rest + toklen >= end && i + 1 < FIELDS_PER_FILE_LINE)
- pg_validator_fatal("manifest file line has too few fields");
- rest += toklen + 1;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected array end");
+ break;
}
+}
- /* We expect to have used the entire line. */
- if (rest < end)
- pg_validator_fatal("manifest file line has too many fields");
-
- /* Parse the size. */
- filesize = strtoul(field[1], &ep, 10);
- if (*ep)
- pg_validator_fatal("manifest file size for file \"%s\" is not a number",
- field[0]);
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
- /* Parse the checksum type. */
- for (s = field[3]; s[0] != '\0' && s[0] != ':'; ++s)
- ;
- if (*s)
+ switch (parse->state)
{
- raw_checksum_payload = s + 1;
- raw_checksum_length = strlen(raw_checksum_payload);
- *s = '\0';
+ case JM_EXPECT_VERSION_FIELD:
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure("expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure("expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure("unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure("expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected object field");
+ break;
}
- if (!pg_checksum_parse_type(field[3], &checksum_type))
- pg_validator_fatal("unrecognized checksum algorithm for file \"%s\": \"%s\"",
- field[0], field[3]);
-
- /* Decode the checksum payload. */
- checksum_length = raw_checksum_length / 2;
- if (checksum_length == 0)
- checksum_payload = NULL;
- else
+}
+
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
{
- checksum_payload = palloc(checksum_length);
- if (!hexdecode_string(checksum_payload, raw_checksum_payload,
- checksum_length))
- pg_validator_fatal("invalid checksum for file \"%s\": \"%s\"",
- field[0], raw_checksum_payload);
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure("unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure("unexpected scalar");
+ break;
}
-
- /* Fill the output struct. */
- f->pathname = field[0];
- f->size = filesize;
- f->checksum_type = checksum_type;
- f->checksum_length = checksum_length;
- f->checksum_payload = checksum_payload;
}
/*
@@ -912,122 +1046,6 @@ should_ignore_relpath(validator_context *context, char *relpath)
return false;
}
-/*
- * Extract a NUL-terminated string from a larger buffer.
- */
-static char *
-extractstr(char *buffer, int length)
-{
- char *s = palloc(length + 1);
-
- memcpy(s, buffer, length);
- s[length] = '\0';
-
- return s;
-}
-
-/*
- * Find the next instance of a given character within a buffer that
- * occurs at or after start_position. If there is none, returns -1; else
- * returns the difference between the position at which the character was
- * found and the start position.
- */
-static int
-findchar(char *buffer, int size, char c, int start_position)
-{
- int i;
-
- for (i = start_position; i < size; ++i)
- if (buffer[i] == c)
- return i - start_position;
- return -1;
-}
-
-/*
- * Extract the next field from a line of text read from the manifest file.
- */
-static int
-findfield(char *buffer, char *end, char **result)
-{
- int qoffset = 1;
- int dqcount = 0;
- int toklen;
- int bufpos;
- int resultpos;
-
- /*
- * If this field is unquoted, we just stop at the next tab; if there's
- * none, we stop at the end of the line. Note that if buffer == end, it
- * just means that the last field on the line is empty.
- */
- if (buffer == end || *buffer != '"')
- {
- toklen = findchar(buffer, end - buffer, '\t', 0);
-
- if (toklen == -1)
- toklen = end - buffer;
- *result = extractstr(buffer, toklen);
- return toklen;
- }
-
- /*
- * Our escaping convention is that if the field contains a tab, it must be
- * surrounded by double-quotes and any internal double-quotes must be
- * doubled.
- */
- while (1)
- {
- /* Where's the next double quote? */
- qoffset += findchar(buffer, end - buffer, '"', qoffset);
- if (qoffset == -1)
- pg_validator_fatal("quoted field in backup manifest is not terminated");
-
- /*
- * If the double-quote we found is the last character on the line or
- * if it's followed by a tab, we've reached the end of this field.
- */
- if (buffer + qoffset >= end || buffer[qoffset + 1] == '\t')
- break;
-
- /* Otherwise, the next character should be another double-quote. */
- if (buffer[qoffset + 1] != '"')
- {
- pg_log_fatal("invalid quoted field in backup manifest");
- exit(1);
- }
-
- /* Skip both double-quotes and go around again. */
- qoffset += 2;
- ++dqcount;
- }
-
- /*
- * At this point, we know that qoffset is the offset, relative to buffer,
- * of the closing double-quote, and that dqcount is the number of escaped
- * double-quotes within the field, and that all of those escape sequences
- * are proper. Extract and de-escape the data in the field.
- *
- * The amount of space needed for the result is equal to the raw token
- * length, minus two for the double quotes at the start and end, minus one
- * for each doubled double-quote within the token, plus one for the
- * trailing zero byte.
- */
- toklen = qoffset + 1;
- *result = palloc(toklen - dqcount - 1);
- bufpos = 1;
- resultpos = 0;
- while (bufpos < qoffset)
- {
- (*result)[resultpos] = buffer[bufpos];
- bufpos += (buffer[bufpos] == '"' ? 2 : 1);
- ++resultpos;
- }
- (*result)[resultpos] = '\0';
- Assert(resultpos == toklen - dqcount - 2);
-
- return toklen;
-}
-
/*
* Helper function for manifestfiles hash table.
*/
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-06 08:58 ` Suraj Kharage <[email protected]>
2020-03-11 20:08 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Suraj Kharage @ 2020-03-06 08:58 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Thanks, Robert.
1: Getting below error while compiling 0002 patch.
edb@localhost:postgres$ mi > mi.log
basebackup.c: In function ‘AddFileToManifest’:
basebackup.c:1052:6: error: ‘pathname’ undeclared (first use in this
function)
pathname);
^
basebackup.c:1052:6: note: each undeclared identifier is reported only once
for each function it appears in
make[3]: *** [basebackup.o] Error 1
make[2]: *** [replication-recursive] Error 2
make[1]: *** [install-backend-recurse] Error 2
make: *** [install-src-recurse] Error 2
I can see you have renamed the filename argument of AddFileToManifest() to
pathname, but those changes are part of 0003 (validator patch).
I think the changes related to src/backend/replication/basebackup.c should
not be there in the validator patch (0003). We can move these changes to
backup manifest patch, either in 0002 or 0004 for better readability of
patch set.
2:
#define KW_MANIFEST_VERSION "PostgreSQL-Backup-Manifest-Version"
#define KW_MANIFEST_FILE "File"
#define KW_MANIFEST_CHECKSUM "Manifest-Checksum"
#define KWL_MANIFEST_VERSION (sizeof(KW_MANIFEST_VERSION)-1)
#define KWL_MANIFEST_FILE (sizeof(KW_MANIFEST_FILE)-1)
#define KWL_MANIFEST_CHECKSUM (sizeof(KW_MANIFEST_CHECKSUM)-1)
#define FIELDS_PER_FILE_LINE 4
Few macros defined in 0003 patch not used anywhere in 0005 patch. Either we
can replace these with hard-coded values or remove them.
On Thu, Mar 5, 2020 at 10:25 PM Robert Haas <[email protected]> wrote:
> On Thu, Mar 5, 2020 at 7:05 AM tushar <[email protected]>
> wrote:
> > There is one small observation if we use slash (/) with option -i then
> not getting the desired result
>
> Here's an updated patch set responding to many of the comments
> received thus far. Since there are quite a few emails, let me
> consolidate my comments and responses here.
>
> Report: Segmentation fault if -m is used to point to a valid manifest,
> but actual backup directory is nonexistent.
> Response: Fixed; thanks for the report.
>
> Report: pg_validatebackup doesn't complain about problems within the
> pg_wal directory.
> Response: That's out of scope. The WAL files are fetched separately
> and are therefore not part of the manifest.
>
> Report: Inaccessible file in data directory being validated leads to a
> double free.
> Response: Fixed; thanks for the report.
>
> Report: Patch 0005 doesn't validate the manifest checksum.
> Response: I know. I mentioned that when posting the previous patch
> set. Fixed in this version, though.
>
> Report: Removing an empty directory doesn't make backup validation
> fail, even though it might cause problems for the server.
> Response: That's a little unfortunate, but I'm not sure it's really
> worth complicating the patch to deal with it. It's something of a
> corner case.
>
> Report: Negative file sizes in the backup manifest are interpreted as
> large integers.
> Response: That's also a little unfortunate, but I doubt it's worth
> adding code to catch it, since any such manifest is corrupt. Also,
> it's not like we're ignoring it; the error just isn't ideal.
>
> Report: If I take the backup label from backup #1 and stick it into
> otherwise-identical backup #2, validation succeeds but the server
> won't start.
> Response: That's because we can't validate the pg_wal directory. As
> noted above, that's out of scope.
>
> Report: Using --ignore with a slash-terminated pathname doesn't work
> as expected.
> Response: Fixed, thanks for the report.
>
> Off-List Report: You forgot a PG_BINARY flag.
> Response: Fixed. I thought I'd done this before but there were two
> places and I'd only fixed one of them.
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
--
--
Thanks & Regards,
Suraj kharage,
EnterpriseDB Corporation,
The Postgres Database Company.
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-06 08:58 ` Re: backup manifests Suraj Kharage <[email protected]>
@ 2020-03-11 20:08 ` Robert Haas <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-11 20:08 UTC (permalink / raw)
To: Suraj Kharage <[email protected]>; +Cc: tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 6, 2020 at 3:58 AM Suraj Kharage
<[email protected]> wrote:
> 1: Getting below error while compiling 0002 patch.
> 2:
>
> Few macros defined in 0003 patch not used anywhere in 0005 patch. Either we can replace these with hard-coded values or remove them.
Thanks. I hope that I have straightened those things out in the new
version which is attached. This version also includes some other
changes. The non-JSON code is now completely gone. Also, I've
refactored the code that does parses the JSON manifest to make it
cleaner, and I've moved it out into a separate file. This might be
useful if anyone ends up wanting to reuse that code for some other
purpose, and I think it makes it easier to understand, too, since the
manifest parsing is now much better separated from the task of
actually validating the given directory against the manifest. I've
also added some tests, which are based in part on testing ideas from
Rajkumar Raghuwanshi and Mark Dilger, but this test code was written
by me. So now it's like this:
0001 - checksum helper functions. same as before.
0002 - patch the server to generate and send a manifest, and
pg_basebackup to receive it
0003 - add pg_validatebackup
0004 - TAP tests
Comments?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v10-0004-Regression-tests-for-pg_validatebackup.patch (9.5K, ../../CA+TgmoYUVyp_ydjAtgm+3b4X0bu26FvyZWe8e8gM6GGrpbYbsQ@mail.gmail.com/2-v10-0004-Regression-tests-for-pg_validatebackup.patch)
download | inline diff:
From 0fcfaa3691f681a5470399f97ab674873567bf48 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Tue, 10 Mar 2020 13:44:46 -0400
Subject: [PATCH v10 4/4] Regression tests for pg_validatebackup.
Patch by me, based in part on ideas from Mark Dilger and
Rajkumar Raghuwanshi.
---
src/bin/pg_validatebackup/.gitignore | 1 +
src/bin/pg_validatebackup/Makefile | 6 +
src/bin/pg_validatebackup/t/001_basic.pl | 27 +++
src/bin/pg_validatebackup/t/002_algorithm.pl | 55 ++++++
src/bin/pg_validatebackup/t/003_corruption.pl | 178 ++++++++++++++++++
5 files changed, 267 insertions(+)
create mode 100644 src/bin/pg_validatebackup/t/001_basic.pl
create mode 100644 src/bin/pg_validatebackup/t/002_algorithm.pl
create mode 100644 src/bin/pg_validatebackup/t/003_corruption.pl
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
index 3ae1c1f03a..21e0a92429 100644
--- a/src/bin/pg_validatebackup/.gitignore
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -1 +1,2 @@
/pg_validatebackup
+/tmp_check/
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
index dde7eb3c02..04ef7d3051 100644
--- a/src/bin/pg_validatebackup/Makefile
+++ b/src/bin/pg_validatebackup/Makefile
@@ -31,3 +31,9 @@ uninstall:
clean distclean maintainer-clean:
rm -f pg_validatebackup$(X) $(OBJS)
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_validatebackup/t/001_basic.pl b/src/bin/pg_validatebackup/t/001_basic.pl
new file mode 100644
index 0000000000..49153f89ac
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/001_basic.pl
@@ -0,0 +1,27 @@
+use strict;
+use warnings;
+use TestLib;
+use Test::More tests => 14;
+
+my $tempdir = TestLib::tempdir;
+
+program_help_ok('pg_validatebackup');
+program_version_ok('pg_validatebackup');
+program_options_handling_ok('pg_validatebackup');
+
+command_fails_like(['pg_validatebackup'],
+ qr/no backup directory specified/,
+ 'pg_validatebackup needs target directory specified');
+command_fails_like(['pg_validatebackup', $tempdir],
+ qr/could not open file.*\/backup_manifest\"/,
+ 'pg_validatebackup requires a manifest');
+
+# create fake manifest file
+open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+close($fh);
+
+# but then try to use an alternate, nonexisting manifest
+command_fails_like(['pg_validatebackup', '-m', "$tempdir/not_the_manifest",
+ $tempdir],
+ qr/could not open file.*\/not_the_manifest\"/,
+ 'pg_validatebackup respects -m flag');
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
new file mode 100644
index 0000000000..9bb89d3855
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -0,0 +1,55 @@
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 19;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
+{
+ my $backup_path = $master->backup_dir . '/' . $algorithm;
+ my @backup = ('pg_basebackup', '-D', $backup_path, '-m', $algorithm,
+ '--no-sync');
+ my @validate = ('pg_validatebackup', '-e', $backup_path);
+
+ # A backup with a bogus algorithm should fail.
+ if ($algorithm eq 'bogus')
+ {
+ $master->command_fails(\@backup,
+ "backup fails with algorithm \"$algorithm\"");
+ next;
+ }
+
+ # A backup with a valid algorithm should work.
+ $master->command_ok(\@backup, "backup ok with algorithm \"$algorithm\"");
+
+ # We expect each real checksum algorithm to be mentioned on every line of
+ # the backup manifest file except the first and last; for simplicity, we
+ # just check that it shows up lots of times. When the checksum algorithm
+ # is none, we just check that the manifest exists.
+ if ($algorithm eq 'none')
+ {
+ ok(-f "$backup_path/backup_manifest", "backup manifest exists");
+ }
+ else
+ {
+ my $manifest = slurp_file("$backup_path/backup_manifest");
+ my $count_of_algorithm_in_manifest =
+ (() = $manifest =~ /$algorithm/mig);
+ cmp_ok($count_of_algorithm_in_manifest, '>', 100,
+ "$algorithm is mentioned many times in the manifest");
+ }
+
+ # Make sure that it validates OK.
+ $master->command_ok(\@validate,
+ "validate backup with algorithm \"$algorithm\"");
+
+ # Remove backup immediately to save disk space.
+ rmtree($backup_path);
+}
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
new file mode 100644
index 0000000000..07ed7b6657
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -0,0 +1,178 @@
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 32;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+# Include a user-defined tablespace in the hopes of detecting problems in that
+# area.
+my $source_ts_path = TestLib::tempdir;
+$master->safe_psql('postgres', <<EOM);
+CREATE TABLE x1 (a int);
+INSERT INTO x1 VALUES (111);
+CREATE TABLESPACE ts1 LOCATION '$source_ts_path';
+CREATE TABLE x2 (a int) TABLESPACE ts1;
+INSERT INTO x1 VALUES (222);
+EOM
+
+my @scenario = (
+ {
+ 'name' => 'extra_file',
+ 'mutilate' => \&mutilate_extra_file,
+ 'fails_like' =>
+ qr/extra_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'extra_tablespace_file',
+ 'mutilate' => \&mutilate_extra_tablespace_file,
+ 'fails_like' =>
+ qr/extra_ts_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'missing_file',
+ 'mutilate' => \&mutilate_missing_file,
+ 'fails_like' =>
+ qr/pg_xact\/0000.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'missing_tablespace',
+ 'mutilate' => \&mutilate_missing_tablespace,
+ 'fails_like' =>
+ qr/pg_tblspc.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'append_to_file',
+ 'mutilate' => \&mutilate_append_to_file,
+ 'fails_like' =>
+ qr/has size \d+ on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'truncate_file',
+ 'mutilate' => \&mutilate_truncate_file,
+ 'fails_like' =>
+ qr/has size 0 on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'replace_file',
+ 'mutilate' => \&mutilate_replace_file,
+ 'fails_like' => qr/checksum mismatch for file/
+ },
+ {
+ 'name' => 'bad_manifest',
+ 'mutilate' => \&mutilate_bad_manifest,
+ 'fails_like' => qr/manifest checksum mismatch/
+ }
+);
+
+for my $scenario (@scenario)
+{
+ my $name = $scenario->{'name'};
+
+ # Take a backup and check that it validates OK.
+ my $backup_path = $master->backup_dir . '/' . $name;
+ my $backup_ts_path = TestLib::tempdir;
+ $master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '-T', "${source_ts_path}=${backup_ts_path}"],
+ "base backup ok");
+ command_ok(['pg_validatebackup', $backup_path ],
+ "intact backup validated");
+
+ # Mutilate the backup in some way.
+ $scenario->{'mutilate'}->($backup_path);
+
+ # Now check that the backup no longer validates.
+ command_fails_like(['pg_validatebackup', $backup_path ],
+ $scenario->{'fails_like'},
+ "corrupt backup fails validation: $name");
+}
+
+sub create_extra_file
+{
+ my ($backup_path, $relative_path) = @_;
+ my $pathname = "$backup_path/$relative_path";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh "This is an extra file.\n";
+ close($fh);
+}
+
+# Add a file into the root directory of the backup.
+sub mutilate_extra_file
+{
+ my ($backup_path) = @_;
+ create_extra_file($backup_path, "extra_file");
+}
+
+# Add a file inside the user-defined tablespace.
+sub mutilate_extra_tablespace_file
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my ($catvdir) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid");
+ my ($tsdboid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid/$catvdir");
+ create_extra_file($backup_path,
+ "pg_tblspc/$tsoid/$catvdir/$tsdboid/extra_ts_file");
+}
+
+# Remove a file.
+sub mutilate_missing_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_xact/0000";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Remove the symlink to the user-defined tablespace.
+sub mutilate_missing_tablespace
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my $pathname = "$backup_path/pg_tblspc/$tsoid";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Append an additional bytes to a file.
+sub mutilate_append_to_file
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/global/pg_control", 'x';
+}
+
+# Truncate a file to zero length.
+sub mutilate_truncate_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/global/pg_control";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ close($fh);
+}
+
+# Replace a file's contents without changing the length of the file. This is
+# not a particularly efficient way to do this, so we pick a file that's
+# expected to be short.
+sub mutilate_replace_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ my $contents = slurp_file($pathname);
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh 'q' x length($contents);
+ close($fh);
+}
+
+# Corrupt the backup manifest.
+sub mutilate_bad_manifest
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/backup_manifest", "\n";
+}
--
2.17.2 (Apple Git-113)
[application/octet-stream] v10-0001-Add-checksum-helper-functions.patch (9.8K, ../../CA+TgmoYUVyp_ydjAtgm+3b4X0bu26FvyZWe8e8gM6GGrpbYbsQ@mail.gmail.com/3-v10-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From 7a53f5a6c5dc617f34a70dd9ac67f8d236185207 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 5 Feb 2020 13:59:45 -0500
Subject: [PATCH v10 1/4] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
(In my version, I decided to support all of the SHA variants for
which we have code, added a function to translate values of the
enum type back to strings so that users of this code don't need
to do that, used different naming, and tried to be more careful
amount formatting and comments.)
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index ce01df68b9..e199ed7acb 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -47,6 +47,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index f89a8a4fdb..ee04fbafa6 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -120,8 +120,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
- f2s.c file_perm.c hashfn.c ip.c jsonapi.c
+ base64.c checksum_helper.c config_info.c controldata_utils.c d2s.c
+ encnames.c exec.c f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c username.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v10-0003-pg_validatebackup-Validate-a-backup-against-the-.patch (40.9K, ../../CA+TgmoYUVyp_ydjAtgm+3b4X0bu26FvyZWe8e8gM6GGrpbYbsQ@mail.gmail.com/4-v10-0003-pg_validatebackup-Validate-a-backup-against-the-.patch)
download | inline diff:
From a104f1eb24cbe94b1349f42f5237c7748eca8a4b Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 9 Mar 2020 15:23:24 -0400
Subject: [PATCH v10 3/4] pg_validatebackup: Validate a backup against the
backup manifest.
Patch by me; review by Tushar Ahuja and Rajkumar Raghuwanshi, and also
off-list by Mark Dilger, Davinder Singh, and Jeevan Chalke.
---
src/bin/Makefile | 1 +
src/bin/pg_validatebackup/.gitignore | 1 +
src/bin/pg_validatebackup/Makefile | 33 +
src/bin/pg_validatebackup/parse_manifest.c | 546 +++++++++++++
src/bin/pg_validatebackup/parse_manifest.h | 40 +
src/bin/pg_validatebackup/pg_validatebackup.c | 719 ++++++++++++++++++
6 files changed, 1340 insertions(+)
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/parse_manifest.c
create mode 100644 src/bin/pg_validatebackup/parse_manifest.h
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..3ae1c1f03a
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1 @@
+/pg_validatebackup
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..dde7eb3c02
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,33 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ parse_manifest.o \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
new file mode 100644
index 0000000000..d8680f96e1
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -0,0 +1,546 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.c
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "parse_manifest.h"
+#include "common/jsonapi.h"
+
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+/*
+ * Internal state used while decoding the JSON-format backup manifest.
+ */
+typedef struct
+{
+ JsonManifestParseContext *context;
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ char *pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void verify_manifest_checksum(JsonManifestParseState *parse,
+ char *buffer, size_t size);
+static void json_manifest_parse_failure(JsonManifestParseContext *context,
+ char *msg);
+
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+
+/*
+ * Main entrypoint to parse a JSON-format backup manifest.
+ *
+ * Caller should set up the parsing context and then invoke this function.
+ * For each file whose information is extracted from the manifest,
+ * context->perfile_cb is invoked. In case of trouble, context->error_cb is
+ * invoked and is expected not to return.
+ */
+void
+json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ size_t size)
+{
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
+
+ /* Set up our private parsing context. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.context = context;
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+ /* Validate the checksum. */
+ verify_manifest_checksum(&parse, buffer, size);
+}
+
+/*
+ * Invoked at the start of each object in the JSON document.
+ *
+ * The document as a whole is expected to be an object with three keys
+ * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
+ * file is expected to be an object with various keys (Path, Size, etc.).
+ * If we're not at the beginning of either the toplevel object or the object
+ * for a particular file, it's an error.
+ */
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each object in the JSON document.
+ *
+ * The possible cases here are the same as for json_manifest_object_start.
+ * There's nothing special to do at the end of the document, but when we
+ * reach the end of an object representing a particular file, we must call
+ * json_manifest_finalize_file() to save the associated details.
+ */
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ json_manifest_finalize_file(parse);
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each array in the JSON document.
+ *
+ * Within the toplevel object, the value associated with the "Files" key
+ * should be an array. No other arrays are expected.
+ */
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each array in the JSON document.
+ *
+ * Just like json_manifest_array_start, there's only one expected case
+ * here.
+ */
+static void
+json_manifest_array_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each object field in the JSON document.
+ */
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_FIELD:
+ /* Inside toplevel object, expecting version indicator. */
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ /* Inside toplevel object, expecting "Files" next. */
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure(parse->context,
+ "unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ /* Inside toplevel object, expecting "Manifest-Checksum" next. */
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object field");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each scalar in the JSON document.
+ *
+ * Object field names don't reach this code; those are handled by
+ * json_manifest_object_field_start. When we're inside of the object for
+ * a particular file, that function will have noticed the name of the field,
+ * and we'll get the corresponding value here. When we're in the toplevel
+ * object, the parse state itself tells us which field this is.
+ *
+ * In all cases except for PostgreSQL-Backup-Manifest-Version, which we
+ * can just check on the spot, the goal here is just to save the value in
+ * the parse state for later use. We don't actually do anything until we
+ * reach either the end of the object representing this file, or the end
+ * of the manifest, as the case may be.
+ */
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure(parse->context,
+ "unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context, "unexpected scalar");
+ break;
+ }
+}
+
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * file, and invoke the per-file callback so that the caller gets those
+ * details. This happens for each file when the corresponding JSON object is
+ * completely parsed.
+ */
+static void
+json_manifest_finalize_file(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t size;
+ char *ep;
+ int checksum_string_length;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL)
+ json_manifest_parse_failure(parse->context, "missing pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure(parse->context, "missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure(parse->context,
+ "checksum without algorithm");
+
+ /* Parse size. */
+ size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm, &checksum_type))
+ context->error_cb(context, "unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ checksum_length = 0;
+ checksum_payload = NULL;
+ }
+ else
+ {
+ checksum_length = checksum_string_length / 2;
+ checksum_payload = palloc(checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(checksum_payload, parse->checksum,
+ checksum_length))
+ context->error_cb(context,
+ "invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, checksum_payload);
+ }
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perfile_cb(context, parse->pathname, size,
+ checksum_type, checksum_length, checksum_payload);
+
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
+}
+
+/*
+ * Verify that the manifest checksum is correct.
+ *
+ * The last line of the manifest file is excluded from the manifest checksum,
+ * because the last line is expected to contain the checksum that covers
+ * the rest of the file.
+ */
+static void
+verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
+ size_t size)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t i;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+
+ /* Find the last two newlines in the file. */
+ for (i = 0; i < size; ++i)
+ {
+ if (buffer[i] == '\n')
+ {
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
+ }
+ }
+
+ /*
+ * Make sure that the last newline is right at the end, and that there are
+ * at least two lines total. We need this to be true in order for the
+ * following code, which computes the manifest checksum, to work properly.
+ */
+ if (number_of_newlines < 2)
+ json_manifest_parse_failure(parse->context,
+ "expected at least 2 lines");
+ if (ultimate_newline != size - 1)
+ json_manifest_parse_failure(parse->context,
+ "last line not newline-terminated");
+
+ /* Checksum the rest. */
+ pg_sha256_init(&manifest_ctx);
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+
+ /* Now verify it. */
+ if (parse->manifest_checksum == NULL)
+ context->error_cb(parse->context, "manifest has no checksum");
+ if (strlen(parse->manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse->manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ context->error_cb(context, "invalid manifest checksum: \"%s\"",
+ parse->manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ context->error_cb(context, "manifest checksum mismatch");
+}
+
+/*
+ * Report a parse error.
+ *
+ * This is intended to be used for fairly low-level failures that probably
+ * shouldn't occur unless somebody has deliberately constructed a bad manifest,
+ * or unless the server is generating bad manifests due to some bug. msg should
+ * be a short string giving some hint as to what the problem is.
+ */
+static void
+json_manifest_parse_failure(JsonManifestParseContext *context, char *msg)
+{
+ context->error_cb(context, "could not parse backup manifest: %s", msg);
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
new file mode 100644
index 0000000000..b0b18a57ca
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -0,0 +1,40 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.h
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PARSE_MANIFEST_H
+#define PARSE_MANIFEST_H
+
+#include "common/checksum_helper.h"
+#include "mb/pg_wchar.h"
+
+struct JsonManifestParseContext;
+typedef struct JsonManifestParseContext JsonManifestParseContext;
+
+typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
+ char *pathname,
+ size_t size, pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
+ char *fmt, ...);
+
+struct JsonManifestParseContext
+{
+ void *private_data;
+ json_manifest_perfile_callback perfile_cb;
+ json_manifest_error_callback error_cb;
+};
+
+extern void json_parse_manifest(JsonManifestParseContext *context,
+ char *buffer, size_t size);
+
+#endif
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..e75e74c8ad
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,719 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+#include "parse_manifest.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+
+static void record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length,
+ uint8 *checksum_payload);
+static void report_manifest_error(JsonManifestParseContext *context,
+ char *fmt, ...);
+
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void pg_validator_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void pg_validator_fatal(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ pg_log_info("backup successfully verified");
+
+ exit(context.saw_any_error ? 1 : 0);
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ int rc;
+ JsonManifestParseContext context;
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ pg_validator_fatal("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ pg_validator_fatal("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /*
+ * Slurp in the whole file.
+ *
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
+ */
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
+ {
+ if (rc < 0)
+ pg_validator_fatal("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ pg_validator_fatal("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
+
+ /* Close the manifest file. */
+ close(fd);
+
+ /* Parse the manifest as JSON. */
+ context.private_data = ht;
+ context.perfile_cb = record_manifest_details_for_file;
+ context.error_cb = report_manifest_error;
+ json_parse_manifest(&context, buffer, statbuf.st_size);
+
+ /* Done with the buffer. */
+ pfree(buffer);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+static void
+report_manifest_error(JsonManifestParseContext *context, char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_validator_fatal(fmt, ap);
+ va_end(ap);
+}
+
+static void
+record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload)
+{
+ manifestfiles_hash *ht = context->private_data;
+ manifestfile *tabent;
+ bool found;
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(ht, pathname, &found);
+ if (found)
+ pg_validator_fatal("duplicate pathname in backup manifest: \"%s\"",
+ pathname);
+
+ /* Initialize the entry. */
+ tabent->size = size;
+ tabent->checksum_type = checksum_type;
+ tabent->checksum_length = checksum_length;
+ tabent->checksum_payload = checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ pg_validator_fatal("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ pg_validator_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ pg_validator_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ pg_validator_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ pg_validator_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ pg_validator_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ pg_validator_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ pg_validator_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ pg_validator_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ pg_validator_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ pg_validator_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ pg_validator_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ pg_validator_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <[email protected]>.\n"));
+}
+
+/*
+ * Report an error. Update the context to indicate that we saw an error, and
+ * exit if the context says we should.
+ */
+static void
+pg_validator_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+pg_validator_fatal(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
--
2.17.2 (Apple Git-113)
[application/octet-stream] v10-0002-Generate-backup-manifests-for-base-backups.patch (33.4K, ../../CA+TgmoYUVyp_ydjAtgm+3b4X0bu26FvyZWe8e8gM6GGrpbYbsQ@mail.gmail.com/5-v10-0002-Generate-backup-manifests-for-base-backups.patch)
download | inline diff:
From 7e63c0ed8ffeb669759fa760012a511c2a5e647f Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 9 Mar 2020 17:59:38 -0400
Subject: [PATCH v10 2/4] Generate backup manifests for base backups.
A manifest is a JSON document which includes the file name, size, last
modification time, and a checksum for each file backed up, as well as
a checksum for the manifest itself. By default, we use CRC-32C for the
checksum algorithm, because we are trying to detect corruption and
user error, not foil an adversary. However, pg_basebackup and the
server-side BASE_BACKUP command now have options to select the
checksum algorithm, so users wanting a cryptographic hash function can
select SHA-224, SHA-256, SHA-384, or SHA-512; and users not wanting
any checksums at all can disable them. Using a cryptographic hash
function in place of CRC-32C consumes significantly more CPU cycles,
which may slow down backups in some cases.
Robert Haas with help from Rushabh Lathia and Suraj Kharage.
---
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 363 +++++++++++++++++++++++--
src/backend/replication/repl_gram.y | 6 +
src/backend/replication/repl_scanner.l | 1 +
src/backend/replication/walsender.c | 30 ++
src/bin/pg_basebackup/pg_basebackup.c | 130 ++++++++-
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
8 files changed, 513 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index b0e953f894..3f9908427f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10551,7 +10551,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index f66cbc2428..77f15f6233 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -39,8 +41,10 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
typedef struct
@@ -52,20 +56,40 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ pg_checksum_type checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ bool first_file;
+ bool still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest, pg_checksum_type);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +126,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -251,6 +285,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -258,12 +293,17 @@ perform_base_backup(basebackup_options *opt)
backup_streamed = 0;
pgstat_progress_start_command(PROGRESS_COMMAND_BASEBACKUP, InvalidOid);
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt->checksum_type);
total_checksum_failures = 0;
@@ -301,7 +341,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -380,7 +423,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -388,11 +432,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -400,10 +447,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -632,7 +680,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -655,16 +703,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -678,6 +730,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -709,8 +764,11 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -797,6 +855,21 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval, &opt->checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
@@ -918,6 +991,228 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, pg_checksum_type checksum_type)
+{
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->first_file = true;
+ manifest->still_checksumming = true;
+
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ int pathlen;
+ StringInfoData buf;
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
+
+ /*
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
+ */
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
+
+ /* Add checksum information. */
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
+ }
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
+
+ /* Avoid leaking memory. */
+ pfree(buf.data);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\"}\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -978,11 +1273,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1017,6 +1316,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1027,7 +1330,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1060,7 +1364,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1079,7 +1384,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1359,7 +1664,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1367,7 +1673,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1437,8 +1744,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1455,6 +1763,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1625,6 +1936,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1649,6 +1963,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1657,7 +1972,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1682,6 +1998,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..0621884ad8 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,7 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -214,6 +215,11 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..5653d233b5 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,7 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index ae4a9cbe11..cc0b97627c 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index 48bd838803..235416a7c2 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -135,6 +141,7 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -180,6 +187,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -386,6 +399,8 @@ usage(void)
printf(_(" --no-slot prevent creation of temporary replication slot\n"));
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1184,6 +1199,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1655,6 +1695,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1665,6 +1763,7 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_checksums_clause = NULL;
int i;
char xlogstart[64];
char xlogend[64];
@@ -1672,6 +1771,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1725,6 +1825,9 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1739,7 +1842,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s",
escaped_label,
showprogress ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1747,7 +1850,8 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_checksums_clause ? manifest_checksums_clause : "");
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1835,7 +1939,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1864,6 +1969,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && serverMajor >= 1300)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2066,6 +2184,7 @@ main(int argc, char **argv)
{"waldir", required_argument, NULL, 1},
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
+ {"manifest-checksums", required_argument, NULL, 'm'},
{NULL, 0, NULL, 0}
};
int c;
@@ -2093,7 +2212,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2234,6 +2353,9 @@ main(int argc, char **argv)
case 3:
verify_checksums = false;
break;
+ case 'm':
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-09 16:22 ` tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-09 16:22 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/5/20 10:25 PM, Robert Haas wrote:
> Here's an updated patch set responding to many of the comments
> received thus far.
Thanks Robert. There is a scenario - if user provide port of v11 server
at the time of creating 'base backup' against pg_basebackup(v13+ your
patch applied)
with option --manifest-checksums,will lead to this below error
[centos@tushar-ldap-docker bin]$ ./pg_basebackup -R -p 9045
--manifest-checksums=SHA224 -D dc1
pg_basebackup: error: could not initiate base backup: ERROR: syntax error
pg_basebackup: removing data directory "dc1"
[centos@tushar-ldap-docker bin]$
Steps to reproduce -
PG v11 is running
run pg_basebackup against that with option --manifest-checksums
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-09 17:16 ` Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-09 17:16 UTC (permalink / raw)
To: tushar <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 9, 2020 at 12:22 PM tushar <[email protected]> wrote:
> On 3/5/20 10:25 PM, Robert Haas wrote:
> > Here's an updated patch set responding to many of the comments
> > received thus far.
> Thanks Robert. There is a scenario - if user provide port of v11 server
> at the time of creating 'base backup' against pg_basebackup(v13+ your
> patch applied)
> with option --manifest-checksums,will lead to this below error
>
> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -R -p 9045
> --manifest-checksums=SHA224 -D dc1
> pg_basebackup: error: could not initiate base backup: ERROR: syntax error
> pg_basebackup: removing data directory "dc1"
> [centos@tushar-ldap-docker bin]$
>
> Steps to reproduce -
> PG v11 is running
> run pg_basebackup against that with option --manifest-checksums
Seems like expected behavior to me. We could consider providing a more
descriptive error message, but there's now way for it to work.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-12 14:46 ` tushar <[email protected]>
2020-03-13 13:53 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: tushar @ 2020-03-12 14:46 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/9/20 10:46 PM, Robert Haas wrote:
> Seems like expected behavior to me. We could consider providing a more
> descriptive error message, but there's now way for it to work.
Right , Error message need to be more user friendly .
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-13 13:53 ` tushar <[email protected]>
2020-03-13 16:54 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: tushar @ 2020-03-13 13:53 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/12/20 8:16 PM, tushar wrote:
>> Seems like expected behavior to me. We could consider providing a more
>> descriptive error message, but there's now way for it to work.
>
> Right , Error message need to be more user friendly .
One scenario which i feel - should error out even if -s option is
specified.
create base backup directory ( ./pg_basebackup data1)
Connect to root user and take out the permission from pg_hba.conf file
( chmod 004 pg_hba.conf)
run pg_validatebackup -
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data1
pg_validatebackup: error: could not open file "pg_hba.conf": Permission
denied
run pg_validatebackup with switch -s
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup data1 -s
pg_validatebackup: backup successfully verified
here file is not accessible so i think - it should throw you an error (
the same above one) instead of blindly skipping it.
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 13:53 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-13 16:54 ` Robert Haas <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-13 16:54 UTC (permalink / raw)
To: tushar <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 13, 2020 at 9:53 AM tushar <[email protected]> wrote:
> run pg_validatebackup -
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data1
> pg_validatebackup: error: could not open file "pg_hba.conf": Permission denied
>
> run pg_validatebackup with switch -s
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data1 -s
> pg_validatebackup: backup successfully verified
>
> here file is not accessible so i think - it should throw you an error ( the same above one) instead of blindly skipping it.
I don't really want to do that. That would require it to open every
file even if it doesn't need to read the data in the files. I think in
most cases that would just slow it down for no real benefit. If you've
specified -s, you have to be OK with getting a less complete check for
problems.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-13 20:34 ` Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 10:22 ` Re: backup manifests tushar <[email protected]>
1 sibling, 2 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-13 20:34 UTC (permalink / raw)
To: tushar <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Thu, Mar 12, 2020 at 10:47 AM tushar <[email protected]> wrote:
> On 3/9/20 10:46 PM, Robert Haas wrote:
> > Seems like expected behavior to me. We could consider providing a more
> > descriptive error message, but there's now way for it to work.
>
> Right , Error message need to be more user friendly .
OK. Done in the attached version, which also includes a few other changes:
- I expanded the regression tests. They now cover every line of code
in parse_manifest.c except for a few that I believe to be unreachable
(though I might be mistaken). Coverage for pg_validatebackup.c is also
improved, but it's not 100%; there are some cases that I don't know
how to hit outside of a kernel malfunction, and others that I only
know how to hit on non-Windows systems. For instance, it's easy to use
perl to make a file inaccessible on Linux with chmod(0, $filename),
but I gather that doesn't work on Windows. I'm going to spend a bit
more time looking at this, but I think it's already reasonably good.
- I fixed a couple of very minor bugs which I discovered by writing those tests.
- I added documentation, in part based on a draft Mark Dilger shared
with me off-list.
I don't think this is committable just yet, but I think it's getting
fairly close, so if anyone has major objections please speak up soon.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v11-0001-Add-checksum-helper-functions.patch (9.8K, ../../CA+TgmoYguaj65+cq=HPcZAXwu7PQJz6PAUe7Qg9PaKY_g6ca9w@mail.gmail.com/2-v11-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From a8267f9374fe43ef4d122fb8f2be3eb1c32e3fdf Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 5 Feb 2020 13:59:45 -0500
Subject: [PATCH v11 1/5] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
(In my version, I decided to support all of the SHA variants for
which we have code, added a function to translate values of the
enum type back to strings so that users of this code don't need
to do that, used different naming, and tried to be more careful
amount formatting and comments.)
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index ce01df68b9..e199ed7acb 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -47,6 +47,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index f89a8a4fdb..ee04fbafa6 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -120,8 +120,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
- f2s.c file_perm.c hashfn.c ip.c jsonapi.c
+ base64.c checksum_helper.c config_info.c controldata_utils.c d2s.c
+ encnames.c exec.c f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c username.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v11-0004-Regression-tests-for-pg_validatebackup.patch (17.5K, ../../CA+TgmoYguaj65+cq=HPcZAXwu7PQJz6PAUe7Qg9PaKY_g6ca9w@mail.gmail.com/3-v11-0004-Regression-tests-for-pg_validatebackup.patch)
download | inline diff:
From ddceb29bd57b6782ef51166dde95e9d3f01e5092 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Tue, 10 Mar 2020 13:44:46 -0400
Subject: [PATCH v11 4/5] Regression tests for pg_validatebackup.
Patch by me, based in part on ideas from Mark Dilger and
Rajkumar Raghuwanshi.
---
src/bin/pg_validatebackup/.gitignore | 1 +
src/bin/pg_validatebackup/Makefile | 6 +
src/bin/pg_validatebackup/t/001_basic.pl | 30 +++
src/bin/pg_validatebackup/t/002_algorithm.pl | 57 ++++++
src/bin/pg_validatebackup/t/003_corruption.pl | 180 ++++++++++++++++++
src/bin/pg_validatebackup/t/004_options.pl | 89 +++++++++
.../pg_validatebackup/t/005_bad_manifest.pl | 139 ++++++++++++++
7 files changed, 502 insertions(+)
create mode 100644 src/bin/pg_validatebackup/t/001_basic.pl
create mode 100644 src/bin/pg_validatebackup/t/002_algorithm.pl
create mode 100644 src/bin/pg_validatebackup/t/003_corruption.pl
create mode 100644 src/bin/pg_validatebackup/t/004_options.pl
create mode 100644 src/bin/pg_validatebackup/t/005_bad_manifest.pl
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
index 3ae1c1f03a..21e0a92429 100644
--- a/src/bin/pg_validatebackup/.gitignore
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -1 +1,2 @@
/pg_validatebackup
+/tmp_check/
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
index dde7eb3c02..04ef7d3051 100644
--- a/src/bin/pg_validatebackup/Makefile
+++ b/src/bin/pg_validatebackup/Makefile
@@ -31,3 +31,9 @@ uninstall:
clean distclean maintainer-clean:
rm -f pg_validatebackup$(X) $(OBJS)
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_validatebackup/t/001_basic.pl b/src/bin/pg_validatebackup/t/001_basic.pl
new file mode 100644
index 0000000000..6d4b8ea01a
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/001_basic.pl
@@ -0,0 +1,30 @@
+use strict;
+use warnings;
+use TestLib;
+use Test::More tests => 16;
+
+my $tempdir = TestLib::tempdir;
+
+program_help_ok('pg_validatebackup');
+program_version_ok('pg_validatebackup');
+program_options_handling_ok('pg_validatebackup');
+
+command_fails_like(['pg_validatebackup'],
+ qr/no backup directory specified/,
+ 'target directory must be specified');
+command_fails_like(['pg_validatebackup', $tempdir],
+ qr/could not open file.*\/backup_manifest\"/,
+ 'pg_validatebackup requires a manifest');
+command_fails_like(['pg_validatebackup', $tempdir, $tempdir],
+ qr/too many command-line arguments/,
+ 'multiple target directories not allowed');
+
+# create fake manifest file
+open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+close($fh);
+
+# but then try to use an alternate, nonexisting manifest
+command_fails_like(['pg_validatebackup', '-m', "$tempdir/not_the_manifest",
+ $tempdir],
+ qr/could not open file.*\/not_the_manifest\"/,
+ 'pg_validatebackup respects -m flag');
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
new file mode 100644
index 0000000000..e411923411
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -0,0 +1,57 @@
+# Verify that we can take and validate backups with various checksum types.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 19;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
+{
+ my $backup_path = $master->backup_dir . '/' . $algorithm;
+ my @backup = ('pg_basebackup', '-D', $backup_path, '-m', $algorithm,
+ '--no-sync');
+ my @validate = ('pg_validatebackup', '-e', $backup_path);
+
+ # A backup with a bogus algorithm should fail.
+ if ($algorithm eq 'bogus')
+ {
+ $master->command_fails(\@backup,
+ "backup fails with algorithm \"$algorithm\"");
+ next;
+ }
+
+ # A backup with a valid algorithm should work.
+ $master->command_ok(\@backup, "backup ok with algorithm \"$algorithm\"");
+
+ # We expect each real checksum algorithm to be mentioned on every line of
+ # the backup manifest file except the first and last; for simplicity, we
+ # just check that it shows up lots of times. When the checksum algorithm
+ # is none, we just check that the manifest exists.
+ if ($algorithm eq 'none')
+ {
+ ok(-f "$backup_path/backup_manifest", "backup manifest exists");
+ }
+ else
+ {
+ my $manifest = slurp_file("$backup_path/backup_manifest");
+ my $count_of_algorithm_in_manifest =
+ (() = $manifest =~ /$algorithm/mig);
+ cmp_ok($count_of_algorithm_in_manifest, '>', 100,
+ "$algorithm is mentioned many times in the manifest");
+ }
+
+ # Make sure that it validates OK.
+ $master->command_ok(\@validate,
+ "validate backup with algorithm \"$algorithm\"");
+
+ # Remove backup immediately to save disk space.
+ rmtree($backup_path);
+}
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
new file mode 100644
index 0000000000..e845a3fee9
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -0,0 +1,180 @@
+# Verify that various forms of corruption are detected by pg_validatebackup.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 32;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+# Include a user-defined tablespace in the hopes of detecting problems in that
+# area.
+my $source_ts_path = TestLib::tempdir;
+$master->safe_psql('postgres', <<EOM);
+CREATE TABLE x1 (a int);
+INSERT INTO x1 VALUES (111);
+CREATE TABLESPACE ts1 LOCATION '$source_ts_path';
+CREATE TABLE x2 (a int) TABLESPACE ts1;
+INSERT INTO x1 VALUES (222);
+EOM
+
+my @scenario = (
+ {
+ 'name' => 'extra_file',
+ 'mutilate' => \&mutilate_extra_file,
+ 'fails_like' =>
+ qr/extra_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'extra_tablespace_file',
+ 'mutilate' => \&mutilate_extra_tablespace_file,
+ 'fails_like' =>
+ qr/extra_ts_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'missing_file',
+ 'mutilate' => \&mutilate_missing_file,
+ 'fails_like' =>
+ qr/pg_xact\/0000.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'missing_tablespace',
+ 'mutilate' => \&mutilate_missing_tablespace,
+ 'fails_like' =>
+ qr/pg_tblspc.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'append_to_file',
+ 'mutilate' => \&mutilate_append_to_file,
+ 'fails_like' =>
+ qr/has size \d+ on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'truncate_file',
+ 'mutilate' => \&mutilate_truncate_file,
+ 'fails_like' =>
+ qr/has size 0 on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'replace_file',
+ 'mutilate' => \&mutilate_replace_file,
+ 'fails_like' => qr/checksum mismatch for file/
+ },
+ {
+ 'name' => 'bad_manifest',
+ 'mutilate' => \&mutilate_bad_manifest,
+ 'fails_like' => qr/manifest checksum mismatch/
+ }
+);
+
+for my $scenario (@scenario)
+{
+ my $name = $scenario->{'name'};
+
+ # Take a backup and check that it validates OK.
+ my $backup_path = $master->backup_dir . '/' . $name;
+ my $backup_ts_path = TestLib::tempdir;
+ $master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '-T', "${source_ts_path}=${backup_ts_path}"],
+ "base backup ok");
+ command_ok(['pg_validatebackup', $backup_path ],
+ "intact backup validated");
+
+ # Mutilate the backup in some way.
+ $scenario->{'mutilate'}->($backup_path);
+
+ # Now check that the backup no longer validates.
+ command_fails_like(['pg_validatebackup', $backup_path ],
+ $scenario->{'fails_like'},
+ "corrupt backup fails validation: $name");
+}
+
+sub create_extra_file
+{
+ my ($backup_path, $relative_path) = @_;
+ my $pathname = "$backup_path/$relative_path";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh "This is an extra file.\n";
+ close($fh);
+}
+
+# Add a file into the root directory of the backup.
+sub mutilate_extra_file
+{
+ my ($backup_path) = @_;
+ create_extra_file($backup_path, "extra_file");
+}
+
+# Add a file inside the user-defined tablespace.
+sub mutilate_extra_tablespace_file
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my ($catvdir) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid");
+ my ($tsdboid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid/$catvdir");
+ create_extra_file($backup_path,
+ "pg_tblspc/$tsoid/$catvdir/$tsdboid/extra_ts_file");
+}
+
+# Remove a file.
+sub mutilate_missing_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_xact/0000";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Remove the symlink to the user-defined tablespace.
+sub mutilate_missing_tablespace
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my $pathname = "$backup_path/pg_tblspc/$tsoid";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Append an additional bytes to a file.
+sub mutilate_append_to_file
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/global/pg_control", 'x';
+}
+
+# Truncate a file to zero length.
+sub mutilate_truncate_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/global/pg_control";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ close($fh);
+}
+
+# Replace a file's contents without changing the length of the file. This is
+# not a particularly efficient way to do this, so we pick a file that's
+# expected to be short.
+sub mutilate_replace_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ my $contents = slurp_file($pathname);
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh 'q' x length($contents);
+ close($fh);
+}
+
+# Corrupt the backup manifest.
+sub mutilate_bad_manifest
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/backup_manifest", "\n";
+}
diff --git a/src/bin/pg_validatebackup/t/004_options.pl b/src/bin/pg_validatebackup/t/004_options.pl
new file mode 100644
index 0000000000..8f185626ed
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/004_options.pl
@@ -0,0 +1,89 @@
+# Verify the behavior of assorted pg_validatebackup options.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 25;
+
+# Start up the server and take a backup.
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_options';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync' ],
+ "base backup ok");
+
+# Verify that pg_validatebackup -q succeeds and produces no output.
+my $stdout;
+my $stderr;
+my $result = IPC::Run::run ['pg_validatebackup', '-q', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok($result, "-q succeeds: exit code 0");
+is($stdout, '', "-q succeeds: no stdout");
+is($stderr, '', "-q succeeds: no stderr");
+
+# Corrupt the PG_VERSION file.
+my $version_pathname = "$backup_path/PG_VERSION";
+my $version_contents = slurp_file($version_pathname);
+open(my $fh, '>', $version_pathname) || die "open $version_pathname: $!";
+print $fh 'q' x length($version_contents);
+close($fh);
+
+# Verify that pg_validatebackup -q now fails.
+command_fails_like(['pg_validatebackup', '-q', $backup_path ],
+ qr/checksum mismatch for file \"PG_VERSION\"/,
+ '-q checksum mismatch');
+
+# Since we didn't change the length of the file, validation should succeed
+# if we ignore checksums. Check that we get the right message, too.
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ '-s skips checksumming');
+
+# Validation should succeed if we ignore the problem file.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/backup successfully verified/,
+ '-i ignores problem file');
+
+# PG_VERSION is already corrupt; let's try also removing all of pg_xact.
+rmtree($backup_path . "/pg_xact");
+
+# We're ignoring the problem with PG_VERSION, but not the problem with
+# pg_xact, so validation should fail here.
+command_fails_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/pg_xact.*is present in the manifest but not on disk/,
+ '-i does not ignore all problems');
+
+# If we use -i twice, we should be able to ignore all of the problems.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', '-i', 'pg_xact',
+ $backup_path ],
+ qr/backup successfully verified/,
+ 'multiple -i options work');
+
+# Verify that when -i is not used, both problems are reported.
+$result = IPC::Run::run ['pg_validatebackup', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "multiple problems: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "multiple problems: missing files reported");
+like($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "multiple problems: checksum mismatch reported");
+
+# Verify that when -e is used, only the problem detected first is reported.
+$result = IPC::Run::run ['pg_validatebackup', '-e', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "-e reports 1 error: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "-e reports 1 error: missing files reported");
+unlike($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "-e reports 1 error: checksum mismatch not reported");
+
+# Test valid manifest with nonexistent backup directory.
+command_fails_like(['pg_validatebackup', '-m', "$backup_path/backup_manifest",
+ "$backup_path/fake" ],
+ qr/could not open directory/,
+ 'nonexistent backup directory');
diff --git a/src/bin/pg_validatebackup/t/005_bad_manifest.pl b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
new file mode 100644
index 0000000000..8449c527d3
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
@@ -0,0 +1,139 @@
+# Test the behavior of pg_validatebackup when the backup manifest has
+# problems.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 38;
+
+my $tempdir = TestLib::tempdir;
+
+test_bad_manifest('input string ended unexpctedly',
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
+{
+EOM
+
+test_parse_error('unexpected object end', <<EOM);
+{}
+EOM
+
+test_parse_error('unexpected array start', <<EOM);
+[]
+EOM
+
+test_parse_error('expected version indicator', <<EOM);
+{"not-expected": 1}
+EOM
+
+test_parse_error('unexpected manifest version', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": "phooey"}
+EOM
+
+test_parse_error('unexpected scalar', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": true}
+EOM
+
+test_parse_error('expected file list', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Oops": 1}
+EOM
+
+test_parse_error('unexpected object start', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": {}}
+EOM
+
+test_parse_error('missing pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [{}]}
+EOM
+
+test_parse_error('unexpected file field', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Oops": 1}
+]}
+EOM
+
+test_parse_error('missing size', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x"}
+]}
+EOM
+
+test_parse_error('file size is not an integer', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": "Oops"}
+]}
+EOM
+
+test_parse_error('checksum without algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum": "Oops"}
+]}
+EOM
+
+test_fatal_error('unrecognized checksum algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "Oops", "Checksum": "00"}
+]}
+EOM
+
+test_fatal_error('invalid checksum for file', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "CRC32C", "Checksum": "0"}
+]}
+EOM
+
+test_parse_error('expected manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Oops": 1}
+EOM
+
+test_parse_error('expected at least 2 lines', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [], "Manifest-Checksum": null}
+EOM
+
+my $manifest_without_newline = <<EOM;
+{"PostgreSQL-Backup-Manifest-Version": 1,
+ "Files": [],
+ "Manifest-Checksum": null}
+EOM
+chomp($manifest_without_newline);
+test_parse_error('last line not newline-terminated',
+ $manifest_without_newline);
+
+test_fatal_error('invalid manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Manifest-Checksum": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890-"}
+EOM
+
+sub test_parse_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/could not parse backup manifest: $test_name/,
+ $manifest_contents);
+}
+
+sub test_fatal_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/fatal: $test_name/,
+ $manifest_contents);
+}
+
+sub test_bad_manifest
+{
+ my ($test_name, $regexp, $manifest_contents) = @_;
+
+ open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+ print $fh $manifest_contents;
+ close($fh);
+
+ command_fails_like(['pg_validatebackup', $tempdir], $regexp,
+ $test_name);
+}
--
2.17.2 (Apple Git-113)
[application/octet-stream] v11-0003-pg_validatebackup-Validate-a-backup-against-the-.patch (41.2K, ../../CA+TgmoYguaj65+cq=HPcZAXwu7PQJz6PAUe7Qg9PaKY_g6ca9w@mail.gmail.com/4-v11-0003-pg_validatebackup-Validate-a-backup-against-the-.patch)
download | inline diff:
From 30952505a3428a161f94d720cd08cae9d06afbde Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 9 Mar 2020 15:23:24 -0400
Subject: [PATCH v11 3/5] pg_validatebackup: Validate a backup against the
backup manifest.
Patch by me; review by Tushar Ahuja and Rajkumar Raghuwanshi, and also
off-list by Mark Dilger, Davinder Singh, and Jeevan Chalke.
---
src/bin/Makefile | 1 +
src/bin/pg_validatebackup/.gitignore | 1 +
src/bin/pg_validatebackup/Makefile | 33 +
src/bin/pg_validatebackup/parse_manifest.c | 547 +++++++++++++
src/bin/pg_validatebackup/parse_manifest.h | 40 +
src/bin/pg_validatebackup/pg_validatebackup.c | 732 ++++++++++++++++++
6 files changed, 1354 insertions(+)
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/parse_manifest.c
create mode 100644 src/bin/pg_validatebackup/parse_manifest.h
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..3ae1c1f03a
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1 @@
+/pg_validatebackup
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..dde7eb3c02
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,33 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ parse_manifest.o \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
new file mode 100644
index 0000000000..b8ee889c50
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -0,0 +1,547 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.c
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "parse_manifest.h"
+#include "common/jsonapi.h"
+
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+/*
+ * Internal state used while decoding the JSON-format backup manifest.
+ */
+typedef struct
+{
+ JsonManifestParseContext *context;
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ char *pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void verify_manifest_checksum(JsonManifestParseState *parse,
+ char *buffer, size_t size);
+static void json_manifest_parse_failure(JsonManifestParseContext *context,
+ char *msg);
+
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+
+/*
+ * Main entrypoint to parse a JSON-format backup manifest.
+ *
+ * Caller should set up the parsing context and then invoke this function.
+ * For each file whose information is extracted from the manifest,
+ * context->perfile_cb is invoked. In case of trouble, context->error_cb is
+ * invoked and is expected not to return.
+ */
+void
+json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ size_t size)
+{
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
+
+ /* Set up our private parsing context. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.context = context;
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+ /* Validate the checksum. */
+ verify_manifest_checksum(&parse, buffer, size);
+}
+
+/*
+ * Invoked at the start of each object in the JSON document.
+ *
+ * The document as a whole is expected to be an object with three keys
+ * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
+ * file is expected to be an object with various keys (Path, Size, etc.).
+ * If we're not at the beginning of either the toplevel object or the object
+ * for a particular file, it's an error.
+ */
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->size = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each object in the JSON document.
+ *
+ * The possible cases here are the same as for json_manifest_object_start.
+ * There's nothing special to do at the end of the document, but when we
+ * reach the end of an object representing a particular file, we must call
+ * json_manifest_finalize_file() to save the associated details.
+ */
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ json_manifest_finalize_file(parse);
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each array in the JSON document.
+ *
+ * Within the toplevel object, the value associated with the "Files" key
+ * should be an array. No other arrays are expected.
+ */
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each array in the JSON document.
+ *
+ * Just like json_manifest_array_start, there's only one expected case
+ * here.
+ */
+static void
+json_manifest_array_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each object field in the JSON document.
+ */
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_FIELD:
+ /* Inside toplevel object, expecting version indicator. */
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ /* Inside toplevel object, expecting "Files" next. */
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure(parse->context,
+ "unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ /* Inside toplevel object, expecting "Manifest-Checksum" next. */
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object field");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each scalar in the JSON document.
+ *
+ * Object field names don't reach this code; those are handled by
+ * json_manifest_object_field_start. When we're inside of the object for
+ * a particular file, that function will have noticed the name of the field,
+ * and we'll get the corresponding value here. When we're in the toplevel
+ * object, the parse state itself tells us which field this is.
+ *
+ * In all cases except for PostgreSQL-Backup-Manifest-Version, which we
+ * can just check on the spot, the goal here is just to save the value in
+ * the parse state for later use. We don't actually do anything until we
+ * reach either the end of the object representing this file, or the end
+ * of the manifest, as the case may be.
+ */
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure(parse->context,
+ "unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context, "unexpected scalar");
+ break;
+ }
+}
+
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * file, and invoke the per-file callback so that the caller gets those
+ * details. This happens for each file when the corresponding JSON object is
+ * completely parsed.
+ */
+static void
+json_manifest_finalize_file(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t size;
+ char *ep;
+ int checksum_string_length;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL)
+ json_manifest_parse_failure(parse->context, "missing pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure(parse->context, "missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure(parse->context,
+ "checksum without algorithm");
+
+ /* Parse size. */
+ size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm, &checksum_type))
+ context->error_cb(context, "unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ checksum_length = 0;
+ checksum_payload = NULL;
+ }
+ else
+ {
+ checksum_length = checksum_string_length / 2;
+ checksum_payload = palloc(checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(checksum_payload, parse->checksum,
+ checksum_length))
+ context->error_cb(context,
+ "invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, parse->checksum);
+ }
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perfile_cb(context, parse->pathname, size,
+ checksum_type, checksum_length, checksum_payload);
+
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
+}
+
+/*
+ * Verify that the manifest checksum is correct.
+ *
+ * The last line of the manifest file is excluded from the manifest checksum,
+ * because the last line is expected to contain the checksum that covers
+ * the rest of the file.
+ */
+static void
+verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
+ size_t size)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t i;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+
+ /* Find the last two newlines in the file. */
+ for (i = 0; i < size; ++i)
+ {
+ if (buffer[i] == '\n')
+ {
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
+ }
+ }
+
+ /*
+ * Make sure that the last newline is right at the end, and that there are
+ * at least two lines total. We need this to be true in order for the
+ * following code, which computes the manifest checksum, to work properly.
+ */
+ if (number_of_newlines < 2)
+ json_manifest_parse_failure(parse->context,
+ "expected at least 2 lines");
+ if (ultimate_newline != size - 1)
+ json_manifest_parse_failure(parse->context,
+ "last line not newline-terminated");
+
+ /* Checksum the rest. */
+ pg_sha256_init(&manifest_ctx);
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+
+ /* Now verify it. */
+ if (parse->manifest_checksum == NULL)
+ context->error_cb(parse->context, "manifest has no checksum");
+ if (strlen(parse->manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse->manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ context->error_cb(context, "invalid manifest checksum: \"%s\"",
+ parse->manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ context->error_cb(context, "manifest checksum mismatch");
+}
+
+/*
+ * Report a parse error.
+ *
+ * This is intended to be used for fairly low-level failures that probably
+ * shouldn't occur unless somebody has deliberately constructed a bad manifest,
+ * or unless the server is generating bad manifests due to some bug. msg should
+ * be a short string giving some hint as to what the problem is.
+ */
+static void
+json_manifest_parse_failure(JsonManifestParseContext *context, char *msg)
+{
+ context->error_cb(context, "could not parse backup manifest: %s", msg);
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
new file mode 100644
index 0000000000..b0b18a57ca
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -0,0 +1,40 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.h
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PARSE_MANIFEST_H
+#define PARSE_MANIFEST_H
+
+#include "common/checksum_helper.h"
+#include "mb/pg_wchar.h"
+
+struct JsonManifestParseContext;
+typedef struct JsonManifestParseContext JsonManifestParseContext;
+
+typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
+ char *pathname,
+ size_t size, pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
+ char *fmt, ...);
+
+struct JsonManifestParseContext
+{
+ void *private_data;
+ json_manifest_perfile_callback perfile_cb;
+ json_manifest_error_callback error_cb;
+};
+
+extern void json_parse_manifest(JsonManifestParseContext *context,
+ char *buffer, size_t size);
+
+#endif
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..0e7299b1b9
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,732 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+#include "parse_manifest.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+
+static void record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length,
+ uint8 *checksum_payload);
+static void report_manifest_error(JsonManifestParseContext *context,
+ char *fmt, ...);
+
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void report_backup_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void report_fatal_error(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ printf("backup successfully verified\n");
+
+ return context.saw_any_error ? 1 : 0;
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ int rc;
+ JsonManifestParseContext context;
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ report_fatal_error("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ report_fatal_error("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /*
+ * Slurp in the whole file.
+ *
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
+ */
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
+ {
+ if (rc < 0)
+ report_fatal_error("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ report_fatal_error("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
+
+ /* Close the manifest file. */
+ close(fd);
+
+ /* Parse the manifest as JSON. */
+ context.private_data = ht;
+ context.perfile_cb = record_manifest_details_for_file;
+ context.error_cb = report_manifest_error;
+ json_parse_manifest(&context, buffer, statbuf.st_size);
+
+ /* Done with the buffer. */
+ pfree(buffer);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * Report an error while parsing the manifest.
+ *
+ * We consider all such errors to be fatal errors. The manifest parser
+ * expects this function not to return.
+ */
+static void
+report_manifest_error(JsonManifestParseContext *context, char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Record details extracted from the backup manifest for one file.
+ */
+static void
+record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload)
+{
+ manifestfiles_hash *ht = context->private_data;
+ manifestfile *tabent;
+ bool found;
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(ht, pathname, &found);
+ if (found)
+ report_fatal_error("duplicate pathname in backup manifest: \"%s\"",
+ pathname);
+
+ /* Initialize the entry. */
+ tabent->size = size;
+ tabent->checksum_type = checksum_type;
+ tabent->checksum_length = checksum_length;
+ tabent->checksum_payload = checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ report_fatal_error("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ report_backup_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ report_backup_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ report_backup_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ report_backup_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ report_backup_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ report_backup_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ report_backup_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ report_backup_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ report_backup_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ report_backup_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ report_backup_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ report_backup_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Report a problem with the backup.
+ *
+ * Update the context to indicate that we saw an error, and exit if the
+ * context says we should.
+ */
+static void
+report_backup_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+report_fatal_error(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <[email protected]>.\n"));
+}
--
2.17.2 (Apple Git-113)
[application/octet-stream] v11-0005-Documentation.patch (12.0K, ../../CA+TgmoYguaj65+cq=HPcZAXwu7PQJz6PAUe7Qg9PaKY_g6ca9w@mail.gmail.com/5-v11-0005-Documentation.patch)
download | inline diff:
From 77ee6586cb58bdcf3acf198baf35086fc0c4e62f Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 13 Mar 2020 11:54:58 -0400
Subject: [PATCH v11 5/5] Documentation.
- Add documentation for pg_validatebackup.
- Adjust documentation for pg_basebackup.
Robert Haas and Mark Dilger
---
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_basebackup.sgml | 40 ++++
doc/src/sgml/ref/pg_validatebackup.sgml | 232 ++++++++++++++++++++++++
doc/src/sgml/reference.sgml | 1 +
4 files changed, 274 insertions(+)
create mode 100644 doc/src/sgml/ref/pg_validatebackup.sgml
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..ab71176cdf 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -211,6 +211,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
<!ENTITY pgRestore SYSTEM "pg_restore.sgml">
<!ENTITY pgRewind SYSTEM "pg_rewind.sgml">
+<!ENTITY pgValidateBackup SYSTEM "pg_validatebackup.sgml">
<!ENTITY pgtestfsync SYSTEM "pgtestfsync.sgml">
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index 29bf2f9b97..f6175dd968 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -552,6 +552,46 @@ PostgreSQL documentation
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be used to checksum each file
+ for purposes of the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ <para>
+ If <literal>NONE</literal> is selected, the backup manifest will
+ not contain any checksums. Otherwise, it will contain a checksum
+ of each file in the backup using the specified algorithm. In addition,
+ the manifest itself will always contain a <literal>SHA256</literal>
+ checksum of its own contents. The <literal>SHA</literal> algorithms
+ are significantly more CPU-intensive than <literal>CRC32C</literal>,
+ so selecting one of them may increase the time required to complete
+ the backup.
+ </para>
+ <para>
+ On the other hand, <literal>CRC32C</literal> is not a cryptographic
+ hash function, so it is only suitable for protecting against
+ inadvertent or random modifications to a backup. An adversary
+ who can modify the backup could easily do so in such a way that
+ the CRC does not change, whereas a SHA collision will be hard
+ to manufacture. (However, note that if the attacker also has access
+ to modify the backup manifest itself, no checksum algorithm will
+ provide any protection.) An additional advantage of the
+ <literal>SHA</literal> family of functions is that they output
+ a much larger number of bits.
+ </para>
+ <para>
+ <xref linkend="app-pgvalidatebackup" /> can be used to check the
+ integrity of a backup against the backup manifest.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
new file mode 100644
index 0000000000..678a14a2f1
--- /dev/null
+++ b/doc/src/sgml/ref/pg_validatebackup.sgml
@@ -0,0 +1,232 @@
+<!--
+doc/src/sgml/ref/pg_validatebackup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgvalidatebackup">
+ <indexterm zone="app-pgvalidatebackup">
+ <primary>pg_validatebackup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>pg_validatebackup</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_validatebackup</refname>
+ <refpurpose>verify the integrity of a base backup of a
+ <productname>PostgreSQL</productname> cluster</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_validatebackup</command>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>
+ Description
+ </title>
+ <para>
+ <application>pg_validatebackup</application> is used to check the integrity
+ of a database cluster backup. The backup being checked should have been
+ created by <command>pg_basebackup</command> or some other tool that includes
+ a <literal>backup_manifest</literal> file with the bacup. The backup
+ must be stored in the "plain" format; a "tar" format backup can be checked
+ after extracting it. Backup manifests are created by the server beginning
+ with <productname>PostgreSQL</productname> version 13, so older backups
+ cannot be validated using this tool.
+ </para>
+
+ <para>
+ <application>pg_validatebackup</application> reads the manifest file of a
+ backup, verifies the manifest against its own internal checksum, and then
+ verifies that the same files are present in the target directory as in the
+ manifest itself. It then verifies that each file has the expected checksum,
+ unless the backup was taken the checksum algorithm set to
+ <literal>none</literal>, in which case checksum verification is not
+ performed. The presence or absence of directories is not checked, except
+ indirectly: if a directory is missing, any files it should have contained
+ will necessarily also be missing. Certain files and directories are
+ excluded from verification:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>backup_manifest</literal> is ignored because the backup
+ manifest is logically not part of the backup and does not include
+ any entry for itself.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>pg_wal</literal> is ignored because WAL files are sent
+ separately from the backup, and are therefore not described by the
+ backup manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>postgesql.auto.conf</literal>,
+ <literal>standby.signal</literal>,
+ and <literal>recovery.signal</literal> are ignored because they may
+ sometimes be created or modified by the backup client itself.
+ (For example, <literal>pg_basebackup -R</literal> will modify
+ <literal>postgresql.auto.conf</literal> and create
+ <literal>standby.signal</literal>.)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ The following command-line options control the behavior.
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-e</option></term>
+ <term><option>--exit-on-error</option></term>
+ <listitem>
+ <para>
+ Exit as soon as a problem with the backup is detected. If this option
+ is not specified, <literal>pg_basebackup</literal> will continue
+ checking the backup even after a problem has been detected, and will
+ report all problems detected as errors.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-i <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--ignore=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Ignore the specified file or directory, which should be expressed
+ as a relative pathname. If the backup contains extra files, is
+ missing files, or has files that have been modified as compared with
+ what is described in the manifest, this option can be used to suppress
+ the errors that would otherwise occur. If a directory is specified,
+ this option affects the entire subtree rooted at that location.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-m <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--manifest-path=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Use the manifest file at the specified path, rather than one located
+ in the root of the backup directory.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-q</option></term>
+ <term><option>--quiet</option></term>
+ <listitem>
+ <para>
+ Don't print anything when a backup is successfully validated.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-s</option></term>
+ <term><option>--skip-checksums</option></term>
+ <listitem>
+ <para>
+ Do not validate checksums. The presence or absence of files and the
+ sizes of those files will still be checked. This is much faster,
+ because the files themselves do not need to read.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other options are also available:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_validatebackup</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_validatebackup</application> command
+ line arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal> and
+ validate the integrity of the backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal>, move
+ the manifest somewhere outside the backup directory, and validate the
+ backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234</userinput>
+<prompt>$</prompt> <userinput>mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234</userinput>
+</screen>
+ </para>
+
+ <para>
+ To validate a backup while ignoring a file that was added manually to the
+ backup directory, and also skipping checksum verification:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>edit /usr/local/pgsql/data/note.to.self</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="app-pgbasebackup"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..d25a77b13c 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -255,6 +255,7 @@
&pgReceivewal;
&pgRecvlogical;
&pgRestore;
+ &pgValidateBackup;
&psqlRef;
&reindexdb;
&vacuumdb;
--
2.17.2 (Apple Git-113)
[application/octet-stream] v11-0002-Generate-backup-manifests-for-base-backups.patch (33.8K, ../../CA+TgmoYguaj65+cq=HPcZAXwu7PQJz6PAUe7Qg9PaKY_g6ca9w@mail.gmail.com/6-v11-0002-Generate-backup-manifests-for-base-backups.patch)
download | inline diff:
From c8f8d516b3416bc2df03f69544061472f29071b6 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 9 Mar 2020 17:59:38 -0400
Subject: [PATCH v11 2/5] Generate backup manifests for base backups.
A manifest is a JSON document which includes the file name, size, last
modification time, and a checksum for each file backed up, as well as
a checksum for the manifest itself. By default, we use CRC-32C for the
checksum algorithm, because we are trying to detect corruption and
user error, not foil an adversary. However, pg_basebackup and the
server-side BASE_BACKUP command now have options to select the
checksum algorithm, so users wanting a cryptographic hash function can
select SHA-224, SHA-256, SHA-384, or SHA-512; and users not wanting
any checksums at all can disable them. Using a cryptographic hash
function in place of CRC-32C consumes significantly more CPU cycles,
which may slow down backups in some cases.
Robert Haas, with help, review, and testing from Rushabh Lathia,
Suraj Kharage, Tushar Ahuja, and Rajkumar Raghuwanshi.
---
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 363 +++++++++++++++++++++++--
src/backend/replication/repl_gram.y | 6 +
src/backend/replication/repl_scanner.l | 1 +
src/backend/replication/walsender.c | 30 ++
src/bin/pg_basebackup/pg_basebackup.c | 140 +++++++++-
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
8 files changed, 523 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 4fa446ffa4..10a6714843 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10551,7 +10551,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 806d013108..dd1ec1a688 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -39,8 +41,10 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
typedef struct
@@ -52,20 +56,40 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ pg_checksum_type checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ bool first_file;
+ bool still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest, pg_checksum_type);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +126,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -251,6 +285,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -258,12 +293,17 @@ perform_base_backup(basebackup_options *opt)
backup_streamed = 0;
pgstat_progress_start_command(PROGRESS_COMMAND_BASEBACKUP, InvalidOid);
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt->checksum_type);
total_checksum_failures = 0;
@@ -301,7 +341,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -380,7 +423,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -388,11 +432,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -400,10 +447,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -632,7 +680,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -655,16 +703,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -678,6 +730,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -709,8 +764,11 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -797,6 +855,21 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval, &opt->checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
@@ -918,6 +991,228 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, pg_checksum_type checksum_type)
+{
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->first_file = true;
+ manifest->still_checksumming = true;
+
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ int pathlen;
+ StringInfoData buf;
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
+
+ /*
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
+ */
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
+
+ /* Add checksum information. */
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
+ }
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
+
+ /* Avoid leaking memory. */
+ pfree(buf.data);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\"}\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -978,11 +1273,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1017,6 +1316,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1027,7 +1330,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1060,7 +1364,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1079,7 +1384,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1359,7 +1664,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1367,7 +1673,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1437,8 +1744,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1455,6 +1763,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1625,6 +1936,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1649,6 +1963,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1657,7 +1972,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1682,6 +1998,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..0621884ad8 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,7 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -214,6 +215,11 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..5653d233b5 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,7 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 3f74bc8493..88c73dc21c 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -316,6 +316,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -323,6 +325,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index 48bd838803..942e377bbc 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -135,6 +141,7 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -180,6 +187,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -386,6 +399,8 @@ usage(void)
printf(_(" --no-slot prevent creation of temporary replication slot\n"));
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1184,6 +1199,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1655,6 +1695,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1665,6 +1763,7 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_checksums_clause = NULL;
int i;
char xlogstart[64];
char xlogend[64];
@@ -1672,6 +1771,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1725,6 +1825,19 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest_checksums != NULL)
+ {
+ if (serverMajor < 1300)
+ {
+ const char *serverver = PQparameterStatus(conn, "server_version");
+
+ pg_log_error("manifest checksums are not supported by server version %s",
+ serverver ? serverver : "'unknown'");
+ exit(1);
+ }
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
+ }
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1739,7 +1852,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s",
escaped_label,
showprogress ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1747,7 +1860,8 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_checksums_clause ? manifest_checksums_clause : "");
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1835,7 +1949,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1864,6 +1979,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && serverMajor >= 1300)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2066,6 +2194,7 @@ main(int argc, char **argv)
{"waldir", required_argument, NULL, 1},
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
+ {"manifest-checksums", required_argument, NULL, 'm'},
{NULL, 0, NULL, 0}
};
int c;
@@ -2093,7 +2222,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2234,6 +2363,9 @@ main(int argc, char **argv)
case 3:
verify_checksums = false;
break;
+ case 'm':
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-16 05:07 ` Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Suraj Kharage @ 2020-03-16 05:07 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Thank you, Robert.
Getting below warning while compiling the
v11-0003-pg_validatebackup-Validate-a-backup-against-the-.patch.
*pg_validatebackup.c: In function
‘report_manifest_error’:pg_validatebackup.c:356:2: warning: function might
be possible candidate for ‘gnu_printf’ format attribute
[-Wsuggest-attribute=format] pg_log_generic_v(PG_LOG_FATAL, fmt, ap);*
To resolve this, can we use "pg_attribute_printf(2, 3)" in function
declaration something like below?
e.g:
diff --git a/src/bin/pg_validatebackup/parse_manifest.h
b/src/bin/pg_validatebackup/parse_manifest.h
index b0b18a5..25d140f 100644
--- a/src/bin/pg_validatebackup/parse_manifest.h
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -25,7 +25,7 @@ typedef void
(*json_manifest_perfile_callback)(JsonManifestParseContext *,
size_t
size, pg_checksum_type checksum_type,
int
checksum_length, uint8 *checksum_payload);
typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
- char *fmt,
...);
+ char
*fmt,...) pg_attribute_printf(2, 3);
struct JsonManifestParseContext
{
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c
b/src/bin/pg_validatebackup/pg_validatebackup.c
index 0e7299b..6ccbe59 100644
--- a/src/bin/pg_validatebackup/pg_validatebackup.c
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -95,7 +95,7 @@ static void
record_manifest_details_for_file(JsonManifestParseContext *context,
int checksum_length,
uint8 *checksum_payload);
static void report_manifest_error(JsonManifestParseContext *context,
- char
*fmt, ...);
+ char
*fmt,...) pg_attribute_printf(2, 3);
static void validate_backup_directory(validator_context *context,
char *relpath, char *fullpath);
Typos:
0004 patch
unexpctedly => unexpectedly
0005 patch
bacup => backup
On Sat, Mar 14, 2020 at 2:04 AM Robert Haas <[email protected]> wrote:
> On Thu, Mar 12, 2020 at 10:47 AM tushar <[email protected]>
> wrote:
> > On 3/9/20 10:46 PM, Robert Haas wrote:
> > > Seems like expected behavior to me. We could consider providing a more
> > > descriptive error message, but there's now way for it to work.
> >
> > Right , Error message need to be more user friendly .
>
> OK. Done in the attached version, which also includes a few other changes:
>
> - I expanded the regression tests. They now cover every line of code
> in parse_manifest.c except for a few that I believe to be unreachable
> (though I might be mistaken). Coverage for pg_validatebackup.c is also
> improved, but it's not 100%; there are some cases that I don't know
> how to hit outside of a kernel malfunction, and others that I only
> know how to hit on non-Windows systems. For instance, it's easy to use
> perl to make a file inaccessible on Linux with chmod(0, $filename),
> but I gather that doesn't work on Windows. I'm going to spend a bit
> more time looking at this, but I think it's already reasonably good.
>
> - I fixed a couple of very minor bugs which I discovered by writing those
> tests.
>
> - I added documentation, in part based on a draft Mark Dilger shared
> with me off-list.
>
> I don't think this is committable just yet, but I think it's getting
> fairly close, so if anyone has major objections please speak up soon.
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
--
--
Thanks & Regards,
Suraj kharage,
EnterpriseDB Corporation,
The Postgres Database Company.
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
@ 2020-03-16 06:03 ` Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Suraj Kharage @ 2020-03-16 06:03 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
One more suggestion, recent commit (1933ae62) has added the PostgreSQL home
page to --help output.
e.g:
*PostgreSQL home page: <https://www.postgresql.org/
<https://www.postgresql.org/>>*
We might need to consider this change for pg_validatebackup binary.
On Mon, Mar 16, 2020 at 10:37 AM Suraj Kharage <
[email protected]> wrote:
> Thank you, Robert.
>
> Getting below warning while compiling the
> v11-0003-pg_validatebackup-Validate-a-backup-against-the-.patch.
>
>
>
> *pg_validatebackup.c: In function
> ‘report_manifest_error’:pg_validatebackup.c:356:2: warning: function might
> be possible candidate for ‘gnu_printf’ format attribute
> [-Wsuggest-attribute=format] pg_log_generic_v(PG_LOG_FATAL, fmt, ap);*
>
>
> To resolve this, can we use "pg_attribute_printf(2, 3)" in function
> declaration something like below?
> e.g:
>
> diff --git a/src/bin/pg_validatebackup/parse_manifest.h
> b/src/bin/pg_validatebackup/parse_manifest.h
> index b0b18a5..25d140f 100644
> --- a/src/bin/pg_validatebackup/parse_manifest.h
> +++ b/src/bin/pg_validatebackup/parse_manifest.h
> @@ -25,7 +25,7 @@ typedef void
> (*json_manifest_perfile_callback)(JsonManifestParseContext *,
> size_t
> size, pg_checksum_type checksum_type,
> int
> checksum_length, uint8 *checksum_payload);
> typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
> - char
> *fmt, ...);
> + char
> *fmt,...) pg_attribute_printf(2, 3);
>
> struct JsonManifestParseContext
> {
> diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c
> b/src/bin/pg_validatebackup/pg_validatebackup.c
> index 0e7299b..6ccbe59 100644
> --- a/src/bin/pg_validatebackup/pg_validatebackup.c
> +++ b/src/bin/pg_validatebackup/pg_validatebackup.c
> @@ -95,7 +95,7 @@ static void
> record_manifest_details_for_file(JsonManifestParseContext *context,
>
> int checksum_length,
>
> uint8 *checksum_payload);
> static void report_manifest_error(JsonManifestParseContext *context,
> - char
> *fmt, ...);
> + char
> *fmt,...) pg_attribute_printf(2, 3);
>
> static void validate_backup_directory(validator_context *context,
>
> char *relpath, char *fullpath);
>
>
> Typos:
>
> 0004 patch
> unexpctedly => unexpectedly
>
> 0005 patch
> bacup => backup
>
> On Sat, Mar 14, 2020 at 2:04 AM Robert Haas <[email protected]> wrote:
>
>> On Thu, Mar 12, 2020 at 10:47 AM tushar <[email protected]>
>> wrote:
>> > On 3/9/20 10:46 PM, Robert Haas wrote:
>> > > Seems like expected behavior to me. We could consider providing a more
>> > > descriptive error message, but there's now way for it to work.
>> >
>> > Right , Error message need to be more user friendly .
>>
>> OK. Done in the attached version, which also includes a few other changes:
>>
>> - I expanded the regression tests. They now cover every line of code
>> in parse_manifest.c except for a few that I believe to be unreachable
>> (though I might be mistaken). Coverage for pg_validatebackup.c is also
>> improved, but it's not 100%; there are some cases that I don't know
>> how to hit outside of a kernel malfunction, and others that I only
>> know how to hit on non-Windows systems. For instance, it's easy to use
>> perl to make a file inaccessible on Linux with chmod(0, $filename),
>> but I gather that doesn't work on Windows. I'm going to spend a bit
>> more time looking at this, but I think it's already reasonably good.
>>
>> - I fixed a couple of very minor bugs which I discovered by writing those
>> tests.
>>
>> - I added documentation, in part based on a draft Mark Dilger shared
>> with me off-list.
>>
>> I don't think this is committable just yet, but I think it's getting
>> fairly close, so if anyone has major objections please speak up soon.
>>
>> --
>> Robert Haas
>> EnterpriseDB: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
> --
> --
>
> Thanks & Regards,
> Suraj kharage,
> EnterpriseDB Corporation,
> The Postgres Database Company.
>
--
--
Thanks & Regards,
Suraj kharage,
EnterpriseDB Corporation,
The Postgres Database Company.
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
@ 2020-03-20 22:29 ` Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-20 22:29 UTC (permalink / raw)
To: Suraj Kharage <[email protected]>; +Cc: tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 16, 2020 at 2:03 AM Suraj Kharage
<[email protected]> wrote:
> One more suggestion, recent commit (1933ae62) has added the PostgreSQL home page to --help output.
Good catch. Fixed. I also attempted to address the compiler warning
you mentioned in your other email.
Also, I realized that the previous patch versions didn't handle the
hex-encoded path format that we need to use for non-UTF8 filenames,
and that there was no easy way to test that format. So, in this
version I added an option to force all pathnames to be encoded in that
format. I also made that option capable of suppressing the backup
manifest altogether. Other than that, this version is pretty much the
same as the last version, except for a few additional test cases which
I added to get the code coverage up even a little more. It would be
nice if someone could test whether the tests pass on Windows.
I have squashed the series down to just 2 commits, since that seems
like the way that this should probably be committed. Barring strong
objections and/or the end of the world, I plan to do that next week.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v12-0001-Add-checksum-helper-functions.patch (9.6K, ../../CA+TgmobhNAHF1jqi9swroOqZgkcaGXNZG_bm8bdtHEJV=fmF3w@mail.gmail.com/2-v12-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From 78613b46ebf7602f44b944ff4c6333b455ef647c Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 20 Mar 2020 14:48:33 -0400
Subject: [PATCH v12 1/2] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index ce01df68b9..e199ed7acb 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -47,6 +47,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index f89a8a4fdb..ee04fbafa6 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -120,8 +120,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
- f2s.c file_perm.c hashfn.c ip.c jsonapi.c
+ base64.c checksum_helper.c config_info.c controldata_utils.c d2s.c
+ encnames.c exec.c f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c username.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v12-0002-Generate-backup-manifests-for-base-backups-and-v.patch (116.9K, ../../CA+TgmobhNAHF1jqi9swroOqZgkcaGXNZG_bm8bdtHEJV=fmF3w@mail.gmail.com/3-v12-0002-Generate-backup-manifests-for-base-backups-and-v.patch)
download | inline diff:
From 1a72a37219abdd764d74859e941cbc4dbef559e0 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 20 Mar 2020 18:01:52 -0400
Subject: [PATCH v12 2/2] Generate backup manifests for base backups; and
validate them.
A manifest is a JSON document which includes the file name, size, last
modification time, and a checksum for each file backed up, as well as
a checksum for the manifest itself. By default, we use CRC-32C for the
checksum algorithm, because we are trying to detect corruption and
user error, not foil an adversary. However, pg_basebackup and the
server-side BASE_BACKUP command now have options to select the
checksum algorithm, so users wanting a cryptographic hash function can
select SHA-224, SHA-256, SHA-384, or SHA-512. Users not wanting any
checksums at all can disable them, or disable generating of the backup
manifest altogether. Using a cryptographic hash function in place of
CRC-32C consumes significantly more CPU cycles, which may slow down
backups in some cases.
A new tool called pg_validatebackup can validate a backup against the
manifest. If no checksums are present, it can still check that the
right files exist and that they have the expected sizes. If checksums
are present, it can also verify that each file has the expected
checksum. Only plain format backups can be validated directly, but tar
format backups can be validated after extracting them.
Robert Haas, with help, ideas, review, and testing from David Steele,
Stephen Frost, Andrew Dunstan, Rushabh Lathia, Suraj Kharage, Tushar
Ahuja, Rajkumar Raghuwanshi, Mark Dilger, Davinder Singh, and Jeevan
Chalke.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
doc/src/sgml/protocol.sgml | 33 +-
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_basebackup.sgml | 63 ++
doc/src/sgml/ref/pg_validatebackup.sgml | 232 ++++++
doc/src/sgml/reference.sgml | 1 +
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 430 +++++++++-
src/backend/replication/repl_gram.y | 13 +
src/backend/replication/repl_scanner.l | 2 +
src/backend/replication/walsender.c | 30 +
src/bin/Makefile | 1 +
src/bin/pg_basebackup/pg_basebackup.c | 184 ++++-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 8 +-
src/bin/pg_validatebackup/.gitignore | 2 +
src/bin/pg_validatebackup/Makefile | 39 +
src/bin/pg_validatebackup/parse_manifest.c | 575 ++++++++++++++
src/bin/pg_validatebackup/parse_manifest.h | 40 +
src/bin/pg_validatebackup/pg_validatebackup.c | 734 ++++++++++++++++++
src/bin/pg_validatebackup/t/001_basic.pl | 30 +
src/bin/pg_validatebackup/t/002_algorithm.pl | 58 ++
src/bin/pg_validatebackup/t/003_corruption.pl | 244 ++++++
src/bin/pg_validatebackup/t/004_options.pl | 89 +++
.../pg_validatebackup/t/005_bad_manifest.pl | 158 ++++
src/bin/pg_validatebackup/t/006_encoding.pl | 27 +
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
26 files changed, 2973 insertions(+), 32 deletions(-)
create mode 100644 doc/src/sgml/ref/pg_validatebackup.sgml
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/parse_manifest.c
create mode 100644 src/bin/pg_validatebackup/parse_manifest.h
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
create mode 100644 src/bin/pg_validatebackup/t/001_basic.pl
create mode 100644 src/bin/pg_validatebackup/t/002_algorithm.pl
create mode 100644 src/bin/pg_validatebackup/t/003_corruption.pl
create mode 100644 src/bin/pg_validatebackup/t/004_options.pl
create mode 100644 src/bin/pg_validatebackup/t/005_bad_manifest.pl
create mode 100644 src/bin/pg_validatebackup/t/006_encoding.pl
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index f139ba0231..d1ff53e8e8 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2466,7 +2466,7 @@ The commands accepted in replication mode are:
</varlistentry>
<varlistentry id="protocol-replication-base-backup" xreflabel="BASE_BACKUP">
- <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ]
+ <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ] [ <literal>MANIFEST</literal> <replaceable>manifest_option</replaceable> ] [ <literal>MANIFEST_CHECKSUMS</literal> <replaceable>checksum_algorithm</replaceable> ]
<indexterm><primary>BASE_BACKUP</primary></indexterm>
</term>
<listitem>
@@ -2576,6 +2576,37 @@ The commands accepted in replication mode are:
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST</literal></term>
+ <listitem>
+ <para>
+ When this option is specified with a value of <literal>ye'</literal>
+ or <literal>force-escape</literal>, a backup manifest is created
+ and sent along with the backup. The latter value forces all filenames
+ to be hex-encoded; otherwise, this type of encoding is performed only
+ for files whose names are non-UTF8 octet sequences.
+ <literal>force-escape</literal> is intended primarily for testing
+ purposes, to be sure that clients which read the backup manifest
+ can handle this case. For compatibility with previous releases,
+ the default is <literal>MANIFEST 'no'</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST_CHECKSUMS</literal></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be used to checksum each file
+ for purposes of the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
<para>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..ab71176cdf 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -211,6 +211,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
<!ENTITY pgRestore SYSTEM "pg_restore.sgml">
<!ENTITY pgRewind SYSTEM "pg_rewind.sgml">
+<!ENTITY pgValidateBackup SYSTEM "pg_validatebackup.sgml">
<!ENTITY pgtestfsync SYSTEM "pgtestfsync.sgml">
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index 90638aad0e..bf6963a595 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -561,6 +561,69 @@ PostgreSQL documentation
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>--no-manifest</option></term>
+ <listitem>
+ <para>
+ Disables generation of a backup manifest. If this option is not
+ specified, the server will and send generate a backup manifest
+ which can be verified using <xref linkend="app-pgvalidatebackup" />.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-force-encode</option></term>
+ <listitem>
+ <para>
+ Forces all filenames in the backup manifest to be hex-encoded.
+ If this option is not specified, only non-UTF8 filenames are
+ hex-encoded. This option is mostly intended to test that tools which
+ read a backup manifest file properly handle this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be used to checksum each file
+ for purposes of the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ <para>
+ If <literal>NONE</literal> is selected, the backup manifest will
+ not contain any checksums. Otherwise, it will contain a checksum
+ of each file in the backup using the specified algorithm. In addition,
+ the manifest itself will always contain a <literal>SHA256</literal>
+ checksum of its own contents. The <literal>SHA</literal> algorithms
+ are significantly more CPU-intensive than <literal>CRC32C</literal>,
+ so selecting one of them may increase the time required to complete
+ the backup.
+ </para>
+ <para>
+ On the other hand, <literal>CRC32C</literal> is not a cryptographic
+ hash function, so it is only suitable for protecting against
+ inadvertent or random modifications to a backup. An adversary
+ who can modify the backup could easily do so in such a way that
+ the CRC does not change, whereas a SHA collision will be hard
+ to manufacture. (However, note that if the attacker also has access
+ to modify the backup manifest itself, no checksum algorithm will
+ provide any protection.) An additional advantage of the
+ <literal>SHA</literal> family of functions is that they output
+ a much larger number of bits.
+ </para>
+ <para>
+ <xref linkend="app-pgvalidatebackup" /> can be used to check the
+ integrity of a backup against the backup manifest.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
new file mode 100644
index 0000000000..1c171f6970
--- /dev/null
+++ b/doc/src/sgml/ref/pg_validatebackup.sgml
@@ -0,0 +1,232 @@
+<!--
+doc/src/sgml/ref/pg_validatebackup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgvalidatebackup">
+ <indexterm zone="app-pgvalidatebackup">
+ <primary>pg_validatebackup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>pg_validatebackup</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_validatebackup</refname>
+ <refpurpose>verify the integrity of a base backup of a
+ <productname>PostgreSQL</productname> cluster</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_validatebackup</command>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>
+ Description
+ </title>
+ <para>
+ <application>pg_validatebackup</application> is used to check the integrity
+ of a database cluster backup. The backup being checked should have been
+ created by <command>pg_basebackup</command> or some other tool that includes
+ a <literal>backup_manifest</literal> file with the backup. The backup
+ must be stored in the "plain" format; a "tar" format backup can be checked
+ after extracting it. Backup manifests are created by the server beginning
+ with <productname>PostgreSQL</productname> version 13, so older backups
+ cannot be validated using this tool.
+ </para>
+
+ <para>
+ <application>pg_validatebackup</application> reads the manifest file of a
+ backup, verifies the manifest against its own internal checksum, and then
+ verifies that the same files are present in the target directory as in the
+ manifest itself. It then verifies that each file has the expected checksum,
+ unless the backup was taken the checksum algorithm set to
+ <literal>none</literal>, in which case checksum verification is not
+ performed. The presence or absence of directories is not checked, except
+ indirectly: if a directory is missing, any files it should have contained
+ will necessarily also be missing. Certain files and directories are
+ excluded from verification:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>backup_manifest</literal> is ignored because the backup
+ manifest is logically not part of the backup and does not include
+ any entry for itself.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>pg_wal</literal> is ignored because WAL files are sent
+ separately from the backup, and are therefore not described by the
+ backup manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>postgesql.auto.conf</literal>,
+ <literal>standby.signal</literal>,
+ and <literal>recovery.signal</literal> are ignored because they may
+ sometimes be created or modified by the backup client itself.
+ (For example, <literal>pg_basebackup -R</literal> will modify
+ <literal>postgresql.auto.conf</literal> and create
+ <literal>standby.signal</literal>.)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ The following command-line options control the behavior.
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-e</option></term>
+ <term><option>--exit-on-error</option></term>
+ <listitem>
+ <para>
+ Exit as soon as a problem with the backup is detected. If this option
+ is not specified, <literal>pg_basebackup</literal> will continue
+ checking the backup even after a problem has been detected, and will
+ report all problems detected as errors.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-i <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--ignore=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Ignore the specified file or directory, which should be expressed
+ as a relative pathname. If the backup contains extra files, is
+ missing files, or has files that have been modified as compared with
+ what is described in the manifest, this option can be used to suppress
+ the errors that would otherwise occur. If a directory is specified,
+ this option affects the entire subtree rooted at that location.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-m <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--manifest-path=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Use the manifest file at the specified path, rather than one located
+ in the root of the backup directory.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-q</option></term>
+ <term><option>--quiet</option></term>
+ <listitem>
+ <para>
+ Don't print anything when a backup is successfully validated.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-s</option></term>
+ <term><option>--skip-checksums</option></term>
+ <listitem>
+ <para>
+ Do not validate checksums. The presence or absence of files and the
+ sizes of those files will still be checked. This is much faster,
+ because the files themselves do not need to read.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other options are also available:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_validatebackup</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_validatebackup</application> command
+ line arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal> and
+ validate the integrity of the backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal>, move
+ the manifest somewhere outside the backup directory, and validate the
+ backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234</userinput>
+<prompt>$</prompt> <userinput>mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234</userinput>
+</screen>
+ </para>
+
+ <para>
+ To validate a backup while ignoring a file that was added manually to the
+ backup directory, and also skipping checksum verification:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>edit /usr/local/pgsql/data/note.to.self</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="app-pgbasebackup"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..d25a77b13c 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -255,6 +255,7 @@
&pgReceivewal;
&pgRecvlogical;
&pgRestore;
+ &pgValidateBackup;
&psqlRef;
&reindexdb;
&vacuumdb;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 793c076da6..b3917bc526 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10551,7 +10551,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 806d013108..6dffc6ef5b 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -39,10 +41,19 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
+typedef enum manifest_option
+{
+ MANIFEST_OPTION_YES,
+ MANIFEST_OPTION_NO,
+ MANIFEST_OPTION_FORCE_ENCODE
+} manifest_option;
+
typedef struct
{
const char *label;
@@ -52,20 +63,43 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ manifest_option manifest;
+ pg_checksum_type manifest_checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ bool force_encode;
+ bool first_file;
+ bool still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest,
+ basebackup_options *opt);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +136,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -251,6 +295,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -258,12 +303,17 @@ perform_base_backup(basebackup_options *opt)
backup_streamed = 0;
pgstat_progress_start_command(PROGRESS_COMMAND_BASEBACKUP, InvalidOid);
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt);
total_checksum_failures = 0;
@@ -301,7 +351,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -380,7 +433,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -388,11 +442,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -400,10 +457,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -632,7 +690,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -655,16 +713,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -678,6 +740,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -709,8 +774,13 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->manifest = MANIFEST_OPTION_NO;
+ opt->manifest_checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -797,12 +867,61 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest") == 0)
+ {
+ char *optval = strVal(defel->arg);
+ bool manifest_bool;
+
+ if (o_manifest)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (parse_bool(optval, &manifest_bool))
+ {
+ if (manifest_bool)
+ opt->manifest = MANIFEST_OPTION_YES;
+ else
+ opt->manifest = MANIFEST_OPTION_NO;
+ }
+ else if (pg_strcasecmp(optval, "force-encode") == 0)
+ opt->manifest = MANIFEST_OPTION_FORCE_ENCODE;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized manifest option: \"%s\"",
+ optval)));
+ o_manifest = true;
+ }
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval,
+ &opt->manifest_checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
}
if (opt->label == NULL)
opt->label = "base backup";
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ {
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("manifest checksums require a backup manifest")));
+ opt->manifest_checksum_type = CHECKSUM_TYPE_NONE;
+ }
}
@@ -918,6 +1037,249 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, basebackup_options *opt)
+{
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ manifest->buffile = NULL;
+ else
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = opt->manifest_checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->force_encode = (opt->manifest == MANIFEST_OPTION_FORCE_ENCODE);
+ manifest->first_file = true;
+ manifest->still_checksumming = true;
+
+ if (opt->manifest != MANIFEST_OPTION_NO)
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ Assert(manifest != NULL);
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ int pathlen;
+ StringInfoData buf;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (!manifest->force_encode &&
+ pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
+
+ /*
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
+ */
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
+
+ /* Add checksum information. */
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
+ }
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
+
+ /* Avoid leaking memory. */
+ pfree(buf.data);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\"}\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -978,11 +1340,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1017,6 +1383,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1027,7 +1397,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1060,7 +1431,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1079,7 +1451,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1359,7 +1731,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1367,7 +1740,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1437,8 +1811,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1455,6 +1830,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1625,6 +2003,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1649,6 +2030,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1657,7 +2039,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1682,6 +2065,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..f93a0de218 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,8 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -156,6 +158,7 @@ var_name: IDENT { $$ = $1; }
/*
* BASE_BACKUP [LABEL '<label>'] [PROGRESS] [FAST] [WAL] [NOWAIT]
* [MAX_RATE %d] [TABLESPACE_MAP] [NOVERIFY_CHECKSUMS]
+ * [MANIFEST %s] [MANIFEST_CHECKSUMS %s]
*/
base_backup:
K_BASE_BACKUP base_backup_opt_list
@@ -214,6 +217,16 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST SCONST
+ {
+ $$ = makeDefElem("manifest",
+ (Node *)makeString($2), -1);
+ }
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..452ad9fc27 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,8 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST { return K_MANIFEST; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 76ec3c7dd0..3b117d8367 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index c5d95958b2..f355eb612c 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -136,6 +142,9 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static bool manifest = true;
+static bool manifest_force_encode = false;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -181,6 +190,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -388,6 +403,11 @@ usage(void)
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
printf(_(" --no-estimate-size do not estimate backup size in server side\n"));
+ printf(_(" --no-manifest suppress generation of backup manifest\n"));
+ printf(_(" --manifest-force-encode\n"
+ " hex encode all filenames in manifest\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1186,6 +1206,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1657,6 +1702,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1667,6 +1770,8 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_clause;
+ char *manifest_checksums_clause = "";
int i;
char xlogstart[64];
char xlogend[64];
@@ -1674,6 +1779,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1727,6 +1833,32 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest)
+ {
+ if (serverMajor < 1300)
+ {
+ const char *serverver = PQparameterStatus(conn, "server_version");
+
+ pg_log_error("backup manifests are not supported by server version %s",
+ serverver ? serverver : "'unknown'");
+ exit(1);
+ }
+
+ if (manifest_force_encode)
+ manifest_clause = "MANIFEST 'force-encode'";
+ else
+ manifest_clause = "MANIFEST 'yes'";
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
+ }
+ else
+ {
+ if (serverMajor < 1300)
+ manifest_clause = "";
+ else
+ manifest_clause = "MANIFEST 'no'";
+ }
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1741,7 +1873,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s %s",
escaped_label,
estimatesize ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1749,7 +1881,9 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_clause,
+ manifest_checksums_clause);
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1837,7 +1971,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1866,6 +2001,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && manifest)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2069,6 +2217,9 @@ main(int argc, char **argv)
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
{"no-estimate-size", no_argument, NULL, 4},
+ {"no-manifest", no_argument, NULL, 5},
+ {"manifest-force-encode", no_argument, NULL, 6},
+ {"manifest-checksums", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
int c;
@@ -2096,7 +2247,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2240,6 +2391,15 @@ main(int argc, char **argv)
case 4:
estimatesize = false;
break;
+ case 5:
+ manifest = false;
+ break;
+ case 6:
+ manifest_force_encode = true;
+ break;
+ case 7:
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
@@ -2370,6 +2530,22 @@ main(int argc, char **argv)
exit(1);
}
+ if (!manifest && manifest_checksums != NULL)
+ {
+ pg_log_error("--no-manifest and --manifest-checksums are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (!manifest && manifest_force_encode)
+ {
+ pg_log_error("--no-manifest and --manifest-force-encode are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
/* connection in replication mode to server */
conn = GetConnection();
if (!conn)
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..63381764e9 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 109;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -104,6 +104,7 @@ foreach my $filename (@tempRelationFiles)
$node->command_ok([ 'pg_basebackup', '-D', "$tempdir/backup", '-X', 'none' ],
'pg_basebackup runs');
ok(-f "$tempdir/backup/PG_VERSION", 'backup was created');
+ok(-f "$tempdir/backup/backup_manifest", 'backup manifest included');
# Permissions on backup should be default
SKIP:
@@ -160,11 +161,12 @@ rmtree("$tempdir/backup");
$node->command_ok(
[
- 'pg_basebackup', '-D', "$tempdir/backup2", '--waldir',
- "$tempdir/xlog2"
+ 'pg_basebackup', '-D', "$tempdir/backup2", '--no-manifest',
+ '--waldir', "$tempdir/xlog2"
],
'separate xlog directory');
ok(-f "$tempdir/backup2/PG_VERSION", 'backup was created');
+ok(! -f "$tempdir/backup2/backup_manifest", 'manifest was suppressed');
ok(-d "$tempdir/xlog2/", 'xlog directory was created');
rmtree("$tempdir/backup2");
rmtree("$tempdir/xlog2");
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..21e0a92429
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1,2 @@
+/pg_validatebackup
+/tmp_check/
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..04ef7d3051
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,39 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ parse_manifest.o \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
new file mode 100644
index 0000000000..f2e043077e
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -0,0 +1,575 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.c
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "parse_manifest.h"
+#include "common/jsonapi.h"
+
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_ENCODED_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+/*
+ * Internal state used while decoding the JSON-format backup manifest.
+ */
+typedef struct
+{
+ JsonManifestParseContext *context;
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ char *pathname;
+ char *encoded_pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void verify_manifest_checksum(JsonManifestParseState *parse,
+ char *buffer, size_t size);
+static void json_manifest_parse_failure(JsonManifestParseContext *context,
+ char *msg);
+
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+
+/*
+ * Main entrypoint to parse a JSON-format backup manifest.
+ *
+ * Caller should set up the parsing context and then invoke this function.
+ * For each file whose information is extracted from the manifest,
+ * context->perfile_cb is invoked. In case of trouble, context->error_cb is
+ * invoked and is expected not to return.
+ */
+void
+json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ size_t size)
+{
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
+
+ /* Set up our private parsing context. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.context = context;
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+ /* Validate the checksum. */
+ verify_manifest_checksum(&parse, buffer, size);
+}
+
+/*
+ * Invoked at the start of each object in the JSON document.
+ *
+ * The document as a whole is expected to be an object with three keys
+ * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
+ * file is expected to be an object with various keys (Path, Size, etc.).
+ * If we're not at the beginning of either the toplevel object or the object
+ * for a particular file, it's an error.
+ */
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->size = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each object in the JSON document.
+ *
+ * The possible cases here are the same as for json_manifest_object_start.
+ * There's nothing special to do at the end of the document, but when we
+ * reach the end of an object representing a particular file, we must call
+ * json_manifest_finalize_file() to save the associated details.
+ */
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ json_manifest_finalize_file(parse);
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each array in the JSON document.
+ *
+ * Within the toplevel object, the value associated with the "Files" key
+ * should be an array. No other arrays are expected.
+ */
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each array in the JSON document.
+ *
+ * Just like json_manifest_array_start, there's only one expected case
+ * here.
+ */
+static void
+json_manifest_array_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each object field in the JSON document.
+ */
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_FIELD:
+ /* Inside toplevel object, expecting version indicator. */
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ /* Inside toplevel object, expecting "Files" next. */
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Encoded-Path") == 0)
+ parse->field = JMFF_ENCODED_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure(parse->context,
+ "unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ /* Inside toplevel object, expecting "Manifest-Checksum" next. */
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object field");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each scalar in the JSON document.
+ *
+ * Object field names don't reach this code; those are handled by
+ * json_manifest_object_field_start. When we're inside of the object for
+ * a particular file, that function will have noticed the name of the field,
+ * and we'll get the corresponding value here. When we're in the toplevel
+ * object, the parse state itself tells us which field this is.
+ *
+ * In all cases except for PostgreSQL-Backup-Manifest-Version, which we
+ * can just check on the spot, the goal here is just to save the value in
+ * the parse state for later use. We don't actually do anything until we
+ * reach either the end of the object representing this file, or the end
+ * of the manifest, as the case may be.
+ */
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure(parse->context,
+ "unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_ENCODED_PATH:
+ parse->encoded_pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context, "unexpected scalar");
+ break;
+ }
+}
+
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * file, and invoke the per-file callback so that the caller gets those
+ * details. This happens for each file when the corresponding JSON object is
+ * completely parsed.
+ */
+static void
+json_manifest_finalize_file(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t size;
+ char *ep;
+ int checksum_string_length;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL && parse->encoded_pathname == NULL)
+ json_manifest_parse_failure(parse->context, "missing pathname");
+ if (parse->pathname != NULL && parse->encoded_pathname != NULL)
+ json_manifest_parse_failure(parse->context,
+ "both pathname and encoded pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure(parse->context, "missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure(parse->context,
+ "checksum without algorithm");
+
+ /* Decode encoded pathname, if that's what we have. */
+ if (parse->encoded_pathname != NULL)
+ {
+ int encoded_length = strlen(parse->encoded_pathname);
+ int raw_length = encoded_length / 2;
+
+ parse->pathname = palloc(raw_length + 1);
+ if (encoded_length % 2 != 0 ||
+ !hexdecode_string((uint8 *) parse->pathname,
+ parse->encoded_pathname,
+ raw_length))
+ json_manifest_parse_failure(parse->context,
+ "unable to decode filename");
+ parse->pathname[raw_length] = '\0';
+ pfree(parse->encoded_pathname);
+ parse->encoded_pathname = NULL;
+ }
+
+ /* Parse size. */
+ size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm, &checksum_type))
+ context->error_cb(context, "unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ checksum_length = 0;
+ checksum_payload = NULL;
+ }
+ else
+ {
+ checksum_length = checksum_string_length / 2;
+ checksum_payload = palloc(checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(checksum_payload, parse->checksum,
+ checksum_length))
+ context->error_cb(context,
+ "invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, parse->checksum);
+ }
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perfile_cb(context, parse->pathname, size,
+ checksum_type, checksum_length, checksum_payload);
+
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
+}
+
+/*
+ * Verify that the manifest checksum is correct.
+ *
+ * The last line of the manifest file is excluded from the manifest checksum,
+ * because the last line is expected to contain the checksum that covers
+ * the rest of the file.
+ */
+static void
+verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
+ size_t size)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t i;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+
+ /* Find the last two newlines in the file. */
+ for (i = 0; i < size; ++i)
+ {
+ if (buffer[i] == '\n')
+ {
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
+ }
+ }
+
+ /*
+ * Make sure that the last newline is right at the end, and that there are
+ * at least two lines total. We need this to be true in order for the
+ * following code, which computes the manifest checksum, to work properly.
+ */
+ if (number_of_newlines < 2)
+ json_manifest_parse_failure(parse->context,
+ "expected at least 2 lines");
+ if (ultimate_newline != size - 1)
+ json_manifest_parse_failure(parse->context,
+ "last line not newline-terminated");
+
+ /* Checksum the rest. */
+ pg_sha256_init(&manifest_ctx);
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+
+ /* Now verify it. */
+ if (parse->manifest_checksum == NULL)
+ context->error_cb(parse->context, "manifest has no checksum");
+ if (strlen(parse->manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse->manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ context->error_cb(context, "invalid manifest checksum: \"%s\"",
+ parse->manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ context->error_cb(context, "manifest checksum mismatch");
+}
+
+/*
+ * Report a parse error.
+ *
+ * This is intended to be used for fairly low-level failures that probably
+ * shouldn't occur unless somebody has deliberately constructed a bad manifest,
+ * or unless the server is generating bad manifests due to some bug. msg should
+ * be a short string giving some hint as to what the problem is.
+ */
+static void
+json_manifest_parse_failure(JsonManifestParseContext *context, char *msg)
+{
+ context->error_cb(context, "could not parse backup manifest: %s", msg);
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
new file mode 100644
index 0000000000..b0b18a57ca
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -0,0 +1,40 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.h
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PARSE_MANIFEST_H
+#define PARSE_MANIFEST_H
+
+#include "common/checksum_helper.h"
+#include "mb/pg_wchar.h"
+
+struct JsonManifestParseContext;
+typedef struct JsonManifestParseContext JsonManifestParseContext;
+
+typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
+ char *pathname,
+ size_t size, pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
+ char *fmt, ...);
+
+struct JsonManifestParseContext
+{
+ void *private_data;
+ json_manifest_perfile_callback perfile_cb;
+ json_manifest_error_callback error_cb;
+};
+
+extern void json_parse_manifest(JsonManifestParseContext *context,
+ char *buffer, size_t size);
+
+#endif
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..eb1473d9d0
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,734 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+#include "parse_manifest.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+
+static void record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length,
+ uint8 *checksum_payload);
+static void report_manifest_error(JsonManifestParseContext *context,
+ char *fmt, ...)
+ pg_attribute_printf(2, 3) pg_attribute_noreturn();
+
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void report_backup_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void report_fatal_error(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ printf("backup successfully verified\n");
+
+ return context.saw_any_error ? 1 : 0;
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ int rc;
+ JsonManifestParseContext context;
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ report_fatal_error("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ report_fatal_error("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /*
+ * Slurp in the whole file.
+ *
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
+ */
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
+ {
+ if (rc < 0)
+ report_fatal_error("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ report_fatal_error("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
+
+ /* Close the manifest file. */
+ close(fd);
+
+ /* Parse the manifest as JSON. */
+ context.private_data = ht;
+ context.perfile_cb = record_manifest_details_for_file;
+ context.error_cb = report_manifest_error;
+ json_parse_manifest(&context, buffer, statbuf.st_size);
+
+ /* Done with the buffer. */
+ pfree(buffer);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * Report an error while parsing the manifest.
+ *
+ * We consider all such errors to be fatal errors. The manifest parser
+ * expects this function not to return.
+ */
+static void
+report_manifest_error(JsonManifestParseContext *context, char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Record details extracted from the backup manifest for one file.
+ */
+static void
+record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload)
+{
+ manifestfiles_hash *ht = context->private_data;
+ manifestfile *tabent;
+ bool found;
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(ht, pathname, &found);
+ if (found)
+ report_fatal_error("duplicate pathname in backup manifest: \"%s\"",
+ pathname);
+
+ /* Initialize the entry. */
+ tabent->size = size;
+ tabent->checksum_type = checksum_type;
+ tabent->checksum_length = checksum_length;
+ tabent->checksum_payload = checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ report_fatal_error("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ report_backup_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ report_backup_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ report_backup_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ report_backup_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ report_backup_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ report_backup_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ report_backup_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ report_backup_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ report_backup_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ report_backup_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ report_backup_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ report_backup_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Report a problem with the backup.
+ *
+ * Update the context to indicate that we saw an error, and exit if the
+ * context says we should.
+ */
+static void
+report_backup_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+report_fatal_error(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+ printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
+}
diff --git a/src/bin/pg_validatebackup/t/001_basic.pl b/src/bin/pg_validatebackup/t/001_basic.pl
new file mode 100644
index 0000000000..6d4b8ea01a
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/001_basic.pl
@@ -0,0 +1,30 @@
+use strict;
+use warnings;
+use TestLib;
+use Test::More tests => 16;
+
+my $tempdir = TestLib::tempdir;
+
+program_help_ok('pg_validatebackup');
+program_version_ok('pg_validatebackup');
+program_options_handling_ok('pg_validatebackup');
+
+command_fails_like(['pg_validatebackup'],
+ qr/no backup directory specified/,
+ 'target directory must be specified');
+command_fails_like(['pg_validatebackup', $tempdir],
+ qr/could not open file.*\/backup_manifest\"/,
+ 'pg_validatebackup requires a manifest');
+command_fails_like(['pg_validatebackup', $tempdir, $tempdir],
+ qr/too many command-line arguments/,
+ 'multiple target directories not allowed');
+
+# create fake manifest file
+open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+close($fh);
+
+# but then try to use an alternate, nonexisting manifest
+command_fails_like(['pg_validatebackup', '-m', "$tempdir/not_the_manifest",
+ $tempdir],
+ qr/could not open file.*\/not_the_manifest\"/,
+ 'pg_validatebackup respects -m flag');
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
new file mode 100644
index 0000000000..2046a709b4
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -0,0 +1,58 @@
+# Verify that we can take and validate backups with various checksum types.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 19;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
+{
+ my $backup_path = $master->backup_dir . '/' . $algorithm;
+ my @backup = ('pg_basebackup', '-D', $backup_path,
+ '--manifest-checksum', $algorithm,
+ '--no-sync');
+ my @validate = ('pg_validatebackup', '-e', $backup_path);
+
+ # A backup with a bogus algorithm should fail.
+ if ($algorithm eq 'bogus')
+ {
+ $master->command_fails(\@backup,
+ "backup fails with algorithm \"$algorithm\"");
+ next;
+ }
+
+ # A backup with a valid algorithm should work.
+ $master->command_ok(\@backup, "backup ok with algorithm \"$algorithm\"");
+
+ # We expect each real checksum algorithm to be mentioned on every line of
+ # the backup manifest file except the first and last; for simplicity, we
+ # just check that it shows up lots of times. When the checksum algorithm
+ # is none, we just check that the manifest exists.
+ if ($algorithm eq 'none')
+ {
+ ok(-f "$backup_path/backup_manifest", "backup manifest exists");
+ }
+ else
+ {
+ my $manifest = slurp_file("$backup_path/backup_manifest");
+ my $count_of_algorithm_in_manifest =
+ (() = $manifest =~ /$algorithm/mig);
+ cmp_ok($count_of_algorithm_in_manifest, '>', 100,
+ "$algorithm is mentioned many times in the manifest");
+ }
+
+ # Make sure that it validates OK.
+ $master->command_ok(\@validate,
+ "validate backup with algorithm \"$algorithm\"");
+
+ # Remove backup immediately to save disk space.
+ rmtree($backup_path);
+}
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
new file mode 100644
index 0000000000..45e2220c38
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -0,0 +1,244 @@
+# Verify that various forms of corruption are detected by pg_validatebackup.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+# Include a user-defined tablespace in the hopes of detecting problems in that
+# area.
+my $source_ts_path = TestLib::tempdir;
+$master->safe_psql('postgres', <<EOM);
+CREATE TABLE x1 (a int);
+INSERT INTO x1 VALUES (111);
+CREATE TABLESPACE ts1 LOCATION '$source_ts_path';
+CREATE TABLE x2 (a int) TABLESPACE ts1;
+INSERT INTO x1 VALUES (222);
+EOM
+
+my @scenario = (
+ {
+ 'name' => 'extra_file',
+ 'mutilate' => \&mutilate_extra_file,
+ 'fails_like' =>
+ qr/extra_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'extra_tablespace_file',
+ 'mutilate' => \&mutilate_extra_tablespace_file,
+ 'fails_like' =>
+ qr/extra_ts_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'missing_file',
+ 'mutilate' => \&mutilate_missing_file,
+ 'fails_like' =>
+ qr/pg_xact\/0000.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'missing_tablespace',
+ 'mutilate' => \&mutilate_missing_tablespace,
+ 'fails_like' =>
+ qr/pg_tblspc.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'append_to_file',
+ 'mutilate' => \&mutilate_append_to_file,
+ 'fails_like' =>
+ qr/has size \d+ on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'truncate_file',
+ 'mutilate' => \&mutilate_truncate_file,
+ 'fails_like' =>
+ qr/has size 0 on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'replace_file',
+ 'mutilate' => \&mutilate_replace_file,
+ 'fails_like' => qr/checksum mismatch for file/
+ },
+ {
+ 'name' => 'bad_manifest',
+ 'mutilate' => \&mutilate_bad_manifest,
+ 'fails_like' => qr/manifest checksum mismatch/
+ },
+ {
+ 'name' => 'open_file_fails',
+ 'mutilate' => \&mutilate_open_file_fails,
+ 'fails_like' => qr/could not open file/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'open_directory_fails',
+ 'mutilate' => \&mutilate_open_directory_fails,
+ 'fails_like' => qr/could not open directory/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'search_directory_fails',
+ 'mutilate' => \&mutilate_search_directory_fails,
+ 'cleanup' => \&cleanup_search_directory_fails,
+ 'fails_like' => qr/could not stat file or directory/,
+ 'skip_on_windows' => 1
+ }
+);
+
+for my $scenario (@scenario)
+{
+ my $name = $scenario->{'name'};
+
+ SKIP:
+ {
+ skip "unix-style permissions not supported on Windows", 4
+ if $scenario->{'skip_on_windows'} && $windows_os;
+
+ # Take a backup and check that it validates OK.
+ my $backup_path = $master->backup_dir . '/' . $name;
+ my $backup_ts_path = TestLib::tempdir;
+ $master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '-T', "${source_ts_path}=${backup_ts_path}"],
+ "base backup ok");
+ command_ok(['pg_validatebackup', $backup_path ],
+ "intact backup validated");
+
+ # Mutilate the backup in some way.
+ $scenario->{'mutilate'}->($backup_path);
+
+ # Now check that the backup no longer validates.
+ command_fails_like(['pg_validatebackup', $backup_path ],
+ $scenario->{'fails_like'},
+ "corrupt backup fails validation: $name");
+
+ # Run cleanup hook, if provided.
+ $scenario->{'cleanup'}->($backup_path)
+ if exists $scenario->{'cleanup'};
+
+ # Finally, use rmtree to reclaim space.
+ rmtree($backup_path);
+ }
+}
+
+sub create_extra_file
+{
+ my ($backup_path, $relative_path) = @_;
+ my $pathname = "$backup_path/$relative_path";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh "This is an extra file.\n";
+ close($fh);
+}
+
+# Add a file into the root directory of the backup.
+sub mutilate_extra_file
+{
+ my ($backup_path) = @_;
+ create_extra_file($backup_path, "extra_file");
+}
+
+# Add a file inside the user-defined tablespace.
+sub mutilate_extra_tablespace_file
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my ($catvdir) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid");
+ my ($tsdboid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid/$catvdir");
+ create_extra_file($backup_path,
+ "pg_tblspc/$tsoid/$catvdir/$tsdboid/extra_ts_file");
+}
+
+# Remove a file.
+sub mutilate_missing_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_xact/0000";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Remove the symlink to the user-defined tablespace.
+sub mutilate_missing_tablespace
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my $pathname = "$backup_path/pg_tblspc/$tsoid";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Append an additional bytes to a file.
+sub mutilate_append_to_file
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/global/pg_control", 'x';
+}
+
+# Truncate a file to zero length.
+sub mutilate_truncate_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/global/pg_control";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ close($fh);
+}
+
+# Replace a file's contents without changing the length of the file. This is
+# not a particularly efficient way to do this, so we pick a file that's
+# expected to be short.
+sub mutilate_replace_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ my $contents = slurp_file($pathname);
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh 'q' x length($contents);
+ close($fh);
+}
+
+# Corrupt the backup manifest.
+sub mutilate_bad_manifest
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/backup_manifest", "\n";
+}
+
+# Create a file that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_file_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_subtrans";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be searched. (This is skipped on Windows.)
+sub mutilate_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0400, $pathname) || die "chmod $pathname: $!";
+}
+
+# rmtree can't cope with a mode 400 directory, so change back to 700.
+sub cleanup_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0700, $pathname) || die "chmod $pathname: $!";
+}
diff --git a/src/bin/pg_validatebackup/t/004_options.pl b/src/bin/pg_validatebackup/t/004_options.pl
new file mode 100644
index 0000000000..8f185626ed
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/004_options.pl
@@ -0,0 +1,89 @@
+# Verify the behavior of assorted pg_validatebackup options.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 25;
+
+# Start up the server and take a backup.
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_options';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync' ],
+ "base backup ok");
+
+# Verify that pg_validatebackup -q succeeds and produces no output.
+my $stdout;
+my $stderr;
+my $result = IPC::Run::run ['pg_validatebackup', '-q', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok($result, "-q succeeds: exit code 0");
+is($stdout, '', "-q succeeds: no stdout");
+is($stderr, '', "-q succeeds: no stderr");
+
+# Corrupt the PG_VERSION file.
+my $version_pathname = "$backup_path/PG_VERSION";
+my $version_contents = slurp_file($version_pathname);
+open(my $fh, '>', $version_pathname) || die "open $version_pathname: $!";
+print $fh 'q' x length($version_contents);
+close($fh);
+
+# Verify that pg_validatebackup -q now fails.
+command_fails_like(['pg_validatebackup', '-q', $backup_path ],
+ qr/checksum mismatch for file \"PG_VERSION\"/,
+ '-q checksum mismatch');
+
+# Since we didn't change the length of the file, validation should succeed
+# if we ignore checksums. Check that we get the right message, too.
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ '-s skips checksumming');
+
+# Validation should succeed if we ignore the problem file.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/backup successfully verified/,
+ '-i ignores problem file');
+
+# PG_VERSION is already corrupt; let's try also removing all of pg_xact.
+rmtree($backup_path . "/pg_xact");
+
+# We're ignoring the problem with PG_VERSION, but not the problem with
+# pg_xact, so validation should fail here.
+command_fails_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/pg_xact.*is present in the manifest but not on disk/,
+ '-i does not ignore all problems');
+
+# If we use -i twice, we should be able to ignore all of the problems.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', '-i', 'pg_xact',
+ $backup_path ],
+ qr/backup successfully verified/,
+ 'multiple -i options work');
+
+# Verify that when -i is not used, both problems are reported.
+$result = IPC::Run::run ['pg_validatebackup', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "multiple problems: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "multiple problems: missing files reported");
+like($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "multiple problems: checksum mismatch reported");
+
+# Verify that when -e is used, only the problem detected first is reported.
+$result = IPC::Run::run ['pg_validatebackup', '-e', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "-e reports 1 error: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "-e reports 1 error: missing files reported");
+unlike($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "-e reports 1 error: checksum mismatch not reported");
+
+# Test valid manifest with nonexistent backup directory.
+command_fails_like(['pg_validatebackup', '-m', "$backup_path/backup_manifest",
+ "$backup_path/fake" ],
+ qr/could not open directory/,
+ 'nonexistent backup directory');
diff --git a/src/bin/pg_validatebackup/t/005_bad_manifest.pl b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
new file mode 100644
index 0000000000..9c503600d2
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
@@ -0,0 +1,158 @@
+# Test the behavior of pg_validatebackup when the backup manifest has
+# problems.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $tempdir = TestLib::tempdir;
+
+test_bad_manifest('input string ended unexpectedly',
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
+{
+EOM
+
+test_parse_error('unexpected object end', <<EOM);
+{}
+EOM
+
+test_parse_error('unexpected array start', <<EOM);
+[]
+EOM
+
+test_parse_error('expected version indicator', <<EOM);
+{"not-expected": 1}
+EOM
+
+test_parse_error('unexpected manifest version', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": "phooey"}
+EOM
+
+test_parse_error('unexpected scalar', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": true}
+EOM
+
+test_parse_error('expected file list', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Oops": 1}
+EOM
+
+test_parse_error('unexpected object start', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": {}}
+EOM
+
+test_parse_error('missing pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [{}]}
+EOM
+
+test_parse_error('both pathname and encoded pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Encoded-Path": "1234"}
+]}
+EOM
+
+test_parse_error('unexpected file field', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Oops": 1}
+]}
+EOM
+
+test_parse_error('missing size', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x"}
+]}
+EOM
+
+test_parse_error('file size is not an integer', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": "Oops"}
+]}
+EOM
+
+test_parse_error('unable to decode filename', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Encoded-Path": "123", "Size": 0}
+]}
+EOM
+
+test_fatal_error('duplicate pathname in backup manifest', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 0},
+ {"Path": "x", "Size": 0}
+]}
+EOM
+
+test_parse_error('checksum without algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum": "Oops"}
+]}
+EOM
+
+test_fatal_error('unrecognized checksum algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "Oops", "Checksum": "00"}
+]}
+EOM
+
+test_fatal_error('invalid checksum for file', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "CRC32C", "Checksum": "0"}
+]}
+EOM
+
+test_parse_error('expected manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Oops": 1}
+EOM
+
+test_parse_error('expected at least 2 lines', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [], "Manifest-Checksum": null}
+EOM
+
+my $manifest_without_newline = <<EOM;
+{"PostgreSQL-Backup-Manifest-Version": 1,
+ "Files": [],
+ "Manifest-Checksum": null}
+EOM
+chomp($manifest_without_newline);
+test_parse_error('last line not newline-terminated',
+ $manifest_without_newline);
+
+test_fatal_error('invalid manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Manifest-Checksum": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890-"}
+EOM
+
+sub test_parse_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/could not parse backup manifest: $test_name/,
+ $manifest_contents);
+}
+
+sub test_fatal_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/fatal: $test_name/,
+ $manifest_contents);
+}
+
+sub test_bad_manifest
+{
+ my ($test_name, $regexp, $manifest_contents) = @_;
+
+ open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+ print $fh $manifest_contents;
+ close($fh);
+
+ command_fails_like(['pg_validatebackup', $tempdir], $regexp,
+ $test_name);
+}
diff --git a/src/bin/pg_validatebackup/t/006_encoding.pl b/src/bin/pg_validatebackup/t/006_encoding.pl
new file mode 100644
index 0000000000..5e3e7152a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/006_encoding.pl
@@ -0,0 +1,27 @@
+# Verify that pg_validatebackup handles hex-encoded filenames correctly.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 5;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_encoding';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '--manifest-force-encode' ],
+ "backup ok with forced hex encoding");
+
+my $manifest = slurp_file("$backup_path/backup_manifest");
+my $count_of_encoded_path_in_manifest =
+ (() = $manifest =~ /Encoded-Path/mig);
+cmp_ok($count_of_encoded_path_in_manifest, '>', 100,
+ "many paths are encoded in the manifest");
+
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ 'backup with forced encoding validated');
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-21 12:26 ` Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Amit Kapila @ 2020-03-21 12:26 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Sat, Mar 21, 2020 at 4:00 AM Robert Haas <[email protected]> wrote:
>
> On Mon, Mar 16, 2020 at 2:03 AM Suraj Kharage
> <[email protected]> wrote:
> > One more suggestion, recent commit (1933ae62) has added the PostgreSQL home page to --help output.
>
> Good catch. Fixed. I also attempted to address the compiler warning
> you mentioned in your other email.
>
> Also, I realized that the previous patch versions didn't handle the
> hex-encoded path format that we need to use for non-UTF8 filenames,
> and that there was no easy way to test that format. So, in this
> version I added an option to force all pathnames to be encoded in that
> format. I also made that option capable of suppressing the backup
> manifest altogether. Other than that, this version is pretty much the
> same as the last version, except for a few additional test cases which
> I added to get the code coverage up even a little more. It would be
> nice if someone could test whether the tests pass on Windows.
>
On my CentOS, the patch gives below compilation failure:
pg_validatebackup.c: In function ‘parse_manifest_file’:
pg_validatebackup.c:335:19: error: assignment left-hand side might be
a candidate for a format attribute [-Werror=suggest-attribute=format]
context.error_cb = report_manifest_error;
I have tested it on Windows and found there are multiple failures.
The failures are as below:
Test Summary Report
---------------------------------------
t/002_algorithm.pl (Wstat: 512 Tests: 5 Failed: 4)
Failed tests: 2-5
Non-zero exit status: 2
Parse errors: Bad plan. You planned 19 tests but ran 5.
t/003_corruption.pl (Wstat: 256 Tests: 14 Failed: 7)
Failed tests: 2, 4, 6, 8, 10, 12, 14
Non-zero exit status: 1
Parse errors: Bad plan. You planned 44 tests but ran 14.
t/004_options.pl (Wstat: 4352 Tests: 25 Failed: 17)
Failed tests: 2, 4, 6-12, 14-17, 19-20, 22, 25
Non-zero exit status: 17
t/005_bad_manifest.pl (Wstat: 1792 Tests: 44 Failed: 7)
Failed tests: 18, 24, 26, 30, 32, 34, 36
Non-zero exit status: 7
Files=6, Tests=109, 72 wallclock secs ( 0.05 usr + 0.01 sys = 0.06 CPU)
Result: FAIL
Failure Report
------------------------
t/002_algorithm.pl ..... 1/19
# Failed test 'backup ok with algorithm "none"'
# at t/002_algorithm.pl line 33.
# Failed test 'backup manifest exists'
# at t/002_algorithm.pl line 39.
t/002_algorithm.pl ..... 4/19 # Failed test 'validate backup with
algorithm "none"'
# at t/002_algorithm.pl line 53.
# Failed test 'backup ok with algorithm "crc32c"'
# at t/002_algorithm.pl line 33.
# Looks like you planned 19 tests but ran 5.
# Looks like you failed 4 tests of 5 run.
# Looks like your test exited with 2 just after 5.
t/002_algorithm.pl ..... Dubious, test returned 2 (wstat 512, 0x200)
Failed 18/19 subtests
t/003_corruption.pl .... 1/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.
# Failed test 'corrupt backup fails validation: extra_file: matches'
# at t/003_corruption.pl line 117.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:extra_file.*present on disk but not in the manifest)'
t/003_corruption.pl .... 5/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.
t/003_corruption.pl .... 7/44
# Failed test 'corrupt backup fails validation:
extra_tablespace_file: matches'
# at t/003_corruption.pl line 117.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:extra_ts_file.*present on disk but not in the
manifest)'
t/003_corruption.pl .... 9/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.
# Failed test 'corrupt backup fails validation: missing_file: matches'
# at t/003_corruption.pl line 117.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact/0000.*present in the manifest but not on disk)'
t/003_corruption.pl .... 13/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.
# Looks like you planned 44 tests but ran 14.
# Looks like you failed 7 tests of 14 run.
# Looks like your test exited with 1 just after 14.
t/003_corruption.pl .... Dubious, test returned 1 (wstat 256, 0x100)
Failed 37/44 subtests
t/004_options.pl ....... 1/25
# Failed test '-q succeeds: exit code 0'
# at t/004_options.pl line 25.
# Failed test '-q succeeds: no stderr'
# at t/004_options.pl line 27.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''
# Failed test '-q checksum mismatch: matches'
# at t/004_options.pl line 37.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:checksum mismatch for file \"PG_VERSION\")'
t/004_options.pl ....... 7/25
# Failed test '-s skips checksumming: exit code 0'
# at t/004_options.pl line 43.
# Failed test '-s skips checksumming: no stderr'
# at t/004_options.pl line 43.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''
# Failed test '-s skips checksumming: matches'
# at t/004_options.pl line 43.
# ''
# doesn't match '(?^:backup successfully verified)'
# Failed test '-i ignores problem file: exit code 0'
# at t/004_options.pl line 48.
# Failed test '-i ignores problem file: no stderr'
# at t/004_options.pl line 48.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''
# Failed test '-i ignores problem file: matches'
# at t/004_options.pl line 48.
# ''
# doesn't match '(?^:backup successfully verified)'
# Failed test '-i does not ignore all problems: matches'
# at t/004_options.pl line 57.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'
# Failed test 'multiple -i options work: exit code 0'
# at t/004_options.pl line 62.
# Failed test 'multiple -i options work: no stderr'
# at t/004_options.pl line 62.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''
# Failed test 'multiple -i options work: matches'
# at t/004_options.pl line 62.
# ''
# doesn't match '(?^:backup successfully verified)'
# Failed test 'multiple problems: missing files reported'
# at t/004_options.pl line 71.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'
# Failed test 'multiple problems: checksum mismatch reported'
# at t/004_options.pl line 73.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:checksum mismatch for file \"PG_VERSION\")'
# Failed test '-e reports 1 error: missing files reported'
# at t/004_options.pl line 80.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'
# Failed test 'nonexistent backup directory: matches'
# at t/004_options.pl line 86.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not open directory)'
# Looks like you failed 17 tests of 25.
t/004_options.pl ....... Dubious, test returned 17 (wstat 4352, 0x1100)
Failed 17/25 subtests
t/005_bad_manifest.pl .. 1/44
# Failed test 'missing pathname: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: missing size
# '
# doesn't match '(?^:could not parse backup manifest: missing pathname)'
# Failed test 'missing size: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not parse backup manifest: missing size)'
# Failed test 'file size is not an integer: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not parse backup manifest: file size is
not an integer)'
# Failed test 'duplicate pathname in backup manifest: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:fatal: duplicate pathname in backup manifest)'
t/005_bad_manifest.pl .. 31/44
# Failed test 'checksum without algorithm: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not parse backup manifest: checksum
without algorithm)'
# Failed test 'unrecognized checksum algorithm: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:fatal: unrecognized checksum algorithm)'
# Failed test 'invalid checksum for file: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:fatal: invalid checksum for file)'
# Looks like you failed 7 tests of 44.
t/005_bad_manifest.pl .. Dubious, test returned 7 (wstat 1792, 0x700)
Failed 7/44 subtests
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
@ 2020-03-23 11:04 ` Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Amit Kapila @ 2020-03-23 11:04 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Sat, Mar 21, 2020 at 5:56 PM Amit Kapila <[email protected]> wrote:
>
>
> On my CentOS, the patch gives below compilation failure:
> pg_validatebackup.c: In function ‘parse_manifest_file’:
> pg_validatebackup.c:335:19: error: assignment left-hand side might be
> a candidate for a format attribute [-Werror=suggest-attribute=format]
> context.error_cb = report_manifest_error;
>
> I have tested it on Windows and found there are multiple failures.
> The failures are as below:
>
I have started to investigate the failures.
>
> Failure Report
> ------------------------
> t/002_algorithm.pl ..... 1/19
> # Failed test 'backup ok with algorithm "none"'
> # at t/002_algorithm.pl line 33.
>
I checked the log and it was giving error:
/src/bin/pg_validatebackup/tmp_check/t_002_algorithm_master_data/backup/none
--manifest-checksum none --no-sync
\tmp_install\bin\pg_basebackup.EXE: illegal option -- manifest-checksum
It seems the option to be used should be --manifest-checksums. The
attached patch fixes this problem for me.
> t/002_algorithm.pl ..... 4/19 # Failed test 'validate backup with
> algorithm "none"'
> # at t/002_algorithm.pl line 53.
>
The error message for the above failure is:
pg_validatebackup: fatal: could not parse backup manifest: both
pathname and encoded pathname
I don't know at this stage what could cause this? Any pointers?
Attached are logs of failed runs (regression.tar.gz).
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] fix_log_002_algorithm.patch (615B, ../../CAA4eK1+WyCVLZ03ZPH+POAY_NE80p+QxXDUnF10iryHj3vFjTA@mail.gmail.com/2-fix_log_002_algorithm.patch)
download | inline diff:
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
index 2046a709b4..98871e12a5 100644
--- a/src/bin/pg_validatebackup/t/002_algorithm.pl
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -17,7 +17,7 @@ for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
{
my $backup_path = $master->backup_dir . '/' . $algorithm;
my @backup = ('pg_basebackup', '-D', $backup_path,
- '--manifest-checksum', $algorithm,
+ '--manifest-checksums', $algorithm,
'--no-sync');
my @validate = ('pg_validatebackup', '-e', $backup_path);
[application/x-gzip] regression.tar.gz (5.3K, ../../CAA4eK1+WyCVLZ03ZPH+POAY_NE80p+QxXDUnF10iryHj3vFjTA@mail.gmail.com/3-regression.tar.gz)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
@ 2020-03-23 16:15 ` Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 03:43 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 3 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-23 16:15 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 23, 2020 at 7:04 AM Amit Kapila <[email protected]> wrote:
> /src/bin/pg_validatebackup/tmp_check/t_002_algorithm_master_data/backup/none
> --manifest-checksum none --no-sync
> \tmp_install\bin\pg_basebackup.EXE: illegal option -- manifest-checksum
>
> It seems the option to be used should be --manifest-checksums. The
> attached patch fixes this problem for me.
OK, incorporated that.
> > t/002_algorithm.pl ..... 4/19 # Failed test 'validate backup with
> > algorithm "none"'
> > # at t/002_algorithm.pl line 53.
> >
>
> The error message for the above failure is:
> pg_validatebackup: fatal: could not parse backup manifest: both
> pathname and encoded pathname
>
> I don't know at this stage what could cause this? Any pointers?
I think I forgot an initializer. Try this version.
I also incorporated a fix previously proposed by Suraj for the
compiler warning you mentioned in the other email.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v13-0001-Add-checksum-helper-functions.patch (9.6K, ../../CA+TgmoZBrwYvfQ=ymXk669+hgdQxpoypQVYdWiGwpOE_yuMW4A@mail.gmail.com/2-v13-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From 78613b46ebf7602f44b944ff4c6333b455ef647c Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 20 Mar 2020 14:48:33 -0400
Subject: [PATCH v13 1/2] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index ce01df68b9..e199ed7acb 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -47,6 +47,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index f89a8a4fdb..ee04fbafa6 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -120,8 +120,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
- f2s.c file_perm.c hashfn.c ip.c jsonapi.c
+ base64.c checksum_helper.c config_info.c controldata_utils.c d2s.c
+ encnames.c exec.c f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c username.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v13-0002-Generate-backup-manifests-for-base-backups-and-v.patch (117.0K, ../../CA+TgmoZBrwYvfQ=ymXk669+hgdQxpoypQVYdWiGwpOE_yuMW4A@mail.gmail.com/3-v13-0002-Generate-backup-manifests-for-base-backups-and-v.patch)
download | inline diff:
From f4c7e93f761de1a480bd729f6ce37484be3fa41b Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 23 Mar 2020 12:07:20 -0400
Subject: [PATCH v13 2/2] Generate backup manifests for base backups, and
validate them.
A manifest is a JSON document which includes the file name, size, last
modification time, and a checksum for each file backed up, as well as
a checksum for the manifest itself. By default, we use CRC-32C for the
checksum algorithm, because we are trying to detect corruption and
user error, not foil an adversary. However, pg_basebackup and the
server-side BASE_BACKUP command now have options to select the
checksum algorithm, so users wanting a cryptographic hash function can
select SHA-224, SHA-256, SHA-384, or SHA-512. Users not wanting any
checksums at all can disable them, or disable generating of the backup
manifest altogether. Using a cryptographic hash function in place of
CRC-32C consumes significantly more CPU cycles, which may slow down
backups in some cases.
A new tool called pg_validatebackup can validate a backup against the
manifest. If no checksums are present, it can still check that the
right files exist and that they have the expected sizes. If checksums
are present, it can also verify that each file has the expected
checksum. Only plain format backups can be validated directly, but tar
format backups can be validated after extracting them.
Robert Haas, with help, ideas, review, and testing from David Steele,
Stephen Frost, Andrew Dunstan, Rushabh Lathia, Suraj Kharage, Tushar
Ahuja, Rajkumar Raghuwanshi, Mark Dilger, Davinder Singh, Jeevan
Chalke, and Amit Kapila.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
doc/src/sgml/protocol.sgml | 33 +-
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_basebackup.sgml | 63 ++
doc/src/sgml/ref/pg_validatebackup.sgml | 232 ++++++
doc/src/sgml/reference.sgml | 1 +
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 430 +++++++++-
src/backend/replication/repl_gram.y | 13 +
src/backend/replication/repl_scanner.l | 2 +
src/backend/replication/walsender.c | 30 +
src/bin/Makefile | 1 +
src/bin/pg_basebackup/pg_basebackup.c | 184 ++++-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 8 +-
src/bin/pg_validatebackup/.gitignore | 2 +
src/bin/pg_validatebackup/Makefile | 39 +
src/bin/pg_validatebackup/parse_manifest.c | 576 ++++++++++++++
src/bin/pg_validatebackup/parse_manifest.h | 40 +
src/bin/pg_validatebackup/pg_validatebackup.c | 734 ++++++++++++++++++
src/bin/pg_validatebackup/t/001_basic.pl | 30 +
src/bin/pg_validatebackup/t/002_algorithm.pl | 58 ++
src/bin/pg_validatebackup/t/003_corruption.pl | 244 ++++++
src/bin/pg_validatebackup/t/004_options.pl | 89 +++
.../pg_validatebackup/t/005_bad_manifest.pl | 158 ++++
src/bin/pg_validatebackup/t/006_encoding.pl | 27 +
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
26 files changed, 2974 insertions(+), 32 deletions(-)
create mode 100644 doc/src/sgml/ref/pg_validatebackup.sgml
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/parse_manifest.c
create mode 100644 src/bin/pg_validatebackup/parse_manifest.h
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
create mode 100644 src/bin/pg_validatebackup/t/001_basic.pl
create mode 100644 src/bin/pg_validatebackup/t/002_algorithm.pl
create mode 100644 src/bin/pg_validatebackup/t/003_corruption.pl
create mode 100644 src/bin/pg_validatebackup/t/004_options.pl
create mode 100644 src/bin/pg_validatebackup/t/005_bad_manifest.pl
create mode 100644 src/bin/pg_validatebackup/t/006_encoding.pl
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index f139ba0231..d1ff53e8e8 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2466,7 +2466,7 @@ The commands accepted in replication mode are:
</varlistentry>
<varlistentry id="protocol-replication-base-backup" xreflabel="BASE_BACKUP">
- <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ]
+ <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ] [ <literal>MANIFEST</literal> <replaceable>manifest_option</replaceable> ] [ <literal>MANIFEST_CHECKSUMS</literal> <replaceable>checksum_algorithm</replaceable> ]
<indexterm><primary>BASE_BACKUP</primary></indexterm>
</term>
<listitem>
@@ -2576,6 +2576,37 @@ The commands accepted in replication mode are:
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST</literal></term>
+ <listitem>
+ <para>
+ When this option is specified with a value of <literal>ye'</literal>
+ or <literal>force-escape</literal>, a backup manifest is created
+ and sent along with the backup. The latter value forces all filenames
+ to be hex-encoded; otherwise, this type of encoding is performed only
+ for files whose names are non-UTF8 octet sequences.
+ <literal>force-escape</literal> is intended primarily for testing
+ purposes, to be sure that clients which read the backup manifest
+ can handle this case. For compatibility with previous releases,
+ the default is <literal>MANIFEST 'no'</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST_CHECKSUMS</literal></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be used to checksum each file
+ for purposes of the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
<para>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..ab71176cdf 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -211,6 +211,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
<!ENTITY pgRestore SYSTEM "pg_restore.sgml">
<!ENTITY pgRewind SYSTEM "pg_rewind.sgml">
+<!ENTITY pgValidateBackup SYSTEM "pg_validatebackup.sgml">
<!ENTITY pgtestfsync SYSTEM "pgtestfsync.sgml">
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index 90638aad0e..bf6963a595 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -561,6 +561,69 @@ PostgreSQL documentation
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>--no-manifest</option></term>
+ <listitem>
+ <para>
+ Disables generation of a backup manifest. If this option is not
+ specified, the server will and send generate a backup manifest
+ which can be verified using <xref linkend="app-pgvalidatebackup" />.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-force-encode</option></term>
+ <listitem>
+ <para>
+ Forces all filenames in the backup manifest to be hex-encoded.
+ If this option is not specified, only non-UTF8 filenames are
+ hex-encoded. This option is mostly intended to test that tools which
+ read a backup manifest file properly handle this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be used to checksum each file
+ for purposes of the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ <para>
+ If <literal>NONE</literal> is selected, the backup manifest will
+ not contain any checksums. Otherwise, it will contain a checksum
+ of each file in the backup using the specified algorithm. In addition,
+ the manifest itself will always contain a <literal>SHA256</literal>
+ checksum of its own contents. The <literal>SHA</literal> algorithms
+ are significantly more CPU-intensive than <literal>CRC32C</literal>,
+ so selecting one of them may increase the time required to complete
+ the backup.
+ </para>
+ <para>
+ On the other hand, <literal>CRC32C</literal> is not a cryptographic
+ hash function, so it is only suitable for protecting against
+ inadvertent or random modifications to a backup. An adversary
+ who can modify the backup could easily do so in such a way that
+ the CRC does not change, whereas a SHA collision will be hard
+ to manufacture. (However, note that if the attacker also has access
+ to modify the backup manifest itself, no checksum algorithm will
+ provide any protection.) An additional advantage of the
+ <literal>SHA</literal> family of functions is that they output
+ a much larger number of bits.
+ </para>
+ <para>
+ <xref linkend="app-pgvalidatebackup" /> can be used to check the
+ integrity of a backup against the backup manifest.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
new file mode 100644
index 0000000000..1c171f6970
--- /dev/null
+++ b/doc/src/sgml/ref/pg_validatebackup.sgml
@@ -0,0 +1,232 @@
+<!--
+doc/src/sgml/ref/pg_validatebackup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgvalidatebackup">
+ <indexterm zone="app-pgvalidatebackup">
+ <primary>pg_validatebackup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>pg_validatebackup</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_validatebackup</refname>
+ <refpurpose>verify the integrity of a base backup of a
+ <productname>PostgreSQL</productname> cluster</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_validatebackup</command>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>
+ Description
+ </title>
+ <para>
+ <application>pg_validatebackup</application> is used to check the integrity
+ of a database cluster backup. The backup being checked should have been
+ created by <command>pg_basebackup</command> or some other tool that includes
+ a <literal>backup_manifest</literal> file with the backup. The backup
+ must be stored in the "plain" format; a "tar" format backup can be checked
+ after extracting it. Backup manifests are created by the server beginning
+ with <productname>PostgreSQL</productname> version 13, so older backups
+ cannot be validated using this tool.
+ </para>
+
+ <para>
+ <application>pg_validatebackup</application> reads the manifest file of a
+ backup, verifies the manifest against its own internal checksum, and then
+ verifies that the same files are present in the target directory as in the
+ manifest itself. It then verifies that each file has the expected checksum,
+ unless the backup was taken the checksum algorithm set to
+ <literal>none</literal>, in which case checksum verification is not
+ performed. The presence or absence of directories is not checked, except
+ indirectly: if a directory is missing, any files it should have contained
+ will necessarily also be missing. Certain files and directories are
+ excluded from verification:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>backup_manifest</literal> is ignored because the backup
+ manifest is logically not part of the backup and does not include
+ any entry for itself.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>pg_wal</literal> is ignored because WAL files are sent
+ separately from the backup, and are therefore not described by the
+ backup manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>postgesql.auto.conf</literal>,
+ <literal>standby.signal</literal>,
+ and <literal>recovery.signal</literal> are ignored because they may
+ sometimes be created or modified by the backup client itself.
+ (For example, <literal>pg_basebackup -R</literal> will modify
+ <literal>postgresql.auto.conf</literal> and create
+ <literal>standby.signal</literal>.)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ The following command-line options control the behavior.
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-e</option></term>
+ <term><option>--exit-on-error</option></term>
+ <listitem>
+ <para>
+ Exit as soon as a problem with the backup is detected. If this option
+ is not specified, <literal>pg_basebackup</literal> will continue
+ checking the backup even after a problem has been detected, and will
+ report all problems detected as errors.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-i <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--ignore=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Ignore the specified file or directory, which should be expressed
+ as a relative pathname. If the backup contains extra files, is
+ missing files, or has files that have been modified as compared with
+ what is described in the manifest, this option can be used to suppress
+ the errors that would otherwise occur. If a directory is specified,
+ this option affects the entire subtree rooted at that location.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-m <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--manifest-path=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Use the manifest file at the specified path, rather than one located
+ in the root of the backup directory.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-q</option></term>
+ <term><option>--quiet</option></term>
+ <listitem>
+ <para>
+ Don't print anything when a backup is successfully validated.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-s</option></term>
+ <term><option>--skip-checksums</option></term>
+ <listitem>
+ <para>
+ Do not validate checksums. The presence or absence of files and the
+ sizes of those files will still be checked. This is much faster,
+ because the files themselves do not need to read.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other options are also available:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_validatebackup</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_validatebackup</application> command
+ line arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal> and
+ validate the integrity of the backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal>, move
+ the manifest somewhere outside the backup directory, and validate the
+ backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234</userinput>
+<prompt>$</prompt> <userinput>mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234</userinput>
+</screen>
+ </para>
+
+ <para>
+ To validate a backup while ignoring a file that was added manually to the
+ backup directory, and also skipping checksum verification:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>edit /usr/local/pgsql/data/note.to.self</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="app-pgbasebackup"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..d25a77b13c 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -255,6 +255,7 @@
&pgReceivewal;
&pgRecvlogical;
&pgRestore;
+ &pgValidateBackup;
&psqlRef;
&reindexdb;
&vacuumdb;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 793c076da6..b3917bc526 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10551,7 +10551,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 806d013108..6dffc6ef5b 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -39,10 +41,19 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
+typedef enum manifest_option
+{
+ MANIFEST_OPTION_YES,
+ MANIFEST_OPTION_NO,
+ MANIFEST_OPTION_FORCE_ENCODE
+} manifest_option;
+
typedef struct
{
const char *label;
@@ -52,20 +63,43 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ manifest_option manifest;
+ pg_checksum_type manifest_checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ bool force_encode;
+ bool first_file;
+ bool still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest,
+ basebackup_options *opt);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +136,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -251,6 +295,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -258,12 +303,17 @@ perform_base_backup(basebackup_options *opt)
backup_streamed = 0;
pgstat_progress_start_command(PROGRESS_COMMAND_BASEBACKUP, InvalidOid);
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt);
total_checksum_failures = 0;
@@ -301,7 +351,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -380,7 +433,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -388,11 +442,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -400,10 +457,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -632,7 +690,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -655,16 +713,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -678,6 +740,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -709,8 +774,13 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->manifest = MANIFEST_OPTION_NO;
+ opt->manifest_checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -797,12 +867,61 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest") == 0)
+ {
+ char *optval = strVal(defel->arg);
+ bool manifest_bool;
+
+ if (o_manifest)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (parse_bool(optval, &manifest_bool))
+ {
+ if (manifest_bool)
+ opt->manifest = MANIFEST_OPTION_YES;
+ else
+ opt->manifest = MANIFEST_OPTION_NO;
+ }
+ else if (pg_strcasecmp(optval, "force-encode") == 0)
+ opt->manifest = MANIFEST_OPTION_FORCE_ENCODE;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized manifest option: \"%s\"",
+ optval)));
+ o_manifest = true;
+ }
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval,
+ &opt->manifest_checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
}
if (opt->label == NULL)
opt->label = "base backup";
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ {
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("manifest checksums require a backup manifest")));
+ opt->manifest_checksum_type = CHECKSUM_TYPE_NONE;
+ }
}
@@ -918,6 +1037,249 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, basebackup_options *opt)
+{
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ manifest->buffile = NULL;
+ else
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = opt->manifest_checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->force_encode = (opt->manifest == MANIFEST_OPTION_FORCE_ENCODE);
+ manifest->first_file = true;
+ manifest->still_checksumming = true;
+
+ if (opt->manifest != MANIFEST_OPTION_NO)
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ Assert(manifest != NULL);
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ int pathlen;
+ StringInfoData buf;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (!manifest->force_encode &&
+ pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
+
+ /*
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
+ */
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
+
+ /* Add checksum information. */
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
+ }
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
+
+ /* Avoid leaking memory. */
+ pfree(buf.data);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\"}\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -978,11 +1340,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1017,6 +1383,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1027,7 +1397,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1060,7 +1431,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1079,7 +1451,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1359,7 +1731,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1367,7 +1740,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1437,8 +1811,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1455,6 +1830,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1625,6 +2003,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1649,6 +2030,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1657,7 +2039,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1682,6 +2065,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..f93a0de218 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,8 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -156,6 +158,7 @@ var_name: IDENT { $$ = $1; }
/*
* BASE_BACKUP [LABEL '<label>'] [PROGRESS] [FAST] [WAL] [NOWAIT]
* [MAX_RATE %d] [TABLESPACE_MAP] [NOVERIFY_CHECKSUMS]
+ * [MANIFEST %s] [MANIFEST_CHECKSUMS %s]
*/
base_backup:
K_BASE_BACKUP base_backup_opt_list
@@ -214,6 +217,16 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST SCONST
+ {
+ $$ = makeDefElem("manifest",
+ (Node *)makeString($2), -1);
+ }
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..452ad9fc27 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,8 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST { return K_MANIFEST; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 76ec3c7dd0..3b117d8367 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index c5d95958b2..f355eb612c 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -136,6 +142,9 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static bool manifest = true;
+static bool manifest_force_encode = false;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -181,6 +190,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -388,6 +403,11 @@ usage(void)
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
printf(_(" --no-estimate-size do not estimate backup size in server side\n"));
+ printf(_(" --no-manifest suppress generation of backup manifest\n"));
+ printf(_(" --manifest-force-encode\n"
+ " hex encode all filenames in manifest\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1186,6 +1206,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1657,6 +1702,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1667,6 +1770,8 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_clause;
+ char *manifest_checksums_clause = "";
int i;
char xlogstart[64];
char xlogend[64];
@@ -1674,6 +1779,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1727,6 +1833,32 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest)
+ {
+ if (serverMajor < 1300)
+ {
+ const char *serverver = PQparameterStatus(conn, "server_version");
+
+ pg_log_error("backup manifests are not supported by server version %s",
+ serverver ? serverver : "'unknown'");
+ exit(1);
+ }
+
+ if (manifest_force_encode)
+ manifest_clause = "MANIFEST 'force-encode'";
+ else
+ manifest_clause = "MANIFEST 'yes'";
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
+ }
+ else
+ {
+ if (serverMajor < 1300)
+ manifest_clause = "";
+ else
+ manifest_clause = "MANIFEST 'no'";
+ }
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1741,7 +1873,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s %s",
escaped_label,
estimatesize ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1749,7 +1881,9 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_clause,
+ manifest_checksums_clause);
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1837,7 +1971,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1866,6 +2001,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && manifest)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2069,6 +2217,9 @@ main(int argc, char **argv)
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
{"no-estimate-size", no_argument, NULL, 4},
+ {"no-manifest", no_argument, NULL, 5},
+ {"manifest-force-encode", no_argument, NULL, 6},
+ {"manifest-checksums", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
int c;
@@ -2096,7 +2247,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2240,6 +2391,15 @@ main(int argc, char **argv)
case 4:
estimatesize = false;
break;
+ case 5:
+ manifest = false;
+ break;
+ case 6:
+ manifest_force_encode = true;
+ break;
+ case 7:
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
@@ -2370,6 +2530,22 @@ main(int argc, char **argv)
exit(1);
}
+ if (!manifest && manifest_checksums != NULL)
+ {
+ pg_log_error("--no-manifest and --manifest-checksums are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (!manifest && manifest_force_encode)
+ {
+ pg_log_error("--no-manifest and --manifest-force-encode are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
/* connection in replication mode to server */
conn = GetConnection();
if (!conn)
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..63381764e9 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 109;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -104,6 +104,7 @@ foreach my $filename (@tempRelationFiles)
$node->command_ok([ 'pg_basebackup', '-D', "$tempdir/backup", '-X', 'none' ],
'pg_basebackup runs');
ok(-f "$tempdir/backup/PG_VERSION", 'backup was created');
+ok(-f "$tempdir/backup/backup_manifest", 'backup manifest included');
# Permissions on backup should be default
SKIP:
@@ -160,11 +161,12 @@ rmtree("$tempdir/backup");
$node->command_ok(
[
- 'pg_basebackup', '-D', "$tempdir/backup2", '--waldir',
- "$tempdir/xlog2"
+ 'pg_basebackup', '-D', "$tempdir/backup2", '--no-manifest',
+ '--waldir', "$tempdir/xlog2"
],
'separate xlog directory');
ok(-f "$tempdir/backup2/PG_VERSION", 'backup was created');
+ok(! -f "$tempdir/backup2/backup_manifest", 'manifest was suppressed');
ok(-d "$tempdir/xlog2/", 'xlog directory was created');
rmtree("$tempdir/backup2");
rmtree("$tempdir/xlog2");
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..21e0a92429
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1,2 @@
+/pg_validatebackup
+/tmp_check/
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..04ef7d3051
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,39 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ parse_manifest.o \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
new file mode 100644
index 0000000000..e6b42adfda
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -0,0 +1,576 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.c
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "parse_manifest.h"
+#include "common/jsonapi.h"
+
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_ENCODED_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+/*
+ * Internal state used while decoding the JSON-format backup manifest.
+ */
+typedef struct
+{
+ JsonManifestParseContext *context;
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ char *pathname;
+ char *encoded_pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void verify_manifest_checksum(JsonManifestParseState *parse,
+ char *buffer, size_t size);
+static void json_manifest_parse_failure(JsonManifestParseContext *context,
+ char *msg);
+
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+
+/*
+ * Main entrypoint to parse a JSON-format backup manifest.
+ *
+ * Caller should set up the parsing context and then invoke this function.
+ * For each file whose information is extracted from the manifest,
+ * context->perfile_cb is invoked. In case of trouble, context->error_cb is
+ * invoked and is expected not to return.
+ */
+void
+json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ size_t size)
+{
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
+
+ /* Set up our private parsing context. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.context = context;
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+ /* Validate the checksum. */
+ verify_manifest_checksum(&parse, buffer, size);
+}
+
+/*
+ * Invoked at the start of each object in the JSON document.
+ *
+ * The document as a whole is expected to be an object with three keys
+ * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
+ * file is expected to be an object with various keys (Path, Size, etc.).
+ * If we're not at the beginning of either the toplevel object or the object
+ * for a particular file, it's an error.
+ */
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->encoded_pathname = NULL;
+ parse->size = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each object in the JSON document.
+ *
+ * The possible cases here are the same as for json_manifest_object_start.
+ * There's nothing special to do at the end of the document, but when we
+ * reach the end of an object representing a particular file, we must call
+ * json_manifest_finalize_file() to save the associated details.
+ */
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ json_manifest_finalize_file(parse);
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each array in the JSON document.
+ *
+ * Within the toplevel object, the value associated with the "Files" key
+ * should be an array. No other arrays are expected.
+ */
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each array in the JSON document.
+ *
+ * Just like json_manifest_array_start, there's only one expected case
+ * here.
+ */
+static void
+json_manifest_array_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each object field in the JSON document.
+ */
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_FIELD:
+ /* Inside toplevel object, expecting version indicator. */
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ /* Inside toplevel object, expecting "Files" next. */
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Encoded-Path") == 0)
+ parse->field = JMFF_ENCODED_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure(parse->context,
+ "unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ /* Inside toplevel object, expecting "Manifest-Checksum" next. */
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object field");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each scalar in the JSON document.
+ *
+ * Object field names don't reach this code; those are handled by
+ * json_manifest_object_field_start. When we're inside of the object for
+ * a particular file, that function will have noticed the name of the field,
+ * and we'll get the corresponding value here. When we're in the toplevel
+ * object, the parse state itself tells us which field this is.
+ *
+ * In all cases except for PostgreSQL-Backup-Manifest-Version, which we
+ * can just check on the spot, the goal here is just to save the value in
+ * the parse state for later use. We don't actually do anything until we
+ * reach either the end of the object representing this file, or the end
+ * of the manifest, as the case may be.
+ */
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure(parse->context,
+ "unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_ENCODED_PATH:
+ parse->encoded_pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context, "unexpected scalar");
+ break;
+ }
+}
+
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * file, and invoke the per-file callback so that the caller gets those
+ * details. This happens for each file when the corresponding JSON object is
+ * completely parsed.
+ */
+static void
+json_manifest_finalize_file(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t size;
+ char *ep;
+ int checksum_string_length;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL && parse->encoded_pathname == NULL)
+ json_manifest_parse_failure(parse->context, "missing pathname");
+ if (parse->pathname != NULL && parse->encoded_pathname != NULL)
+ json_manifest_parse_failure(parse->context,
+ "both pathname and encoded pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure(parse->context, "missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure(parse->context,
+ "checksum without algorithm");
+
+ /* Decode encoded pathname, if that's what we have. */
+ if (parse->encoded_pathname != NULL)
+ {
+ int encoded_length = strlen(parse->encoded_pathname);
+ int raw_length = encoded_length / 2;
+
+ parse->pathname = palloc(raw_length + 1);
+ if (encoded_length % 2 != 0 ||
+ !hexdecode_string((uint8 *) parse->pathname,
+ parse->encoded_pathname,
+ raw_length))
+ json_manifest_parse_failure(parse->context,
+ "unable to decode filename");
+ parse->pathname[raw_length] = '\0';
+ pfree(parse->encoded_pathname);
+ parse->encoded_pathname = NULL;
+ }
+
+ /* Parse size. */
+ size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm, &checksum_type))
+ context->error_cb(context, "unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ checksum_length = 0;
+ checksum_payload = NULL;
+ }
+ else
+ {
+ checksum_length = checksum_string_length / 2;
+ checksum_payload = palloc(checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(checksum_payload, parse->checksum,
+ checksum_length))
+ context->error_cb(context,
+ "invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, parse->checksum);
+ }
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perfile_cb(context, parse->pathname, size,
+ checksum_type, checksum_length, checksum_payload);
+
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
+}
+
+/*
+ * Verify that the manifest checksum is correct.
+ *
+ * The last line of the manifest file is excluded from the manifest checksum,
+ * because the last line is expected to contain the checksum that covers
+ * the rest of the file.
+ */
+static void
+verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
+ size_t size)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t i;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+
+ /* Find the last two newlines in the file. */
+ for (i = 0; i < size; ++i)
+ {
+ if (buffer[i] == '\n')
+ {
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
+ }
+ }
+
+ /*
+ * Make sure that the last newline is right at the end, and that there are
+ * at least two lines total. We need this to be true in order for the
+ * following code, which computes the manifest checksum, to work properly.
+ */
+ if (number_of_newlines < 2)
+ json_manifest_parse_failure(parse->context,
+ "expected at least 2 lines");
+ if (ultimate_newline != size - 1)
+ json_manifest_parse_failure(parse->context,
+ "last line not newline-terminated");
+
+ /* Checksum the rest. */
+ pg_sha256_init(&manifest_ctx);
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+
+ /* Now verify it. */
+ if (parse->manifest_checksum == NULL)
+ context->error_cb(parse->context, "manifest has no checksum");
+ if (strlen(parse->manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse->manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ context->error_cb(context, "invalid manifest checksum: \"%s\"",
+ parse->manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ context->error_cb(context, "manifest checksum mismatch");
+}
+
+/*
+ * Report a parse error.
+ *
+ * This is intended to be used for fairly low-level failures that probably
+ * shouldn't occur unless somebody has deliberately constructed a bad manifest,
+ * or unless the server is generating bad manifests due to some bug. msg should
+ * be a short string giving some hint as to what the problem is.
+ */
+static void
+json_manifest_parse_failure(JsonManifestParseContext *context, char *msg)
+{
+ context->error_cb(context, "could not parse backup manifest: %s", msg);
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
new file mode 100644
index 0000000000..25d140f72f
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -0,0 +1,40 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.h
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PARSE_MANIFEST_H
+#define PARSE_MANIFEST_H
+
+#include "common/checksum_helper.h"
+#include "mb/pg_wchar.h"
+
+struct JsonManifestParseContext;
+typedef struct JsonManifestParseContext JsonManifestParseContext;
+
+typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
+ char *pathname,
+ size_t size, pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
+ char *fmt, ...) pg_attribute_printf(2, 3);
+
+struct JsonManifestParseContext
+{
+ void *private_data;
+ json_manifest_perfile_callback perfile_cb;
+ json_manifest_error_callback error_cb;
+};
+
+extern void json_parse_manifest(JsonManifestParseContext *context,
+ char *buffer, size_t size);
+
+#endif
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..eb1473d9d0
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,734 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+#include "parse_manifest.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+
+static void record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length,
+ uint8 *checksum_payload);
+static void report_manifest_error(JsonManifestParseContext *context,
+ char *fmt, ...)
+ pg_attribute_printf(2, 3) pg_attribute_noreturn();
+
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void report_backup_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void report_fatal_error(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ printf("backup successfully verified\n");
+
+ return context.saw_any_error ? 1 : 0;
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ int rc;
+ JsonManifestParseContext context;
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ report_fatal_error("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ report_fatal_error("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /*
+ * Slurp in the whole file.
+ *
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
+ */
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
+ {
+ if (rc < 0)
+ report_fatal_error("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ report_fatal_error("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
+
+ /* Close the manifest file. */
+ close(fd);
+
+ /* Parse the manifest as JSON. */
+ context.private_data = ht;
+ context.perfile_cb = record_manifest_details_for_file;
+ context.error_cb = report_manifest_error;
+ json_parse_manifest(&context, buffer, statbuf.st_size);
+
+ /* Done with the buffer. */
+ pfree(buffer);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * Report an error while parsing the manifest.
+ *
+ * We consider all such errors to be fatal errors. The manifest parser
+ * expects this function not to return.
+ */
+static void
+report_manifest_error(JsonManifestParseContext *context, char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Record details extracted from the backup manifest for one file.
+ */
+static void
+record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload)
+{
+ manifestfiles_hash *ht = context->private_data;
+ manifestfile *tabent;
+ bool found;
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(ht, pathname, &found);
+ if (found)
+ report_fatal_error("duplicate pathname in backup manifest: \"%s\"",
+ pathname);
+
+ /* Initialize the entry. */
+ tabent->size = size;
+ tabent->checksum_type = checksum_type;
+ tabent->checksum_length = checksum_length;
+ tabent->checksum_payload = checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ report_fatal_error("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ report_backup_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ report_backup_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ report_backup_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ report_backup_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ report_backup_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ report_backup_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ report_backup_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ report_backup_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ report_backup_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ report_backup_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ report_backup_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ report_backup_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Report a problem with the backup.
+ *
+ * Update the context to indicate that we saw an error, and exit if the
+ * context says we should.
+ */
+static void
+report_backup_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+report_fatal_error(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+ printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
+}
diff --git a/src/bin/pg_validatebackup/t/001_basic.pl b/src/bin/pg_validatebackup/t/001_basic.pl
new file mode 100644
index 0000000000..6d4b8ea01a
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/001_basic.pl
@@ -0,0 +1,30 @@
+use strict;
+use warnings;
+use TestLib;
+use Test::More tests => 16;
+
+my $tempdir = TestLib::tempdir;
+
+program_help_ok('pg_validatebackup');
+program_version_ok('pg_validatebackup');
+program_options_handling_ok('pg_validatebackup');
+
+command_fails_like(['pg_validatebackup'],
+ qr/no backup directory specified/,
+ 'target directory must be specified');
+command_fails_like(['pg_validatebackup', $tempdir],
+ qr/could not open file.*\/backup_manifest\"/,
+ 'pg_validatebackup requires a manifest');
+command_fails_like(['pg_validatebackup', $tempdir, $tempdir],
+ qr/too many command-line arguments/,
+ 'multiple target directories not allowed');
+
+# create fake manifest file
+open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+close($fh);
+
+# but then try to use an alternate, nonexisting manifest
+command_fails_like(['pg_validatebackup', '-m', "$tempdir/not_the_manifest",
+ $tempdir],
+ qr/could not open file.*\/not_the_manifest\"/,
+ 'pg_validatebackup respects -m flag');
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
new file mode 100644
index 0000000000..98871e12a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -0,0 +1,58 @@
+# Verify that we can take and validate backups with various checksum types.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 19;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
+{
+ my $backup_path = $master->backup_dir . '/' . $algorithm;
+ my @backup = ('pg_basebackup', '-D', $backup_path,
+ '--manifest-checksums', $algorithm,
+ '--no-sync');
+ my @validate = ('pg_validatebackup', '-e', $backup_path);
+
+ # A backup with a bogus algorithm should fail.
+ if ($algorithm eq 'bogus')
+ {
+ $master->command_fails(\@backup,
+ "backup fails with algorithm \"$algorithm\"");
+ next;
+ }
+
+ # A backup with a valid algorithm should work.
+ $master->command_ok(\@backup, "backup ok with algorithm \"$algorithm\"");
+
+ # We expect each real checksum algorithm to be mentioned on every line of
+ # the backup manifest file except the first and last; for simplicity, we
+ # just check that it shows up lots of times. When the checksum algorithm
+ # is none, we just check that the manifest exists.
+ if ($algorithm eq 'none')
+ {
+ ok(-f "$backup_path/backup_manifest", "backup manifest exists");
+ }
+ else
+ {
+ my $manifest = slurp_file("$backup_path/backup_manifest");
+ my $count_of_algorithm_in_manifest =
+ (() = $manifest =~ /$algorithm/mig);
+ cmp_ok($count_of_algorithm_in_manifest, '>', 100,
+ "$algorithm is mentioned many times in the manifest");
+ }
+
+ # Make sure that it validates OK.
+ $master->command_ok(\@validate,
+ "validate backup with algorithm \"$algorithm\"");
+
+ # Remove backup immediately to save disk space.
+ rmtree($backup_path);
+}
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
new file mode 100644
index 0000000000..45e2220c38
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -0,0 +1,244 @@
+# Verify that various forms of corruption are detected by pg_validatebackup.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+# Include a user-defined tablespace in the hopes of detecting problems in that
+# area.
+my $source_ts_path = TestLib::tempdir;
+$master->safe_psql('postgres', <<EOM);
+CREATE TABLE x1 (a int);
+INSERT INTO x1 VALUES (111);
+CREATE TABLESPACE ts1 LOCATION '$source_ts_path';
+CREATE TABLE x2 (a int) TABLESPACE ts1;
+INSERT INTO x1 VALUES (222);
+EOM
+
+my @scenario = (
+ {
+ 'name' => 'extra_file',
+ 'mutilate' => \&mutilate_extra_file,
+ 'fails_like' =>
+ qr/extra_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'extra_tablespace_file',
+ 'mutilate' => \&mutilate_extra_tablespace_file,
+ 'fails_like' =>
+ qr/extra_ts_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'missing_file',
+ 'mutilate' => \&mutilate_missing_file,
+ 'fails_like' =>
+ qr/pg_xact\/0000.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'missing_tablespace',
+ 'mutilate' => \&mutilate_missing_tablespace,
+ 'fails_like' =>
+ qr/pg_tblspc.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'append_to_file',
+ 'mutilate' => \&mutilate_append_to_file,
+ 'fails_like' =>
+ qr/has size \d+ on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'truncate_file',
+ 'mutilate' => \&mutilate_truncate_file,
+ 'fails_like' =>
+ qr/has size 0 on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'replace_file',
+ 'mutilate' => \&mutilate_replace_file,
+ 'fails_like' => qr/checksum mismatch for file/
+ },
+ {
+ 'name' => 'bad_manifest',
+ 'mutilate' => \&mutilate_bad_manifest,
+ 'fails_like' => qr/manifest checksum mismatch/
+ },
+ {
+ 'name' => 'open_file_fails',
+ 'mutilate' => \&mutilate_open_file_fails,
+ 'fails_like' => qr/could not open file/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'open_directory_fails',
+ 'mutilate' => \&mutilate_open_directory_fails,
+ 'fails_like' => qr/could not open directory/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'search_directory_fails',
+ 'mutilate' => \&mutilate_search_directory_fails,
+ 'cleanup' => \&cleanup_search_directory_fails,
+ 'fails_like' => qr/could not stat file or directory/,
+ 'skip_on_windows' => 1
+ }
+);
+
+for my $scenario (@scenario)
+{
+ my $name = $scenario->{'name'};
+
+ SKIP:
+ {
+ skip "unix-style permissions not supported on Windows", 4
+ if $scenario->{'skip_on_windows'} && $windows_os;
+
+ # Take a backup and check that it validates OK.
+ my $backup_path = $master->backup_dir . '/' . $name;
+ my $backup_ts_path = TestLib::tempdir;
+ $master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '-T', "${source_ts_path}=${backup_ts_path}"],
+ "base backup ok");
+ command_ok(['pg_validatebackup', $backup_path ],
+ "intact backup validated");
+
+ # Mutilate the backup in some way.
+ $scenario->{'mutilate'}->($backup_path);
+
+ # Now check that the backup no longer validates.
+ command_fails_like(['pg_validatebackup', $backup_path ],
+ $scenario->{'fails_like'},
+ "corrupt backup fails validation: $name");
+
+ # Run cleanup hook, if provided.
+ $scenario->{'cleanup'}->($backup_path)
+ if exists $scenario->{'cleanup'};
+
+ # Finally, use rmtree to reclaim space.
+ rmtree($backup_path);
+ }
+}
+
+sub create_extra_file
+{
+ my ($backup_path, $relative_path) = @_;
+ my $pathname = "$backup_path/$relative_path";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh "This is an extra file.\n";
+ close($fh);
+}
+
+# Add a file into the root directory of the backup.
+sub mutilate_extra_file
+{
+ my ($backup_path) = @_;
+ create_extra_file($backup_path, "extra_file");
+}
+
+# Add a file inside the user-defined tablespace.
+sub mutilate_extra_tablespace_file
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my ($catvdir) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid");
+ my ($tsdboid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid/$catvdir");
+ create_extra_file($backup_path,
+ "pg_tblspc/$tsoid/$catvdir/$tsdboid/extra_ts_file");
+}
+
+# Remove a file.
+sub mutilate_missing_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_xact/0000";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Remove the symlink to the user-defined tablespace.
+sub mutilate_missing_tablespace
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my $pathname = "$backup_path/pg_tblspc/$tsoid";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Append an additional bytes to a file.
+sub mutilate_append_to_file
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/global/pg_control", 'x';
+}
+
+# Truncate a file to zero length.
+sub mutilate_truncate_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/global/pg_control";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ close($fh);
+}
+
+# Replace a file's contents without changing the length of the file. This is
+# not a particularly efficient way to do this, so we pick a file that's
+# expected to be short.
+sub mutilate_replace_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ my $contents = slurp_file($pathname);
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh 'q' x length($contents);
+ close($fh);
+}
+
+# Corrupt the backup manifest.
+sub mutilate_bad_manifest
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/backup_manifest", "\n";
+}
+
+# Create a file that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_file_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_subtrans";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be searched. (This is skipped on Windows.)
+sub mutilate_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0400, $pathname) || die "chmod $pathname: $!";
+}
+
+# rmtree can't cope with a mode 400 directory, so change back to 700.
+sub cleanup_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0700, $pathname) || die "chmod $pathname: $!";
+}
diff --git a/src/bin/pg_validatebackup/t/004_options.pl b/src/bin/pg_validatebackup/t/004_options.pl
new file mode 100644
index 0000000000..8f185626ed
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/004_options.pl
@@ -0,0 +1,89 @@
+# Verify the behavior of assorted pg_validatebackup options.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 25;
+
+# Start up the server and take a backup.
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_options';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync' ],
+ "base backup ok");
+
+# Verify that pg_validatebackup -q succeeds and produces no output.
+my $stdout;
+my $stderr;
+my $result = IPC::Run::run ['pg_validatebackup', '-q', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok($result, "-q succeeds: exit code 0");
+is($stdout, '', "-q succeeds: no stdout");
+is($stderr, '', "-q succeeds: no stderr");
+
+# Corrupt the PG_VERSION file.
+my $version_pathname = "$backup_path/PG_VERSION";
+my $version_contents = slurp_file($version_pathname);
+open(my $fh, '>', $version_pathname) || die "open $version_pathname: $!";
+print $fh 'q' x length($version_contents);
+close($fh);
+
+# Verify that pg_validatebackup -q now fails.
+command_fails_like(['pg_validatebackup', '-q', $backup_path ],
+ qr/checksum mismatch for file \"PG_VERSION\"/,
+ '-q checksum mismatch');
+
+# Since we didn't change the length of the file, validation should succeed
+# if we ignore checksums. Check that we get the right message, too.
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ '-s skips checksumming');
+
+# Validation should succeed if we ignore the problem file.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/backup successfully verified/,
+ '-i ignores problem file');
+
+# PG_VERSION is already corrupt; let's try also removing all of pg_xact.
+rmtree($backup_path . "/pg_xact");
+
+# We're ignoring the problem with PG_VERSION, but not the problem with
+# pg_xact, so validation should fail here.
+command_fails_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/pg_xact.*is present in the manifest but not on disk/,
+ '-i does not ignore all problems');
+
+# If we use -i twice, we should be able to ignore all of the problems.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', '-i', 'pg_xact',
+ $backup_path ],
+ qr/backup successfully verified/,
+ 'multiple -i options work');
+
+# Verify that when -i is not used, both problems are reported.
+$result = IPC::Run::run ['pg_validatebackup', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "multiple problems: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "multiple problems: missing files reported");
+like($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "multiple problems: checksum mismatch reported");
+
+# Verify that when -e is used, only the problem detected first is reported.
+$result = IPC::Run::run ['pg_validatebackup', '-e', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "-e reports 1 error: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "-e reports 1 error: missing files reported");
+unlike($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "-e reports 1 error: checksum mismatch not reported");
+
+# Test valid manifest with nonexistent backup directory.
+command_fails_like(['pg_validatebackup', '-m', "$backup_path/backup_manifest",
+ "$backup_path/fake" ],
+ qr/could not open directory/,
+ 'nonexistent backup directory');
diff --git a/src/bin/pg_validatebackup/t/005_bad_manifest.pl b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
new file mode 100644
index 0000000000..9c503600d2
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
@@ -0,0 +1,158 @@
+# Test the behavior of pg_validatebackup when the backup manifest has
+# problems.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $tempdir = TestLib::tempdir;
+
+test_bad_manifest('input string ended unexpectedly',
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
+{
+EOM
+
+test_parse_error('unexpected object end', <<EOM);
+{}
+EOM
+
+test_parse_error('unexpected array start', <<EOM);
+[]
+EOM
+
+test_parse_error('expected version indicator', <<EOM);
+{"not-expected": 1}
+EOM
+
+test_parse_error('unexpected manifest version', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": "phooey"}
+EOM
+
+test_parse_error('unexpected scalar', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": true}
+EOM
+
+test_parse_error('expected file list', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Oops": 1}
+EOM
+
+test_parse_error('unexpected object start', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": {}}
+EOM
+
+test_parse_error('missing pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [{}]}
+EOM
+
+test_parse_error('both pathname and encoded pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Encoded-Path": "1234"}
+]}
+EOM
+
+test_parse_error('unexpected file field', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Oops": 1}
+]}
+EOM
+
+test_parse_error('missing size', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x"}
+]}
+EOM
+
+test_parse_error('file size is not an integer', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": "Oops"}
+]}
+EOM
+
+test_parse_error('unable to decode filename', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Encoded-Path": "123", "Size": 0}
+]}
+EOM
+
+test_fatal_error('duplicate pathname in backup manifest', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 0},
+ {"Path": "x", "Size": 0}
+]}
+EOM
+
+test_parse_error('checksum without algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum": "Oops"}
+]}
+EOM
+
+test_fatal_error('unrecognized checksum algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "Oops", "Checksum": "00"}
+]}
+EOM
+
+test_fatal_error('invalid checksum for file', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "CRC32C", "Checksum": "0"}
+]}
+EOM
+
+test_parse_error('expected manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Oops": 1}
+EOM
+
+test_parse_error('expected at least 2 lines', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [], "Manifest-Checksum": null}
+EOM
+
+my $manifest_without_newline = <<EOM;
+{"PostgreSQL-Backup-Manifest-Version": 1,
+ "Files": [],
+ "Manifest-Checksum": null}
+EOM
+chomp($manifest_without_newline);
+test_parse_error('last line not newline-terminated',
+ $manifest_without_newline);
+
+test_fatal_error('invalid manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Manifest-Checksum": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890-"}
+EOM
+
+sub test_parse_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/could not parse backup manifest: $test_name/,
+ $manifest_contents);
+}
+
+sub test_fatal_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/fatal: $test_name/,
+ $manifest_contents);
+}
+
+sub test_bad_manifest
+{
+ my ($test_name, $regexp, $manifest_contents) = @_;
+
+ open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+ print $fh $manifest_contents;
+ close($fh);
+
+ command_fails_like(['pg_validatebackup', $tempdir], $regexp,
+ $test_name);
+}
diff --git a/src/bin/pg_validatebackup/t/006_encoding.pl b/src/bin/pg_validatebackup/t/006_encoding.pl
new file mode 100644
index 0000000000..5e3e7152a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/006_encoding.pl
@@ -0,0 +1,27 @@
+# Verify that pg_validatebackup handles hex-encoded filenames correctly.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 5;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_encoding';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '--manifest-force-encode' ],
+ "backup ok with forced hex encoding");
+
+my $manifest = slurp_file("$backup_path/backup_manifest");
+my $count_of_encoded_path_in_manifest =
+ (() = $manifest =~ /Encoded-Path/mig);
+cmp_ok($count_of_encoded_path_in_manifest, '>', 100,
+ "many paths are encoded in the manifest");
+
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ 'backup with forced encoding validated');
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-23 22:42 ` Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-23 22:42 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> I think I forgot an initializer. Try this version.
Just took a quick look through this. I'm pretty sure David wants to
look at it too. Anyway, some comments below.
> diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
> index f139ba0231..d1ff53e8e8 100644
> --- a/doc/src/sgml/protocol.sgml
> +++ b/doc/src/sgml/protocol.sgml
> @@ -2466,7 +2466,7 @@ The commands accepted in replication mode are:
> </varlistentry>
>
> <varlistentry id="protocol-replication-base-backup" xreflabel="BASE_BACKUP">
> - <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ]
> + <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ] [ <literal>MANIFEST</literal> <replaceable>manifest_option</replaceable> ] [ <literal>MANIFEST_CHECKSUMS</literal> <replaceable>checksum_algorithm</replaceable> ]
> <indexterm><primary>BASE_BACKUP</primary></indexterm>
> </term>
> <listitem>
> @@ -2576,6 +2576,37 @@ The commands accepted in replication mode are:
> </para>
> </listitem>
> </varlistentry>
> +
> + <varlistentry>
> + <term><literal>MANIFEST</literal></term>
> + <listitem>
> + <para>
> + When this option is specified with a value of <literal>ye'</literal>
> + or <literal>force-escape</literal>, a backup manifest is created
> + and sent along with the backup. The latter value forces all filenames
> + to be hex-encoded; otherwise, this type of encoding is performed only
> + for files whose names are non-UTF8 octet sequences.
> + <literal>force-escape</literal> is intended primarily for testing
> + purposes, to be sure that clients which read the backup manifest
> + can handle this case. For compatibility with previous releases,
> + the default is <literal>MANIFEST 'no'</literal>.
> + </para>
> + </listitem>
> + </varlistentry>
> +
> + <varlistentry>
> + <term><literal>MANIFEST_CHECKSUMS</literal></term>
> + <listitem>
> + <para>
> + Specifies the algorithm that should be used to checksum each file
> + for purposes of the backup manifest. Currently, the available
> + algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
> + <literal>SHA224</literal>, <literal>SHA256</literal>,
> + <literal>SHA384</literal>, and <literal>SHA512</literal>.
> + The default is <literal>CRC32C</literal>.
> + </para>
> + </listitem>
> + </varlistentry>
> </variablelist>
> </para>
> <para>
While I get the desire to have a default here that includes checksums,
the way the command is structured, it strikes me as odd that the lack of
MANIFEST_CHECKSUMS in the command actually results in checksums being
included. I would think that we'd either:
- have the lack of MANIFEST_CHECKSUMS mean 'No checksums'
or
- Require MANIFEST_CHECKSUMS to be specified and not have it be optional
We aren't expecting people to actually be typing these commands out and
so I don't think it's a *huge* deal to have it the way you've written
it, but it still strikes me as odd. I don't think I have a real
preference between the two options that I suggest above, maybe very
slightly in favor of the first.
> diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
> index 90638aad0e..bf6963a595 100644
> --- a/doc/src/sgml/ref/pg_basebackup.sgml
> +++ b/doc/src/sgml/ref/pg_basebackup.sgml
> @@ -561,6 +561,69 @@ PostgreSQL documentation
> </para>
> </listitem>
> </varlistentry>
> +
> + <varlistentry>
> + <term><option>--no-manifest</option></term>
> + <listitem>
> + <para>
> + Disables generation of a backup manifest. If this option is not
> + specified, the server will and send generate a backup manifest
> + which can be verified using <xref linkend="app-pgvalidatebackup" />.
> + </para>
> + </listitem>
> + </varlistentry>
How about "If this option is not specified, the server will generate and
send a backup manifest which can be verified using ..."
> + <varlistentry>
> + <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
> + <listitem>
> + <para>
> + Specifies the algorithm that should be used to checksum each file
> + for purposes of the backup manifest. Currently, the available
> + algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
> + <literal>SHA224</literal>, <literal>SHA256</literal>,
> + <literal>SHA384</literal>, and <literal>SHA512</literal>.
> + The default is <literal>CRC32C</literal>.
> + </para>
As I recall, there was an invitation to argue about the defaults at one
point, and so I'm going to say here that I would advocate for a
different default than 'crc32c'. Specifically, I would think sha256 or
512 would be better. I don't recall seeing a debate about this that
conclusively found crc32c to be better, but I'm happy to go back and
reread anything someone wants to point me at.
> + <para>
> + If <literal>NONE</literal> is selected, the backup manifest will
> + not contain any checksums. Otherwise, it will contain a checksum
> + of each file in the backup using the specified algorithm. In addition,
> + the manifest itself will always contain a <literal>SHA256</literal>
> + checksum of its own contents. The <literal>SHA</literal> algorithms
> + are significantly more CPU-intensive than <literal>CRC32C</literal>,
> + so selecting one of them may increase the time required to complete
> + the backup.
> + </para>
It also seems a bit silly to me that using the defaults means having to
deal with two different algorithms- crc32c and sha256. Considering how
fast these algorithms are, compared to everything else involved in a
backup (particularly one that's likely going across a network...), I
wonder if we should say "may slightly increase" above.
> + <para>
> + On the other hand, <literal>CRC32C</literal> is not a cryptographic
> + hash function, so it is only suitable for protecting against
> + inadvertent or random modifications to a backup. An adversary
> + who can modify the backup could easily do so in such a way that
> + the CRC does not change, whereas a SHA collision will be hard
> + to manufacture. (However, note that if the attacker also has access
> + to modify the backup manifest itself, no checksum algorithm will
> + provide any protection.) An additional advantage of the
> + <literal>SHA</literal> family of functions is that they output
> + a much larger number of bits.
> + </para>
I'm not really sure that this paragraph is sensible to include.. We
certainly don't talk about adversaries and cryptographic hash functions
when we talk about our page-level checksums, for example. I'm not
completely against including it, but I don't want to give the impression
that this is something we routinely consider or that lack of discussion
elsewhere implies we have protections against a determined attacker.
> diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
> new file mode 100644
> index 0000000000..1c171f6970
> --- /dev/null
> +++ b/doc/src/sgml/ref/pg_validatebackup.sgml
> @@ -0,0 +1,232 @@
> +<!--
> +doc/src/sgml/ref/pg_validatebackup.sgml
> +PostgreSQL documentation
> +-->
> +
> +<refentry id="app-pgvalidatebackup">
> + <indexterm zone="app-pgvalidatebackup">
> + <primary>pg_validatebackup</primary>
> + </indexterm>
> +
> + <refmeta>
> + <refentrytitle>pg_validatebackup</refentrytitle>
> + <manvolnum>1</manvolnum>
> + <refmiscinfo>Application</refmiscinfo>
> + </refmeta>
> +
> + <refnamediv>
> + <refname>pg_validatebackup</refname>
> + <refpurpose>verify the integrity of a base backup of a
> + <productname>PostgreSQL</productname> cluster</refpurpose>
> + </refnamediv>
"verify the integrity of a backup taken using pg_basebackup"
> + <refsect1>
> + <title>
> + Description
> + </title>
> + <para>
> + <application>pg_validatebackup</application> is used to check the integrity
> + of a database cluster backup. The backup being checked should have been
> + created by <command>pg_basebackup</command> or some other tool that includes
> + a <literal>backup_manifest</literal> file with the backup. The backup
> + must be stored in the "plain" format; a "tar" format backup can be checked
> + after extracting it. Backup manifests are created by the server beginning
> + with <productname>PostgreSQL</productname> version 13, so older backups
> + cannot be validated using this tool.
> + </para>
This seems to invite the idea that pg_validatebackup should be able to
work with external backup solutions- but I'm a bit concerned by that
idea because it seems like it would then mean we'd have to be
particularly careful when changing things in this area, and I'm not
thrilled by that. I'd like to make sure that new versions of
pg_validatebackup work with older backups, and, ideally, older versions
of pg_validatebackup would work even with newer backups, all of which I
think the json structure of the manifest helps us with, but that's when
we're building the manifest and know what it's going to look like.
Maybe to put it another way- would a patch be accepted to make
pg_validatebackup work with other manifests..? If not, then I'd keep
this to the more specific "this tool is used to validate backups taken
using pg_basebackup".
> + <para>
> + <application>pg_validatebackup</application> reads the manifest file of a
> + backup, verifies the manifest against its own internal checksum, and then
> + verifies that the same files are present in the target directory as in the
> + manifest itself. It then verifies that each file has the expected checksum,
> + unless the backup was taken the checksum algorithm set to
"was taken with the checksum algorithm"...
> + <literal>none</literal>, in which case checksum verification is not
> + performed. The presence or absence of directories is not checked, except
> + indirectly: if a directory is missing, any files it should have contained
> + will necessarily also be missing. Certain files and directories are
> + excluded from verification:
> + </para>
> +
> + <itemizedlist>
> + <listitem>
> + <para>
> + <literal>backup_manifest</literal> is ignored because the backup
> + manifest is logically not part of the backup and does not include
> + any entry for itself.
> + </para>
> + </listitem>
This seems a bit confusing, doesn't it? The backup_manifest must exist,
and its checksum is internal, and is checked, isn't it? Why say that
it's excluded..?
> + <listitem>
> + <para>
> + <literal>pg_wal</literal> is ignored because WAL files are sent
> + separately from the backup, and are therefore not described by the
> + backup manifest.
> + </para>
> + </listitem>
I don't agree with the choice to exclude the WAL files, considering
they're an integral part of a backup, to exclude them means that if
they've been corrupted at all then the entire backup is invalid. You
don't want to be discovering that when you're trying to do a restore of
a backup that you took with pg_basebackup and which pg_validatebackup
says is valid. After all, the tool being used here, pg_basebackup,
*does* also stream the WAL files- there's no reason why we can't
calculate a checksum on them and store that checksum somewhere and use
it to validate the WAL files. This, in my opinion, is actually a
show-stopper for this feature. Claiming it's a valid backup when we
don't check the absolutely necessary-for-restore WAL is making a false
claim, no matter how well it's documented.
I do understand that it's possible to run pg_basebackup without the WAL
files being grabbed as part of that run- in such a case, we should be
able to detect that was the case for the backup and when running
pg_validatebackup we should issue a WARNING that the WAL files weren't
able to be verified (we could have an option to suppress that warning if
people feel that's needed).
> + <listitem>
> + <para>
> + <literal>postgesql.auto.conf</literal>,
> + <literal>standby.signal</literal>,
> + and <literal>recovery.signal</literal> are ignored because they may
> + sometimes be created or modified by the backup client itself.
> + (For example, <literal>pg_basebackup -R</literal> will modify
> + <literal>postgresql.auto.conf</literal> and create
> + <literal>standby.signal</literal>.)
> + </para>
> + </listitem>
> + </itemizedlist>
> + </refsect1>
Not really thrilled with this (pg_basebackup certainly could figure out
the checksum for those files...), but I also don't think it's a huge
issue as they can be recreated by a user (unlike a WAL file..).
I got through most of the pg_basebackup changes, and they looked pretty
good in general. Will try to review more tomorrow.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-24 18:04 ` Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-24 18:04 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 23, 2020 at 6:42 PM Stephen Frost <[email protected]> wrote:
> While I get the desire to have a default here that includes checksums,
> the way the command is structured, it strikes me as odd that the lack of
> MANIFEST_CHECKSUMS in the command actually results in checksums being
> included.
I don't think that's quite accurate, because the default for the
MANIFEST option is 'no', so the actual default if you say nothing
about manifests at all, you don't get one. However, it is true that if
you ask for a manifest and you don't specify the type of checksums,
you get CRC-32C. We could change it so that if you ask for a manifest
you must also specify the type of checksum, but I don't see any
advantage in that approach. Nothing prevents the client from
specifying the value if it cares, but making the default "I don't
care, you pick" seems pretty sensible. It could be really helpful if,
for example, we decide to remove the initial default in a future
release for some reason. Then the client just keeps working without
needing to change anything, but anyone who explicitly specified the
old default gets an error.
> > + Disables generation of a backup manifest. If this option is not
> > + specified, the server will and send generate a backup manifest
> > + which can be verified using <xref linkend="app-pgvalidatebackup" />.
> > + </para>
> > + </listitem>
> > + </varlistentry>
>
> How about "If this option is not specified, the server will generate and
> send a backup manifest which can be verified using ..."
Good suggestion. :-)
> > + <varlistentry>
> > + <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
> > + <listitem>
> > + <para>
> > + Specifies the algorithm that should be used to checksum each file
> > + for purposes of the backup manifest. Currently, the available
> > + algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
> > + <literal>SHA224</literal>, <literal>SHA256</literal>,
> > + <literal>SHA384</literal>, and <literal>SHA512</literal>.
> > + The default is <literal>CRC32C</literal>.
> > + </para>
>
> As I recall, there was an invitation to argue about the defaults at one
> point, and so I'm going to say here that I would advocate for a
> different default than 'crc32c'. Specifically, I would think sha256 or
> 512 would be better. I don't recall seeing a debate about this that
> conclusively found crc32c to be better, but I'm happy to go back and
> reread anything someone wants to point me at.
It was discussed upthread. Andrew Dunstan argued that there was no
reason to use a cryptographic checksum here and that we shouldn't do
so gratuitously. Suraj Kharage found that CRC-32C has very little
performance impact but that any of the SHA functions slow down backups
considerably. David Steele pointed out that you'd need a better
checksum if you wanted to use it for purposes such as delta restore,
with which I agree, but that's not the design center for this feature.
I concluded that different people wanted different things, so that we
ought to make this configurable, but that CRC-32C is a good default.
It has approximately a 99.9999999767169% chance of detecting a random
error, which is pretty good, and it doesn't drastically slow down
backups, which is also good.
> It also seems a bit silly to me that using the defaults means having to
> deal with two different algorithms- crc32c and sha256. Considering how
> fast these algorithms are, compared to everything else involved in a
> backup (particularly one that's likely going across a network...), I
> wonder if we should say "may slightly increase" above.
Actually, Suraj's results upthread show that it's a pretty big hit.
> > + <para>
> > + On the other hand, <literal>CRC32C</literal> is not a cryptographic
> > + hash function, so it is only suitable for protecting against
> > + inadvertent or random modifications to a backup. An adversary
> > + who can modify the backup could easily do so in such a way that
> > + the CRC does not change, whereas a SHA collision will be hard
> > + to manufacture. (However, note that if the attacker also has access
> > + to modify the backup manifest itself, no checksum algorithm will
> > + provide any protection.) An additional advantage of the
> > + <literal>SHA</literal> family of functions is that they output
> > + a much larger number of bits.
> > + </para>
>
> I'm not really sure that this paragraph is sensible to include.. We
> certainly don't talk about adversaries and cryptographic hash functions
> when we talk about our page-level checksums, for example. I'm not
> completely against including it, but I don't want to give the impression
> that this is something we routinely consider or that lack of discussion
> elsewhere implies we have protections against a determined attacker.
Given the skepticism from some quarters about CRC-32C on this thread,
I didn't want to oversell it. Also, I do think that these things are
possibly things that we should consider more widely. I agree with
Andrew's complaint that it's far too easy to just throw SHA<lots> at
problems that don't really require it without any actually good
reason. Spelling out our reasons for choosing certain algorithms for
certain purposes seems like a good habit to get into, and if we
haven't done it in other places, maybe we should. On the other hand,
while I'm inclined to keep this paragraph, I won't lose much sleep if
we decide to remove it.
> > + <refnamediv>
> > + <refname>pg_validatebackup</refname>
> > + <refpurpose>verify the integrity of a base backup of a
> > + <productname>PostgreSQL</productname> cluster</refpurpose>
> > + </refnamediv>
>
> "verify the integrity of a backup taken using pg_basebackup"
OK.
> This seems to invite the idea that pg_validatebackup should be able to
> work with external backup solutions- but I'm a bit concerned by that
> idea because it seems like it would then mean we'd have to be
> particularly careful when changing things in this area, and I'm not
> thrilled by that. I'd like to make sure that new versions of
> pg_validatebackup work with older backups, and, ideally, older versions
> of pg_validatebackup would work even with newer backups, all of which I
> think the json structure of the manifest helps us with, but that's when
> we're building the manifest and know what it's going to look like.
Both you and David made forceful arguments that this needed to be JSON
rather than an ad-hoc text format precisely so that other tools could
parse it more easily, and I just spent *a lot* of time making the JSON
parsing stuff work precisely so that you could have that. This project
would've been done a month ago if not for that. I don't care all that
much whether we remove the mention here, but the idea that using JSON
was so that pg_validatebackup could manage compatibility issues is
just not correct. The version number on line 1 of the file was more
than sufficient for that purpose.
> > + <para>
> > + <application>pg_validatebackup</application> reads the manifest file of a
> > + backup, verifies the manifest against its own internal checksum, and then
> > + verifies that the same files are present in the target directory as in the
> > + manifest itself. It then verifies that each file has the expected checksum,
> > + unless the backup was taken the checksum algorithm set to
>
> "was taken with the checksum algorithm"...
Oops. Will fix.
> > + <itemizedlist>
> > + <listitem>
> > + <para>
> > + <literal>backup_manifest</literal> is ignored because the backup
> > + manifest is logically not part of the backup and does not include
> > + any entry for itself.
> > + </para>
> > + </listitem>
>
> This seems a bit confusing, doesn't it? The backup_manifest must exist,
> and its checksum is internal, and is checked, isn't it? Why say that
> it's excluded..?
Well, there's no entry in the backup manifest for backup_manifest
itself. Normally, the presence of a file not mentioned in
backup_manifest would cause a complaint about an extra file, but
because backup_manifest is in the ignore list, it doesn't.
> > + <listitem>
> > + <para>
> > + <literal>pg_wal</literal> is ignored because WAL files are sent
> > + separately from the backup, and are therefore not described by the
> > + backup manifest.
> > + </para>
> > + </listitem>
>
> I don't agree with the choice to exclude the WAL files, considering
> they're an integral part of a backup, to exclude them means that if
> they've been corrupted at all then the entire backup is invalid. You
> don't want to be discovering that when you're trying to do a restore of
> a backup that you took with pg_basebackup and which pg_validatebackup
> says is valid. After all, the tool being used here, pg_basebackup,
> *does* also stream the WAL files- there's no reason why we can't
> calculate a checksum on them and store that checksum somewhere and use
> it to validate the WAL files. This, in my opinion, is actually a
> show-stopper for this feature. Claiming it's a valid backup when we
> don't check the absolutely necessary-for-restore WAL is making a false
> claim, no matter how well it's documented.
The default for pg_basebackup is -Xstream, which means that the WAL
files are being sent over a separate connection that has no connection
to the original session. The server, when generating the backup
manifest, has no idea what WAL files are being sent over that separate
connection, and thus cannot include them in the manifest. This problem
could be "solved" by having the client generate the manifest rather
than the server, but I think that cure would be worse than the
disease. As it stands, the manifest provides some protection against
transmission errors, which would be lost with that design. As you
point out, this clearly can't be done with -Xnone. I think it would be
possible to support this with -Xfetch, but we'd have to have the
manifest itself specify whether or not it included files in pg_wal,
which would require complicating the format a bit. I don't think that
makes sense. I assume -Xstream is the most commonly-used mode, because
the default used to be -Xfetch and we changed it, which I think we
would not have done unless people liked -Xstream significantly better.
Adding complexity to cater to a non-default case which I suspect is
not widely used doesn't really make sense to me.
In the future, we might want to consider improvements which could make
validation of pg_wal feasible in common cases. Specifically, suppose
that pg_basebackup could receive the manifest from the server, keep
all the entries for the existing files just as they are, but add
entries for WAL files and anything else it may have added to the
backup, recompute the manifest checksum, and store the resulting
revised manifest with the backup. That, I think, would be fairly cool,
but it's a significant body of additional development work, and this
is already quite a large patch. The patch itself has grown to about
3000 lines, and has already 10 preparatory commits doing another ~1500
lines of refactoring to prepare for it.
> Not really thrilled with this (pg_basebackup certainly could figure out
> the checksum for those files...), but I also don't think it's a huge
> issue as they can be recreated by a user (unlike a WAL file..).
Yeah, same issues, though. Here again, there are several possible
fixes: (1) make the server modify those files rather than letting
pg_basebackup do it; (2) make the client compute the manifest rather
than the server; (3) have the client revise the manifest. (3) makes
most sense to me, but I think that it would be better to return to
that topic at a later date. This is certainly not a perfect feature as
things stand but I believe it is good enough to provide significant
benefits.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-25 13:31 ` Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-25 13:31 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Mon, Mar 23, 2020 at 6:42 PM Stephen Frost <[email protected]> wrote:
> > While I get the desire to have a default here that includes checksums,
> > the way the command is structured, it strikes me as odd that the lack of
> > MANIFEST_CHECKSUMS in the command actually results in checksums being
> > included.
>
> I don't think that's quite accurate, because the default for the
> MANIFEST option is 'no', so the actual default if you say nothing
> about manifests at all, you don't get one. However, it is true that if
> you ask for a manifest and you don't specify the type of checksums,
> you get CRC-32C. We could change it so that if you ask for a manifest
> you must also specify the type of checksum, but I don't see any
> advantage in that approach. Nothing prevents the client from
> specifying the value if it cares, but making the default "I don't
> care, you pick" seems pretty sensible. It could be really helpful if,
> for example, we decide to remove the initial default in a future
> release for some reason. Then the client just keeps working without
> needing to change anything, but anyone who explicitly specified the
> old default gets an error.
I get that the default for manifest is 'no', but I don't really see how
that means that the lack of saying anything about checksums should mean
"give me crc32c checksums". It's really rather common that if we don't
specify something, it means don't do that thing- like an 'ORDER BY'
clause. We aren't designing SQL here, so I'm not going to get terribly
upset if you push forward with "if you don't want checksums, you have to
explicitly say MANIFEST_CHECKSUMS no", but I don't agree with the
reasoning here.
> > > + <varlistentry>
> > > + <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
> > > + <listitem>
> > > + <para>
> > > + Specifies the algorithm that should be used to checksum each file
> > > + for purposes of the backup manifest. Currently, the available
> > > + algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
> > > + <literal>SHA224</literal>, <literal>SHA256</literal>,
> > > + <literal>SHA384</literal>, and <literal>SHA512</literal>.
> > > + The default is <literal>CRC32C</literal>.
> > > + </para>
> >
> > As I recall, there was an invitation to argue about the defaults at one
> > point, and so I'm going to say here that I would advocate for a
> > different default than 'crc32c'. Specifically, I would think sha256 or
> > 512 would be better. I don't recall seeing a debate about this that
> > conclusively found crc32c to be better, but I'm happy to go back and
> > reread anything someone wants to point me at.
>
> It was discussed upthread. Andrew Dunstan argued that there was no
> reason to use a cryptographic checksum here and that we shouldn't do
> so gratuitously. Suraj Kharage found that CRC-32C has very little
> performance impact but that any of the SHA functions slow down backups
> considerably. David Steele pointed out that you'd need a better
> checksum if you wanted to use it for purposes such as delta restore,
> with which I agree, but that's not the design center for this feature.
> I concluded that different people wanted different things, so that we
> ought to make this configurable, but that CRC-32C is a good default.
> It has approximately a 99.9999999767169% chance of detecting a random
> error, which is pretty good, and it doesn't drastically slow down
> backups, which is also good.
There were also comments made up-thread about how it might not be great
for larger (eg: 1GB files, like we tend to have quite a few of...), and
something about it being a 40 year old algorithm.. Having re-read some
of the discussion, I'm actually more inclined to say we should be using
sha256 instead of crc32c.
> > It also seems a bit silly to me that using the defaults means having to
> > deal with two different algorithms- crc32c and sha256. Considering how
> > fast these algorithms are, compared to everything else involved in a
> > backup (particularly one that's likely going across a network...), I
> > wonder if we should say "may slightly increase" above.
>
> Actually, Suraj's results upthread show that it's a pretty big hit.
So, I went back and re-read part of the thread and looked at the
(seemingly, only one..?) post regarding timing and didn't understand
what, exactly, was being timed there, because I didn't see the actual
commands/script/whatever that was used to get those results included.
I'm sure that sha256 takes a lot more time than crc32c, I'm certainly
not trying to dispute that, but what's relevent here is how much it
impacts the time required to run the overall backup (including sync'ing
it to disk, and possibly network transmission time.. if we're just
comparing the time to run it through memory then, sure, the sha256
computation time might end up being quite a bit of the time, but that's
not really that interesting of a test..).
> > > + <para>
> > > + On the other hand, <literal>CRC32C</literal> is not a cryptographic
> > > + hash function, so it is only suitable for protecting against
> > > + inadvertent or random modifications to a backup. An adversary
> > > + who can modify the backup could easily do so in such a way that
> > > + the CRC does not change, whereas a SHA collision will be hard
> > > + to manufacture. (However, note that if the attacker also has access
> > > + to modify the backup manifest itself, no checksum algorithm will
> > > + provide any protection.) An additional advantage of the
> > > + <literal>SHA</literal> family of functions is that they output
> > > + a much larger number of bits.
> > > + </para>
> >
> > I'm not really sure that this paragraph is sensible to include.. We
> > certainly don't talk about adversaries and cryptographic hash functions
> > when we talk about our page-level checksums, for example. I'm not
> > completely against including it, but I don't want to give the impression
> > that this is something we routinely consider or that lack of discussion
> > elsewhere implies we have protections against a determined attacker.
>
> Given the skepticism from some quarters about CRC-32C on this thread,
> I didn't want to oversell it. Also, I do think that these things are
> possibly things that we should consider more widely. I agree with
> Andrew's complaint that it's far too easy to just throw SHA<lots> at
> problems that don't really require it without any actually good
> reason. Spelling out our reasons for choosing certain algorithms for
> certain purposes seems like a good habit to get into, and if we
> haven't done it in other places, maybe we should. On the other hand,
> while I'm inclined to keep this paragraph, I won't lose much sleep if
> we decide to remove it.
I don't mind spelling out reasoning for certain algorithms over others,
in general, this just seems a bit much. I'm not sure we need to be
going into what being a cryptographic hash function means every time we
talk about any hash or checksum. Those who actually care about
cryptographic hash function usage really don't need someone to explain
to them that crc32c isn't cryptographically secure. The last sentence
also seems kind of odd (why is a much larger number of bits, alone, an
advantage..?).
I tried to figure out a way to rewrite this and I feel like I keep
ending up coming back to something like "CRC32C is a CRC, not a hash"
and that kind of truism just doesn't feel terribly useful to include in
our documentation.
Maybe:
"Using a SHA hash function provides a cryptographically secure digest
of each file for users who wish to verify that the backup has not been
tampered with, while the CRC32C algorithm provides a checksum which is
much faster to calculate and good at catching errors due to accidental
changes but is not resistent to targeted modifications. Note that, to
be useful against an adversary who has access to the backup, the backup
manifest would need to be stored securely elsewhere or otherwise
verified to have not been modified since the backup was taken."
This at least talks about things in a positive direction (SHA hash
functions do this, CRC32C does that) rather than in a negative tone.
> > This seems to invite the idea that pg_validatebackup should be able to
> > work with external backup solutions- but I'm a bit concerned by that
> > idea because it seems like it would then mean we'd have to be
> > particularly careful when changing things in this area, and I'm not
> > thrilled by that. I'd like to make sure that new versions of
> > pg_validatebackup work with older backups, and, ideally, older versions
> > of pg_validatebackup would work even with newer backups, all of which I
> > think the json structure of the manifest helps us with, but that's when
> > we're building the manifest and know what it's going to look like.
>
> Both you and David made forceful arguments that this needed to be JSON
> rather than an ad-hoc text format precisely so that other tools could
> parse it more easily, and I just spent *a lot* of time making the JSON
> parsing stuff work precisely so that you could have that. This project
> would've been done a month ago if not for that. I don't care all that
> much whether we remove the mention here, but the idea that using JSON
> was so that pg_validatebackup could manage compatibility issues is
> just not correct. The version number on line 1 of the file was more
> than sufficient for that purpose.
I stand by the decision that the manifest should be in JSON, but that's
what is produced by the backend server as part of a base backup, which
is quite likely going to be used by some external tools, and isn't at
all the same as the external pg_validatebackup command that the
discussion here is about. I also did make the argument up-thread,
though I'll admit that it seemed to be mostly ignored, but I make it
still, that a simple version number sucks and using JSON does avoid some
of the downsides from it. Particularly, I'd love to see a v13
pg_validatebackup able to work with a v14 pg_basebackup, even if that
v14 pg_basebackup added some extra stuff to the manifest. That's
possible to do with a generic structure like JSON and not something that
a simple version number would allow. Yes, I admit that we might change
the structure or the contents in a way where that wouldn't be possible
and I'm not going to raise a fuss if we do so, but this approach gives
us more options.
Anyway, my point here was really just that *pg_validatebackup* is about
validating backups taken with pg_basebackup. While it's possible that
it could be used for backups taken with other tools, I don't think
that's really part of its actual mandate or that we're going to actively
work to add such support in the future.
> > > + <itemizedlist>
> > > + <listitem>
> > > + <para>
> > > + <literal>backup_manifest</literal> is ignored because the backup
> > > + manifest is logically not part of the backup and does not include
> > > + any entry for itself.
> > > + </para>
> > > + </listitem>
> >
> > This seems a bit confusing, doesn't it? The backup_manifest must exist,
> > and its checksum is internal, and is checked, isn't it? Why say that
> > it's excluded..?
>
> Well, there's no entry in the backup manifest for backup_manifest
> itself. Normally, the presence of a file not mentioned in
> backup_manifest would cause a complaint about an extra file, but
> because backup_manifest is in the ignore list, it doesn't.
Yes, I get why it's excluded from the manifest and why we have code to
avoid complaining about it being an extra file, but this is
documentation and, in this part of the docs, we seem to be saying that
we're not checking/validating the manifest, and that's certainly not
actually true.
In particular, the sentence right above this list is:
"Certain files and directories are excluded from verification:"
but we actually do verify the manifest, that's all I'm saying here.
Maybe rewording that a bit is what would help, say:
"Certain files and directories are not included in the manifest:"
then have the entry for backup_manifest be something like:
"backup_manifest is not included as it is the manifest itself and is not
logically part of the backup; backup_manifest is checked using its own
internal validation digest" or something along those lines.
> > > + <listitem>
> > > + <para>
> > > + <literal>pg_wal</literal> is ignored because WAL files are sent
> > > + separately from the backup, and are therefore not described by the
> > > + backup manifest.
> > > + </para>
> > > + </listitem>
> >
> > I don't agree with the choice to exclude the WAL files, considering
> > they're an integral part of a backup, to exclude them means that if
> > they've been corrupted at all then the entire backup is invalid. You
> > don't want to be discovering that when you're trying to do a restore of
> > a backup that you took with pg_basebackup and which pg_validatebackup
> > says is valid. After all, the tool being used here, pg_basebackup,
> > *does* also stream the WAL files- there's no reason why we can't
> > calculate a checksum on them and store that checksum somewhere and use
> > it to validate the WAL files. This, in my opinion, is actually a
> > show-stopper for this feature. Claiming it's a valid backup when we
> > don't check the absolutely necessary-for-restore WAL is making a false
> > claim, no matter how well it's documented.
>
> The default for pg_basebackup is -Xstream, which means that the WAL
> files are being sent over a separate connection that has no connection
> to the original session. The server, when generating the backup
> manifest, has no idea what WAL files are being sent over that separate
> connection, and thus cannot include them in the manifest. This problem
> could be "solved" by having the client generate the manifest rather
> than the server, but I think that cure would be worse than the
> disease. As it stands, the manifest provides some protection against
> transmission errors, which would be lost with that design. As you
> point out, this clearly can't be done with -Xnone. I think it would be
> possible to support this with -Xfetch, but we'd have to have the
> manifest itself specify whether or not it included files in pg_wal,
> which would require complicating the format a bit. I don't think that
> makes sense. I assume -Xstream is the most commonly-used mode, because
> the default used to be -Xfetch and we changed it, which I think we
> would not have done unless people liked -Xstream significantly better.
> Adding complexity to cater to a non-default case which I suspect is
> not widely used doesn't really make sense to me.
Yeah, I get that it's not easy to figure out how to validate the WAL,
but I stand by my opinion that it's simply not acceptable to exclude the
necessary WAL from verification so and to claim that a backup is valid
when we haven't checked the WAL.
I agree that -Xfetch isn't commonly used and only supporting validation
of WAL when that's used isn't a good answer.
> In the future, we might want to consider improvements which could make
> validation of pg_wal feasible in common cases. Specifically, suppose
> that pg_basebackup could receive the manifest from the server, keep
> all the entries for the existing files just as they are, but add
> entries for WAL files and anything else it may have added to the
> backup, recompute the manifest checksum, and store the resulting
> revised manifest with the backup. That, I think, would be fairly cool,
> but it's a significant body of additional development work, and this
> is already quite a large patch. The patch itself has grown to about
> 3000 lines, and has already 10 preparatory commits doing another ~1500
> lines of refactoring to prepare for it.
Having the client calculate the checksums for the WAL and add them to
the manifest is one approach and could work, but there's others-
- Have the WAL checksums be calculated during the base backup and kept
somewhere, and then included in the manifest sent by the server- the
backup_manifest is the last thing we send anyway, isn't it? And
surely at the end of the backup we actually do know all of the WAL
that's needed for the backup to be valid, because we pass that
information to pg_basebackup to construct the necessary backup_label
file.
- Validate the WAL using its own internal checksums instead of having
the manifest involved at all. That's not ideal since we wouldn't have
cryptographically secure digests for the WAL, but at least we will
have validated it and raised the chances that the backup will be able
to actually be restored using PG a whole bunch.
- With the 'checksum none' option, we aren't really validating contents
of anything, so in that case it'd actually be alright to simply scan
the WAL and make sure that we've at least got all of the WAL files
needed to go from the start of the backup to the end. I don't think
just checking that the WAL files exist is a proper solution when it
comes to a backup where the user has asked for checksums to be
included though. I will say that I'm really very surprised that
pg_validatebackup wasn't already checking that we at least had the WAL
that is needed, but I don't see any code for that.
> > Not really thrilled with this (pg_basebackup certainly could figure out
> > the checksum for those files...), but I also don't think it's a huge
> > issue as they can be recreated by a user (unlike a WAL file..).
>
> Yeah, same issues, though. Here again, there are several possible
> fixes: (1) make the server modify those files rather than letting
> pg_basebackup do it; (2) make the client compute the manifest rather
> than the server; (3) have the client revise the manifest. (3) makes
> most sense to me, but I think that it would be better to return to
> that topic at a later date. This is certainly not a perfect feature as
> things stand but I believe it is good enough to provide significant
> benefits.
As I said, I don't consider these files to be as much of an issue and
therefore excluding them and documenting that we do would be alright. I
don't feel that's an acceptable option for the WAL though.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-25 16:50 ` Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-25 16:50 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Wed, Mar 25, 2020 at 9:31 AM Stephen Frost <[email protected]> wrote:
> I get that the default for manifest is 'no', but I don't really see how
> that means that the lack of saying anything about checksums should mean
> "give me crc32c checksums". It's really rather common that if we don't
> specify something, it means don't do that thing- like an 'ORDER BY'
> clause.
That's a fair argument, but I think the other relevant principle is
that we try to give people useful defaults for things. I think that
checksums are a sufficiently useful thing that having the default be
not to do it doesn't make sense. I had the impression that you and
David were in agreement on that point, actually.
> There were also comments made up-thread about how it might not be great
> for larger (eg: 1GB files, like we tend to have quite a few of...), and
> something about it being a 40 year old algorithm..
Well, the 512MB "limit" for CRC-32C means only that for certain very
specific types of errors, detection is not guaranteed above that file
size. So if you have a single flipped bit, for example, and the file
size is greater than 512MB, then CRC-32C has only a 99.9999999767169%
chance of detecting the error, whereas if the file size is less than
512MB, it is 100% certain, because of the design of the algorithm. But
nine nines is plenty, and neither SHA nor our page-level checksums
provide guaranteed error detection properties anyway.
I'm not sure why the fact that it's a 40-year-old algorithm is
relevant. There are many 40-year-old algorithms that are very good.
Generally, if we discover that we're using bad 40-year-old algorithms,
like Knuth's tape sorting stuff, we eventually figure out how to
replace them with something else that's better. But there's no reason
to retire an algorithm simply because it's old. I have not heard
anyone say, for example, that we should stop using CRC-32C for XLOG
checksums. We continue to use it for that purpose because it (1) is
highly likely to detect any errors and (2) is very fast. Those are the
same reasons why I think it's a good fit for this case.
My guess is that if this patch is adopted as currently proposed, we
will eventually need to replace the cryptographic hash functions due
to the march of time. As I'm sure you realize, the problem with hash
functions that are designed to foil an adversary is that adversaries
keep getting smarter. So, eventually someone will probably figure out
how to do something nefarious with SHA-512. Some other technique that
nobody's cracked yet will need to be adopted, and then people will
begin trying to crack that, and the whole thing will repeat. But I
suspect that we can keep using the same non-cryptographic hash
function essentially forever. It does not matter that people know how
the algorithm works because it makes no pretensions of trying to foil
an opponent. It is just trying to mix up the bits in such a way that a
change to the file is likely to cause a change in the checksum. The
bit-mixing properties of the algorithm do not degrade with the passage
of time.
> I'm sure that sha256 takes a lot more time than crc32c, I'm certainly
> not trying to dispute that, but what's relevent here is how much it
> impacts the time required to run the overall backup (including sync'ing
> it to disk, and possibly network transmission time.. if we're just
> comparing the time to run it through memory then, sure, the sha256
> computation time might end up being quite a bit of the time, but that's
> not really that interesting of a test..).
I think that http://postgr.es/m/[email protected]
is one of the more interesting emails on this topic. My conclusion
from that email, and the ones that led up to it, was that there is a
40-50% overhead from doing a SHA checksum, but in pgbackrest, users
don't see it because backups are compressed. Because the compression
uses so much CPU time, the additional overhead from the SHA checksum
is only a few percent more. But I don't think that it would be smart
to slow down uncompressed backups by 40-50%. That's going to cause a
problem for somebody, almost for sure.
> Maybe:
>
> "Using a SHA hash function provides a cryptographically secure digest
> of each file for users who wish to verify that the backup has not been
> tampered with, while the CRC32C algorithm provides a checksum which is
> much faster to calculate and good at catching errors due to accidental
> changes but is not resistent to targeted modifications. Note that, to
> be useful against an adversary who has access to the backup, the backup
> manifest would need to be stored securely elsewhere or otherwise
> verified to have not been modified since the backup was taken."
>
> This at least talks about things in a positive direction (SHA hash
> functions do this, CRC32C does that) rather than in a negative tone.
Cool. I like it.
> Anyway, my point here was really just that *pg_validatebackup* is about
> validating backups taken with pg_basebackup. While it's possible that
> it could be used for backups taken with other tools, I don't think
> that's really part of its actual mandate or that we're going to actively
> work to add such support in the future.
I think you're kind just nitpicking here, because the statement that
pg_validatebackup can validate not only a backup taken by
pg_basebackup but also a backup taken in using some compatible method
is just a tautology. But I'll remove the reference.
> In particular, the sentence right above this list is:
>
> "Certain files and directories are excluded from verification:"
>
> but we actually do verify the manifest, that's all I'm saying here.
>
> Maybe rewording that a bit is what would help, say:
>
> "Certain files and directories are not included in the manifest:"
Well, that'd be wrong, though. It's true that backup_manifest won't
have an entry in the manifest, and neither will WAL files, but
postgresql.auto.conf will. We'll just skip complaining about it if the
checksum doesn't match or whatever. The server generates manifest
entries for everything, and the client decides not to pay attention to
some of them because it knows that pg_basebackup may have made certain
changes that were not known to the server.
> Yeah, I get that it's not easy to figure out how to validate the WAL,
> but I stand by my opinion that it's simply not acceptable to exclude the
> necessary WAL from verification so and to claim that a backup is valid
> when we haven't checked the WAL.
I hear that, but I don't agree that having nothing is better than
having this much committed. I would be fine with renaming the tool
(pg_validatebackupmanifest? pg_validatemanifest?), or with updating
the documentation to be more clear about what is and is not checked,
but I'm not going to extent the tool to do totally new things for
which we don't even have an agreed design yet. I believe in trying to
create patches that do one thing and do it well, and this patch does
that. The fact that it doesn't do some other thing that is
conceptually related yet different is a good thing, not a bad one.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-25 20:54 ` Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-25 20:54 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Wed, Mar 25, 2020 at 9:31 AM Stephen Frost <[email protected]> wrote:
> > I get that the default for manifest is 'no', but I don't really see how
> > that means that the lack of saying anything about checksums should mean
> > "give me crc32c checksums". It's really rather common that if we don't
> > specify something, it means don't do that thing- like an 'ORDER BY'
> > clause.
>
> That's a fair argument, but I think the other relevant principle is
> that we try to give people useful defaults for things. I think that
> checksums are a sufficiently useful thing that having the default be
> not to do it doesn't make sense. I had the impression that you and
> David were in agreement on that point, actually.
I agree with wanting to have useful defaults and that checksums should
be included by default, and I'm alright even with letting people pick
what algorithms they'd like to have too. The construct here is made odd
because we've got this idea that "no checksum" is an option, which is
actually something that I don't particularly like, but that's what's
making this particular syntax weird. I don't suppose you'd be open to
the idea of just dropping that though..? There wouldn't be any issue
with this syntax if we just always had checksums included when a
manifest is requested. :)
Somehow, I don't think I'm going to win that argument.
> > There were also comments made up-thread about how it might not be great
> > for larger (eg: 1GB files, like we tend to have quite a few of...), and
> > something about it being a 40 year old algorithm..
>
> Well, the 512MB "limit" for CRC-32C means only that for certain very
> specific types of errors, detection is not guaranteed above that file
> size. So if you have a single flipped bit, for example, and the file
> size is greater than 512MB, then CRC-32C has only a 99.9999999767169%
> chance of detecting the error, whereas if the file size is less than
> 512MB, it is 100% certain, because of the design of the algorithm. But
> nine nines is plenty, and neither SHA nor our page-level checksums
> provide guaranteed error detection properties anyway.
Right, so we know that CRC-32C has an upper-bound of 512MB to be useful
for exactly what it's designed to be useful for, but we also know that
we're going to have larger files- at least 1GB ones, and quite possibly
larger, so why are we choosing this?
At the least, wouldn't it make sense to consider a larger CRC, one whose
limit is above the size of commonly expected files, if we're going to
use a CRC?
> I'm not sure why the fact that it's a 40-year-old algorithm is
> relevant. There are many 40-year-old algorithms that are very good.
Sure there are, but there probably wasn't a lot of thought about
GB-sized files, and this doesn't really seem to be the direction people
are going in for larger objects. s3, as an example, uses sha256.
Google, it seems, suggests folks use "HighwayHash" (from their crc32c
github repo- https://github.com/google/crc32c). Most CRC uses seem to
be for much smaller data sets.
> My guess is that if this patch is adopted as currently proposed, we
> will eventually need to replace the cryptographic hash functions due
> to the march of time. As I'm sure you realize, the problem with hash
> functions that are designed to foil an adversary is that adversaries
> keep getting smarter. So, eventually someone will probably figure out
> how to do something nefarious with SHA-512. Some other technique that
> nobody's cracked yet will need to be adopted, and then people will
> begin trying to crack that, and the whole thing will repeat. But I
> suspect that we can keep using the same non-cryptographic hash
> function essentially forever. It does not matter that people know how
> the algorithm works because it makes no pretensions of trying to foil
> an opponent. It is just trying to mix up the bits in such a way that a
> change to the file is likely to cause a change in the checksum. The
> bit-mixing properties of the algorithm do not degrade with the passage
> of time.
Sure, there's a good chance we'll need newer algorithms in the future, I
don't doubt that. On the other hand, if crc32c, or CRC whatever, was
the perfect answer and no one will ever need something better, then
what's with folks like Google suggesting something else..?
> > I'm sure that sha256 takes a lot more time than crc32c, I'm certainly
> > not trying to dispute that, but what's relevent here is how much it
> > impacts the time required to run the overall backup (including sync'ing
> > it to disk, and possibly network transmission time.. if we're just
> > comparing the time to run it through memory then, sure, the sha256
> > computation time might end up being quite a bit of the time, but that's
> > not really that interesting of a test..).
>
> I think that http://postgr.es/m/[email protected]
> is one of the more interesting emails on this topic. My conclusion
> from that email, and the ones that led up to it, was that there is a
> 40-50% overhead from doing a SHA checksum, but in pgbackrest, users
> don't see it because backups are compressed. Because the compression
> uses so much CPU time, the additional overhead from the SHA checksum
> is only a few percent more. But I don't think that it would be smart
> to slow down uncompressed backups by 40-50%. That's going to cause a
> problem for somebody, almost for sure.
I like that email on the topic also, as it points out again (as I tried
to do earlier also..) that it depends on what we're actually including
in the test- and it seems, again, that those tests didn't consider the
time to actually write the data somewhere, either network or disk.
As for folks who are that close to the edge on their backup timing that
they can't have it slow down- chances are pretty darn good that they're
not far from ending up needing to find a better solution than
pg_basebackup anyway. Or they don't need to generate a manifest (or, I
suppose, they could have one but not have checksums..).
> > In particular, the sentence right above this list is:
> >
> > "Certain files and directories are excluded from verification:"
> >
> > but we actually do verify the manifest, that's all I'm saying here.
> >
> > Maybe rewording that a bit is what would help, say:
> >
> > "Certain files and directories are not included in the manifest:"
>
> Well, that'd be wrong, though. It's true that backup_manifest won't
> have an entry in the manifest, and neither will WAL files, but
> postgresql.auto.conf will. We'll just skip complaining about it if the
> checksum doesn't match or whatever. The server generates manifest
> entries for everything, and the client decides not to pay attention to
> some of them because it knows that pg_basebackup may have made certain
> changes that were not known to the server.
Ok, but it's also wrong to say that the backup_label is excluded from
verification.
> > Yeah, I get that it's not easy to figure out how to validate the WAL,
> > but I stand by my opinion that it's simply not acceptable to exclude the
> > necessary WAL from verification so and to claim that a backup is valid
> > when we haven't checked the WAL.
>
> I hear that, but I don't agree that having nothing is better than
> having this much committed. I would be fine with renaming the tool
> (pg_validatebackupmanifest? pg_validatemanifest?), or with updating
> the documentation to be more clear about what is and is not checked,
> but I'm not going to extent the tool to do totally new things for
> which we don't even have an agreed design yet. I believe in trying to
> create patches that do one thing and do it well, and this patch does
> that. The fact that it doesn't do some other thing that is
> conceptually related yet different is a good thing, not a bad one.
I fail to see the usefulness of a tool that doesn't actually verify that
the backup is able to be restored from.
Even pg_basebackup (in both fetch and stream modes...) checks that we at
least got all the WAL that's needed for the backup from the server
before considering the backup to be valid and telling the user that
there was a successful backup. With what you're proposing here, we
could have someone do a pg_basebackup, get back an ERROR saying the
backup wasn't valid, and then run pg_validatebackup and be told that the
backup is valid. I don't get how that's sensible.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-26 15:37 ` Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 3 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-26 15:37 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Wed, Mar 25, 2020 at 4:54 PM Stephen Frost <[email protected]> wrote:
> > That's a fair argument, but I think the other relevant principle is
> > that we try to give people useful defaults for things. I think that
> > checksums are a sufficiently useful thing that having the default be
> > not to do it doesn't make sense. I had the impression that you and
> > David were in agreement on that point, actually.
>
> I agree with wanting to have useful defaults and that checksums should
> be included by default, and I'm alright even with letting people pick
> what algorithms they'd like to have too. The construct here is made odd
> because we've got this idea that "no checksum" is an option, which is
> actually something that I don't particularly like, but that's what's
> making this particular syntax weird. I don't suppose you'd be open to
> the idea of just dropping that though..? There wouldn't be any issue
> with this syntax if we just always had checksums included when a
> manifest is requested. :)
>
> Somehow, I don't think I'm going to win that argument.
Well, it's not a crazy idea. So, at some point, I had the idea that
you were always going to get a manifest, and therefore you should at
least ought to have the option of not checksumming to avoid the
overhead. But, as things stand now, you can suppress the manifest
altogether, so that you can still take a backup even if you've got no
disk space to spool the manifest on the master. So, if you really want
no overhead from manifests, just don't have a manifest. And if you are
OK with some overhead, why not at least have a CRC-32C checksum, which
is, after all, pretty cheap?
Now, on the other hand, I don't have any strong evidence that the
manifest-without-checksums mode is useless. You can still use it to
verify that you have the correct files and that those files have the
expected sizes. And, verifying those things is very cheap, because you
only need to stat() each file, not open and read them all. True, you
can do those things by using pg_validatebackup -s. But, you'd still
incur the (admittedly fairly low) overhead of computing checksums that
you don't intend to use.
This is where I feel like I'm trying to make decisions in a vacuum. If
we had a few more people weighing in on the thread on this point, I'd
be happy to go with whatever the consensus was. If most people think
having both --no-manifest (suppressing the manifest completely) and
--manifest-checksums=none (suppressing only the checksums) is useless
and confusing, then sure, let's rip the latter one out. If most people
like the flexibility, let's keep it: it's already implemented and
tested. But I hate to base the decision on what one or two people
think.
> > Well, the 512MB "limit" for CRC-32C means only that for certain very
> > specific types of errors, detection is not guaranteed above that file
> > size. So if you have a single flipped bit, for example, and the file
> > size is greater than 512MB, then CRC-32C has only a 99.9999999767169%
> > chance of detecting the error, whereas if the file size is less than
> > 512MB, it is 100% certain, because of the design of the algorithm. But
> > nine nines is plenty, and neither SHA nor our page-level checksums
> > provide guaranteed error detection properties anyway.
>
> Right, so we know that CRC-32C has an upper-bound of 512MB to be useful
> for exactly what it's designed to be useful for, but we also know that
> we're going to have larger files- at least 1GB ones, and quite possibly
> larger, so why are we choosing this?
>
> At the least, wouldn't it make sense to consider a larger CRC, one whose
> limit is above the size of commonly expected files, if we're going to
> use a CRC?
I mean, you're just repeating the same argument here, and it's just
not valid. Regardless of the file size, the chances of a false
checksum match are literally less than one in a billion. There is
every reason to believe that users will be happy with a low-overhead
method that has a 99.9999999+% chance of detecting corrupt files. I do
agree that a 64-bit CRC would probably be not much more expensive and
improve the probability of detecting errors even further, but I wanted
to restrict this patch to using infrastructure we already have. The
choices there are the various SHA functions (so I supported those),
MD5 (which I deliberately omitted, for reasons I hope you'll be the
first to agree with), CRC-32C (which is fast), a couple of other
CRC-32 variants (which I omitted because they seemed redundant and one
of them only ever existed in PostgreSQL because of a coding mistake),
and the hacked-up version of FNV that we use for page-level checksums
(which is only 16 bits and seems to have no advantages for this
purpose).
> > I'm not sure why the fact that it's a 40-year-old algorithm is
> > relevant. There are many 40-year-old algorithms that are very good.
>
> Sure there are, but there probably wasn't a lot of thought about
> GB-sized files, and this doesn't really seem to be the direction people
> are going in for larger objects. s3, as an example, uses sha256.
> Google, it seems, suggests folks use "HighwayHash" (from their crc32c
> github repo- https://github.com/google/crc32c). Most CRC uses seem to
> be for much smaller data sets.
Again, I really want to stick with infrastructure we already have.
Trying to find a hash function that will please everybody is a hole
with no bottom, or more to the point, a bikeshed in need of painting.
There are TONS of great hash functions out there on the Internet, and
as previous discussions of pgsql-hackers will attest, as soon as you
go down that road, somebody will say "well, what about xxhash" or
whatever, and then you spend the rest of your life trying to figure
out what hash function we could try to commit that is fast and secure
and doesn't have copyright or patent problems. There have been
multiple efforts to introduce such hash functions in the past, and I
think basically all of those have crashed into a brick wall.
I don't think that's because introducing new hash functions is a bad
idea. I think that there are various reasons why it might be a good
idea. For instance, highwayhash purports to be a cryptographic hash
function that is fast enough to replace non-cryptographic hash
functions. It's easy to see why someone might want that, here. For
example, it would be entirely reasonable to copy the backup manifest
onto a USB key and store it in a vault. Later, if you get the USB key
back out of the vault and validate it against the backup, you pretty
much know that none of the data files have been tampered with,
provided that you used a cryptographic hash. So, SHA is a good option
for people who have a USB key and a vault, and a faster cryptographic
might be even better. I don't have any desire to block such proposals,
and I would be thrilled if this work inspires other people to add such
options. However, I also don't want this patch to get blocked by an
interminable argument about which hash functions we ought to use. The
ones we have in core now are good enough for a start, and more can be
added later.
> Sure, there's a good chance we'll need newer algorithms in the future, I
> don't doubt that. On the other hand, if crc32c, or CRC whatever, was
> the perfect answer and no one will ever need something better, then
> what's with folks like Google suggesting something else..?
I have never said that CRC was the perfect answer, and the reason why
Google is suggesting something different is because they wanted a fast
hash (not SHA) that still has cryptographic properties. What I have
said is that using CRC-32C by default means that there is very little
downside as compared with current releases. Backups will not get
slower, and error detection will get better. If you pick any other
default from the menu of options currently available, then either
backups get noticeably slower, or we get less error detection
capability than that option gives us.
> As for folks who are that close to the edge on their backup timing that
> they can't have it slow down- chances are pretty darn good that they're
> not far from ending up needing to find a better solution than
> pg_basebackup anyway. Or they don't need to generate a manifest (or, I
> suppose, they could have one but not have checksums..).
40-50% is a lot more than "if you were on the edge."
> > Well, that'd be wrong, though. It's true that backup_manifest won't
> > have an entry in the manifest, and neither will WAL files, but
> > postgresql.auto.conf will. We'll just skip complaining about it if the
> > checksum doesn't match or whatever. The server generates manifest
> > entries for everything, and the client decides not to pay attention to
> > some of them because it knows that pg_basebackup may have made certain
> > changes that were not known to the server.
>
> Ok, but it's also wrong to say that the backup_label is excluded from
> verification.
The docs don't say that backup_label is excluded from verification.
They do say that backup_manifest is excluded from verification
*against the manifest*, because it is. I'm not sure if you're honestly
confused here or if we're just devolving into arguing for the sake of
argument, but right now the code looks like this:
simple_string_list_append(&context.ignore_list, "backup_manifest");
simple_string_list_append(&context.ignore_list, "pg_wal");
simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
simple_string_list_append(&context.ignore_list, "recovery.signal");
simple_string_list_append(&context.ignore_list, "standby.signal");
Notice that this is the same list of files mentioned in the
documentation. Now let's suppose we remove the first of those lines of
code, so that backup_manifest is not in the exclude list by default.
Now let's try to validate a backup:
[rhaas pgsql]$ src/bin/pg_validatebackup/pg_validatebackup ~/pgslave
pg_validatebackup: error: "backup_manifest" is present on disk but not
in the manifest
Oops. If you read that error carefully, you can see that the complaint
is 100% valid. backup_manifest is indeed present on disk, but not in
the manifest. However, because this situation is expected and known
not to be a problem, the right thing to do is suppress the error. That
is why it is in the ignore_list by default. The documentation is
attempting to explain this. If it's unclear, we should try to make it
better, but it is absolutely NOT saying that there is no internal
validation of the backup_manifest. In fact, the previous paragraph
tries to explain that:
+ <application>pg_validatebackup</application> reads the manifest file of a
+ backup, verifies the manifest against its own internal checksum, and then
It is, however, saying, and *entirely correctly*, that
pg_validatebackup will not check the backup_manifest file against the
backup_manifest. If it did, it would find that it's not there. It
would then emit an error message like the one above even though
there's no problem with the backup.
> I fail to see the usefulness of a tool that doesn't actually verify that
> the backup is able to be restored from.
>
> Even pg_basebackup (in both fetch and stream modes...) checks that we at
> least got all the WAL that's needed for the backup from the server
> before considering the backup to be valid and telling the user that
> there was a successful backup. With what you're proposing here, we
> could have someone do a pg_basebackup, get back an ERROR saying the
> backup wasn't valid, and then run pg_validatebackup and be told that the
> backup is valid. I don't get how that's sensible.
I'm sorry that you can't see how that's sensible, but it doesn't mean
that it isn't sensible. It is totally unrealistic to expect that any
backup verification tool can verify that you won't get an error when
trying to use the backup. That would require that everything that the
validation tool try to do everything that PostgreSQL will try to do
when the backup is used, including running recovery and updating the
data files. Anything less than that creates a real possibility that
the backup will verify good but fail when used. This tool has a much
narrower purpose, which is to try to verify that we (still) have the
files the server sent as part of the backup and that, to the best of
our ability to detect such things, they have not been modified. As you
know, or should know, the WAL files are not sent as part of the
backup, and so are not verified. Other things that would also be
useful to check are also not verified. It would be fantastic to have
more verification tools in the future, but it is difficult to see why
anyone would bother trying if an attempt to get the first one
committed gets blocked because it does not yet do everything. Very few
patches try to do everything, and those that do usually get blocked
because, by trying to do too much, they get some of it badly wrong.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-26 16:34 ` Stephen Frost <[email protected]>
2020-03-26 17:40 ` Re: backup manifests Mark Dilger <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 22:36 ` Re: backup manifests Bruce Momjian <[email protected]>
2 siblings, 3 replies; 372+ messages in thread
From: Stephen Frost @ 2020-03-26 16:34 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Wed, Mar 25, 2020 at 4:54 PM Stephen Frost <[email protected]> wrote:
> > > That's a fair argument, but I think the other relevant principle is
> > > that we try to give people useful defaults for things. I think that
> > > checksums are a sufficiently useful thing that having the default be
> > > not to do it doesn't make sense. I had the impression that you and
> > > David were in agreement on that point, actually.
> >
> > I agree with wanting to have useful defaults and that checksums should
> > be included by default, and I'm alright even with letting people pick
> > what algorithms they'd like to have too. The construct here is made odd
> > because we've got this idea that "no checksum" is an option, which is
> > actually something that I don't particularly like, but that's what's
> > making this particular syntax weird. I don't suppose you'd be open to
> > the idea of just dropping that though..? There wouldn't be any issue
> > with this syntax if we just always had checksums included when a
> > manifest is requested. :)
> >
> > Somehow, I don't think I'm going to win that argument.
>
> Well, it's not a crazy idea. So, at some point, I had the idea that
> you were always going to get a manifest, and therefore you should at
> least ought to have the option of not checksumming to avoid the
> overhead. But, as things stand now, you can suppress the manifest
> altogether, so that you can still take a backup even if you've got no
> disk space to spool the manifest on the master. So, if you really want
> no overhead from manifests, just don't have a manifest. And if you are
> OK with some overhead, why not at least have a CRC-32C checksum, which
> is, after all, pretty cheap?
>
> Now, on the other hand, I don't have any strong evidence that the
> manifest-without-checksums mode is useless. You can still use it to
> verify that you have the correct files and that those files have the
> expected sizes. And, verifying those things is very cheap, because you
> only need to stat() each file, not open and read them all. True, you
> can do those things by using pg_validatebackup -s. But, you'd still
> incur the (admittedly fairly low) overhead of computing checksums that
> you don't intend to use.
>
> This is where I feel like I'm trying to make decisions in a vacuum. If
> we had a few more people weighing in on the thread on this point, I'd
> be happy to go with whatever the consensus was. If most people think
> having both --no-manifest (suppressing the manifest completely) and
> --manifest-checksums=none (suppressing only the checksums) is useless
> and confusing, then sure, let's rip the latter one out. If most people
> like the flexibility, let's keep it: it's already implemented and
> tested. But I hate to base the decision on what one or two people
> think.
I'm frustrated at the lack of involvement from others also.
Just to be clear- I'm not completely against having a 'manifest but no
checksum' option, but if that's what we're going to have then it seems
like the syntax should be such that if you don't specify checksums then
you don't get checksums and "MANIFEST_CHECKSUM none" shouldn't be a
thing.
All that said, as I said up-thread, I appreciate that we aren't
designing SQL here and that this is pretty special syntax to begin with,
so if you ended up committing it the way you have it now, so be it, I
wouldn't be asking for it to be reverted over this. It's a bit awkward
and kind of a thorn, but it's not entirely unreasonable, and we'd
probably end up there anyway if we started out without a 'none' option
and someone did come up with a good argument and a patch to add such an
option in the future.
> > > Well, the 512MB "limit" for CRC-32C means only that for certain very
> > > specific types of errors, detection is not guaranteed above that file
> > > size. So if you have a single flipped bit, for example, and the file
> > > size is greater than 512MB, then CRC-32C has only a 99.9999999767169%
> > > chance of detecting the error, whereas if the file size is less than
> > > 512MB, it is 100% certain, because of the design of the algorithm. But
> > > nine nines is plenty, and neither SHA nor our page-level checksums
> > > provide guaranteed error detection properties anyway.
> >
> > Right, so we know that CRC-32C has an upper-bound of 512MB to be useful
> > for exactly what it's designed to be useful for, but we also know that
> > we're going to have larger files- at least 1GB ones, and quite possibly
> > larger, so why are we choosing this?
> >
> > At the least, wouldn't it make sense to consider a larger CRC, one whose
> > limit is above the size of commonly expected files, if we're going to
> > use a CRC?
>
> I mean, you're just repeating the same argument here, and it's just
> not valid. Regardless of the file size, the chances of a false
> checksum match are literally less than one in a billion. There is
> every reason to believe that users will be happy with a low-overhead
> method that has a 99.9999999+% chance of detecting corrupt files. I do
> agree that a 64-bit CRC would probably be not much more expensive and
> improve the probability of detecting errors even further, but I wanted
> to restrict this patch to using infrastructure we already have. The
> choices there are the various SHA functions (so I supported those),
> MD5 (which I deliberately omitted, for reasons I hope you'll be the
> first to agree with), CRC-32C (which is fast), a couple of other
> CRC-32 variants (which I omitted because they seemed redundant and one
> of them only ever existed in PostgreSQL because of a coding mistake),
> and the hacked-up version of FNV that we use for page-level checksums
> (which is only 16 bits and seems to have no advantages for this
> purpose).
The argument that "well, we happened to already have it, even though we
used it for much smaller data sets, which are well within the
100%-single-bit-error detection limit" certainly doesn't make me be in
more support of this. Choosing the right algorithm to use maybe
shouldn't be based on the age of that algorithm, but it also certainly
shouldn't be "just because we already have it" when we're using it for a
very different use-case.
I'm guessing folks have already seen it, but I thought this was an
interesting run-down of actual collisions based on various checksum
lengths using one data set (though it's not clear exactly how big it is,
from what I can see)-
http://www.backplane.com/matt/crc64.html
I do agree with excluding things like md5 and others that aren't good
options. I wasn't saying we should necessarily exclude crc32c either..
but rather saying that it shouldn't be the default.
Here's another way to look at it- where do we use crc32c today, and how
much data might we possibly be covering with that crc? Why was crc32c
picked for that purpose? If the individual who decided to pick crc32c
for that case was contemplating a checksum for up-to-1GB files, would
they have picked crc32c? Seems unlikely to me.
> > > I'm not sure why the fact that it's a 40-year-old algorithm is
> > > relevant. There are many 40-year-old algorithms that are very good.
> >
> > Sure there are, but there probably wasn't a lot of thought about
> > GB-sized files, and this doesn't really seem to be the direction people
> > are going in for larger objects. s3, as an example, uses sha256.
> > Google, it seems, suggests folks use "HighwayHash" (from their crc32c
> > github repo- https://github.com/google/crc32c). Most CRC uses seem to
> > be for much smaller data sets.
>
> Again, I really want to stick with infrastructure we already have.
I don't agree with that as a sensible justification for picking it for
this case, because it's clearly not the same use-case.
> Trying to find a hash function that will please everybody is a hole
> with no bottom, or more to the point, a bikeshed in need of painting.
> There are TONS of great hash functions out there on the Internet, and
> as previous discussions of pgsql-hackers will attest, as soon as you
> go down that road, somebody will say "well, what about xxhash" or
> whatever, and then you spend the rest of your life trying to figure
> out what hash function we could try to commit that is fast and secure
> and doesn't have copyright or patent problems. There have been
> multiple efforts to introduce such hash functions in the past, and I
> think basically all of those have crashed into a brick wall.
>
> I don't think that's because introducing new hash functions is a bad
> idea. I think that there are various reasons why it might be a good
> idea. For instance, highwayhash purports to be a cryptographic hash
> function that is fast enough to replace non-cryptographic hash
> functions. It's easy to see why someone might want that, here. For
> example, it would be entirely reasonable to copy the backup manifest
> onto a USB key and store it in a vault. Later, if you get the USB key
> back out of the vault and validate it against the backup, you pretty
> much know that none of the data files have been tampered with,
> provided that you used a cryptographic hash. So, SHA is a good option
> for people who have a USB key and a vault, and a faster cryptographic
> might be even better. I don't have any desire to block such proposals,
> and I would be thrilled if this work inspires other people to add such
> options. However, I also don't want this patch to get blocked by an
> interminable argument about which hash functions we ought to use. The
> ones we have in core now are good enough for a start, and more can be
> added later.
I'm not actually argueing about which hash functions we should support,
but rather what the default is and if crc32c, specifically, is actually
a reasonable choice. Just because it's fast and we already had an
implementation of it doesn't justify its use as the default. Given that
it doesn't actually provide the check that is generally expected of
CRC checksums (100% detection of single-bit errors) when the file size
gets over 512MB makes me wonder if we should have it at all, yes, but it
definitely makes me think it shouldn't be our default.
Folks look to PG as being pretty good at figuring things out and doing
the thing that makes sense to minimize risk of data loss or corruption.
I can understand and agree with the desire to have a faster alternative
to sha256 for those who don't need a cryptographically safe hash, but if
we're going to provide that option, it should be the right answer and
it's pretty clear, at least to me, that crc32c isn't a good choice for
gigabyte-size files.
> > Sure, there's a good chance we'll need newer algorithms in the future, I
> > don't doubt that. On the other hand, if crc32c, or CRC whatever, was
> > the perfect answer and no one will ever need something better, then
> > what's with folks like Google suggesting something else..?
>
> I have never said that CRC was the perfect answer, and the reason why
> Google is suggesting something different is because they wanted a fast
> hash (not SHA) that still has cryptographic properties. What I have
> said is that using CRC-32C by default means that there is very little
> downside as compared with current releases. Backups will not get
> slower, and error detection will get better. If you pick any other
> default from the menu of options currently available, then either
> backups get noticeably slower, or we get less error detection
> capability than that option gives us.
I don't agree with limiting our view to only those algorithms that we've
already got implemented in PG.
> > As for folks who are that close to the edge on their backup timing that
> > they can't have it slow down- chances are pretty darn good that they're
> > not far from ending up needing to find a better solution than
> > pg_basebackup anyway. Or they don't need to generate a manifest (or, I
> > suppose, they could have one but not have checksums..).
>
> 40-50% is a lot more than "if you were on the edge."
We can agree to disagree on this, it's not particularly relevant in the
end.
> > > Well, that'd be wrong, though. It's true that backup_manifest won't
> > > have an entry in the manifest, and neither will WAL files, but
> > > postgresql.auto.conf will. We'll just skip complaining about it if the
> > > checksum doesn't match or whatever. The server generates manifest
> > > entries for everything, and the client decides not to pay attention to
> > > some of them because it knows that pg_basebackup may have made certain
> > > changes that were not known to the server.
> >
> > Ok, but it's also wrong to say that the backup_label is excluded from
> > verification.
>
> The docs don't say that backup_label is excluded from verification.
> They do say that backup_manifest is excluded from verification
> *against the manifest*, because it is. I'm not sure if you're honestly
> confused here or if we're just devolving into arguing for the sake of
> argument, but right now the code looks like this:
That you're bringing up code here is really just not sensible- we're
talking about the documentation, not about the code here. I do
understand what the code is doing and I don't have any complaint about
the code.
> Oops. If you read that error carefully, you can see that the complaint
> is 100% valid. backup_manifest is indeed present on disk, but not in
> the manifest. However, because this situation is expected and known
> not to be a problem, the right thing to do is suppress the error. That
> is why it is in the ignore_list by default. The documentation is
> attempting to explain this. If it's unclear, we should try to make it
> better, but it is absolutely NOT saying that there is no internal
> validation of the backup_manifest. In fact, the previous paragraph
> tries to explain that:
Yes, I think the documentation is unclear, as I said before, because it
purports to list things that aren't being validated and then includes
backup_manifest in that list, which doesn't make sense. The sentence in
question does *not* say "Certain files and directories are excluded from
the manifest" (which is wording that I actually proposed up-thread, to
try to address this...), it says, from the patch:
"Certain files and directories are excluded from verification:"
Excluded from verification. Then lists backup_manifest. Even though,
earlier in that same paragraph it says that the manifest is verified
against its own checksum.
> + <application>pg_validatebackup</application> reads the manifest file of a
> + backup, verifies the manifest against its own internal checksum, and then
>
> It is, however, saying, and *entirely correctly*, that
> pg_validatebackup will not check the backup_manifest file against the
> backup_manifest. If it did, it would find that it's not there. It
> would then emit an error message like the one above even though
> there's no problem with the backup.
It's saying, removing the listing aspect, exactly that "backup_label is
excluded from verification". That's what I am taking issue with. I've
made multiple attempts to suggest other language to avoid saying that
because it's clearly wrong- the manifest is verified.
> > I fail to see the usefulness of a tool that doesn't actually verify that
> > the backup is able to be restored from.
> >
> > Even pg_basebackup (in both fetch and stream modes...) checks that we at
> > least got all the WAL that's needed for the backup from the server
> > before considering the backup to be valid and telling the user that
> > there was a successful backup. With what you're proposing here, we
> > could have someone do a pg_basebackup, get back an ERROR saying the
> > backup wasn't valid, and then run pg_validatebackup and be told that the
> > backup is valid. I don't get how that's sensible.
>
> I'm sorry that you can't see how that's sensible, but it doesn't mean
> that it isn't sensible. It is totally unrealistic to expect that any
> backup verification tool can verify that you won't get an error when
> trying to use the backup. That would require that everything that the
> validation tool try to do everything that PostgreSQL will try to do
> when the backup is used, including running recovery and updating the
> data files. Anything less than that creates a real possibility that
> the backup will verify good but fail when used. This tool has a much
> narrower purpose, which is to try to verify that we (still) have the
> files the server sent as part of the backup and that, to the best of
> our ability to detect such things, they have not been modified. As you
> know, or should know, the WAL files are not sent as part of the
> backup, and so are not verified. Other things that would also be
> useful to check are also not verified. It would be fantastic to have
> more verification tools in the future, but it is difficult to see why
> anyone would bother trying if an attempt to get the first one
> committed gets blocked because it does not yet do everything. Very few
> patches try to do everything, and those that do usually get blocked
> because, by trying to do too much, they get some of it badly wrong.
I'm not talking about making sure that no error ever happens when doing
a restore of a particular backup. You're arguing against something that
I have not advocated for and which I don't advocate for.
I'm saying that the existing tool that takes the backup has a *really*
*important* verification check that this proposed "validate backup" tool
doesn't have, and that isn't sensible. It leads to situations where the
backup tool itself, pg_basebackup, can fail or be killed before it's
actually completed, and the "validate backup" tool would say that the
backup is perfectly fine. That is not sensible.
That there might be other reasons why a backup can't be restored isn't
relevant and I'm not asking for a tool that is perfect and does some
kind of proof that the backup is able to be restored.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-26 17:40 ` Mark Dilger <[email protected]>
2020-03-26 19:37 ` Re: backup manifests Stephen Frost <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Mark Dilger @ 2020-03-26 17:40 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
> On Mar 26, 2020, at 9:34 AM, Stephen Frost <[email protected]> wrote:
>
> I'm not actually argueing about which hash functions we should support,
> but rather what the default is and if crc32c, specifically, is actually
> a reasonable choice. Just because it's fast and we already had an
> implementation of it doesn't justify its use as the default. Given that
> it doesn't actually provide the check that is generally expected of
> CRC checksums (100% detection of single-bit errors) when the file size
> gets over 512MB makes me wonder if we should have it at all, yes, but it
> definitely makes me think it shouldn't be our default.
I don't understand your focus on the single-bit error issue. If you are sending your backup across the wire, single bit errors during transmission should already be detected as part of the networking protocol. The real issue has to be detection of the kinds of errors or modifications that are most likely to happen in practice. Which are those? People manually mucking with the files? Bugs in backup scripts? Corruption on the storage device? Truncated files? The more bits in the checksum (assuming a well designed checksum algorithm), the more likely we are to detect accidental modification, so it is no surprise if a 64-bit crc does better than 32-bit crc. But that logic can be taken arbitrarily far. I don't see the connection between, on the one hand, an analysis of single-bit error detection against file size, and on the other hand, the verification of backups.
From a support perspective, I think the much more important issue is making certain that checksums are turned on. A one in a billion chance of missing an error seems pretty acceptable compared to the, let's say, one in two chance that your customer didn't use checksums. Why are we even allowing this to be turned off? Is there a usage case compelling that option?
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 17:40 ` Re: backup manifests Mark Dilger <[email protected]>
@ 2020-03-26 19:37 ` Stephen Frost <[email protected]>
2020-03-26 20:38 ` Re: backup manifests Mark Dilger <[email protected]>
2020-03-27 05:31 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: Stephen Frost @ 2020-03-26 19:37 UTC (permalink / raw)
To: Mark Dilger <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Mark Dilger ([email protected]) wrote:
> > On Mar 26, 2020, at 9:34 AM, Stephen Frost <[email protected]> wrote:
> > I'm not actually argueing about which hash functions we should support,
> > but rather what the default is and if crc32c, specifically, is actually
> > a reasonable choice. Just because it's fast and we already had an
> > implementation of it doesn't justify its use as the default. Given that
> > it doesn't actually provide the check that is generally expected of
> > CRC checksums (100% detection of single-bit errors) when the file size
> > gets over 512MB makes me wonder if we should have it at all, yes, but it
> > definitely makes me think it shouldn't be our default.
>
> I don't understand your focus on the single-bit error issue.
Maybe I'm wrong, but my understanding was that detecting single-bit
errors was one of the primary design goals of CRC and why people talk
about CRCs of certain sizes having 'limits'- that's the size at which
single-bit errors will no longer, necessarily, be picked up and
therefore that's where the CRC of that size starts falling down on that
goal.
> If you are sending your backup across the wire, single bit errors during transmission should already be detected as part of the networking protocol. The real issue has to be detection of the kinds of errors or modifications that are most likely to happen in practice. Which are those? People manually mucking with the files? Bugs in backup scripts? Corruption on the storage device? Truncated files? The more bits in the checksum (assuming a well designed checksum algorithm), the more likely we are to detect accidental modification, so it is no surprise if a 64-bit crc does better than 32-bit crc. But that logic can be taken arbitrarily far. I don't see the connection between, on the one hand, an analysis of single-bit error detection against file size, and on the other hand, the verification of backups.
We'd like something that does a good job at detecting any differences
between when the file was copied off of the server and when the command
is run- potentially weeks or months later. I would expect most issues
to end up being storage-level corruption over time where the backup is
stored, which could be single bit flips or whole pages getting zeroed or
various other things. Files changing size probably is one of the less
common things, but, sure, that too.
That we could take this "arbitrarily far" is actually entirely fine-
that's a good reason to have alternatives, which this patch does have,
but that doesn't mean we should have a default that's not suitable for
the files that we know we're going to be storing.
Consider that we could have used a 16-bit CRC instead, but does that
actually make sense? Ok, sure, maybe someone really wants something
super fast- but should that be our default? If not, then what criteria
should we use for the default?
> From a support perspective, I think the much more important issue is making certain that checksums are turned on. A one in a billion chance of missing an error seems pretty acceptable compared to the, let's say, one in two chance that your customer didn't use checksums. Why are we even allowing this to be turned off? Is there a usage case compelling that option?
The argument is that adding checksums takes more time. I can understand
that argument, though I don't really agree with it. Certainly a few
percent really shouldn't be that big of an issue, and in many cases even
a sha256 hash isn't going to have that dramatic of an impact on the
actual overall time.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 17:40 ` Re: backup manifests Mark Dilger <[email protected]>
2020-03-26 19:37 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-26 20:38 ` Mark Dilger <[email protected]>
2020-03-26 21:00 ` Re: backup manifests Stephen Frost <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Mark Dilger @ 2020-03-26 20:38 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
> On Mar 26, 2020, at 12:37 PM, Stephen Frost <[email protected]> wrote:
>
> Greetings,
>
> * Mark Dilger ([email protected]) wrote:
>>> On Mar 26, 2020, at 9:34 AM, Stephen Frost <[email protected]> wrote:
>>> I'm not actually argueing about which hash functions we should support,
>>> but rather what the default is and if crc32c, specifically, is actually
>>> a reasonable choice. Just because it's fast and we already had an
>>> implementation of it doesn't justify its use as the default. Given that
>>> it doesn't actually provide the check that is generally expected of
>>> CRC checksums (100% detection of single-bit errors) when the file size
>>> gets over 512MB makes me wonder if we should have it at all, yes, but it
>>> definitely makes me think it shouldn't be our default.
>>
>> I don't understand your focus on the single-bit error issue.
>
> Maybe I'm wrong, but my understanding was that detecting single-bit
> errors was one of the primary design goals of CRC and why people talk
> about CRCs of certain sizes having 'limits'- that's the size at which
> single-bit errors will no longer, necessarily, be picked up and
> therefore that's where the CRC of that size starts falling down on that
> goal.
I think I agree with all that. I'm not sure it is relevant. When people use CRCs to detect things *other than* transmission errors, they are in some sense using a hammer to drive a screw. At that point, the analysis of how good the hammer is, and how big a nail it can drive, is no longer relevant. The relevant discussion here is how appropriate a CRC is for our purpose. I don't know the answer to that, but it doesn't seem the single-bit error analysis is the right analysis.
>> If you are sending your backup across the wire, single bit errors during transmission should already be detected as part of the networking protocol. The real issue has to be detection of the kinds of errors or modifications that are most likely to happen in practice. Which are those? People manually mucking with the files? Bugs in backup scripts? Corruption on the storage device? Truncated files? The more bits in the checksum (assuming a well designed checksum algorithm), the more likely we are to detect accidental modification, so it is no surprise if a 64-bit crc does better than 32-bit crc. But that logic can be taken arbitrarily far. I don't see the connection between, on the one hand, an analysis of single-bit error detection against file size, and on the other hand, the verification of backups.
>
> We'd like something that does a good job at detecting any differences
> between when the file was copied off of the server and when the command
> is run- potentially weeks or months later. I would expect most issues
> to end up being storage-level corruption over time where the backup is
> stored, which could be single bit flips or whole pages getting zeroed or
> various other things. Files changing size probably is one of the less
> common things, but, sure, that too.
>
> That we could take this "arbitrarily far" is actually entirely fine-
> that's a good reason to have alternatives, which this patch does have,
> but that doesn't mean we should have a default that's not suitable for
> the files that we know we're going to be storing.
>
> Consider that we could have used a 16-bit CRC instead, but does that
> actually make sense? Ok, sure, maybe someone really wants something
> super fast- but should that be our default? If not, then what criteria
> should we use for the default?
I'll answer this below....
>> From a support perspective, I think the much more important issue is making certain that checksums are turned on. A one in a billion chance of missing an error seems pretty acceptable compared to the, let's say, one in two chance that your customer didn't use checksums. Why are we even allowing this to be turned off? Is there a usage case compelling that option?
>
> The argument is that adding checksums takes more time. I can understand
> that argument, though I don't really agree with it. Certainly a few
> percent really shouldn't be that big of an issue, and in many cases even
> a sha256 hash isn't going to have that dramatic of an impact on the
> actual overall time.
I see two dangers here:
(1) The user enables checksums of some type, and due to checksums not being perfect, corruption happens but goes undetected, leaving her in a bad place.
(2) The user makes no checksum selection at all, gets checksums of the *default* type, determines it is too slow for her purposes, and instead of adjusting the checksum algorithm to something faster, simply turns checksums off; corruption happens and of course is undetected, leaving her in a bad place.
I think the risk of (2) is far worse, which makes me tend towards a default that is fast enough not to encourage anybody to disable checksums altogether. I have no opinion about which algorithm is best suited to that purpose, because I haven't benchmarked any. I'm pretty much going off what Robert said, in terms of how big an impact using a heavier algorithm would be. Perhaps you'd like to run benchmarks and make a concrete proposal for another algorithm, with numbers showing the runtime changes? You mentioned up-thread that prior timings which showed a 40-50% slowdown were not including all the relevant stuff, so perhaps you could fix that in your benchmark and let us know what is included in the timings?
I don't think we should be contemplating for v13 any checksum algorithms for the default except the ones already in the options list. Doing that just derails the patch. If you want highwayhash or similar to be the default, can't we hold off until v14 and think about changing the default? Maybe I'm missing something, but I don't see any reason why it would be hard to change this after the first version has already been released.
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 17:40 ` Re: backup manifests Mark Dilger <[email protected]>
2020-03-26 19:37 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 20:38 ` Re: backup manifests Mark Dilger <[email protected]>
@ 2020-03-26 21:00 ` Stephen Frost <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Stephen Frost @ 2020-03-26 21:00 UTC (permalink / raw)
To: Mark Dilger <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Mark Dilger ([email protected]) wrote:
> > On Mar 26, 2020, at 12:37 PM, Stephen Frost <[email protected]> wrote:
> > * Mark Dilger ([email protected]) wrote:
> >>> On Mar 26, 2020, at 9:34 AM, Stephen Frost <[email protected]> wrote:
> >>> I'm not actually argueing about which hash functions we should support,
> >>> but rather what the default is and if crc32c, specifically, is actually
> >>> a reasonable choice. Just because it's fast and we already had an
> >>> implementation of it doesn't justify its use as the default. Given that
> >>> it doesn't actually provide the check that is generally expected of
> >>> CRC checksums (100% detection of single-bit errors) when the file size
> >>> gets over 512MB makes me wonder if we should have it at all, yes, but it
> >>> definitely makes me think it shouldn't be our default.
> >>
> >> I don't understand your focus on the single-bit error issue.
> >
> > Maybe I'm wrong, but my understanding was that detecting single-bit
> > errors was one of the primary design goals of CRC and why people talk
> > about CRCs of certain sizes having 'limits'- that's the size at which
> > single-bit errors will no longer, necessarily, be picked up and
> > therefore that's where the CRC of that size starts falling down on that
> > goal.
>
> I think I agree with all that. I'm not sure it is relevant. When people use CRCs to detect things *other than* transmission errors, they are in some sense using a hammer to drive a screw. At that point, the analysis of how good the hammer is, and how big a nail it can drive, is no longer relevant. The relevant discussion here is how appropriate a CRC is for our purpose. I don't know the answer to that, but it doesn't seem the single-bit error analysis is the right analysis.
I disagree that it's not relevant- it's, in fact, the one really clear
thing we can get a pretty straight-forward answer on, and that seems
really useful to me.
> >> If you are sending your backup across the wire, single bit errors during transmission should already be detected as part of the networking protocol. The real issue has to be detection of the kinds of errors or modifications that are most likely to happen in practice. Which are those? People manually mucking with the files? Bugs in backup scripts? Corruption on the storage device? Truncated files? The more bits in the checksum (assuming a well designed checksum algorithm), the more likely we are to detect accidental modification, so it is no surprise if a 64-bit crc does better than 32-bit crc. But that logic can be taken arbitrarily far. I don't see the connection between, on the one hand, an analysis of single-bit error detection against file size, and on the other hand, the verification of backups.
> >
> > We'd like something that does a good job at detecting any differences
> > between when the file was copied off of the server and when the command
> > is run- potentially weeks or months later. I would expect most issues
> > to end up being storage-level corruption over time where the backup is
> > stored, which could be single bit flips or whole pages getting zeroed or
> > various other things. Files changing size probably is one of the less
> > common things, but, sure, that too.
> >
> > That we could take this "arbitrarily far" is actually entirely fine-
> > that's a good reason to have alternatives, which this patch does have,
> > but that doesn't mean we should have a default that's not suitable for
> > the files that we know we're going to be storing.
> >
> > Consider that we could have used a 16-bit CRC instead, but does that
> > actually make sense? Ok, sure, maybe someone really wants something
> > super fast- but should that be our default? If not, then what criteria
> > should we use for the default?
>
> I'll answer this below....
>
> >> From a support perspective, I think the much more important issue is making certain that checksums are turned on. A one in a billion chance of missing an error seems pretty acceptable compared to the, let's say, one in two chance that your customer didn't use checksums. Why are we even allowing this to be turned off? Is there a usage case compelling that option?
> >
> > The argument is that adding checksums takes more time. I can understand
> > that argument, though I don't really agree with it. Certainly a few
> > percent really shouldn't be that big of an issue, and in many cases even
> > a sha256 hash isn't going to have that dramatic of an impact on the
> > actual overall time.
>
> I see two dangers here:
>
> (1) The user enables checksums of some type, and due to checksums not being perfect, corruption happens but goes undetected, leaving her in a bad place.
>
> (2) The user makes no checksum selection at all, gets checksums of the *default* type, determines it is too slow for her purposes, and instead of adjusting the checksum algorithm to something faster, simply turns checksums off; corruption happens and of course is undetected, leaving her in a bad place.
Alright, I have tried to avoid referring back to pgbackrest, but I can't
help it here.
We have never, ever, had a user come to us and complain that pgbackrest
is too slow because we're using a SHA hash. We have also had them by
default since absolutely day number one, and we even removed the option
to disable them in 1.0. We've never even been asked if we should
implement some other hash or checksum which is faster.
> I think the risk of (2) is far worse, which makes me tend towards a default that is fast enough not to encourage anybody to disable checksums altogether. I have no opinion about which algorithm is best suited to that purpose, because I haven't benchmarked any. I'm pretty much going off what Robert said, in terms of how big an impact using a heavier algorithm would be. Perhaps you'd like to run benchmarks and make a concrete proposal for another algorithm, with numbers showing the runtime changes? You mentioned up-thread that prior timings which showed a 40-50% slowdown were not including all the relevant stuff, so perhaps you could fix that in your benchmark and let us know what is included in the timings?
I don't even know what the 40-50% slowdown numbers included. Also, the
general expectation in this community is that whomever is pushing a
given patch forward should be providing the benchmarks to justify their
choice.
> I don't think we should be contemplating for v13 any checksum algorithms for the default except the ones already in the options list. Doing that just derails the patch. If you want highwayhash or similar to be the default, can't we hold off until v14 and think about changing the default? Maybe I'm missing something, but I don't see any reason why it would be hard to change this after the first version has already been released.
I'd rather we default to something that we are all confident and happy
with, erroring on the side of it being overkill rather than something
that we know isn't really appropriate for the data volume.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 17:40 ` Re: backup manifests Mark Dilger <[email protected]>
2020-03-26 19:37 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 05:31 ` Andres Freund <[email protected]>
1 sibling, 0 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-27 05:31 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Mark Dilger <[email protected]>; Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-26 15:37:11 -0400, Stephen Frost wrote:
> The argument is that adding checksums takes more time. I can understand
> that argument, though I don't really agree with it. Certainly a few
> percent really shouldn't be that big of an issue, and in many cases even
> a sha256 hash isn't going to have that dramatic of an impact on the
> actual overall time.
I don't understand how you can come to that conclusion? It doesn't take
very long to measure openssl's sha256 performance (which is pretty well
optimized). Note that we do use openssl's sha256, when compiled with
openssl support.
On my workstation, with a pretty new (but not fastest single core perf
model) intel Xeon Gold 5215, I get:
$ openssl speed sha256
...
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
sha256 76711.75k 172036.78k 321566.89k 399008.09k 431423.49k 433689.94k
IOW, ~430MB/s.
On my laptop, with pretty fast cores:
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
sha256 97054.91k 217188.63k 394864.13k 493441.02k 532100.44k 533441.19k
IOW, 530MB/s
530 MB/s is well within the realm of medium sized VMs.
And, as mentioned before. even if you do only half of that, you're still
going to be spending roughly half of the CPU time of sending a base
backup.
What makes you think that a few hundred MB/s is out of reach for a large
fraction of PG installations that actually keep backups?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-26 18:02 ` Robert Haas <[email protected]>
2020-03-26 20:44 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 05:06 ` Re: backup manifests Andres Freund <[email protected]>
2 siblings, 2 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-26 18:02 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Thu, Mar 26, 2020 at 12:34 PM Stephen Frost <[email protected]> wrote:
> I do agree with excluding things like md5 and others that aren't good
> options. I wasn't saying we should necessarily exclude crc32c either..
> but rather saying that it shouldn't be the default.
>
> Here's another way to look at it- where do we use crc32c today, and how
> much data might we possibly be covering with that crc?
WAL record size is a 32-bit unsigned integer, so in theory, up to 4GB
minus 1 byte. In practice, most of them are not more than a few
hundred bytes, the amount we might possibly be covering is a lot more.
> Why was crc32c
> picked for that purpose?
Because it was discovered that 64-bit CRC was too slow, per commit
21fda22ec46deb7734f793ef4d7fa6c226b4c78e.
> If the individual who decided to pick crc32c
> for that case was contemplating a checksum for up-to-1GB files, would
> they have picked crc32c? Seems unlikely to me.
It's hard to be sure what someone who isn't us would have done in some
situation that they didn't face, but we do have the discussion thread:
https://www.postgresql.org/message-id/flat/9291.1117593389%40sss.pgh.pa.us#c4e413bbf3d7fbeced7786da1...
The question of how much data is protected by the CRC was discussed,
mostly in the first few messages, in general terms, but it doesn't
seem to have covered the question very thoroughly. I'm sure we could
each draw things from that discussion that support our view of the
situation, but I'm not sure it would be very productive.
What confuses to me is that you seem to have a view of the upsides and
downsides of these various algorithms that seems to me to be highly
skewed. Like, suppose we change the default from CRC-32C to
SHA-something. On the upside, the error detection rate will increase
from 99.9999999+% to something much closer to 100%. On the downside,
backups will get as much as 40-50% slower for some users. I hope we
can agree that both detecting errors and taking backups quickly are
important. However, it is hard for me to imagine that the typical user
would want to pay even a 5-10% performance penalty when taking a
backup in order to improve an error detection feature which they may
not even use and which already has less than a one-in-a-billion chance
of going wrong. We routinely reject features for causing, say, a 2%
regression on general workloads. Base backup speed is probably less
important than how many SELECT or INSERT queries you can pump through
the system in a second, but it's still a pain point for lots of
people. I think if you said to some users "hey, would you like to have
error detection for your backups? it'll cost 10%" many people would
say "yes, please." But I think if you went to the same users and said
"hey, would you like to make the error detection for your backups
better? it currently has a less than 1-in-a-billion chance of failing
to detect random corruption, and you can reduce that by many orders of
magnitude for an extra 10% on your backup time," I think the results
would be much more mixed. Some people would like it, but it certainly
not everybody.
> I'm not actually argueing about which hash functions we should support,
> but rather what the default is and if crc32c, specifically, is actually
> a reasonable choice. Just because it's fast and we already had an
> implementation of it doesn't justify its use as the default. Given that
> it doesn't actually provide the check that is generally expected of
> CRC checksums (100% detection of single-bit errors) when the file size
> gets over 512MB makes me wonder if we should have it at all, yes, but it
> definitely makes me think it shouldn't be our default.
I mean, the property that I care about is the one where it detects
better than 999,999,999 errors out of every 1,000,000,000, regardless
of input length.
> I don't agree with limiting our view to only those algorithms that we've
> already got implemented in PG.
I mean, opening that giant can of worms ~2 weeks before feature freeze
is not very nice. This patch has been around for months, and the
algorithms were openly discussed a long time ago. I checked and found
out that the CRC-64 code was nuked in commit
404bc51cde9dce1c674abe4695635612f08fe27e, so in theory we could revert
that, but how much confidence do we have that the code in question
actually did the right thing, or that it's actually fast? An awful lot
of work has been done on the CRC-32C code over the years, including
several rounds of speeding it up
(f044d71e331d77a0039cec0a11859b5a3c72bc95,
3dc2d62d0486325bf263655c2d9a96aee0b02abe) and one round of fixing it
because it was producing completely wrong answers
(5028f22f6eb0579890689655285a4778b4ffc460), so I don't have a lot of
confidence about that CRC-64 code being totally without problems.
The commit message for that last commit,
5028f22f6eb0579890689655285a4778b4ffc460, seems pretty relevant in
this context, too. It observes that, because it "does not correspond
to any bit-wise CRC calculation" it is "difficult to reason about its
properties." In other words, the algorithm that we used for WAL
records for many years likely did not have the guaranteed
error-detection properties with which you are so concerned (nor do
most hash functions we might choose; CRC-64 is probably the only
choice that would). Despite that, the commit message also observed
that "it has worked well in practice." I realize I'm not convincing
you of anything here, but the guaranteed error-detection properties of
CRC are almost totally uninteresting in this context. I'm not
concerned that CRC-32C doesn't have those properties. I'm not
concerned that SHA-n wouldn't have those properties. I'm not concerned
that xxhash or HighwayHash don't have that property either. I doubt
the fact that CRC-64 would have that property would give us much
benefit. I think the only things that matter here are (1) how many
bits you get (more bits = better chance of finding errors, but even
*sixteen* bits would give you a pretty fair chance of noticing if
things are broken) and (2) whether you want a cryptographic hash
function so that you can keep the backup manifest in a vault.
> It's saying, removing the listing aspect, exactly that "backup_label is
> excluded from verification". That's what I am taking issue with. I've
> made multiple attempts to suggest other language to avoid saying that
> because it's clearly wrong- the manifest is verified.
Well, it's talking about the particular kind of verification that has
just been discussed, not any form of verification. As one idea,
perhaps instead of:
+ Certain files and directories are
+ excluded from verification:
...I could maybe insert a paragraph break there and then continue with
something like this:
When pg_basebackup compares the files and directories in the manifest
to those which are present on disk, it will ignore the presence of, or
changes to, certain files:
backup_manifest will not be present in the manifest itself, and is
therefore ignored. Note that the manifest is still verified
internally, as described above, but no error will be issued about the
presence of a backup_manifest file in the backup directory even though
it is not listed in the manifest.
Would that be more clear? Do you want to suggest something else?
> I'm not talking about making sure that no error ever happens when doing
> I'm saying that the existing tool that takes the backup has a *really*
> *important* verification check that this proposed "validate backup" tool
> doesn't have, and that isn't sensible. It leads to situations where the
> backup tool itself, pg_basebackup, can fail or be killed before it's
> actually completed, and the "validate backup" tool would say that the
> backup is perfectly fine. That is not sensible.
If someone's procedure for taking and restoring backups involves not
knowing whether or not pg_basebackup completed without error and then
trying to use the backup anyway, they are doing something which is
very foolish, and it's questionable whether any technological solution
has much hope of getting them out of trouble. But on the plus side,
this patch would have a good chance of detecting the problem, which is
a noticeable improvement over what we have now, which has no chance of
detecting the problem, because we have nothing.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-26 20:44 ` Stephen Frost <[email protected]>
2020-03-27 18:53 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-26 20:44 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Thu, Mar 26, 2020 at 12:34 PM Stephen Frost <[email protected]> wrote:
> > I do agree with excluding things like md5 and others that aren't good
> > options. I wasn't saying we should necessarily exclude crc32c either..
> > but rather saying that it shouldn't be the default.
> >
> > Here's another way to look at it- where do we use crc32c today, and how
> > much data might we possibly be covering with that crc?
>
> WAL record size is a 32-bit unsigned integer, so in theory, up to 4GB
> minus 1 byte. In practice, most of them are not more than a few
> hundred bytes, the amount we might possibly be covering is a lot more.
Is it actually possible, today, in PG, to have a 4GB WAL record?
Judging this based on the WAL record size doesn't seem quite right.
> > Why was crc32c
> > picked for that purpose?
>
> Because it was discovered that 64-bit CRC was too slow, per commit
> 21fda22ec46deb7734f793ef4d7fa6c226b4c78e.
... 15 years ago. I actually find it pretty interesting that we started
out with a 64bit CRC there, I didn't know that was the case. Also
interesting is that we had 64bit CRC code already.
> > If the individual who decided to pick crc32c
> > for that case was contemplating a checksum for up-to-1GB files, would
> > they have picked crc32c? Seems unlikely to me.
>
> It's hard to be sure what someone who isn't us would have done in some
> situation that they didn't face, but we do have the discussion thread:
>
> https://www.postgresql.org/message-id/flat/9291.1117593389%40sss.pgh.pa.us#c4e413bbf3d7fbeced7786da1...
>
> The question of how much data is protected by the CRC was discussed,
> mostly in the first few messages, in general terms, but it doesn't
> seem to have covered the question very thoroughly. I'm sure we could
> each draw things from that discussion that support our view of the
> situation, but I'm not sure it would be very productive.
Interesting.
> What confuses to me is that you seem to have a view of the upsides and
> downsides of these various algorithms that seems to me to be highly
> skewed. Like, suppose we change the default from CRC-32C to
> SHA-something. On the upside, the error detection rate will increase
> from 99.9999999+% to something much closer to 100%. On the downside,
> backups will get as much as 40-50% slower for some users. I hope we
> can agree that both detecting errors and taking backups quickly are
> important. However, it is hard for me to imagine that the typical user
> would want to pay even a 5-10% performance penalty when taking a
> backup in order to improve an error detection feature which they may
> not even use and which already has less than a one-in-a-billion chance
> of going wrong. We routinely reject features for causing, say, a 2%
> regression on general workloads. Base backup speed is probably less
> important than how many SELECT or INSERT queries you can pump through
> the system in a second, but it's still a pain point for lots of
> people. I think if you said to some users "hey, would you like to have
> error detection for your backups? it'll cost 10%" many people would
> say "yes, please." But I think if you went to the same users and said
> "hey, would you like to make the error detection for your backups
> better? it currently has a less than 1-in-a-billion chance of failing
> to detect random corruption, and you can reduce that by many orders of
> magnitude for an extra 10% on your backup time," I think the results
> would be much more mixed. Some people would like it, but it certainly
> not everybody.
I think you're right that base backup speed is much less of an issue to
slow down than SELECT or INSERT workloads, but I do also understand
that it isn't completely unimportant, which is why having options isn't
a bad idea here. That said, the options presented for users should all
be reasonable options, and for the default we should pick something
sensible, erroring on the "be safer" side, if anything.
There's lots of options for speeding up base backups, with this patch,
even if the default is to have a manifest with sha256 hashes- it could
be changed to some form of CRC, or changed to not have checksums, or
changed to not have a manifest. Users will have options.
Again, I'm not against having a checksum algorithm as a option. I'm not
saying that it must be SHA512 as the default.
> > I'm not actually argueing about which hash functions we should support,
> > but rather what the default is and if crc32c, specifically, is actually
> > a reasonable choice. Just because it's fast and we already had an
> > implementation of it doesn't justify its use as the default. Given that
> > it doesn't actually provide the check that is generally expected of
> > CRC checksums (100% detection of single-bit errors) when the file size
> > gets over 512MB makes me wonder if we should have it at all, yes, but it
> > definitely makes me think it shouldn't be our default.
>
> I mean, the property that I care about is the one where it detects
> better than 999,999,999 errors out of every 1,000,000,000, regardless
> of input length.
Throwing these kinds of things around I really don't think is useful.
> > I don't agree with limiting our view to only those algorithms that we've
> > already got implemented in PG.
>
> I mean, opening that giant can of worms ~2 weeks before feature freeze
> is not very nice. This patch has been around for months, and the
> algorithms were openly discussed a long time ago.
Yes, they were discussed before, and these issues were brought up before
and there was specifically concern brought up about exactly the same
issues that I'm repeating here. Those concerns seem to have been
largely ignored, apparently because "we don't have that in PG today" as
at least one of the considerations- even though we used to. I don't
think that was the right response and, yeah, I saw that you were
planning to commit and that prompted me to look into it right now. I
don't think that's entirely uncommon around here. I also had hoped that
David's concerns that were raised before had been heeded, as I knew he
was involved in the discussion previously, but that turns out to not
have been the case.
> > It's saying, removing the listing aspect, exactly that "backup_label is
> > excluded from verification". That's what I am taking issue with. I've
> > made multiple attempts to suggest other language to avoid saying that
> > because it's clearly wrong- the manifest is verified.
>
> Well, it's talking about the particular kind of verification that has
> just been discussed, not any form of verification. As one idea,
> perhaps instead of:
>
> + Certain files and directories are
> + excluded from verification:
>
> ...I could maybe insert a paragraph break there and then continue with
> something like this:
>
> When pg_basebackup compares the files and directories in the manifest
> to those which are present on disk, it will ignore the presence of, or
> changes to, certain files:
>
> backup_manifest will not be present in the manifest itself, and is
> therefore ignored. Note that the manifest is still verified
> internally, as described above, but no error will be issued about the
> presence of a backup_manifest file in the backup directory even though
> it is not listed in the manifest.
>
> Would that be more clear? Do you want to suggest something else?
Yes, that looks fine. Feels slightly redundant to include the "as
described above ..." bit, and I think that could be dropped, but up to
you.
> > I'm not talking about making sure that no error ever happens when doing
> > I'm saying that the existing tool that takes the backup has a *really*
> > *important* verification check that this proposed "validate backup" tool
> > doesn't have, and that isn't sensible. It leads to situations where the
> > backup tool itself, pg_basebackup, can fail or be killed before it's
> > actually completed, and the "validate backup" tool would say that the
> > backup is perfectly fine. That is not sensible.
>
> If someone's procedure for taking and restoring backups involves not
> knowing whether or not pg_basebackup completed without error and then
> trying to use the backup anyway, they are doing something which is
> very foolish, and it's questionable whether any technological solution
> has much hope of getting them out of trouble. But on the plus side,
> this patch would have a good chance of detecting the problem, which is
> a noticeable improvement over what we have now, which has no chance of
> detecting the problem, because we have nothing.
This doesn't address my concern at all. Even if it seems ridiculous and
foolish to think that a backup was successful when the system was
rebooted and pg_basebackup was killed before all of the WAL had made it
into pg_wal, there is absolutely zero doubt in my mind that it's going
to happen and users are going to, entirely reasonably, think that
pg_validatebackup at least includes all the checks that pg_basebackup
does about making sure that the backup is valid.
I really don't understand how we can have a backup validation tool that
doesn't do the absolute basics, like making sure that we have all of the
WAL for the backup. I've routinely, almost jokingly, said to folks that
any backup tool that doesn't check that isn't really a backup tool, and
I was glad that pg_basebackup had that check, so, yeah, I'm going to
continue to object to committing a backup validation tool that doesn't
have that absolutely basic and necessary check.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:44 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 18:53 ` Robert Haas <[email protected]>
2020-03-27 19:55 ` Re: backup manifests Stephen Frost <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-27 18:53 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Thu, Mar 26, 2020 at 4:44 PM Stephen Frost <[email protected]> wrote:
> Is it actually possible, today, in PG, to have a 4GB WAL record?
> Judging this based on the WAL record size doesn't seem quite right.
I'm not sure. I mean, most records are quite small, but I think if you
set REPLICA IDENTITY FULL on a table with a bunch of very wide columns
(and also wal_level=logical) it can get really big. I haven't tested
to figure out just how big it can get. (If I have a table with lots of
almost-1GB-blobs in it, does it work without logical replication and
fail with logical replication? I don't know, but I doubt a WAL record
>4GB is possible, because it seems unlikely that the code has a way to
cope with that struct field overflowing.)
> Again, I'm not against having a checksum algorithm as a option. I'm not
> saying that it must be SHA512 as the default.
I think that what we have seen so far is that all of the SHA-n
algorithms that PostgreSQL supports are about equally slow, so it
doesn't really matter which one you pick there from a performance
point of view. If you're not saying it has to be SHA-512 but you do
want it to be SHA-256, I don't think that really fixes anything. Using
CRC-32C does fix the performance issue, but I don't think you like
that, either. We could default to having no checksums at all, or even
no manifest at all, but I didn't get the impression that David, at
least, wanted to go that way, and I don't like it either. It's not the
world's best feature, but I think it's good enough to justify enabling
it by default. So I'm not sure we have any options here that will
satisfy you.
> > > I don't agree with limiting our view to only those algorithms that we've
> > > already got implemented in PG.
> >
> > I mean, opening that giant can of worms ~2 weeks before feature freeze
> > is not very nice. This patch has been around for months, and the
> > algorithms were openly discussed a long time ago.
>
> Yes, they were discussed before, and these issues were brought up before
> and there was specifically concern brought up about exactly the same
> issues that I'm repeating here. Those concerns seem to have been
> largely ignored, apparently because "we don't have that in PG today" as
> at least one of the considerations- even though we used to.
I might have missed something, but I don't remember any suggestion of
CRC-64 or other algorithms for which PG does not currently have
support prior to this week. The only thing I remember having been
suggested previously was SHA, and I responded to that by adding
support for SHA, not by ignoring the suggestion. If there was another
suggestion made earlier, I must have missed it.
> I also had hoped that
> David's concerns that were raised before had been heeded, as I knew he
> was involved in the discussion previously, but that turns out to not
> have been the case.
Well, I mean, I am trying pretty hard here, but I realize that I'm not
succeeding. I don't know which specific suggestion you're talking
about here. I understand that there is a concern about a 32-bit CRC
somehow not being valid for more than 512MB, but based on my research,
I believe that to be incorrect. I've explained the reasons why I
believe it to be incorrect several times now, but I feel like we're
just going around in circles. If my explanation of why it's incorrect
is itself incorrect, tell me why, but let's not just keep saying the
things we've both already said.
> Yes, that looks fine. Feels slightly redundant to include the "as
> described above ..." bit, and I think that could be dropped, but up to
> you.
Done in the version I posted a bit ago.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:44 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 18:53 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 19:55 ` Stephen Frost <[email protected]>
2020-03-27 20:39 ` Re: backup manifests David Steele <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 19:55 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Thu, Mar 26, 2020 at 4:44 PM Stephen Frost <[email protected]> wrote:
> > Is it actually possible, today, in PG, to have a 4GB WAL record?
> > Judging this based on the WAL record size doesn't seem quite right.
>
> I'm not sure. I mean, most records are quite small, but I think if you
> set REPLICA IDENTITY FULL on a table with a bunch of very wide columns
> (and also wal_level=logical) it can get really big. I haven't tested
> to figure out just how big it can get. (If I have a table with lots of
> almost-1GB-blobs in it, does it work without logical replication and
> fail with logical replication? I don't know, but I doubt a WAL record
> >4GB is possible, because it seems unlikely that the code has a way to
> cope with that struct field overflowing.)
Interesting.. Well, topic for another thread, but I'd say if we believe
that's possible then we might want to consider if the crc32c is a good
decision to use still there.
> > Again, I'm not against having a checksum algorithm as a option. I'm not
> > saying that it must be SHA512 as the default.
>
> I think that what we have seen so far is that all of the SHA-n
> algorithms that PostgreSQL supports are about equally slow, so it
> doesn't really matter which one you pick there from a performance
> point of view. If you're not saying it has to be SHA-512 but you do
> want it to be SHA-256, I don't think that really fixes anything. Using
> CRC-32C does fix the performance issue, but I don't think you like
> that, either. We could default to having no checksums at all, or even
> no manifest at all, but I didn't get the impression that David, at
> least, wanted to go that way, and I don't like it either. It's not the
> world's best feature, but I think it's good enough to justify enabling
> it by default. So I'm not sure we have any options here that will
> satisfy you.
I do like having a manifest by default. At this point it's pretty clear
that we've just got a fundamental disagreement that more words aren't
going to fix. I'd rather we play it safe and use a sha256 hash and
accept that it's going to be slower by default, and then give users an
option to make it go faster if they want (though I'd much rather that
alternative be a 64bit CRC than a 32bit one).
Andres seems to agree with you. I'm not sure where David sits on this
specific question.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:44 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 18:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 19:55 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 20:39 ` David Steele <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: David Steele @ 2020-03-27 20:39 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/27/20 3:55 PM, Stephen Frost wrote:
> * Robert Haas ([email protected]) wrote:
>> I think that what we have seen so far is that all of the SHA-n
>> algorithms that PostgreSQL supports are about equally slow, so it
>> doesn't really matter which one you pick there from a performance
>> point of view. If you're not saying it has to be SHA-512 but you do
>> want it to be SHA-256, I don't think that really fixes anything. Using
>> CRC-32C does fix the performance issue, but I don't think you like
>> that, either. We could default to having no checksums at all, or even
>> no manifest at all, but I didn't get the impression that David, at
>> least, wanted to go that way, and I don't like it either. It's not the
>> world's best feature, but I think it's good enough to justify enabling
>> it by default. So I'm not sure we have any options here that will
>> satisfy you.
>
> I do like having a manifest by default. At this point it's pretty clear
> that we've just got a fundamental disagreement that more words aren't
> going to fix. I'd rather we play it safe and use a sha256 hash and
> accept that it's going to be slower by default, and then give users an
> option to make it go faster if they want (though I'd much rather that
> alternative be a 64bit CRC than a 32bit one).
>
> Andres seems to agree with you. I'm not sure where David sits on this
> specific question.
I would prefer a stronger checksum as the default but I would be fine
with SHA1, which is a bit faster.
I believe the overhead of checksums is being overblown. In my experience
the vast majority of users are using compression and running the backup
over a network. Once you have done those two things the cost of SHA1 is
pretty negligible. As I posted way up-thread we found that just gzip -6
pushed the cost of SHA1 below 3% and that did not include network transfer.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 05:06 ` Andres Freund <[email protected]>
2020-03-27 18:13 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-27 05:06 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-26 14:02:29 -0400, Robert Haas wrote:
> On Thu, Mar 26, 2020 at 12:34 PM Stephen Frost <[email protected]> wrote:
> > Why was crc32c
> > picked for that purpose?
>
> Because it was discovered that 64-bit CRC was too slow, per commit
> 21fda22ec46deb7734f793ef4d7fa6c226b4c78e.
Well, a 32bit crc, not crc32c. IIRC it was the ethernet polynomial (+
bug). We switched to crc32c at some point because there are hardware
implementations:
commit 5028f22f6eb0579890689655285a4778b4ffc460
Author: Heikki Linnakangas <[email protected]>
Date: 2014-11-04 11:35:15 +0200
Switch to CRC-32C in WAL and other places.
> Like, suppose we change the default from CRC-32C to SHA-something. On
> the upside, the error detection rate will increase from 99.9999999+%
> to something much closer to 100%.
FWIW, I don't buy the relevancy of 99.9999999+% at all. That's assuming
a single bit error (at relevant lengths, before that it's single burst
errors of a greater length), which isn't that relevant for our purposes.
That's not to say that I don't think a CRC check can provide value. It
does provide a high likelihood of detecting enough errors, including
coding errors in how data is restored (not unimportant), that you're
likely not find out aobut a problem soon.
> On the downside,
> backups will get as much as 40-50% slower for some users. I hope we
> can agree that both detecting errors and taking backups quickly are
> important. However, it is hard for me to imagine that the typical user
> would want to pay even a 5-10% performance penalty when taking a
> backup in order to improve an error detection feature which they may
> not even use and which already has less than a one-in-a-billion chance
> of going wrong.
FWIW, that seems far too large a slowdown to default to for me. Most
people aren't going to be able to figure out that it's the checksum
parameter that causes this slowdown, there just going to feel the pain
of the backup being much slower than their hardware.
A few hundred megabytes of streaming reads/writes really doesn't take a
beefy server these days. Medium sized VMs + a bit larger network block
devices at all the common cloud providers have considerably higher
bandwidth. Even a raid5x of 4 spinning disks can deliver > 500MB/s.
And plenty of even the smaller instances at many providers have >
5gbit/s network. At the upper end it's way more than that.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 05:06 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 18:13 ` Robert Haas <[email protected]>
2020-03-27 19:56 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-27 18:13 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 1:06 AM Andres Freund <[email protected]> wrote:
> > Like, suppose we change the default from CRC-32C to SHA-something. On
> > the upside, the error detection rate will increase from 99.9999999+%
> > to something much closer to 100%.
>
> FWIW, I don't buy the relevancy of 99.9999999+% at all. That's assuming
> a single bit error (at relevant lengths, before that it's single burst
> errors of a greater length), which isn't that relevant for our purposes.
>
> That's not to say that I don't think a CRC check can provide value. It
> does provide a high likelihood of detecting enough errors, including
> coding errors in how data is restored (not unimportant), that you're
> likely not find out aobut a problem soon.
So, I'm glad that you think a CRC check gives a sufficiently good
chance of detection errors, but I don't understand what your objection
to the percentage. Stephen just objected to it again, too:
On Thu, Mar 26, 2020 at 4:44 PM Stephen Frost <[email protected]> wrote:
> > I mean, the property that I care about is the one where it detects
> > better than 999,999,999 errors out of every 1,000,000,000, regardless
> > of input length.
>
> Throwing these kinds of things around I really don't think is useful.
...but I don't understand his reasoning, or yours.
My reasoning for thinking that the number is accurate is that a 32-bit
checksum has 2^32 possible results. If all of those results are
equally probable, then the probability that two files with unequal
contents produce the same result is 2^-32. This does assume that the
hash function is perfect, which no hash function is, so the actual
probability of a collision is likely higher. But if the hash function
is pretty good, it shouldn't be all that much higher. Note that I am
making no assumptions here about how many bits are different, nor am I
making any assumption about the length of a file. I am simply saying
that an n-bit checksum should detect a difference between two files
with a probability of roughly 1-2^{-n}, modulo the imperfections of
the hash function. I thought that this was a well-accepted fact that
would produce little argument from anybody, and I'm confused that
people seem to feel otherwise.
One explanation that would make sense to me is if somebody said, well,
the nature of this particular algorithm means that, although values
are uniformly distributed in general, the kinds of errors that are
likely to occur in practice are likely to cancel out. For instance, if
you imagine trivial algorithms such as adding or xor-ing all the
bytes, adding zero bytes doesn't change the answer, and neither do
transpositions. However, CRC is, AIUI, designed to be resistant to
such problems. Your remark about large blocks of zero bytes is
interesting to me in this context, but in a quick search I couldn't
find anything stating that CRC was weak for such use cases.
The old thread about switching from 64-bit CRC to 32-bit CRC had a
link to a page which has subsequently been moved to here:
https://www.ece.unb.ca/tervo/ee4253/crc.shtml
Down towards the bottom, it says:
"In general, bit errors and bursts up to N-bits long will be detected
for a P(x) of degree N. For arbitrary bit errors longer than N-bits,
the odds are one in 2^{N} than a totally false bit pattern will
nonetheless lead to a zero remainder."
Which I think is the same thing I'm saying: the chances of failing to
detecting an error with a decent n-bit checksum ought to be about
2^{-N}. If that's not right, I'd really like to understand why.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 18:02 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 05:06 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 18:13 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 19:56 ` Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-27 19:56 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-27 14:13:17 -0400, Robert Haas wrote:
> On Thu, Mar 26, 2020 at 4:44 PM Stephen Frost <[email protected]> wrote:
> > > I mean, the property that I care about is the one where it detects
> > > better than 999,999,999 errors out of every 1,000,000,000, regardless
> > > of input length.
> >
> > Throwing these kinds of things around I really don't think is useful.
>
> ...but I don't understand his reasoning, or yours.
>
> My reasoning for thinking that the number is accurate is that a 32-bit
> checksum has 2^32 possible results. If all of those results are
> equally probable, then the probability that two files with unequal
> contents produce the same result is 2^-32. This does assume that the
> hash function is perfect, which no hash function is, so the actual
> probability of a collision is likely higher. But if the hash function
> is pretty good, it shouldn't be all that much higher. Note that I am
> making no assumptions here about how many bits are different, nor am I
> making any assumption about the length of a file. I am simply saying
> that an n-bit checksum should detect a difference between two files
> with a probability of roughly 1-2^{-n}, modulo the imperfections of
> the hash function. I thought that this was a well-accepted fact that
> would produce little argument from anybody, and I'm confused that
> people seem to feel otherwise.
Well: crc32 is a terrible hash, if you're looking for even distribution
of hashed values. That's not too surprising - its design goals included
guaranteed error detection for certain lengths, and error correction of
single bit errors. My understanding of the underlying math is spotty at
best, but from what I understand that does pretty directly imply less
independence between source data -> hash value than what we'd want from
a good hash function.
Here's an smhasher result page for crc32 (at least the latter is crc32
afaict):
https://notabug.org/vaeringjar/smhasher/src/master/doc/crc32
https://notabug.org/vaeringjar/smhasher/src/master/doc/crc32_hw
and then compare that with something like xxhash, or even lookup3 (which
I think is what our hash is a variant of):
https://notabug.org/vaeringjar/smhasher/src/master/doc/xxHash32
https://notabug.org/vaeringjar/smhasher/src/master/doc/lookup3
The birthday paradoxon doesn't apply (otherwise 32bit would never be
enough, at a 50% chance of conflict at around 80k hashes), but still I
do wonder if it matters that we're trying to detect errors in not one,
but commonly tens of thousands to millions of files. But since we just
need to detect one error to call the whole backup corrupt...
> One explanation that would make sense to me is if somebody said, well,
> the nature of this particular algorithm means that, although values
> are uniformly distributed in general, the kinds of errors that are
> likely to occur in practice are likely to cancel out. For instance, if
> you imagine trivial algorithms such as adding or xor-ing all the
> bytes, adding zero bytes doesn't change the answer, and neither do
> transpositions. However, CRC is, AIUI, designed to be resistant to
> such problems. Your remark about large blocks of zero bytes is
> interesting to me in this context, but in a quick search I couldn't
> find anything stating that CRC was weak for such use cases.
My main point was that CRC's error detection guarantees are pretty much
irrelevant for us. I.e. while the right CRC will guarantee that all
single 2 bit errors will be detected, that's not a helpful property for
us. There rarely are single bit errors, and the bursts are too long to
to benefit from any >2 bit guarantees. Nor are multiple failures rare
once you hit a problem.
> The old thread about switching from 64-bit CRC to 32-bit CRC had a
> link to a page which has subsequently been moved to here:
>
> https://www.ece.unb.ca/tervo/ee4253/crc.shtml
>
> Down towards the bottom, it says:
>
> "In general, bit errors and bursts up to N-bits long will be detected
> for a P(x) of degree N. For arbitrary bit errors longer than N-bits,
> the odds are one in 2^{N} than a totally false bit pattern will
> nonetheless lead to a zero remainder."
That's still about a single sequence of bit errors though, as far as I
can tell. I.e. it doesn't hold for CRCs if you have two errors at
different places.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 22:36 ` Bruce Momjian <[email protected]>
2020-03-27 22:38 ` Re: backup manifests Stephen Frost <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Bruce Momjian @ 2020-03-27 22:36 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Thu, Mar 26, 2020 at 12:34:52PM -0400, Stephen Frost wrote:
> * Robert Haas ([email protected]) wrote:
> > This is where I feel like I'm trying to make decisions in a vacuum. If
> > we had a few more people weighing in on the thread on this point, I'd
> > be happy to go with whatever the consensus was. If most people think
> > having both --no-manifest (suppressing the manifest completely) and
> > --manifest-checksums=none (suppressing only the checksums) is useless
> > and confusing, then sure, let's rip the latter one out. If most people
> > like the flexibility, let's keep it: it's already implemented and
> > tested. But I hate to base the decision on what one or two people
> > think.
>
> I'm frustrated at the lack of involvement from others also.
Well, the topic of backup manifests feels like it has generated a lot of
bickering emails, and people don't want to spend their time dealing with
that.
--
Bruce Momjian <[email protected]> https://momjian.us
EnterpriseDB https://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 22:36 ` Re: backup manifests Bruce Momjian <[email protected]>
@ 2020-03-27 22:38 ` Stephen Frost <[email protected]>
2020-03-27 22:39 ` Re: backup manifests Bruce Momjian <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 22:38 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andrew Dunstan <[email protected]>; David Steele <[email protected]>; Jeevan Chalke <[email protected]>; pgsql-hackers; Rajkumar Raghuwanshi <[email protected]>; Robert Haas <[email protected]>; Rushabh Lathia <[email protected]>; Suraj Kharage <[email protected]>; Tels <[email protected]>; tushar <[email protected]>; vignesh C <[email protected]>
Greetings,
On Fri, Mar 27, 2020 at 18:36 Bruce Momjian <[email protected]> wrote:
> On Thu, Mar 26, 2020 at 12:34:52PM -0400, Stephen Frost wrote:
> > * Robert Haas ([email protected]) wrote:
> > > This is where I feel like I'm trying to make decisions in a vacuum. If
> > > we had a few more people weighing in on the thread on this point, I'd
> > > be happy to go with whatever the consensus was. If most people think
> > > having both --no-manifest (suppressing the manifest completely) and
> > > --manifest-checksums=none (suppressing only the checksums) is useless
> > > and confusing, then sure, let's rip the latter one out. If most people
> > > like the flexibility, let's keep it: it's already implemented and
> > > tested. But I hate to base the decision on what one or two people
> > > think.
> >
> > I'm frustrated at the lack of involvement from others also.
>
> Well, the topic of backup manifests feels like it has generated a lot of
> bickering emails, and people don't want to spend their time dealing with
> that.
I’d like to not also. I suppose it’s just an area that I’m particularly
concerned with that allows me to overcome that. Backups are important to me.
Thanks,
Stephen
>
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 16:34 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 22:36 ` Re: backup manifests Bruce Momjian <[email protected]>
2020-03-27 22:38 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 22:39 ` Bruce Momjian <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Bruce Momjian @ 2020-03-27 22:39 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Amit Kapila <[email protected]>; Andrew Dunstan <[email protected]>; David Steele <[email protected]>; Jeevan Chalke <[email protected]>; pgsql-hackers; Rajkumar Raghuwanshi <[email protected]>; Robert Haas <[email protected]>; Rushabh Lathia <[email protected]>; Suraj Kharage <[email protected]>; Tels <[email protected]>; tushar <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 06:38:33PM -0400, Stephen Frost wrote:
> Greetings,
>
> On Fri, Mar 27, 2020 at 18:36 Bruce Momjian <[email protected]> wrote:
>
> On Thu, Mar 26, 2020 at 12:34:52PM -0400, Stephen Frost wrote:
> > * Robert Haas ([email protected]) wrote:
> > > This is where I feel like I'm trying to make decisions in a vacuum. If
> > > we had a few more people weighing in on the thread on this point, I'd
> > > be happy to go with whatever the consensus was. If most people think
> > > having both --no-manifest (suppressing the manifest completely) and
> > > --manifest-checksums=none (suppressing only the checksums) is useless
> > > and confusing, then sure, let's rip the latter one out. If most people
> > > like the flexibility, let's keep it: it's already implemented and
> > > tested. But I hate to base the decision on what one or two people
> > > think.
> >
> > I'm frustrated at the lack of involvement from others also.
>
> Well, the topic of backup manifests feels like it has generated a lot of
> bickering emails, and people don't want to spend their time dealing with
> that.
>
>
> I’d like to not also. I suppose it’s just an area that I’m particularly
> concerned with that allows me to overcome that. Backups are important to me.
The big question is whether the discussion _needs_ to be that way.
--
Bruce Momjian <[email protected]> https://momjian.us
EnterpriseDB https://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-26 20:37 ` David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 18:34 ` Re: backup manifests Robert Haas <[email protected]>
2 siblings, 2 replies; 372+ messages in thread
From: David Steele @ 2020-03-26 20:37 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Stephen Frost <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/26/20 11:37 AM, Robert Haas wrote:
>> On Wed, Mar 25, 2020 at 4:54 PM Stephen Frost <[email protected]> wrot >
> This is where I feel like I'm trying to make decisions in a vacuum. If
> we had a few more people weighing in on the thread on this point, I'd
> be happy to go with whatever the consensus was. If most people think
> having both --no-manifest (suppressing the manifest completely) and
> --manifest-checksums=none (suppressing only the checksums) is useless
> and confusing, then sure, let's rip the latter one out. If most people
> like the flexibility, let's keep it: it's already implemented and
> tested. But I hate to base the decision on what one or two people
> think.
I'm not sure I see a lot of value to being able to build manifest with
no checksums, especially if overhead for the default checksum algorithm
is negligible.
However, I'd still prefer that the default be something more robust and
allow users to tune it down rather than the other way around. But I've
made that pretty clear up-thread and I consider that argument lost at
this point.
>> As for folks who are that close to the edge on their backup timing that
>> they can't have it slow down- chances are pretty darn good that they're
>> not far from ending up needing to find a better solution than
>> pg_basebackup anyway. Or they don't need to generate a manifest (or, I
>> suppose, they could have one but not have checksums..).
>
> 40-50% is a lot more than "if you were on the edge."
For the record I think this is a very misleading number. Sure, if you
are doing your backup to a local SSD on a powerful development laptop it
makes sense.
But backups are generally placed on slower storage, remotely, with
compression. Even without compression the first two are going to bring
this percentage down by a lot.
When you get to page-level incremental backups, which is where this all
started, I'd still recommend using a stronger checksum algorithm to
verify that the file was reconstructed correctly on restore. That much
I believe we have agreed on.
>> Even pg_basebackup (in both fetch and stream modes...) checks that we at
>> least got all the WAL that's needed for the backup from the server
>> before considering the backup to be valid and telling the user that
>> there was a successful backup. With what you're proposing here, we
>> could have someone do a pg_basebackup, get back an ERROR saying the
>> backup wasn't valid, and then run pg_validatebackup and be told that the
>> backup is valid. I don't get how that's sensible.
>
> I'm sorry that you can't see how that's sensible, but it doesn't mean
> that it isn't sensible. It is totally unrealistic to expect that any
> backup verification tool can verify that you won't get an error when
> trying to use the backup. That would require that everything that the
> validation tool try to do everything that PostgreSQL will try to do
> when the backup is used, including running recovery and updating the
> data files. Anything less than that creates a real possibility that
> the backup will verify good but fail when used. This tool has a much
> narrower purpose, which is to try to verify that we (still) have the
> files the server sent as part of the backup and that, to the best of
> our ability to detect such things, they have not been modified. As you
> know, or should know, the WAL files are not sent as part of the
> backup, and so are not verified. Other things that would also be
> useful to check are also not verified. It would be fantastic to have
> more verification tools in the future, but it is difficult to see why
> anyone would bother trying if an attempt to get the first one
> committed gets blocked because it does not yet do everything. Very few
> patches try to do everything, and those that do usually get blocked
> because, by trying to do too much, they get some of it badly wrong.
I agree with Stephen that this should be done, but I agree with you that
it can wait for a future commit. However, I do think:
1) It should be called out rather plainly in the documentation.
2) If there are files in pg_wal then pg_validatebackup should inform the
user that those files have not been validated.
I know you and Stephen have agreed on a number of doc changes, would it
be possible to get a new patch with those included? I finally have time
to do a review of this tomorrow. I saw some mistakes in the docs in the
current patch but I know those patches are not current.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
@ 2020-03-27 17:53 ` Robert Haas <[email protected]>
2020-03-27 19:50 ` Re: backup manifests David Steele <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
1 sibling, 2 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-27 17:53 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Thu, Mar 26, 2020 at 4:37 PM David Steele <[email protected]> wrote:
> I know you and Stephen have agreed on a number of doc changes, would it
> be possible to get a new patch with those included? I finally have time
> to do a review of this tomorrow. I saw some mistakes in the docs in the
> current patch but I know those patches are not current.
Hi David,
Here's a new version with some fixes:
- Fixes for doc typos noted by Stephen Frost and Andres Freund.
- Replace a doc paragraph about the advantages and disadvantages of
CRC-32C with one by Stephen Frost, with a slightly change by me that I
thought made it sound more grammatical.
- Change the pg_validatebackup documentation so that it makes no
mention of compatible tools, per Stephen.
- Reword the discussion of the exclude list in the pg_validatebackup
documentation, per discussion between Stephen and myself.
- Try to make the documentation more clear about the fact that we
check for both extra and missing files.
- Incorporate a fix from Amit Kapila to make 003_corruption.pl pass on Windows.
HTH,
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v14-0001-Add-checksum-helper-functions.patch (9.6K, ../../CA+TgmoZfj3wqcgoaXp+4a7ZSk3coARcxvghT7m=7Hf=8r6st6w@mail.gmail.com/2-v14-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From 78613b46ebf7602f44b944ff4c6333b455ef647c Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 20 Mar 2020 14:48:33 -0400
Subject: [PATCH v14 1/2] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index ce01df68b9..e199ed7acb 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -47,6 +47,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index f89a8a4fdb..ee04fbafa6 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -120,8 +120,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
- f2s.c file_perm.c hashfn.c ip.c jsonapi.c
+ base64.c checksum_helper.c config_info.c controldata_utils.c d2s.c
+ encnames.c exec.c f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c username.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v14-0002-Generate-backup-manifests-for-base-backups-and-v.patch (117.4K, ../../CA+TgmoZfj3wqcgoaXp+4a7ZSk3coARcxvghT7m=7Hf=8r6st6w@mail.gmail.com/3-v14-0002-Generate-backup-manifests-for-base-backups-and-v.patch)
download | inline diff:
From b5782922f0ab7037d7d7dc7ce94ca2fa5217ce56 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 23 Mar 2020 12:07:20 -0400
Subject: [PATCH v14 2/2] Generate backup manifests for base backups, and
validate them.
A manifest is a JSON document which includes the file name, size, last
modification time, and a checksum for each file backed up, as well as
a checksum for the manifest itself. By default, we use CRC-32C for the
checksum algorithm, because we are trying to detect corruption and
user error, not foil an adversary. However, pg_basebackup and the
server-side BASE_BACKUP command now have options to select the
checksum algorithm, so users wanting a cryptographic hash function can
select SHA-224, SHA-256, SHA-384, or SHA-512. Users not wanting any
checksums at all can disable them, or disable generating of the backup
manifest altogether. Using a cryptographic hash function in place of
CRC-32C consumes significantly more CPU cycles, which may slow down
backups in some cases.
A new tool called pg_validatebackup can validate a backup against the
manifest. If no checksums are present, it can still check that the
right files exist and that they have the expected sizes. If checksums
are present, it can also verify that each file has the expected
checksum. Only plain format backups can be validated directly, but tar
format backups can be validated after extracting them.
Robert Haas, with help, ideas, review, and testing from David Steele,
Stephen Frost, Andrew Dunstan, Rushabh Lathia, Suraj Kharage, Tushar
Ahuja, Rajkumar Raghuwanshi, Mark Dilger, Davinder Singh, Jeevan
Chalke, Amit Kapila, and Andres Freund.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
doc/src/sgml/protocol.sgml | 33 +-
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_basebackup.sgml | 61 ++
doc/src/sgml/ref/pg_validatebackup.sgml | 240 ++++++
doc/src/sgml/reference.sgml | 1 +
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 430 +++++++++-
src/backend/replication/repl_gram.y | 13 +
src/backend/replication/repl_scanner.l | 2 +
src/backend/replication/walsender.c | 30 +
src/bin/Makefile | 1 +
src/bin/pg_basebackup/pg_basebackup.c | 184 ++++-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 8 +-
src/bin/pg_validatebackup/.gitignore | 2 +
src/bin/pg_validatebackup/Makefile | 39 +
src/bin/pg_validatebackup/parse_manifest.c | 576 ++++++++++++++
src/bin/pg_validatebackup/parse_manifest.h | 40 +
src/bin/pg_validatebackup/pg_validatebackup.c | 734 ++++++++++++++++++
src/bin/pg_validatebackup/t/001_basic.pl | 30 +
src/bin/pg_validatebackup/t/002_algorithm.pl | 58 ++
src/bin/pg_validatebackup/t/003_corruption.pl | 251 ++++++
src/bin/pg_validatebackup/t/004_options.pl | 89 +++
.../pg_validatebackup/t/005_bad_manifest.pl | 158 ++++
src/bin/pg_validatebackup/t/006_encoding.pl | 27 +
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
26 files changed, 2987 insertions(+), 32 deletions(-)
create mode 100644 doc/src/sgml/ref/pg_validatebackup.sgml
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/parse_manifest.c
create mode 100644 src/bin/pg_validatebackup/parse_manifest.h
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
create mode 100644 src/bin/pg_validatebackup/t/001_basic.pl
create mode 100644 src/bin/pg_validatebackup/t/002_algorithm.pl
create mode 100644 src/bin/pg_validatebackup/t/003_corruption.pl
create mode 100644 src/bin/pg_validatebackup/t/004_options.pl
create mode 100644 src/bin/pg_validatebackup/t/005_bad_manifest.pl
create mode 100644 src/bin/pg_validatebackup/t/006_encoding.pl
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index f139ba0231..4e26c3b391 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2466,7 +2466,7 @@ The commands accepted in replication mode are:
</varlistentry>
<varlistentry id="protocol-replication-base-backup" xreflabel="BASE_BACKUP">
- <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ]
+ <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ] [ <literal>MANIFEST</literal> <replaceable>manifest_option</replaceable> ] [ <literal>MANIFEST_CHECKSUMS</literal> <replaceable>checksum_algorithm</replaceable> ]
<indexterm><primary>BASE_BACKUP</primary></indexterm>
</term>
<listitem>
@@ -2576,6 +2576,37 @@ The commands accepted in replication mode are:
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST</literal></term>
+ <listitem>
+ <para>
+ When this option is specified with a value of <literal>yes</literal>
+ or <literal>force-escape</literal>, a backup manifest is created
+ and sent along with the backup. The latter value forces all filenames
+ to be hex-encoded; otherwise, this type of encoding is performed only
+ for files whose names are non-UTF8 octet sequences.
+ <literal>force-escape</literal> is intended primarily for testing
+ purposes, to be sure that clients which read the backup manifest
+ can handle this case. For compatibility with previous releases,
+ the default is <literal>MANIFEST 'no'</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST_CHECKSUMS</literal></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be used to checksum each file
+ for purposes of the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
<para>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..ab71176cdf 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -211,6 +211,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
<!ENTITY pgRestore SYSTEM "pg_restore.sgml">
<!ENTITY pgRewind SYSTEM "pg_rewind.sgml">
+<!ENTITY pgValidateBackup SYSTEM "pg_validatebackup.sgml">
<!ENTITY pgtestfsync SYSTEM "pgtestfsync.sgml">
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index 90638aad0e..fb749e953f 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -561,6 +561,67 @@ PostgreSQL documentation
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>--no-manifest</option></term>
+ <listitem>
+ <para>
+ Disables generation of a backup manifest. If this option is not
+ specified, the server will generate and send a backup manifest
+ which can be verified using <xref linkend="app-pgvalidatebackup" />.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-force-encode</option></term>
+ <listitem>
+ <para>
+ Forces all filenames in the backup manifest to be hex-encoded.
+ If this option is not specified, only non-UTF8 filenames are
+ hex-encoded. This option is mostly intended to test that tools which
+ read a backup manifest file properly handle this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be used to checksum each file
+ for purposes of the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ <para>
+ If <literal>NONE</literal> is selected, the backup manifest will
+ not contain any checksums. Otherwise, it will contain a checksum
+ of each file in the backup using the specified algorithm. In addition,
+ the manifest itself will always contain a <literal>SHA256</literal>
+ checksum of its own contents. The <literal>SHA</literal> algorithms
+ are significantly more CPU-intensive than <literal>CRC32C</literal>,
+ so selecting one of them may increase the time required to complete
+ the backup.
+ </para>
+ <para>
+ Using a SHA hash function provides a cryptographically secure digest
+ of each file for users who wish to verify that the backup has not been
+ tampered with, while the CRC32C algorithm provides a checksum which is
+ much faster to calculate and good at catching errors due to accidental
+ changes but is not resistent to targeted modifications. Note that, to
+ be useful against an adversary who has access to the backup, the backup
+ manifest would need to be stored securely elsewhere or otherwise
+ verified not to have been modified since the backup was taken.
+ </para>
+ <para>
+ <xref linkend="app-pgvalidatebackup" /> can be used to check the
+ integrity of a backup against the backup manifest.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
new file mode 100644
index 0000000000..aaeb9220c2
--- /dev/null
+++ b/doc/src/sgml/ref/pg_validatebackup.sgml
@@ -0,0 +1,240 @@
+<!--
+doc/src/sgml/ref/pg_validatebackup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgvalidatebackup">
+ <indexterm zone="app-pgvalidatebackup">
+ <primary>pg_validatebackup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>pg_validatebackup</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_validatebackup</refname>
+ <refpurpose>verify the integrity of a base backup of a
+ <productname>PostgreSQL</productname> cluster</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_validatebackup</command>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>
+ Description
+ </title>
+ <para>
+ <application>pg_validatebackup</application> is used to check the integrity
+ of a database cluster backup taken using <command>pg_basebackup</command>.
+ The backup must be stored in the "plain" format; a "tar" format backup can be
+ checked after extracting it. Backup manifests are created by the server beginning
+ with <productname>PostgreSQL</productname> version 13, so older backups
+ cannot be validated using this tool.
+ </para>
+
+ <para>
+ <application>pg_validatebackup</application> reads the manifest file of a
+ backup, verifies the manifest against its own internal checksum, and then
+ verifies that the same files are present in the target directory as in the
+ manifest itself. Note that this process checks both for files which are present
+ in the manifest but not in the backup and also for files which are present
+ in the backup but not mentioned in the manifest. After checking that the right
+ files are present, it then verifies that each file has the expected checksum,
+ unless the backup was taken with the checksum algorithm set to
+ <literal>none</literal>, in which case checksum verification is not
+ performed. The presence or absence of directories is not checked, except
+ indirectly: if a directory is missing, any files it should have contained
+ will necessarily also be missing.
+ </para>
+
+ <para>
+ When pg_basebackup compares the files and directories in the manifest
+ to those which are present on disk, it will ignore the presence of, or
+ changes to, certain files:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>backup_manifest</literal> will not be present in the
+ manifeste itself, and is therefore ignored. Note that the manifest
+ is still verified internally, but no error will be issued about the
+ presence of a backup_manifest file in the backup directory even though
+ it is not listed in the manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>pg_wal</literal> is ignored because WAL files are sent
+ separately from the backup, and are therefore not described by the
+ backup manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>postgesql.auto.conf</literal>,
+ <literal>standby.signal</literal>,
+ and <literal>recovery.signal</literal> are ignored because they may
+ sometimes be created or modified by the backup client itself.
+ (For example, <literal>pg_basebackup -R</literal> will modify
+ <literal>postgresql.auto.conf</literal> and create
+ <literal>standby.signal</literal>.)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ The following command-line options control the behavior.
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-e</option></term>
+ <term><option>--exit-on-error</option></term>
+ <listitem>
+ <para>
+ Exit as soon as a problem with the backup is detected. If this option
+ is not specified, <literal>pg_basebackup</literal> will continue
+ checking the backup even after a problem has been detected, and will
+ report all problems detected as errors.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-i <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--ignore=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Ignore the specified file or directory, which should be expressed
+ as a relative pathname. If the backup contains extra files, is
+ missing files, or has files that have been modified as compared with
+ what is described in the manifest, this option can be used to suppress
+ the errors that would otherwise occur. If a directory is specified,
+ this option affects the entire subtree rooted at that location.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-m <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--manifest-path=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Use the manifest file at the specified path, rather than one located
+ in the root of the backup directory.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-q</option></term>
+ <term><option>--quiet</option></term>
+ <listitem>
+ <para>
+ Don't print anything when a backup is successfully validated.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-s</option></term>
+ <term><option>--skip-checksums</option></term>
+ <listitem>
+ <para>
+ Do not validate checksums. The presence or absence of files and the
+ sizes of those files will still be checked. This is much faster,
+ because the files themselves do not need to read.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other options are also available:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_validatebackup</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_validatebackup</application> command
+ line arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal> and
+ validate the integrity of the backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal>, move
+ the manifest somewhere outside the backup directory, and validate the
+ backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234</userinput>
+<prompt>$</prompt> <userinput>mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234</userinput>
+</screen>
+ </para>
+
+ <para>
+ To validate a backup while ignoring a file that was added manually to the
+ backup directory, and also skipping checksum verification:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>edit /usr/local/pgsql/data/note.to.self</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="app-pgbasebackup"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..d25a77b13c 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -255,6 +255,7 @@
&pgReceivewal;
&pgRecvlogical;
&pgRestore;
+ &pgValidateBackup;
&psqlRef;
&reindexdb;
&vacuumdb;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 793c076da6..b3917bc526 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10551,7 +10551,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 806d013108..6dffc6ef5b 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -39,10 +41,19 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
+typedef enum manifest_option
+{
+ MANIFEST_OPTION_YES,
+ MANIFEST_OPTION_NO,
+ MANIFEST_OPTION_FORCE_ENCODE
+} manifest_option;
+
typedef struct
{
const char *label;
@@ -52,20 +63,43 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ manifest_option manifest;
+ pg_checksum_type manifest_checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ bool force_encode;
+ bool first_file;
+ bool still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest,
+ basebackup_options *opt);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +136,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -251,6 +295,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -258,12 +303,17 @@ perform_base_backup(basebackup_options *opt)
backup_streamed = 0;
pgstat_progress_start_command(PROGRESS_COMMAND_BASEBACKUP, InvalidOid);
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt);
total_checksum_failures = 0;
@@ -301,7 +351,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -380,7 +433,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -388,11 +442,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -400,10 +457,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -632,7 +690,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -655,16 +713,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -678,6 +740,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -709,8 +774,13 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->manifest = MANIFEST_OPTION_NO;
+ opt->manifest_checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -797,12 +867,61 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest") == 0)
+ {
+ char *optval = strVal(defel->arg);
+ bool manifest_bool;
+
+ if (o_manifest)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (parse_bool(optval, &manifest_bool))
+ {
+ if (manifest_bool)
+ opt->manifest = MANIFEST_OPTION_YES;
+ else
+ opt->manifest = MANIFEST_OPTION_NO;
+ }
+ else if (pg_strcasecmp(optval, "force-encode") == 0)
+ opt->manifest = MANIFEST_OPTION_FORCE_ENCODE;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized manifest option: \"%s\"",
+ optval)));
+ o_manifest = true;
+ }
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval,
+ &opt->manifest_checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
}
if (opt->label == NULL)
opt->label = "base backup";
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ {
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("manifest checksums require a backup manifest")));
+ opt->manifest_checksum_type = CHECKSUM_TYPE_NONE;
+ }
}
@@ -918,6 +1037,249 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, basebackup_options *opt)
+{
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ manifest->buffile = NULL;
+ else
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = opt->manifest_checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->force_encode = (opt->manifest == MANIFEST_OPTION_FORCE_ENCODE);
+ manifest->first_file = true;
+ manifest->still_checksumming = true;
+
+ if (opt->manifest != MANIFEST_OPTION_NO)
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ Assert(manifest != NULL);
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ int pathlen;
+ StringInfoData buf;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (!manifest->force_encode &&
+ pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
+
+ /*
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
+ */
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
+
+ /* Add checksum information. */
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
+ }
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
+
+ /* Avoid leaking memory. */
+ pfree(buf.data);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\"}\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -978,11 +1340,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1017,6 +1383,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1027,7 +1397,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1060,7 +1431,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1079,7 +1451,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1359,7 +1731,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1367,7 +1740,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1437,8 +1811,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1455,6 +1830,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1625,6 +2003,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1649,6 +2030,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1657,7 +2039,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1682,6 +2065,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..f93a0de218 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,8 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -156,6 +158,7 @@ var_name: IDENT { $$ = $1; }
/*
* BASE_BACKUP [LABEL '<label>'] [PROGRESS] [FAST] [WAL] [NOWAIT]
* [MAX_RATE %d] [TABLESPACE_MAP] [NOVERIFY_CHECKSUMS]
+ * [MANIFEST %s] [MANIFEST_CHECKSUMS %s]
*/
base_backup:
K_BASE_BACKUP base_backup_opt_list
@@ -214,6 +217,16 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST SCONST
+ {
+ $$ = makeDefElem("manifest",
+ (Node *)makeString($2), -1);
+ }
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..452ad9fc27 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,8 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST { return K_MANIFEST; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 76ec3c7dd0..3b117d8367 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index c5d95958b2..f355eb612c 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -136,6 +142,9 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static bool manifest = true;
+static bool manifest_force_encode = false;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -181,6 +190,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -388,6 +403,11 @@ usage(void)
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
printf(_(" --no-estimate-size do not estimate backup size in server side\n"));
+ printf(_(" --no-manifest suppress generation of backup manifest\n"));
+ printf(_(" --manifest-force-encode\n"
+ " hex encode all filenames in manifest\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1186,6 +1206,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1657,6 +1702,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1667,6 +1770,8 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_clause;
+ char *manifest_checksums_clause = "";
int i;
char xlogstart[64];
char xlogend[64];
@@ -1674,6 +1779,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1727,6 +1833,32 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest)
+ {
+ if (serverMajor < 1300)
+ {
+ const char *serverver = PQparameterStatus(conn, "server_version");
+
+ pg_log_error("backup manifests are not supported by server version %s",
+ serverver ? serverver : "'unknown'");
+ exit(1);
+ }
+
+ if (manifest_force_encode)
+ manifest_clause = "MANIFEST 'force-encode'";
+ else
+ manifest_clause = "MANIFEST 'yes'";
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
+ }
+ else
+ {
+ if (serverMajor < 1300)
+ manifest_clause = "";
+ else
+ manifest_clause = "MANIFEST 'no'";
+ }
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1741,7 +1873,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s %s",
escaped_label,
estimatesize ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1749,7 +1881,9 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_clause,
+ manifest_checksums_clause);
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1837,7 +1971,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1866,6 +2001,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && manifest)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2069,6 +2217,9 @@ main(int argc, char **argv)
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
{"no-estimate-size", no_argument, NULL, 4},
+ {"no-manifest", no_argument, NULL, 5},
+ {"manifest-force-encode", no_argument, NULL, 6},
+ {"manifest-checksums", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
int c;
@@ -2096,7 +2247,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2240,6 +2391,15 @@ main(int argc, char **argv)
case 4:
estimatesize = false;
break;
+ case 5:
+ manifest = false;
+ break;
+ case 6:
+ manifest_force_encode = true;
+ break;
+ case 7:
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
@@ -2370,6 +2530,22 @@ main(int argc, char **argv)
exit(1);
}
+ if (!manifest && manifest_checksums != NULL)
+ {
+ pg_log_error("--no-manifest and --manifest-checksums are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (!manifest && manifest_force_encode)
+ {
+ pg_log_error("--no-manifest and --manifest-force-encode are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
/* connection in replication mode to server */
conn = GetConnection();
if (!conn)
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..63381764e9 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 109;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -104,6 +104,7 @@ foreach my $filename (@tempRelationFiles)
$node->command_ok([ 'pg_basebackup', '-D', "$tempdir/backup", '-X', 'none' ],
'pg_basebackup runs');
ok(-f "$tempdir/backup/PG_VERSION", 'backup was created');
+ok(-f "$tempdir/backup/backup_manifest", 'backup manifest included');
# Permissions on backup should be default
SKIP:
@@ -160,11 +161,12 @@ rmtree("$tempdir/backup");
$node->command_ok(
[
- 'pg_basebackup', '-D', "$tempdir/backup2", '--waldir',
- "$tempdir/xlog2"
+ 'pg_basebackup', '-D', "$tempdir/backup2", '--no-manifest',
+ '--waldir', "$tempdir/xlog2"
],
'separate xlog directory');
ok(-f "$tempdir/backup2/PG_VERSION", 'backup was created');
+ok(! -f "$tempdir/backup2/backup_manifest", 'manifest was suppressed');
ok(-d "$tempdir/xlog2/", 'xlog directory was created');
rmtree("$tempdir/backup2");
rmtree("$tempdir/xlog2");
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..21e0a92429
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1,2 @@
+/pg_validatebackup
+/tmp_check/
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..04ef7d3051
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,39 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ parse_manifest.o \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
new file mode 100644
index 0000000000..e6b42adfda
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -0,0 +1,576 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.c
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "parse_manifest.h"
+#include "common/jsonapi.h"
+
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_ENCODED_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+/*
+ * Internal state used while decoding the JSON-format backup manifest.
+ */
+typedef struct
+{
+ JsonManifestParseContext *context;
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ char *pathname;
+ char *encoded_pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void verify_manifest_checksum(JsonManifestParseState *parse,
+ char *buffer, size_t size);
+static void json_manifest_parse_failure(JsonManifestParseContext *context,
+ char *msg);
+
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+
+/*
+ * Main entrypoint to parse a JSON-format backup manifest.
+ *
+ * Caller should set up the parsing context and then invoke this function.
+ * For each file whose information is extracted from the manifest,
+ * context->perfile_cb is invoked. In case of trouble, context->error_cb is
+ * invoked and is expected not to return.
+ */
+void
+json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ size_t size)
+{
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
+
+ /* Set up our private parsing context. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.context = context;
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+ /* Validate the checksum. */
+ verify_manifest_checksum(&parse, buffer, size);
+}
+
+/*
+ * Invoked at the start of each object in the JSON document.
+ *
+ * The document as a whole is expected to be an object with three keys
+ * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
+ * file is expected to be an object with various keys (Path, Size, etc.).
+ * If we're not at the beginning of either the toplevel object or the object
+ * for a particular file, it's an error.
+ */
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->encoded_pathname = NULL;
+ parse->size = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each object in the JSON document.
+ *
+ * The possible cases here are the same as for json_manifest_object_start.
+ * There's nothing special to do at the end of the document, but when we
+ * reach the end of an object representing a particular file, we must call
+ * json_manifest_finalize_file() to save the associated details.
+ */
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ json_manifest_finalize_file(parse);
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each array in the JSON document.
+ *
+ * Within the toplevel object, the value associated with the "Files" key
+ * should be an array. No other arrays are expected.
+ */
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each array in the JSON document.
+ *
+ * Just like json_manifest_array_start, there's only one expected case
+ * here.
+ */
+static void
+json_manifest_array_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each object field in the JSON document.
+ */
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_FIELD:
+ /* Inside toplevel object, expecting version indicator. */
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ /* Inside toplevel object, expecting "Files" next. */
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Encoded-Path") == 0)
+ parse->field = JMFF_ENCODED_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure(parse->context,
+ "unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ /* Inside toplevel object, expecting "Manifest-Checksum" next. */
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object field");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each scalar in the JSON document.
+ *
+ * Object field names don't reach this code; those are handled by
+ * json_manifest_object_field_start. When we're inside of the object for
+ * a particular file, that function will have noticed the name of the field,
+ * and we'll get the corresponding value here. When we're in the toplevel
+ * object, the parse state itself tells us which field this is.
+ *
+ * In all cases except for PostgreSQL-Backup-Manifest-Version, which we
+ * can just check on the spot, the goal here is just to save the value in
+ * the parse state for later use. We don't actually do anything until we
+ * reach either the end of the object representing this file, or the end
+ * of the manifest, as the case may be.
+ */
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure(parse->context,
+ "unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_ENCODED_PATH:
+ parse->encoded_pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context, "unexpected scalar");
+ break;
+ }
+}
+
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * file, and invoke the per-file callback so that the caller gets those
+ * details. This happens for each file when the corresponding JSON object is
+ * completely parsed.
+ */
+static void
+json_manifest_finalize_file(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t size;
+ char *ep;
+ int checksum_string_length;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL && parse->encoded_pathname == NULL)
+ json_manifest_parse_failure(parse->context, "missing pathname");
+ if (parse->pathname != NULL && parse->encoded_pathname != NULL)
+ json_manifest_parse_failure(parse->context,
+ "both pathname and encoded pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure(parse->context, "missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure(parse->context,
+ "checksum without algorithm");
+
+ /* Decode encoded pathname, if that's what we have. */
+ if (parse->encoded_pathname != NULL)
+ {
+ int encoded_length = strlen(parse->encoded_pathname);
+ int raw_length = encoded_length / 2;
+
+ parse->pathname = palloc(raw_length + 1);
+ if (encoded_length % 2 != 0 ||
+ !hexdecode_string((uint8 *) parse->pathname,
+ parse->encoded_pathname,
+ raw_length))
+ json_manifest_parse_failure(parse->context,
+ "unable to decode filename");
+ parse->pathname[raw_length] = '\0';
+ pfree(parse->encoded_pathname);
+ parse->encoded_pathname = NULL;
+ }
+
+ /* Parse size. */
+ size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm, &checksum_type))
+ context->error_cb(context, "unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ checksum_length = 0;
+ checksum_payload = NULL;
+ }
+ else
+ {
+ checksum_length = checksum_string_length / 2;
+ checksum_payload = palloc(checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(checksum_payload, parse->checksum,
+ checksum_length))
+ context->error_cb(context,
+ "invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, parse->checksum);
+ }
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perfile_cb(context, parse->pathname, size,
+ checksum_type, checksum_length, checksum_payload);
+
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
+}
+
+/*
+ * Verify that the manifest checksum is correct.
+ *
+ * The last line of the manifest file is excluded from the manifest checksum,
+ * because the last line is expected to contain the checksum that covers
+ * the rest of the file.
+ */
+static void
+verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
+ size_t size)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t i;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+
+ /* Find the last two newlines in the file. */
+ for (i = 0; i < size; ++i)
+ {
+ if (buffer[i] == '\n')
+ {
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
+ }
+ }
+
+ /*
+ * Make sure that the last newline is right at the end, and that there are
+ * at least two lines total. We need this to be true in order for the
+ * following code, which computes the manifest checksum, to work properly.
+ */
+ if (number_of_newlines < 2)
+ json_manifest_parse_failure(parse->context,
+ "expected at least 2 lines");
+ if (ultimate_newline != size - 1)
+ json_manifest_parse_failure(parse->context,
+ "last line not newline-terminated");
+
+ /* Checksum the rest. */
+ pg_sha256_init(&manifest_ctx);
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+
+ /* Now verify it. */
+ if (parse->manifest_checksum == NULL)
+ context->error_cb(parse->context, "manifest has no checksum");
+ if (strlen(parse->manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse->manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ context->error_cb(context, "invalid manifest checksum: \"%s\"",
+ parse->manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ context->error_cb(context, "manifest checksum mismatch");
+}
+
+/*
+ * Report a parse error.
+ *
+ * This is intended to be used for fairly low-level failures that probably
+ * shouldn't occur unless somebody has deliberately constructed a bad manifest,
+ * or unless the server is generating bad manifests due to some bug. msg should
+ * be a short string giving some hint as to what the problem is.
+ */
+static void
+json_manifest_parse_failure(JsonManifestParseContext *context, char *msg)
+{
+ context->error_cb(context, "could not parse backup manifest: %s", msg);
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
new file mode 100644
index 0000000000..25d140f72f
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -0,0 +1,40 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.h
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PARSE_MANIFEST_H
+#define PARSE_MANIFEST_H
+
+#include "common/checksum_helper.h"
+#include "mb/pg_wchar.h"
+
+struct JsonManifestParseContext;
+typedef struct JsonManifestParseContext JsonManifestParseContext;
+
+typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
+ char *pathname,
+ size_t size, pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
+ char *fmt, ...) pg_attribute_printf(2, 3);
+
+struct JsonManifestParseContext
+{
+ void *private_data;
+ json_manifest_perfile_callback perfile_cb;
+ json_manifest_error_callback error_cb;
+};
+
+extern void json_parse_manifest(JsonManifestParseContext *context,
+ char *buffer, size_t size);
+
+#endif
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..eb1473d9d0
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,734 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+#include "parse_manifest.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+
+static void record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length,
+ uint8 *checksum_payload);
+static void report_manifest_error(JsonManifestParseContext *context,
+ char *fmt, ...)
+ pg_attribute_printf(2, 3) pg_attribute_noreturn();
+
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void report_backup_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void report_fatal_error(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ printf("backup successfully verified\n");
+
+ return context.saw_any_error ? 1 : 0;
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ int rc;
+ JsonManifestParseContext context;
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ report_fatal_error("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ report_fatal_error("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /*
+ * Slurp in the whole file.
+ *
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
+ */
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
+ {
+ if (rc < 0)
+ report_fatal_error("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ report_fatal_error("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
+
+ /* Close the manifest file. */
+ close(fd);
+
+ /* Parse the manifest as JSON. */
+ context.private_data = ht;
+ context.perfile_cb = record_manifest_details_for_file;
+ context.error_cb = report_manifest_error;
+ json_parse_manifest(&context, buffer, statbuf.st_size);
+
+ /* Done with the buffer. */
+ pfree(buffer);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * Report an error while parsing the manifest.
+ *
+ * We consider all such errors to be fatal errors. The manifest parser
+ * expects this function not to return.
+ */
+static void
+report_manifest_error(JsonManifestParseContext *context, char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Record details extracted from the backup manifest for one file.
+ */
+static void
+record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload)
+{
+ manifestfiles_hash *ht = context->private_data;
+ manifestfile *tabent;
+ bool found;
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(ht, pathname, &found);
+ if (found)
+ report_fatal_error("duplicate pathname in backup manifest: \"%s\"",
+ pathname);
+
+ /* Initialize the entry. */
+ tabent->size = size;
+ tabent->checksum_type = checksum_type;
+ tabent->checksum_length = checksum_length;
+ tabent->checksum_payload = checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ report_fatal_error("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ report_backup_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ report_backup_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ report_backup_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ report_backup_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ report_backup_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ report_backup_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ report_backup_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ report_backup_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ report_backup_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ report_backup_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ report_backup_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ report_backup_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Report a problem with the backup.
+ *
+ * Update the context to indicate that we saw an error, and exit if the
+ * context says we should.
+ */
+static void
+report_backup_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+report_fatal_error(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+ printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
+}
diff --git a/src/bin/pg_validatebackup/t/001_basic.pl b/src/bin/pg_validatebackup/t/001_basic.pl
new file mode 100644
index 0000000000..6d4b8ea01a
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/001_basic.pl
@@ -0,0 +1,30 @@
+use strict;
+use warnings;
+use TestLib;
+use Test::More tests => 16;
+
+my $tempdir = TestLib::tempdir;
+
+program_help_ok('pg_validatebackup');
+program_version_ok('pg_validatebackup');
+program_options_handling_ok('pg_validatebackup');
+
+command_fails_like(['pg_validatebackup'],
+ qr/no backup directory specified/,
+ 'target directory must be specified');
+command_fails_like(['pg_validatebackup', $tempdir],
+ qr/could not open file.*\/backup_manifest\"/,
+ 'pg_validatebackup requires a manifest');
+command_fails_like(['pg_validatebackup', $tempdir, $tempdir],
+ qr/too many command-line arguments/,
+ 'multiple target directories not allowed');
+
+# create fake manifest file
+open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+close($fh);
+
+# but then try to use an alternate, nonexisting manifest
+command_fails_like(['pg_validatebackup', '-m', "$tempdir/not_the_manifest",
+ $tempdir],
+ qr/could not open file.*\/not_the_manifest\"/,
+ 'pg_validatebackup respects -m flag');
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
new file mode 100644
index 0000000000..98871e12a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -0,0 +1,58 @@
+# Verify that we can take and validate backups with various checksum types.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 19;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
+{
+ my $backup_path = $master->backup_dir . '/' . $algorithm;
+ my @backup = ('pg_basebackup', '-D', $backup_path,
+ '--manifest-checksums', $algorithm,
+ '--no-sync');
+ my @validate = ('pg_validatebackup', '-e', $backup_path);
+
+ # A backup with a bogus algorithm should fail.
+ if ($algorithm eq 'bogus')
+ {
+ $master->command_fails(\@backup,
+ "backup fails with algorithm \"$algorithm\"");
+ next;
+ }
+
+ # A backup with a valid algorithm should work.
+ $master->command_ok(\@backup, "backup ok with algorithm \"$algorithm\"");
+
+ # We expect each real checksum algorithm to be mentioned on every line of
+ # the backup manifest file except the first and last; for simplicity, we
+ # just check that it shows up lots of times. When the checksum algorithm
+ # is none, we just check that the manifest exists.
+ if ($algorithm eq 'none')
+ {
+ ok(-f "$backup_path/backup_manifest", "backup manifest exists");
+ }
+ else
+ {
+ my $manifest = slurp_file("$backup_path/backup_manifest");
+ my $count_of_algorithm_in_manifest =
+ (() = $manifest =~ /$algorithm/mig);
+ cmp_ok($count_of_algorithm_in_manifest, '>', 100,
+ "$algorithm is mentioned many times in the manifest");
+ }
+
+ # Make sure that it validates OK.
+ $master->command_ok(\@validate,
+ "validate backup with algorithm \"$algorithm\"");
+
+ # Remove backup immediately to save disk space.
+ rmtree($backup_path);
+}
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
new file mode 100644
index 0000000000..6ad29a031f
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -0,0 +1,251 @@
+# Verify that various forms of corruption are detected by pg_validatebackup.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+# Include a user-defined tablespace in the hopes of detecting problems in that
+# area.
+my $source_ts_path = TestLib::tempdir;
+$master->safe_psql('postgres', <<EOM);
+CREATE TABLE x1 (a int);
+INSERT INTO x1 VALUES (111);
+CREATE TABLESPACE ts1 LOCATION '$source_ts_path';
+CREATE TABLE x2 (a int) TABLESPACE ts1;
+INSERT INTO x1 VALUES (222);
+EOM
+
+my @scenario = (
+ {
+ 'name' => 'extra_file',
+ 'mutilate' => \&mutilate_extra_file,
+ 'fails_like' =>
+ qr/extra_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'extra_tablespace_file',
+ 'mutilate' => \&mutilate_extra_tablespace_file,
+ 'fails_like' =>
+ qr/extra_ts_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'missing_file',
+ 'mutilate' => \&mutilate_missing_file,
+ 'fails_like' =>
+ qr/pg_xact\/0000.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'missing_tablespace',
+ 'mutilate' => \&mutilate_missing_tablespace,
+ 'fails_like' =>
+ qr/pg_tblspc.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'append_to_file',
+ 'mutilate' => \&mutilate_append_to_file,
+ 'fails_like' =>
+ qr/has size \d+ on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'truncate_file',
+ 'mutilate' => \&mutilate_truncate_file,
+ 'fails_like' =>
+ qr/has size 0 on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'replace_file',
+ 'mutilate' => \&mutilate_replace_file,
+ 'fails_like' => qr/checksum mismatch for file/
+ },
+ {
+ 'name' => 'bad_manifest',
+ 'mutilate' => \&mutilate_bad_manifest,
+ 'fails_like' => qr/manifest checksum mismatch/
+ },
+ {
+ 'name' => 'open_file_fails',
+ 'mutilate' => \&mutilate_open_file_fails,
+ 'fails_like' => qr/could not open file/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'open_directory_fails',
+ 'mutilate' => \&mutilate_open_directory_fails,
+ 'fails_like' => qr/could not open directory/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'search_directory_fails',
+ 'mutilate' => \&mutilate_search_directory_fails,
+ 'cleanup' => \&cleanup_search_directory_fails,
+ 'fails_like' => qr/could not stat file or directory/,
+ 'skip_on_windows' => 1
+ }
+);
+
+for my $scenario (@scenario)
+{
+ my $name = $scenario->{'name'};
+
+ SKIP:
+ {
+ skip "unix-style permissions not supported on Windows", 4
+ if $scenario->{'skip_on_windows'} && $windows_os;
+
+ # Take a backup and check that it validates OK.
+ my $backup_path = $master->backup_dir . '/' . $name;
+ my $backup_ts_path = TestLib::tempdir;
+ $master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '-T', "${source_ts_path}=${backup_ts_path}"],
+ "base backup ok");
+ command_ok(['pg_validatebackup', $backup_path ],
+ "intact backup validated");
+
+ # Mutilate the backup in some way.
+ $scenario->{'mutilate'}->($backup_path);
+
+ # Now check that the backup no longer validates.
+ command_fails_like(['pg_validatebackup', $backup_path ],
+ $scenario->{'fails_like'},
+ "corrupt backup fails validation: $name");
+
+ # Run cleanup hook, if provided.
+ $scenario->{'cleanup'}->($backup_path)
+ if exists $scenario->{'cleanup'};
+
+ # Finally, use rmtree to reclaim space.
+ rmtree($backup_path);
+ }
+}
+
+sub create_extra_file
+{
+ my ($backup_path, $relative_path) = @_;
+ my $pathname = "$backup_path/$relative_path";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh "This is an extra file.\n";
+ close($fh);
+}
+
+# Add a file into the root directory of the backup.
+sub mutilate_extra_file
+{
+ my ($backup_path) = @_;
+ create_extra_file($backup_path, "extra_file");
+}
+
+# Add a file inside the user-defined tablespace.
+sub mutilate_extra_tablespace_file
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my ($catvdir) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid");
+ my ($tsdboid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid/$catvdir");
+ create_extra_file($backup_path,
+ "pg_tblspc/$tsoid/$catvdir/$tsdboid/extra_ts_file");
+}
+
+# Remove a file.
+sub mutilate_missing_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_xact/0000";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Remove the symlink to the user-defined tablespace.
+sub mutilate_missing_tablespace
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my $pathname = "$backup_path/pg_tblspc/$tsoid";
+ if ($windows_os)
+ {
+ rmdir($pathname) || die "$pathname: $!";
+ }
+ else
+ {
+ unlink($pathname) || die "$pathname: $!";
+ }
+}
+
+# Append an additional bytes to a file.
+sub mutilate_append_to_file
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/global/pg_control", 'x';
+}
+
+# Truncate a file to zero length.
+sub mutilate_truncate_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/global/pg_control";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ close($fh);
+}
+
+# Replace a file's contents without changing the length of the file. This is
+# not a particularly efficient way to do this, so we pick a file that's
+# expected to be short.
+sub mutilate_replace_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ my $contents = slurp_file($pathname);
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh 'q' x length($contents);
+ close($fh);
+}
+
+# Corrupt the backup manifest.
+sub mutilate_bad_manifest
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/backup_manifest", "\n";
+}
+
+# Create a file that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_file_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_subtrans";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be searched. (This is skipped on Windows.)
+sub mutilate_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0400, $pathname) || die "chmod $pathname: $!";
+}
+
+# rmtree can't cope with a mode 400 directory, so change back to 700.
+sub cleanup_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0700, $pathname) || die "chmod $pathname: $!";
+}
diff --git a/src/bin/pg_validatebackup/t/004_options.pl b/src/bin/pg_validatebackup/t/004_options.pl
new file mode 100644
index 0000000000..8f185626ed
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/004_options.pl
@@ -0,0 +1,89 @@
+# Verify the behavior of assorted pg_validatebackup options.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 25;
+
+# Start up the server and take a backup.
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_options';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync' ],
+ "base backup ok");
+
+# Verify that pg_validatebackup -q succeeds and produces no output.
+my $stdout;
+my $stderr;
+my $result = IPC::Run::run ['pg_validatebackup', '-q', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok($result, "-q succeeds: exit code 0");
+is($stdout, '', "-q succeeds: no stdout");
+is($stderr, '', "-q succeeds: no stderr");
+
+# Corrupt the PG_VERSION file.
+my $version_pathname = "$backup_path/PG_VERSION";
+my $version_contents = slurp_file($version_pathname);
+open(my $fh, '>', $version_pathname) || die "open $version_pathname: $!";
+print $fh 'q' x length($version_contents);
+close($fh);
+
+# Verify that pg_validatebackup -q now fails.
+command_fails_like(['pg_validatebackup', '-q', $backup_path ],
+ qr/checksum mismatch for file \"PG_VERSION\"/,
+ '-q checksum mismatch');
+
+# Since we didn't change the length of the file, validation should succeed
+# if we ignore checksums. Check that we get the right message, too.
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ '-s skips checksumming');
+
+# Validation should succeed if we ignore the problem file.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/backup successfully verified/,
+ '-i ignores problem file');
+
+# PG_VERSION is already corrupt; let's try also removing all of pg_xact.
+rmtree($backup_path . "/pg_xact");
+
+# We're ignoring the problem with PG_VERSION, but not the problem with
+# pg_xact, so validation should fail here.
+command_fails_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/pg_xact.*is present in the manifest but not on disk/,
+ '-i does not ignore all problems');
+
+# If we use -i twice, we should be able to ignore all of the problems.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', '-i', 'pg_xact',
+ $backup_path ],
+ qr/backup successfully verified/,
+ 'multiple -i options work');
+
+# Verify that when -i is not used, both problems are reported.
+$result = IPC::Run::run ['pg_validatebackup', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "multiple problems: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "multiple problems: missing files reported");
+like($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "multiple problems: checksum mismatch reported");
+
+# Verify that when -e is used, only the problem detected first is reported.
+$result = IPC::Run::run ['pg_validatebackup', '-e', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "-e reports 1 error: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "-e reports 1 error: missing files reported");
+unlike($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "-e reports 1 error: checksum mismatch not reported");
+
+# Test valid manifest with nonexistent backup directory.
+command_fails_like(['pg_validatebackup', '-m', "$backup_path/backup_manifest",
+ "$backup_path/fake" ],
+ qr/could not open directory/,
+ 'nonexistent backup directory');
diff --git a/src/bin/pg_validatebackup/t/005_bad_manifest.pl b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
new file mode 100644
index 0000000000..9c503600d2
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
@@ -0,0 +1,158 @@
+# Test the behavior of pg_validatebackup when the backup manifest has
+# problems.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $tempdir = TestLib::tempdir;
+
+test_bad_manifest('input string ended unexpectedly',
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
+{
+EOM
+
+test_parse_error('unexpected object end', <<EOM);
+{}
+EOM
+
+test_parse_error('unexpected array start', <<EOM);
+[]
+EOM
+
+test_parse_error('expected version indicator', <<EOM);
+{"not-expected": 1}
+EOM
+
+test_parse_error('unexpected manifest version', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": "phooey"}
+EOM
+
+test_parse_error('unexpected scalar', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": true}
+EOM
+
+test_parse_error('expected file list', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Oops": 1}
+EOM
+
+test_parse_error('unexpected object start', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": {}}
+EOM
+
+test_parse_error('missing pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [{}]}
+EOM
+
+test_parse_error('both pathname and encoded pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Encoded-Path": "1234"}
+]}
+EOM
+
+test_parse_error('unexpected file field', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Oops": 1}
+]}
+EOM
+
+test_parse_error('missing size', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x"}
+]}
+EOM
+
+test_parse_error('file size is not an integer', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": "Oops"}
+]}
+EOM
+
+test_parse_error('unable to decode filename', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Encoded-Path": "123", "Size": 0}
+]}
+EOM
+
+test_fatal_error('duplicate pathname in backup manifest', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 0},
+ {"Path": "x", "Size": 0}
+]}
+EOM
+
+test_parse_error('checksum without algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum": "Oops"}
+]}
+EOM
+
+test_fatal_error('unrecognized checksum algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "Oops", "Checksum": "00"}
+]}
+EOM
+
+test_fatal_error('invalid checksum for file', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "CRC32C", "Checksum": "0"}
+]}
+EOM
+
+test_parse_error('expected manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Oops": 1}
+EOM
+
+test_parse_error('expected at least 2 lines', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [], "Manifest-Checksum": null}
+EOM
+
+my $manifest_without_newline = <<EOM;
+{"PostgreSQL-Backup-Manifest-Version": 1,
+ "Files": [],
+ "Manifest-Checksum": null}
+EOM
+chomp($manifest_without_newline);
+test_parse_error('last line not newline-terminated',
+ $manifest_without_newline);
+
+test_fatal_error('invalid manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Manifest-Checksum": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890-"}
+EOM
+
+sub test_parse_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/could not parse backup manifest: $test_name/,
+ $manifest_contents);
+}
+
+sub test_fatal_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/fatal: $test_name/,
+ $manifest_contents);
+}
+
+sub test_bad_manifest
+{
+ my ($test_name, $regexp, $manifest_contents) = @_;
+
+ open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+ print $fh $manifest_contents;
+ close($fh);
+
+ command_fails_like(['pg_validatebackup', $tempdir], $regexp,
+ $test_name);
+}
diff --git a/src/bin/pg_validatebackup/t/006_encoding.pl b/src/bin/pg_validatebackup/t/006_encoding.pl
new file mode 100644
index 0000000000..5e3e7152a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/006_encoding.pl
@@ -0,0 +1,27 @@
+# Verify that pg_validatebackup handles hex-encoded filenames correctly.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 5;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_encoding';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '--manifest-force-encode' ],
+ "backup ok with forced hex encoding");
+
+my $manifest = slurp_file("$backup_path/backup_manifest");
+my $count_of_encoded_path_in_manifest =
+ (() = $manifest =~ /Encoded-Path/mig);
+cmp_ok($count_of_encoded_path_in_manifest, '>', 100,
+ "many paths are encoded in the manifest");
+
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ 'backup with forced encoding validated');
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 19:50 ` David Steele <[email protected]>
2020-03-30 20:16 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: David Steele @ 2020-03-27 19:50 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/27/20 1:53 PM, Robert Haas wrote:
> On Thu, Mar 26, 2020 at 4:37 PM David Steele <[email protected]> wrote:
>> I know you and Stephen have agreed on a number of doc changes, would it
>> be possible to get a new patch with those included? I finally have time
>> to do a review of this tomorrow. I saw some mistakes in the docs in the
>> current patch but I know those patches are not current.
>
> Hi David,
>
> Here's a new version with some fixes:
>
> - Fixes for doc typos noted by Stephen Frost and Andres Freund.
> - Replace a doc paragraph about the advantages and disadvantages of
> CRC-32C with one by Stephen Frost, with a slightly change by me that I
> thought made it sound more grammatical.
> - Change the pg_validatebackup documentation so that it makes no
> mention of compatible tools, per Stephen.
> - Reword the discussion of the exclude list in the pg_validatebackup
> documentation, per discussion between Stephen and myself.
> - Try to make the documentation more clear about the fact that we
> check for both extra and missing files.
> - Incorporate a fix from Amit Kapila to make 003_corruption.pl pass on Windows.
Thanks!
There appear to be conflicts with 67e0adfb3f98:
$ git apply -3
../download/v14-0002-Generate-backup-manifests-for-base-backups-and-v.patch
../download/v14-0002-Generate-backup-manifests-for-base-backups-and-v.patch:3396:
trailing whitespace.
sub cleanup_search_directory_fails
error: patch failed: src/backend/replication/basebackup.c:258
Falling back to three-way merge...
Applied patch to 'src/backend/replication/basebackup.c' with conflicts.
U src/backend/replication/basebackup.c
warning: 1 line adds whitespace errors.
> + Specifies the algorithm that should be used to checksum
each file
> + for purposes of the backup manifest. Currently, the available
perhaps "for inclusion in the backup manifest"? Anyway, I think this
sentence is awkward.
> + Specifies the algorithm that should be used to checksum each
file
> + for purposes of the backup manifest. Currently, the available
And again.
> + because the files themselves do not need to read.
should be "need to be read".
> + the manifest itself will always contain a
<literal>SHA256</literal>
I think just "the manifest will always contain" is fine.
> + manifeste itself, and is therefore ignored. Note that the
manifest
typo "manifeste", perhaps remove itself.
> { "Path": "backup_label", "Size": 224, "Last-Modified": "2020-03-27
18:33:18 GMT", "Checksum-Algorithm": "CRC32C", "Checksum": "b914bec9" },
Storing the checksum type with each file seems pretty redundant.
Perhaps that could go in the header? You could always override if a
specific file had a different checksum type, though that seems unlikely.
In general it might be good to go with shorter keys: "mod", "chk", etc.
Manifests can get pretty big and that's a lot of extra bytes.
I'm also partial to using epoch time in the manifest because it is
generally easier for programs to work with. But, human-readable doesn't
suck, either.
> if (maxrate > 0)
> maxrate_clause = psprintf("MAX_RATE %u", maxrate);
> + if (manifest)
A linefeed here would be nice.
> + manifestfile *tabent;
This is an odd name. A holdover from the tab-delimited version?
> + printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
When I ran pg_validatebackup I expected to use -D to specify the backup
dir since pg_basebackup does. On the other hand -D is weird because I
*really* expect that to be the pg data dir.
But, do we want this to be different from pg_basebackup?
> + checksum_length = checksum_string_length / 2;
This check is defeated if a single character is added the to checksum.
Not too big a deal since you still get an error, but still.
> + * Verify that the manifest checksum is correct.
This is not working the way I would expect -- I could freely modify the
manifest without getting a checksum error on the manifest. For example:
$ /home/vagrant/test/pg/bin/pg_validatebackup test/backup3
pg_validatebackup: fatal: invalid checksum for file "backup_label":
"408901e0814f40f8ceb7796309a59c7248458325a21941e7c55568e381f53831?"
So, if I deleted the entry above, I got a manifest checksum error. But
if I just modified the checksum I get a file checksum error with no
manifest checksum error.
I would prefer a manifest checksum error in all cases where it is wrong,
unless --exit-on-error is specified.
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 19:50 ` Re: backup manifests David Steele <[email protected]>
@ 2020-03-30 20:16 ` Robert Haas <[email protected]>
2020-03-30 23:24 ` Re: backup manifests David Steele <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-30 20:16 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 3:51 PM David Steele <[email protected]> wrote:
> There appear to be conflicts with 67e0adfb3f98:
Rebased.
> > + Specifies the algorithm that should be used to checksum
> each file
> > + for purposes of the backup manifest. Currently, the available
>
> perhaps "for inclusion in the backup manifest"? Anyway, I think this
> sentence is awkward.
I changed it to "Specifies the checksum algorithm that should be
applied to each file included in the backup manifest." I hope that's
better. I also added, in both of the places where this text occurs, an
explanation a little higher up of what a backup manifest actually is.
> > + because the files themselves do not need to read.
>
> should be "need to be read".
Fixed.
> > + the manifest itself will always contain a
> <literal>SHA256</literal>
>
> I think just "the manifest will always contain" is fine.
OK.
> > + manifeste itself, and is therefore ignored. Note that the
> manifest
>
> typo "manifeste", perhaps remove itself.
OK, fixed.
> > { "Path": "backup_label", "Size": 224, "Last-Modified": "2020-03-27
> 18:33:18 GMT", "Checksum-Algorithm": "CRC32C", "Checksum": "b914bec9" },
>
> Storing the checksum type with each file seems pretty redundant.
> Perhaps that could go in the header? You could always override if a
> specific file had a different checksum type, though that seems unlikely.
>
> In general it might be good to go with shorter keys: "mod", "chk", etc.
> Manifests can get pretty big and that's a lot of extra bytes.
>
> I'm also partial to using epoch time in the manifest because it is
> generally easier for programs to work with. But, human-readable doesn't
> suck, either.
It doesn't seem impossible for it to come up; for example, consider a
file-level incremental backup facility. You might retain whatever
checksums you have for the unchanged files (to avoid rereading them)
and add checksums for modified or added files.
I am not convinced that minimizing the size of the file here is a
particularly important goal, because I don't think it's going to get
that big in normal cases. I also think having the keys and values be
easily understandable by human being is a plus. If we really want a
minimal format without redundancy, we should've gone with what I
proposed before (though admittedly that could've been tamped down even
further if we'd cared to squeeze, which I didn't think was important
then either).
>
> > if (maxrate > 0)
> > maxrate_clause = psprintf("MAX_RATE %u", maxrate);
> > + if (manifest)
>
> A linefeed here would be nice.
Added.
> > + manifestfile *tabent;
>
> This is an odd name. A holdover from the tab-delimited version?
No, it was meant to stand for table entry. (Now we find out what
happens when I break my own rule against using abbreviated words.)
> > + printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
>
> When I ran pg_validatebackup I expected to use -D to specify the backup
> dir since pg_basebackup does. On the other hand -D is weird because I
> *really* expect that to be the pg data dir.
>
> But, do we want this to be different from pg_basebackup?
I think it's pretty distinguishable, because pg_basebackup needs an
input (server) and an output (directory), whereas pg_validatebackup
only needs one. I don't really care if we want to change it, but I was
thinking of this as being more analogous to, say, pg_resetwal.
Granted, that's a danger-don't-use-this tool and this isn't, but I
don't think we want the -D-is-optional behavior that tools like pg_ctl
have, because having a tool that isn't supposed to be used on a
running cluster default to $PGDATA seems inadvisable. And if the
argument is mandatory then it's not clear to me why we should make
people type -D in front of it.
> > + checksum_length = checksum_string_length / 2;
>
> This check is defeated if a single character is added the to checksum.
>
> Not too big a deal since you still get an error, but still.
I don't see what the problem is here. We speculatively divide by two
and allocate memory assuming the value that it was even, but then
before doing anything critical we bail out if it was actually odd.
That's harmless. We could get around it by saying:
if (checksum_string_length % 2 != 0)
context->error_cb(...);
checksum_length = checksum_string_length / 2;
checksum_payload = palloc(checksum_length);
if (!hexdecode_string(...))
context->error_cb(...);
...but that would be adding additional code, and error messages, for
what's basically a can't-happen-unless-the-user-is-messing-with-us
case.
> > + * Verify that the manifest checksum is correct.
>
> This is not working the way I would expect -- I could freely modify the
> manifest without getting a checksum error on the manifest. For example:
>
> $ /home/vagrant/test/pg/bin/pg_validatebackup test/backup3
> pg_validatebackup: fatal: invalid checksum for file "backup_label":
> "408901e0814f40f8ceb7796309a59c7248458325a21941e7c55568e381f53831?"
>
> So, if I deleted the entry above, I got a manifest checksum error. But
> if I just modified the checksum I get a file checksum error with no
> manifest checksum error.
>
> I would prefer a manifest checksum error in all cases where it is wrong,
> unless --exit-on-error is specified.
I think I would too, but I'm confused as to what you're doing, because
if I just modified the manifest -- by deleting a file, for example, or
changing the checksum of a file, I just get:
pg_validatebackup: fatal: manifest checksum mismatch
I'm confused as to why you're not seeing that. What's the exact
sequence of steps?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v15-0001-Add-checksum-helper-functions.patch (9.5K, ../../CA+TgmobjdP4vyo4qy_U3tP5=TgH==CFJmd8cfJjAfz36kSVYHA@mail.gmail.com/2-v15-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From 9f226ba3b596fb45f16a33100bca4fd1554ad9cc Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 20 Mar 2020 14:48:33 -0400
Subject: [PATCH v15 1/2] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index 6939b9d087..16619e4ba8 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -48,6 +48,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
archive.o \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 39709f20e6..41e7678689 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -120,8 +120,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- archive.c
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
+ archive.c base64.c checksum_helper.c
+ config_info.c controldata_utils.c d2s.c encnames.c exec.c
f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v15-0002-Generate-backup-manifests-for-base-backups-and-v.patch (117.8K, ../../CA+TgmobjdP4vyo4qy_U3tP5=TgH==CFJmd8cfJjAfz36kSVYHA@mail.gmail.com/3-v15-0002-Generate-backup-manifests-for-base-backups-and-v.patch)
download | inline diff:
From b790264fe083919c858d12030075e230b66507ea Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 30 Mar 2020 16:14:00 -0400
Subject: [PATCH v15 2/2] Generate backup manifests for base backups, and
validate them.
A manifest is a JSON document which includes the file name, size, last
modification time, and a checksum for each file backed up, as well as
a checksum for the manifest itself. By default, we use CRC-32C for the
checksum algorithm, because we are trying to detect corruption and
user error, not foil an adversary. However, pg_basebackup and the
server-side BASE_BACKUP command now have options to select the
checksum algorithm, so users wanting a cryptographic hash function can
select SHA-224, SHA-256, SHA-384, or SHA-512. Users not wanting any
checksums at all can disable them, or disable generating of the backup
manifest altogether. Using a cryptographic hash function in place of
CRC-32C consumes significantly more CPU cycles, which may slow down
backups in some cases.
A new tool called pg_validatebackup can validate a backup against the
manifest. If no checksums are present, it can still check that the
right files exist and that they have the expected sizes. If checksums
are present, it can also verify that each file has the expected
checksum. Only plain format backups can be validated directly, but tar
format backups can be validated after extracting them.
Robert Haas, with help, ideas, review, and testing from David Steele,
Stephen Frost, Andrew Dunstan, Rushabh Lathia, Suraj Kharage, Tushar
Ahuja, Rajkumar Raghuwanshi, Mark Dilger, Davinder Singh, Jeevan
Chalke, Amit Kapila, and Andres Freund.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
doc/src/sgml/protocol.sgml | 37 +-
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_basebackup.sgml | 64 ++
doc/src/sgml/ref/pg_validatebackup.sgml | 240 ++++++
doc/src/sgml/reference.sgml | 1 +
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 430 +++++++++-
src/backend/replication/repl_gram.y | 13 +
src/backend/replication/repl_scanner.l | 2 +
src/backend/replication/walsender.c | 30 +
src/bin/Makefile | 1 +
src/bin/pg_basebackup/pg_basebackup.c | 185 ++++-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 8 +-
src/bin/pg_validatebackup/.gitignore | 2 +
src/bin/pg_validatebackup/Makefile | 39 +
src/bin/pg_validatebackup/parse_manifest.c | 576 ++++++++++++++
src/bin/pg_validatebackup/parse_manifest.h | 40 +
src/bin/pg_validatebackup/pg_validatebackup.c | 734 ++++++++++++++++++
src/bin/pg_validatebackup/t/001_basic.pl | 30 +
src/bin/pg_validatebackup/t/002_algorithm.pl | 58 ++
src/bin/pg_validatebackup/t/003_corruption.pl | 251 ++++++
src/bin/pg_validatebackup/t/004_options.pl | 89 +++
.../pg_validatebackup/t/005_bad_manifest.pl | 158 ++++
src/bin/pg_validatebackup/t/006_encoding.pl | 27 +
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
26 files changed, 2995 insertions(+), 32 deletions(-)
create mode 100644 doc/src/sgml/ref/pg_validatebackup.sgml
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/parse_manifest.c
create mode 100644 src/bin/pg_validatebackup/parse_manifest.h
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
create mode 100644 src/bin/pg_validatebackup/t/001_basic.pl
create mode 100644 src/bin/pg_validatebackup/t/002_algorithm.pl
create mode 100644 src/bin/pg_validatebackup/t/003_corruption.pl
create mode 100644 src/bin/pg_validatebackup/t/004_options.pl
create mode 100644 src/bin/pg_validatebackup/t/005_bad_manifest.pl
create mode 100644 src/bin/pg_validatebackup/t/006_encoding.pl
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index f139ba0231..536de9a698 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2466,7 +2466,7 @@ The commands accepted in replication mode are:
</varlistentry>
<varlistentry id="protocol-replication-base-backup" xreflabel="BASE_BACKUP">
- <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ]
+ <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ] [ <literal>MANIFEST</literal> <replaceable>manifest_option</replaceable> ] [ <literal>MANIFEST_CHECKSUMS</literal> <replaceable>checksum_algorithm</replaceable> ]
<indexterm><primary>BASE_BACKUP</primary></indexterm>
</term>
<listitem>
@@ -2576,6 +2576,41 @@ The commands accepted in replication mode are:
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST</literal></term>
+ <listitem>
+ <para>
+ When this option is specified with a value of <literal>yes</literal>
+ or <literal>force-escape</literal>, a backup manifest is created
+ and sent along with the backup. The manifest is a list of every
+ file present in the backup with the exception of any WAL files that
+ may be included. It also stores the size, last modification time, and
+ an optional checksum for each file.
+ A value of <literal>force-escape</literal> forces all filenames
+ to be hex-encoded; otherwise, this type of encoding is performed only
+ for files whose names are non-UTF8 octet sequences.
+ <literal>force-escape</literal> is intended primarily for testing
+ purposes, to be sure that clients which read the backup manifest
+ can handle this case. For compatibility with previous releases,
+ the default is <literal>MANIFEST 'no'</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST_CHECKSUMS</literal></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be applied to each file included
+ in the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
<para>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..ab71176cdf 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -211,6 +211,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
<!ENTITY pgRestore SYSTEM "pg_restore.sgml">
<!ENTITY pgRewind SYSTEM "pg_rewind.sgml">
+<!ENTITY pgValidateBackup SYSTEM "pg_validatebackup.sgml">
<!ENTITY pgtestfsync SYSTEM "pgtestfsync.sgml">
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index c8e040bacf..f5dd4799bc 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -561,6 +561,70 @@ PostgreSQL documentation
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>--no-manifest</option></term>
+ <listitem>
+ <para>
+ Disables generation of a backup manifest. If this option is not
+ specified, the server will generate and send a backup manifest
+ which can be verified using <xref linkend="app-pgvalidatebackup" />.
+ The manifest is a list of every file present in the backup with the
+ exception of any WAL files that may be included. It also stores the
+ size, last modification time, and an optional checksum for each file.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-force-encode</option></term>
+ <listitem>
+ <para>
+ Forces all filenames in the backup manifest to be hex-encoded.
+ If this option is not specified, only non-UTF8 filenames are
+ hex-encoded. This option is mostly intended to test that tools which
+ read a backup manifest file properly handle this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the checksum algorithm that should be applied to each file
+ included in the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ <para>
+ If <literal>NONE</literal> is selected, the backup manifest will
+ not contain any checksums. Otherwise, it will contain a checksum
+ of each file in the backup using the specified algorithm. In addition,
+ the manifest will always contain a <literal>SHA256</literal>
+ checksum of its own contents. The <literal>SHA</literal> algorithms
+ are significantly more CPU-intensive than <literal>CRC32C</literal>,
+ so selecting one of them may increase the time required to complete
+ the backup.
+ </para>
+ <para>
+ Using a SHA hash function provides a cryptographically secure digest
+ of each file for users who wish to verify that the backup has not been
+ tampered with, while the CRC32C algorithm provides a checksum which is
+ much faster to calculate and good at catching errors due to accidental
+ changes but is not resistent to targeted modifications. Note that, to
+ be useful against an adversary who has access to the backup, the backup
+ manifest would need to be stored securely elsewhere or otherwise
+ verified not to have been modified since the backup was taken.
+ </para>
+ <para>
+ <xref linkend="app-pgvalidatebackup" /> can be used to check the
+ integrity of a backup against the backup manifest.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
new file mode 100644
index 0000000000..32547e614a
--- /dev/null
+++ b/doc/src/sgml/ref/pg_validatebackup.sgml
@@ -0,0 +1,240 @@
+<!--
+doc/src/sgml/ref/pg_validatebackup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgvalidatebackup">
+ <indexterm zone="app-pgvalidatebackup">
+ <primary>pg_validatebackup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>pg_validatebackup</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_validatebackup</refname>
+ <refpurpose>verify the integrity of a base backup of a
+ <productname>PostgreSQL</productname> cluster</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_validatebackup</command>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>
+ Description
+ </title>
+ <para>
+ <application>pg_validatebackup</application> is used to check the integrity
+ of a database cluster backup taken using <command>pg_basebackup</command>.
+ The backup must be stored in the "plain" format; a "tar" format backup can
+ be checked after extracting it. Backup manifests are created by the server
+ beginning with <productname>PostgreSQL</productname> version 13, so older
+ backups cannot be validated using this tool.
+ </para>
+
+ <para>
+ <application>pg_validatebackup</application> reads the manifest file of a
+ backup, verifies the manifest against its own internal checksum, and then
+ verifies that the same files are present in the target directory as in the
+ manifest itself. Note that this process checks both for files which are present
+ in the manifest but not in the backup and also for files which are present
+ in the backup but not mentioned in the manifest. After checking that the right
+ files are present, it then verifies that each file has the expected checksum,
+ unless the backup was taken with the checksum algorithm set to
+ <literal>none</literal>, in which case checksum verification is not
+ performed. The presence or absence of directories is not checked, except
+ indirectly: if a directory is missing, any files it should have contained
+ will necessarily also be missing.
+ </para>
+
+ <para>
+ When pg_basebackup compares the files and directories in the manifest
+ to those which are present on disk, it will ignore the presence of, or
+ changes to, certain files:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>backup_manifest</literal> will not be present in the
+ manifest, and is therefore ignored. Note that the manifest
+ is still verified internally, but no error will be issued about the
+ presence of a backup_manifest file in the backup directory even though
+ it is not listed in the manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>pg_wal</literal> is ignored because WAL files are sent
+ separately from the backup, and are therefore not described by the
+ backup manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>postgesql.auto.conf</literal>,
+ <literal>standby.signal</literal>,
+ and <literal>recovery.signal</literal> are ignored because they may
+ sometimes be created or modified by the backup client itself.
+ (For example, <literal>pg_basebackup -R</literal> will modify
+ <literal>postgresql.auto.conf</literal> and create
+ <literal>standby.signal</literal>.)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ The following command-line options control the behavior.
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-e</option></term>
+ <term><option>--exit-on-error</option></term>
+ <listitem>
+ <para>
+ Exit as soon as a problem with the backup is detected. If this option
+ is not specified, <literal>pg_basebackup</literal> will continue
+ checking the backup even after a problem has been detected, and will
+ report all problems detected as errors.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-i <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--ignore=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Ignore the specified file or directory, which should be expressed
+ as a relative pathname. If the backup contains extra files, is
+ missing files, or has files that have been modified as compared with
+ what is described in the manifest, this option can be used to suppress
+ the errors that would otherwise occur. If a directory is specified,
+ this option affects the entire subtree rooted at that location.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-m <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--manifest-path=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Use the manifest file at the specified path, rather than one located
+ in the root of the backup directory.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-q</option></term>
+ <term><option>--quiet</option></term>
+ <listitem>
+ <para>
+ Don't print anything when a backup is successfully validated.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-s</option></term>
+ <term><option>--skip-checksums</option></term>
+ <listitem>
+ <para>
+ Do not validate checksums. The presence or absence of files and the
+ sizes of those files will still be checked. This is much faster,
+ because the files themselves do not need to be read.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other options are also available:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_validatebackup</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_validatebackup</application> command
+ line arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal> and
+ validate the integrity of the backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal>, move
+ the manifest somewhere outside the backup directory, and validate the
+ backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234</userinput>
+<prompt>$</prompt> <userinput>mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234</userinput>
+</screen>
+ </para>
+
+ <para>
+ To validate a backup while ignoring a file that was added manually to the
+ backup directory, and also skipping checksum verification:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>edit /usr/local/pgsql/data/note.to.self</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="app-pgbasebackup"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..d25a77b13c 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -255,6 +255,7 @@
&pgReceivewal;
&pgRecvlogical;
&pgRestore;
+ &pgValidateBackup;
&psqlRef;
&reindexdb;
&vacuumdb;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 1951103b26..5ba3debf82 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10632,7 +10632,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index a2e28b064c..deaa4f1c34 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -39,10 +41,19 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
+typedef enum manifest_option
+{
+ MANIFEST_OPTION_YES,
+ MANIFEST_OPTION_NO,
+ MANIFEST_OPTION_FORCE_ENCODE
+} manifest_option;
+
typedef struct
{
const char *label;
@@ -52,20 +63,43 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ manifest_option manifest;
+ pg_checksum_type manifest_checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ bool force_encode;
+ bool first_file;
+ bool still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest,
+ basebackup_options *opt);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +136,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -254,6 +298,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -273,12 +318,17 @@ perform_base_backup(basebackup_options *opt)
backup_total);
}
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt);
total_checksum_failures = 0;
@@ -316,7 +366,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -395,7 +448,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -403,11 +457,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -415,10 +472,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -647,7 +705,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -670,16 +728,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -693,6 +755,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -724,8 +789,13 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->manifest = MANIFEST_OPTION_NO;
+ opt->manifest_checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -812,12 +882,61 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest") == 0)
+ {
+ char *optval = strVal(defel->arg);
+ bool manifest_bool;
+
+ if (o_manifest)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (parse_bool(optval, &manifest_bool))
+ {
+ if (manifest_bool)
+ opt->manifest = MANIFEST_OPTION_YES;
+ else
+ opt->manifest = MANIFEST_OPTION_NO;
+ }
+ else if (pg_strcasecmp(optval, "force-encode") == 0)
+ opt->manifest = MANIFEST_OPTION_FORCE_ENCODE;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized manifest option: \"%s\"",
+ optval)));
+ o_manifest = true;
+ }
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval,
+ &opt->manifest_checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
}
if (opt->label == NULL)
opt->label = "base backup";
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ {
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("manifest checksums require a backup manifest")));
+ opt->manifest_checksum_type = CHECKSUM_TYPE_NONE;
+ }
}
@@ -933,6 +1052,249 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, basebackup_options *opt)
+{
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ manifest->buffile = NULL;
+ else
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = opt->manifest_checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->force_encode = (opt->manifest == MANIFEST_OPTION_FORCE_ENCODE);
+ manifest->first_file = true;
+ manifest->still_checksumming = true;
+
+ if (opt->manifest != MANIFEST_OPTION_NO)
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ Assert(manifest != NULL);
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ int pathlen;
+ StringInfoData buf;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (!manifest->force_encode &&
+ pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
+
+ /*
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
+ */
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
+
+ /* Add checksum information. */
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
+ }
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
+
+ /* Avoid leaking memory. */
+ pfree(buf.data);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\"}\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -993,11 +1355,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1032,6 +1398,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1042,7 +1412,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1075,7 +1446,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1094,7 +1466,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1374,7 +1746,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1382,7 +1755,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1452,8 +1826,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1470,6 +1845,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1640,6 +2018,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1664,6 +2045,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1672,7 +2054,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1697,6 +2080,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..f93a0de218 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,8 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -156,6 +158,7 @@ var_name: IDENT { $$ = $1; }
/*
* BASE_BACKUP [LABEL '<label>'] [PROGRESS] [FAST] [WAL] [NOWAIT]
* [MAX_RATE %d] [TABLESPACE_MAP] [NOVERIFY_CHECKSUMS]
+ * [MANIFEST %s] [MANIFEST_CHECKSUMS %s]
*/
base_backup:
K_BASE_BACKUP base_backup_opt_list
@@ -214,6 +217,16 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST SCONST
+ {
+ $$ = makeDefElem("manifest",
+ (Node *)makeString($2), -1);
+ }
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..452ad9fc27 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,8 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST { return K_MANIFEST; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 76ec3c7dd0..3b117d8367 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index c5d95958b2..058125be67 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -136,6 +142,9 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static bool manifest = true;
+static bool manifest_force_encode = false;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -181,6 +190,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -388,6 +403,11 @@ usage(void)
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
printf(_(" --no-estimate-size do not estimate backup size in server side\n"));
+ printf(_(" --no-manifest suppress generation of backup manifest\n"));
+ printf(_(" --manifest-force-encode\n"
+ " hex encode all filenames in manifest\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1186,6 +1206,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1657,6 +1702,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1667,6 +1770,8 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_clause;
+ char *manifest_checksums_clause = "";
int i;
char xlogstart[64];
char xlogend[64];
@@ -1674,6 +1779,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1728,6 +1834,33 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest)
+ {
+ if (serverMajor < 1300)
+ {
+ const char *serverver = PQparameterStatus(conn, "server_version");
+
+ pg_log_error("backup manifests are not supported by server version %s",
+ serverver ? serverver : "'unknown'");
+ exit(1);
+ }
+
+ if (manifest_force_encode)
+ manifest_clause = "MANIFEST 'force-encode'";
+ else
+ manifest_clause = "MANIFEST 'yes'";
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
+ }
+ else
+ {
+ if (serverMajor < 1300)
+ manifest_clause = "";
+ else
+ manifest_clause = "MANIFEST 'no'";
+ }
+
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1741,7 +1874,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s %s",
escaped_label,
estimatesize ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1749,7 +1882,9 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_clause,
+ manifest_checksums_clause);
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1837,7 +1972,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1866,6 +2002,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && manifest)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2069,6 +2218,9 @@ main(int argc, char **argv)
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
{"no-estimate-size", no_argument, NULL, 4},
+ {"no-manifest", no_argument, NULL, 5},
+ {"manifest-force-encode", no_argument, NULL, 6},
+ {"manifest-checksums", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
int c;
@@ -2096,7 +2248,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2240,6 +2392,15 @@ main(int argc, char **argv)
case 4:
estimatesize = false;
break;
+ case 5:
+ manifest = false;
+ break;
+ case 6:
+ manifest_force_encode = true;
+ break;
+ case 7:
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
@@ -2370,6 +2531,22 @@ main(int argc, char **argv)
exit(1);
}
+ if (!manifest && manifest_checksums != NULL)
+ {
+ pg_log_error("--no-manifest and --manifest-checksums are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (!manifest && manifest_force_encode)
+ {
+ pg_log_error("--no-manifest and --manifest-force-encode are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
/* connection in replication mode to server */
conn = GetConnection();
if (!conn)
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..63381764e9 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 109;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -104,6 +104,7 @@ foreach my $filename (@tempRelationFiles)
$node->command_ok([ 'pg_basebackup', '-D', "$tempdir/backup", '-X', 'none' ],
'pg_basebackup runs');
ok(-f "$tempdir/backup/PG_VERSION", 'backup was created');
+ok(-f "$tempdir/backup/backup_manifest", 'backup manifest included');
# Permissions on backup should be default
SKIP:
@@ -160,11 +161,12 @@ rmtree("$tempdir/backup");
$node->command_ok(
[
- 'pg_basebackup', '-D', "$tempdir/backup2", '--waldir',
- "$tempdir/xlog2"
+ 'pg_basebackup', '-D', "$tempdir/backup2", '--no-manifest',
+ '--waldir', "$tempdir/xlog2"
],
'separate xlog directory');
ok(-f "$tempdir/backup2/PG_VERSION", 'backup was created');
+ok(! -f "$tempdir/backup2/backup_manifest", 'manifest was suppressed');
ok(-d "$tempdir/xlog2/", 'xlog directory was created');
rmtree("$tempdir/backup2");
rmtree("$tempdir/xlog2");
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..21e0a92429
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1,2 @@
+/pg_validatebackup
+/tmp_check/
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..04ef7d3051
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,39 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ parse_manifest.o \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
new file mode 100644
index 0000000000..e6b42adfda
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -0,0 +1,576 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.c
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "parse_manifest.h"
+#include "common/jsonapi.h"
+
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_ENCODED_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+/*
+ * Internal state used while decoding the JSON-format backup manifest.
+ */
+typedef struct
+{
+ JsonManifestParseContext *context;
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ char *pathname;
+ char *encoded_pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void verify_manifest_checksum(JsonManifestParseState *parse,
+ char *buffer, size_t size);
+static void json_manifest_parse_failure(JsonManifestParseContext *context,
+ char *msg);
+
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+
+/*
+ * Main entrypoint to parse a JSON-format backup manifest.
+ *
+ * Caller should set up the parsing context and then invoke this function.
+ * For each file whose information is extracted from the manifest,
+ * context->perfile_cb is invoked. In case of trouble, context->error_cb is
+ * invoked and is expected not to return.
+ */
+void
+json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ size_t size)
+{
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
+
+ /* Set up our private parsing context. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.context = context;
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+ /* Validate the checksum. */
+ verify_manifest_checksum(&parse, buffer, size);
+}
+
+/*
+ * Invoked at the start of each object in the JSON document.
+ *
+ * The document as a whole is expected to be an object with three keys
+ * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
+ * file is expected to be an object with various keys (Path, Size, etc.).
+ * If we're not at the beginning of either the toplevel object or the object
+ * for a particular file, it's an error.
+ */
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->encoded_pathname = NULL;
+ parse->size = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each object in the JSON document.
+ *
+ * The possible cases here are the same as for json_manifest_object_start.
+ * There's nothing special to do at the end of the document, but when we
+ * reach the end of an object representing a particular file, we must call
+ * json_manifest_finalize_file() to save the associated details.
+ */
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ json_manifest_finalize_file(parse);
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each array in the JSON document.
+ *
+ * Within the toplevel object, the value associated with the "Files" key
+ * should be an array. No other arrays are expected.
+ */
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each array in the JSON document.
+ *
+ * Just like json_manifest_array_start, there's only one expected case
+ * here.
+ */
+static void
+json_manifest_array_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each object field in the JSON document.
+ */
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_FIELD:
+ /* Inside toplevel object, expecting version indicator. */
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ /* Inside toplevel object, expecting "Files" next. */
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Encoded-Path") == 0)
+ parse->field = JMFF_ENCODED_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure(parse->context,
+ "unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ /* Inside toplevel object, expecting "Manifest-Checksum" next. */
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object field");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each scalar in the JSON document.
+ *
+ * Object field names don't reach this code; those are handled by
+ * json_manifest_object_field_start. When we're inside of the object for
+ * a particular file, that function will have noticed the name of the field,
+ * and we'll get the corresponding value here. When we're in the toplevel
+ * object, the parse state itself tells us which field this is.
+ *
+ * In all cases except for PostgreSQL-Backup-Manifest-Version, which we
+ * can just check on the spot, the goal here is just to save the value in
+ * the parse state for later use. We don't actually do anything until we
+ * reach either the end of the object representing this file, or the end
+ * of the manifest, as the case may be.
+ */
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure(parse->context,
+ "unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_ENCODED_PATH:
+ parse->encoded_pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context, "unexpected scalar");
+ break;
+ }
+}
+
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * file, and invoke the per-file callback so that the caller gets those
+ * details. This happens for each file when the corresponding JSON object is
+ * completely parsed.
+ */
+static void
+json_manifest_finalize_file(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t size;
+ char *ep;
+ int checksum_string_length;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL && parse->encoded_pathname == NULL)
+ json_manifest_parse_failure(parse->context, "missing pathname");
+ if (parse->pathname != NULL && parse->encoded_pathname != NULL)
+ json_manifest_parse_failure(parse->context,
+ "both pathname and encoded pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure(parse->context, "missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure(parse->context,
+ "checksum without algorithm");
+
+ /* Decode encoded pathname, if that's what we have. */
+ if (parse->encoded_pathname != NULL)
+ {
+ int encoded_length = strlen(parse->encoded_pathname);
+ int raw_length = encoded_length / 2;
+
+ parse->pathname = palloc(raw_length + 1);
+ if (encoded_length % 2 != 0 ||
+ !hexdecode_string((uint8 *) parse->pathname,
+ parse->encoded_pathname,
+ raw_length))
+ json_manifest_parse_failure(parse->context,
+ "unable to decode filename");
+ parse->pathname[raw_length] = '\0';
+ pfree(parse->encoded_pathname);
+ parse->encoded_pathname = NULL;
+ }
+
+ /* Parse size. */
+ size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm, &checksum_type))
+ context->error_cb(context, "unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ checksum_length = 0;
+ checksum_payload = NULL;
+ }
+ else
+ {
+ checksum_length = checksum_string_length / 2;
+ checksum_payload = palloc(checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(checksum_payload, parse->checksum,
+ checksum_length))
+ context->error_cb(context,
+ "invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, parse->checksum);
+ }
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perfile_cb(context, parse->pathname, size,
+ checksum_type, checksum_length, checksum_payload);
+
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
+}
+
+/*
+ * Verify that the manifest checksum is correct.
+ *
+ * The last line of the manifest file is excluded from the manifest checksum,
+ * because the last line is expected to contain the checksum that covers
+ * the rest of the file.
+ */
+static void
+verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
+ size_t size)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t i;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+
+ /* Find the last two newlines in the file. */
+ for (i = 0; i < size; ++i)
+ {
+ if (buffer[i] == '\n')
+ {
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
+ }
+ }
+
+ /*
+ * Make sure that the last newline is right at the end, and that there are
+ * at least two lines total. We need this to be true in order for the
+ * following code, which computes the manifest checksum, to work properly.
+ */
+ if (number_of_newlines < 2)
+ json_manifest_parse_failure(parse->context,
+ "expected at least 2 lines");
+ if (ultimate_newline != size - 1)
+ json_manifest_parse_failure(parse->context,
+ "last line not newline-terminated");
+
+ /* Checksum the rest. */
+ pg_sha256_init(&manifest_ctx);
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+
+ /* Now verify it. */
+ if (parse->manifest_checksum == NULL)
+ context->error_cb(parse->context, "manifest has no checksum");
+ if (strlen(parse->manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse->manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ context->error_cb(context, "invalid manifest checksum: \"%s\"",
+ parse->manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ context->error_cb(context, "manifest checksum mismatch");
+}
+
+/*
+ * Report a parse error.
+ *
+ * This is intended to be used for fairly low-level failures that probably
+ * shouldn't occur unless somebody has deliberately constructed a bad manifest,
+ * or unless the server is generating bad manifests due to some bug. msg should
+ * be a short string giving some hint as to what the problem is.
+ */
+static void
+json_manifest_parse_failure(JsonManifestParseContext *context, char *msg)
+{
+ context->error_cb(context, "could not parse backup manifest: %s", msg);
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
new file mode 100644
index 0000000000..25d140f72f
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -0,0 +1,40 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.h
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PARSE_MANIFEST_H
+#define PARSE_MANIFEST_H
+
+#include "common/checksum_helper.h"
+#include "mb/pg_wchar.h"
+
+struct JsonManifestParseContext;
+typedef struct JsonManifestParseContext JsonManifestParseContext;
+
+typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
+ char *pathname,
+ size_t size, pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
+ char *fmt, ...) pg_attribute_printf(2, 3);
+
+struct JsonManifestParseContext
+{
+ void *private_data;
+ json_manifest_perfile_callback perfile_cb;
+ json_manifest_error_callback error_cb;
+};
+
+extern void json_parse_manifest(JsonManifestParseContext *context,
+ char *buffer, size_t size);
+
+#endif
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..eb1473d9d0
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,734 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+#include "parse_manifest.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+
+static void record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length,
+ uint8 *checksum_payload);
+static void report_manifest_error(JsonManifestParseContext *context,
+ char *fmt, ...)
+ pg_attribute_printf(2, 3) pg_attribute_noreturn();
+
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void report_backup_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void report_fatal_error(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ printf("backup successfully verified\n");
+
+ return context.saw_any_error ? 1 : 0;
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ int rc;
+ JsonManifestParseContext context;
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ report_fatal_error("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ report_fatal_error("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /*
+ * Slurp in the whole file.
+ *
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
+ */
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
+ {
+ if (rc < 0)
+ report_fatal_error("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ report_fatal_error("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
+
+ /* Close the manifest file. */
+ close(fd);
+
+ /* Parse the manifest as JSON. */
+ context.private_data = ht;
+ context.perfile_cb = record_manifest_details_for_file;
+ context.error_cb = report_manifest_error;
+ json_parse_manifest(&context, buffer, statbuf.st_size);
+
+ /* Done with the buffer. */
+ pfree(buffer);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * Report an error while parsing the manifest.
+ *
+ * We consider all such errors to be fatal errors. The manifest parser
+ * expects this function not to return.
+ */
+static void
+report_manifest_error(JsonManifestParseContext *context, char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Record details extracted from the backup manifest for one file.
+ */
+static void
+record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload)
+{
+ manifestfiles_hash *ht = context->private_data;
+ manifestfile *tabent;
+ bool found;
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(ht, pathname, &found);
+ if (found)
+ report_fatal_error("duplicate pathname in backup manifest: \"%s\"",
+ pathname);
+
+ /* Initialize the entry. */
+ tabent->size = size;
+ tabent->checksum_type = checksum_type;
+ tabent->checksum_length = checksum_length;
+ tabent->checksum_payload = checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ report_fatal_error("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ report_backup_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ report_backup_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ report_backup_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ report_backup_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ report_backup_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ report_backup_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ report_backup_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ report_backup_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ report_backup_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ report_backup_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ report_backup_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ report_backup_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Report a problem with the backup.
+ *
+ * Update the context to indicate that we saw an error, and exit if the
+ * context says we should.
+ */
+static void
+report_backup_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+report_fatal_error(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+ printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
+}
diff --git a/src/bin/pg_validatebackup/t/001_basic.pl b/src/bin/pg_validatebackup/t/001_basic.pl
new file mode 100644
index 0000000000..6d4b8ea01a
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/001_basic.pl
@@ -0,0 +1,30 @@
+use strict;
+use warnings;
+use TestLib;
+use Test::More tests => 16;
+
+my $tempdir = TestLib::tempdir;
+
+program_help_ok('pg_validatebackup');
+program_version_ok('pg_validatebackup');
+program_options_handling_ok('pg_validatebackup');
+
+command_fails_like(['pg_validatebackup'],
+ qr/no backup directory specified/,
+ 'target directory must be specified');
+command_fails_like(['pg_validatebackup', $tempdir],
+ qr/could not open file.*\/backup_manifest\"/,
+ 'pg_validatebackup requires a manifest');
+command_fails_like(['pg_validatebackup', $tempdir, $tempdir],
+ qr/too many command-line arguments/,
+ 'multiple target directories not allowed');
+
+# create fake manifest file
+open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+close($fh);
+
+# but then try to use an alternate, nonexisting manifest
+command_fails_like(['pg_validatebackup', '-m', "$tempdir/not_the_manifest",
+ $tempdir],
+ qr/could not open file.*\/not_the_manifest\"/,
+ 'pg_validatebackup respects -m flag');
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
new file mode 100644
index 0000000000..98871e12a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -0,0 +1,58 @@
+# Verify that we can take and validate backups with various checksum types.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 19;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
+{
+ my $backup_path = $master->backup_dir . '/' . $algorithm;
+ my @backup = ('pg_basebackup', '-D', $backup_path,
+ '--manifest-checksums', $algorithm,
+ '--no-sync');
+ my @validate = ('pg_validatebackup', '-e', $backup_path);
+
+ # A backup with a bogus algorithm should fail.
+ if ($algorithm eq 'bogus')
+ {
+ $master->command_fails(\@backup,
+ "backup fails with algorithm \"$algorithm\"");
+ next;
+ }
+
+ # A backup with a valid algorithm should work.
+ $master->command_ok(\@backup, "backup ok with algorithm \"$algorithm\"");
+
+ # We expect each real checksum algorithm to be mentioned on every line of
+ # the backup manifest file except the first and last; for simplicity, we
+ # just check that it shows up lots of times. When the checksum algorithm
+ # is none, we just check that the manifest exists.
+ if ($algorithm eq 'none')
+ {
+ ok(-f "$backup_path/backup_manifest", "backup manifest exists");
+ }
+ else
+ {
+ my $manifest = slurp_file("$backup_path/backup_manifest");
+ my $count_of_algorithm_in_manifest =
+ (() = $manifest =~ /$algorithm/mig);
+ cmp_ok($count_of_algorithm_in_manifest, '>', 100,
+ "$algorithm is mentioned many times in the manifest");
+ }
+
+ # Make sure that it validates OK.
+ $master->command_ok(\@validate,
+ "validate backup with algorithm \"$algorithm\"");
+
+ # Remove backup immediately to save disk space.
+ rmtree($backup_path);
+}
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
new file mode 100644
index 0000000000..6ad29a031f
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -0,0 +1,251 @@
+# Verify that various forms of corruption are detected by pg_validatebackup.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+# Include a user-defined tablespace in the hopes of detecting problems in that
+# area.
+my $source_ts_path = TestLib::tempdir;
+$master->safe_psql('postgres', <<EOM);
+CREATE TABLE x1 (a int);
+INSERT INTO x1 VALUES (111);
+CREATE TABLESPACE ts1 LOCATION '$source_ts_path';
+CREATE TABLE x2 (a int) TABLESPACE ts1;
+INSERT INTO x1 VALUES (222);
+EOM
+
+my @scenario = (
+ {
+ 'name' => 'extra_file',
+ 'mutilate' => \&mutilate_extra_file,
+ 'fails_like' =>
+ qr/extra_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'extra_tablespace_file',
+ 'mutilate' => \&mutilate_extra_tablespace_file,
+ 'fails_like' =>
+ qr/extra_ts_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'missing_file',
+ 'mutilate' => \&mutilate_missing_file,
+ 'fails_like' =>
+ qr/pg_xact\/0000.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'missing_tablespace',
+ 'mutilate' => \&mutilate_missing_tablespace,
+ 'fails_like' =>
+ qr/pg_tblspc.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'append_to_file',
+ 'mutilate' => \&mutilate_append_to_file,
+ 'fails_like' =>
+ qr/has size \d+ on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'truncate_file',
+ 'mutilate' => \&mutilate_truncate_file,
+ 'fails_like' =>
+ qr/has size 0 on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'replace_file',
+ 'mutilate' => \&mutilate_replace_file,
+ 'fails_like' => qr/checksum mismatch for file/
+ },
+ {
+ 'name' => 'bad_manifest',
+ 'mutilate' => \&mutilate_bad_manifest,
+ 'fails_like' => qr/manifest checksum mismatch/
+ },
+ {
+ 'name' => 'open_file_fails',
+ 'mutilate' => \&mutilate_open_file_fails,
+ 'fails_like' => qr/could not open file/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'open_directory_fails',
+ 'mutilate' => \&mutilate_open_directory_fails,
+ 'fails_like' => qr/could not open directory/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'search_directory_fails',
+ 'mutilate' => \&mutilate_search_directory_fails,
+ 'cleanup' => \&cleanup_search_directory_fails,
+ 'fails_like' => qr/could not stat file or directory/,
+ 'skip_on_windows' => 1
+ }
+);
+
+for my $scenario (@scenario)
+{
+ my $name = $scenario->{'name'};
+
+ SKIP:
+ {
+ skip "unix-style permissions not supported on Windows", 4
+ if $scenario->{'skip_on_windows'} && $windows_os;
+
+ # Take a backup and check that it validates OK.
+ my $backup_path = $master->backup_dir . '/' . $name;
+ my $backup_ts_path = TestLib::tempdir;
+ $master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '-T', "${source_ts_path}=${backup_ts_path}"],
+ "base backup ok");
+ command_ok(['pg_validatebackup', $backup_path ],
+ "intact backup validated");
+
+ # Mutilate the backup in some way.
+ $scenario->{'mutilate'}->($backup_path);
+
+ # Now check that the backup no longer validates.
+ command_fails_like(['pg_validatebackup', $backup_path ],
+ $scenario->{'fails_like'},
+ "corrupt backup fails validation: $name");
+
+ # Run cleanup hook, if provided.
+ $scenario->{'cleanup'}->($backup_path)
+ if exists $scenario->{'cleanup'};
+
+ # Finally, use rmtree to reclaim space.
+ rmtree($backup_path);
+ }
+}
+
+sub create_extra_file
+{
+ my ($backup_path, $relative_path) = @_;
+ my $pathname = "$backup_path/$relative_path";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh "This is an extra file.\n";
+ close($fh);
+}
+
+# Add a file into the root directory of the backup.
+sub mutilate_extra_file
+{
+ my ($backup_path) = @_;
+ create_extra_file($backup_path, "extra_file");
+}
+
+# Add a file inside the user-defined tablespace.
+sub mutilate_extra_tablespace_file
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my ($catvdir) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid");
+ my ($tsdboid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid/$catvdir");
+ create_extra_file($backup_path,
+ "pg_tblspc/$tsoid/$catvdir/$tsdboid/extra_ts_file");
+}
+
+# Remove a file.
+sub mutilate_missing_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_xact/0000";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Remove the symlink to the user-defined tablespace.
+sub mutilate_missing_tablespace
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my $pathname = "$backup_path/pg_tblspc/$tsoid";
+ if ($windows_os)
+ {
+ rmdir($pathname) || die "$pathname: $!";
+ }
+ else
+ {
+ unlink($pathname) || die "$pathname: $!";
+ }
+}
+
+# Append an additional bytes to a file.
+sub mutilate_append_to_file
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/global/pg_control", 'x';
+}
+
+# Truncate a file to zero length.
+sub mutilate_truncate_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/global/pg_control";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ close($fh);
+}
+
+# Replace a file's contents without changing the length of the file. This is
+# not a particularly efficient way to do this, so we pick a file that's
+# expected to be short.
+sub mutilate_replace_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ my $contents = slurp_file($pathname);
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh 'q' x length($contents);
+ close($fh);
+}
+
+# Corrupt the backup manifest.
+sub mutilate_bad_manifest
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/backup_manifest", "\n";
+}
+
+# Create a file that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_file_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_subtrans";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be searched. (This is skipped on Windows.)
+sub mutilate_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0400, $pathname) || die "chmod $pathname: $!";
+}
+
+# rmtree can't cope with a mode 400 directory, so change back to 700.
+sub cleanup_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0700, $pathname) || die "chmod $pathname: $!";
+}
diff --git a/src/bin/pg_validatebackup/t/004_options.pl b/src/bin/pg_validatebackup/t/004_options.pl
new file mode 100644
index 0000000000..8f185626ed
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/004_options.pl
@@ -0,0 +1,89 @@
+# Verify the behavior of assorted pg_validatebackup options.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 25;
+
+# Start up the server and take a backup.
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_options';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync' ],
+ "base backup ok");
+
+# Verify that pg_validatebackup -q succeeds and produces no output.
+my $stdout;
+my $stderr;
+my $result = IPC::Run::run ['pg_validatebackup', '-q', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok($result, "-q succeeds: exit code 0");
+is($stdout, '', "-q succeeds: no stdout");
+is($stderr, '', "-q succeeds: no stderr");
+
+# Corrupt the PG_VERSION file.
+my $version_pathname = "$backup_path/PG_VERSION";
+my $version_contents = slurp_file($version_pathname);
+open(my $fh, '>', $version_pathname) || die "open $version_pathname: $!";
+print $fh 'q' x length($version_contents);
+close($fh);
+
+# Verify that pg_validatebackup -q now fails.
+command_fails_like(['pg_validatebackup', '-q', $backup_path ],
+ qr/checksum mismatch for file \"PG_VERSION\"/,
+ '-q checksum mismatch');
+
+# Since we didn't change the length of the file, validation should succeed
+# if we ignore checksums. Check that we get the right message, too.
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ '-s skips checksumming');
+
+# Validation should succeed if we ignore the problem file.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/backup successfully verified/,
+ '-i ignores problem file');
+
+# PG_VERSION is already corrupt; let's try also removing all of pg_xact.
+rmtree($backup_path . "/pg_xact");
+
+# We're ignoring the problem with PG_VERSION, but not the problem with
+# pg_xact, so validation should fail here.
+command_fails_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/pg_xact.*is present in the manifest but not on disk/,
+ '-i does not ignore all problems');
+
+# If we use -i twice, we should be able to ignore all of the problems.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', '-i', 'pg_xact',
+ $backup_path ],
+ qr/backup successfully verified/,
+ 'multiple -i options work');
+
+# Verify that when -i is not used, both problems are reported.
+$result = IPC::Run::run ['pg_validatebackup', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "multiple problems: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "multiple problems: missing files reported");
+like($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "multiple problems: checksum mismatch reported");
+
+# Verify that when -e is used, only the problem detected first is reported.
+$result = IPC::Run::run ['pg_validatebackup', '-e', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "-e reports 1 error: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "-e reports 1 error: missing files reported");
+unlike($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "-e reports 1 error: checksum mismatch not reported");
+
+# Test valid manifest with nonexistent backup directory.
+command_fails_like(['pg_validatebackup', '-m', "$backup_path/backup_manifest",
+ "$backup_path/fake" ],
+ qr/could not open directory/,
+ 'nonexistent backup directory');
diff --git a/src/bin/pg_validatebackup/t/005_bad_manifest.pl b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
new file mode 100644
index 0000000000..9c503600d2
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
@@ -0,0 +1,158 @@
+# Test the behavior of pg_validatebackup when the backup manifest has
+# problems.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $tempdir = TestLib::tempdir;
+
+test_bad_manifest('input string ended unexpectedly',
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
+{
+EOM
+
+test_parse_error('unexpected object end', <<EOM);
+{}
+EOM
+
+test_parse_error('unexpected array start', <<EOM);
+[]
+EOM
+
+test_parse_error('expected version indicator', <<EOM);
+{"not-expected": 1}
+EOM
+
+test_parse_error('unexpected manifest version', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": "phooey"}
+EOM
+
+test_parse_error('unexpected scalar', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": true}
+EOM
+
+test_parse_error('expected file list', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Oops": 1}
+EOM
+
+test_parse_error('unexpected object start', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": {}}
+EOM
+
+test_parse_error('missing pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [{}]}
+EOM
+
+test_parse_error('both pathname and encoded pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Encoded-Path": "1234"}
+]}
+EOM
+
+test_parse_error('unexpected file field', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Oops": 1}
+]}
+EOM
+
+test_parse_error('missing size', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x"}
+]}
+EOM
+
+test_parse_error('file size is not an integer', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": "Oops"}
+]}
+EOM
+
+test_parse_error('unable to decode filename', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Encoded-Path": "123", "Size": 0}
+]}
+EOM
+
+test_fatal_error('duplicate pathname in backup manifest', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 0},
+ {"Path": "x", "Size": 0}
+]}
+EOM
+
+test_parse_error('checksum without algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum": "Oops"}
+]}
+EOM
+
+test_fatal_error('unrecognized checksum algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "Oops", "Checksum": "00"}
+]}
+EOM
+
+test_fatal_error('invalid checksum for file', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "CRC32C", "Checksum": "0"}
+]}
+EOM
+
+test_parse_error('expected manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Oops": 1}
+EOM
+
+test_parse_error('expected at least 2 lines', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [], "Manifest-Checksum": null}
+EOM
+
+my $manifest_without_newline = <<EOM;
+{"PostgreSQL-Backup-Manifest-Version": 1,
+ "Files": [],
+ "Manifest-Checksum": null}
+EOM
+chomp($manifest_without_newline);
+test_parse_error('last line not newline-terminated',
+ $manifest_without_newline);
+
+test_fatal_error('invalid manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Manifest-Checksum": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890-"}
+EOM
+
+sub test_parse_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/could not parse backup manifest: $test_name/,
+ $manifest_contents);
+}
+
+sub test_fatal_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/fatal: $test_name/,
+ $manifest_contents);
+}
+
+sub test_bad_manifest
+{
+ my ($test_name, $regexp, $manifest_contents) = @_;
+
+ open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+ print $fh $manifest_contents;
+ close($fh);
+
+ command_fails_like(['pg_validatebackup', $tempdir], $regexp,
+ $test_name);
+}
diff --git a/src/bin/pg_validatebackup/t/006_encoding.pl b/src/bin/pg_validatebackup/t/006_encoding.pl
new file mode 100644
index 0000000000..5e3e7152a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/006_encoding.pl
@@ -0,0 +1,27 @@
+# Verify that pg_validatebackup handles hex-encoded filenames correctly.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 5;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_encoding';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '--manifest-force-encode' ],
+ "backup ok with forced hex encoding");
+
+my $manifest = slurp_file("$backup_path/backup_manifest");
+my $count_of_encoded_path_in_manifest =
+ (() = $manifest =~ /Encoded-Path/mig);
+cmp_ok($count_of_encoded_path_in_manifest, '>', 100,
+ "many paths are encoded in the manifest");
+
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ 'backup with forced encoding validated');
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 19:50 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 20:16 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 23:24 ` David Steele <[email protected]>
2020-03-31 11:57 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: David Steele @ 2020-03-30 23:24 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/30/20 4:16 PM, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 3:51 PM David Steele <[email protected]> wrote:
>
>> > { "Path": "backup_label", "Size": 224, "Last-Modified": "2020-03-27
>> 18:33:18 GMT", "Checksum-Algorithm": "CRC32C", "Checksum": "b914bec9" },
>>
>> Storing the checksum type with each file seems pretty redundant.
>> Perhaps that could go in the header? You could always override if a
>> specific file had a different checksum type, though that seems unlikely.
>>
>> In general it might be good to go with shorter keys: "mod", "chk", etc.
>> Manifests can get pretty big and that's a lot of extra bytes.
>>
>> I'm also partial to using epoch time in the manifest because it is
>> generally easier for programs to work with. But, human-readable doesn't
>> suck, either.
>
> It doesn't seem impossible for it to come up; for example, consider a
> file-level incremental backup facility. You might retain whatever
> checksums you have for the unchanged files (to avoid rereading them)
> and add checksums for modified or added files.
OK.
> I am not convinced that minimizing the size of the file here is a
> particularly important goal, because I don't think it's going to get
> that big in normal cases. I also think having the keys and values be
> easily understandable by human being is a plus. If we really want a
> minimal format without redundancy, we should've gone with what I
> proposed before (though admittedly that could've been tamped down even
> further if we'd cared to squeeze, which I didn't think was important
> then either).
Well, normal cases is the key. But fine, in general we have found that
the in memory representation is more important in terms of supporting
clusters with very large numbers of files.
>> When I ran pg_validatebackup I expected to use -D to specify the backup
>> dir since pg_basebackup does. On the other hand -D is weird because I
>> *really* expect that to be the pg data dir.
>>
>> But, do we want this to be different from pg_basebackup?
>
> I think it's pretty distinguishable, because pg_basebackup needs an
> input (server) and an output (directory), whereas pg_validatebackup
> only needs one. I don't really care if we want to change it, but I was
> thinking of this as being more analogous to, say, pg_resetwal.
> Granted, that's a danger-don't-use-this tool and this isn't, but I
> don't think we want the -D-is-optional behavior that tools like pg_ctl
> have, because having a tool that isn't supposed to be used on a
> running cluster default to $PGDATA seems inadvisable. And if the
> argument is mandatory then it's not clear to me why we should make
> people type -D in front of it.
Honestly I think pg_basebackup is the confusing one, because in most
cases -D points at the running cluster dir. So, OK.
>> > + checksum_length = checksum_string_length / 2;
>>
>> This check is defeated if a single character is added the to checksum.
>>
>> Not too big a deal since you still get an error, but still.
>
> I don't see what the problem is here. We speculatively divide by two
> and allocate memory assuming the value that it was even, but then
> before doing anything critical we bail out if it was actually odd.
> That's harmless. We could get around it by saying:
>
> if (checksum_string_length % 2 != 0)
> context->error_cb(...);
> checksum_length = checksum_string_length / 2;
> checksum_payload = palloc(checksum_length);
> if (!hexdecode_string(...))
> context->error_cb(...);
>
> ...but that would be adding additional code, and error messages, for
> what's basically a can't-happen-unless-the-user-is-messing-with-us
> case.
Sorry, pasted the wrong code and even then still didn't get it quite
right.
The problem:
If I remove an even characters from a checksum it appears the checksum
passes but the manifest checksum fails:
$ pg_basebackup -D test/backup5 --manifest-checksums=SHA256
$ vi test/backup5/backup_manifest
* Remove two characters from the checksum of backup_label
$ pg_validatebackup test/backup5
pg_validatebackup: fatal: manifest checksum mismatch
But if I add any number of characters or remove an odd number of
characters I get:
pg_validatebackup: fatal: invalid checksum for file "backup_label":
"a98e9164fd59d498d14cfdf19c67d1c2208a30e7b939d1b4a09f524c7adfc11fXX"
and no manifest checksum failure.
>> > + * Verify that the manifest checksum is correct.
>>
>> This is not working the way I would expect -- I could freely modify the
>> manifest without getting a checksum error on the manifest. For example:
>>
>> $ /home/vagrant/test/pg/bin/pg_validatebackup test/backup3
>> pg_validatebackup: fatal: invalid checksum for file "backup_label":
>> "408901e0814f40f8ceb7796309a59c7248458325a21941e7c55568e381f53831?"
>>
>> So, if I deleted the entry above, I got a manifest checksum error. But
>> if I just modified the checksum I get a file checksum error with no
>> manifest checksum error.
>>
>> I would prefer a manifest checksum error in all cases where it is wrong,
>> unless --exit-on-error is specified.
>
> I think I would too, but I'm confused as to what you're doing, because
> if I just modified the manifest -- by deleting a file, for example, or
> changing the checksum of a file, I just get:
>
> pg_validatebackup: fatal: manifest checksum mismatch
>
> I'm confused as to why you're not seeing that. What's the exact
> sequence of steps?
$ pg_basebackup -D test/backup5 --manifest-checksums=SHA256
$ vi test/backup5/backup_manifest
* Add 'X' to the checksum of backup_label
$ pg_validatebackup test/backup5
pg_validatebackup: fatal: invalid checksum for file "backup_label":
"a98e9164fd59d498d14cfdf19c67d1c2208a30e7b939d1b4a09f524c7adfc11fX"
No mention of the manifest checksum being invalid. But if I remove the
backup label file from the manifest:
pg_validatebackup: fatal: manifest checksum mismatch
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 19:50 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 20:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 23:24 ` Re: backup manifests David Steele <[email protected]>
@ 2020-03-31 11:57 ` Robert Haas <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-31 11:57 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 30, 2020 at 7:24 PM David Steele <[email protected]> wrote:
> > I'm confused as to why you're not seeing that. What's the exact
> > sequence of steps?
>
> $ pg_basebackup -D test/backup5 --manifest-checksums=SHA256
>
> $ vi test/backup5/backup_manifest
> * Add 'X' to the checksum of backup_label
>
> $ pg_validatebackup test/backup5
> pg_validatebackup: fatal: invalid checksum for file "backup_label":
> "a98e9164fd59d498d14cfdf19c67d1c2208a30e7b939d1b4a09f524c7adfc11fX"
>
> No mention of the manifest checksum being invalid. But if I remove the
> backup label file from the manifest:
>
> pg_validatebackup: fatal: manifest checksum mismatch
Oh, I see what's happening now. If the checksum is not an even-length
string of hexademical characters, it's treated as a syntax error, so
it bails out at that point. Generally, a syntax error in the manifest
file is treated as a fatal error, and you just die right there. You'd
get the same behavior if you had malformed JSON, like a stray { or }
or [ or ] someplace that it doesn't belong according to the rules of
JSON. On the other hand, if you corrupt the checksum by adding AA or
EE or 54 or some other even-length string of hex characters, then you
have (in this code's view) a semantic error rather than a syntax
error, so it will finish loading all the manifest data and then bail
because the checksum doesn't match.
We really can't avoid bailing out early sometimes, because if the file
is totally malformed at the JSON level, there's just no way to
continue. We could cause this particular error to get treated as a
semantic error rather than a syntax error, but I don't really see much
advantage in so doing. This way was easier to code, and I don't think
it really matters which error we find first.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-29 03:40 ` Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Noah Misch @ 2020-03-29 03:40 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 01:53:54PM -0400, Robert Haas wrote:
> - Replace a doc paragraph about the advantages and disadvantages of
> CRC-32C with one by Stephen Frost, with a slightly change by me that I
> thought made it sound more grammatical.
Defaulting to CRC-32C seems prudent to me:
- As Andres Freund said, SHA-512 is slow relative to storage now available.
Since gzip is a needlessly-slow choice for backups (or any application that
copies the compressed data just a few times), comparison to "gzip -6" speed
is immaterial.
- While I'm sure some other fast hash would be a superior default, introducing
a new algorithm is a bikeshed, as you said. This design makes it easy,
technically, for someone to introduce a new algorithm later. CRC-32C is not
catastrophically unfit for 1GiB files.
- Defaulting to SHA-512 would, in the absence of a WAL archive that also uses
a cryptographic hash function, give a false sense of having achieved some
coherent cryptographic goal. With the CRC-32C default, WAL and the rest get
similar protection. I'm discounting the case of using BASE_BACKUP without a
WAL archive, because I expect little intersection between sites "worried
enough to hash everything" and those "not worried enough to use an archive".
(On the other hand, the program that manages the WAL archive can reasonably
own hashing base backups; putting ownership in the server isn't achieving
much extra.)
> + <refnamediv>
> + <refname>pg_validatebackup</refname>
> + <refpurpose>verify the integrity of a base backup of a
> + <productname>PostgreSQL</productname> cluster</refpurpose>
> + </refnamediv>
> + <listitem>
> + <para>
> + <literal>pg_wal</literal> is ignored because WAL files are sent
> + separately from the backup, and are therefore not described by the
> + backup manifest.
> + </para>
> + </listitem>
Stephen Frost mentioned that a backup could pass validation even if
pg_basebackup were killed after writing the base backup and before finishing
the writing of pg_wal. One might avoid that by simply writing the manifest to
a temporary name and renaming it to the final name after populating pg_wal.
What do you think of having the verification process also call pg_waldump to
validate the WAL CRCs (shown upthread)? That looked helpful and simple.
I think this functionality doesn't belong in its own program. If you suspect
pg_basebackup or pg_restore will eventually gain the ability to merge
incremental backups into a recovery-ready base backup, I would put the
functionality in that program. Otherwise, I would put it in pg_checksums.
For me, part of the friction here is that the program description indicates
general verification, but the actual functionality merely checks hashes on a
directory tree that happens to represent a PostgreSQL base backup.
> + parse->pathname = palloc(raw_length + 1);
I don't see this freed anywhere; is it? (It's useful to make peak memory
consumption not grow in proportion to the number of files backed up.)
[This message is not a full code review.]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
@ 2020-03-30 00:42 ` Robert Haas <[email protected]>
2020-03-30 00:54 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 05:58 ` Re: backup manifests Noah Misch <[email protected]>
0 siblings, 3 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-30 00:42 UTC (permalink / raw)
To: Noah Misch <[email protected]>; +Cc: David Steele <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Sat, Mar 28, 2020 at 11:40 PM Noah Misch <[email protected]> wrote:
> Stephen Frost mentioned that a backup could pass validation even if
> pg_basebackup were killed after writing the base backup and before finishing
> the writing of pg_wal. One might avoid that by simply writing the manifest to
> a temporary name and renaming it to the final name after populating pg_wal.
Huh, that's an idea. I'll have a look at the code and see what would
be involved.
> What do you think of having the verification process also call pg_waldump to
> validate the WAL CRCs (shown upthread)? That looked helpful and simple.
I don't love calls to external binaries, but I think the thing that
really bothers me is that pg_waldump is practically bound to terminate
with an error, because the last WAL segment will end with a partial
record. For the same reason, I think there's really no such thing as
validating a single WAL file. I suppose you'd need to know the exact
start and end locations for a minimal WAL replay and check that all
records between those LSNs appear OK, ignoring any apparent problems
after the minimum ending point, or at least ignoring any problems due
to an incomplete record in the last file. We don't have a tool for
that currently, and I don't think I can write one this week. Or at
least, not a good one.
> I think this functionality doesn't belong in its own program. If you suspect
> pg_basebackup or pg_restore will eventually gain the ability to merge
> incremental backups into a recovery-ready base backup, I would put the
> functionality in that program. Otherwise, I would put it in pg_checksums.
> For me, part of the friction here is that the program description indicates
> general verification, but the actual functionality merely checks hashes on a
> directory tree that happens to represent a PostgreSQL base backup.
Suraj's original patch made this part of pg_basebackup, but I didn't
really like that, because I wanted it to have its own set of options.
I still think all the options I've added are pretty useful ones, and I
can think of other things somebody might want to do. It feels very
uncomfortable to make pg_basebackup, or pg_checksums, take either
options from set A and do thing X, or options from set B and do thing
Y. But it feels clear that the name pg_validatebackup is not going
over very well with anyone. I think I should rename it to
pg_validatemanifest.
> > + parse->pathname = palloc(raw_length + 1);
>
> I don't see this freed anywhere; is it? (It's useful to make peak memory
> consumption not grow in proportion to the number of files backed up.)
We need the hash table to remain populated for the whole run time of
the tool, because we're essentially doing a full join of the actual
directory contents against the manifest contents. That's a bit
unfortunate but it doesn't seem simple to improve. I think the only
people who are really going to suffer are people who have an enormous
pile of empty or nearly-empty relations. People who have large
databases for the normal reason - i.e. a reasonable number of tables
that hold a lot of data - will have manifests of very manageable size.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 00:54 ` David Steele <[email protected]>
2 siblings, 0 replies; 372+ messages in thread
From: David Steele @ 2020-03-30 00:54 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Noah Misch <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/29/20 8:42 PM, Robert Haas wrote:
> On Sat, Mar 28, 2020 at 11:40 PM Noah Misch <[email protected]> wrote:
>> I don't see this freed anywhere; is it? (It's useful to make peak memory
>> consumption not grow in proportion to the number of files backed up.)
>
> We need the hash table to remain populated for the whole run time of
> the tool, because we're essentially doing a full join of the actual
> directory contents against the manifest contents. That's a bit
> unfortunate but it doesn't seem simple to improve. I think the only
> people who are really going to suffer are people who have an enormous
> pile of empty or nearly-empty relations. People who have large
> databases for the normal reason - i.e. a reasonable number of tables
> that hold a lot of data - will have manifests of very manageable size.
+1
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 01:07 ` Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-30 01:07 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Noah Misch <[email protected]>; David Steele <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-29 20:42:35 -0400, Robert Haas wrote:
> > What do you think of having the verification process also call pg_waldump to
> > validate the WAL CRCs (shown upthread)? That looked helpful and simple.
>
> I don't love calls to external binaries, but I think the thing that
> really bothers me is that pg_waldump is practically bound to terminate
> with an error, because the last WAL segment will end with a partial
> record.
I don't think that's the case here. You should know the last required
record, which should allow to specify the precise end for pg_waldump. If
it errors out reading to that point, we'd be in trouble.
> For the same reason, I think there's really no such thing as
> validating a single WAL file. I suppose you'd need to know the exact
> start and end locations for a minimal WAL replay and check that all
> records between those LSNs appear OK, ignoring any apparent problems
> after the minimum ending point, or at least ignoring any problems due
> to an incomplete record in the last file. We don't have a tool for
> that currently, and I don't think I can write one this week. Or at
> least, not a good one.
pg_waldump -s / -e?
> > > + parse->pathname = palloc(raw_length + 1);
> >
> > I don't see this freed anywhere; is it? (It's useful to make peak memory
> > consumption not grow in proportion to the number of files backed up.)
>
> We need the hash table to remain populated for the whole run time of
> the tool, because we're essentially doing a full join of the actual
> directory contents against the manifest contents. That's a bit
> unfortunate but it doesn't seem simple to improve. I think the only
> people who are really going to suffer are people who have an enormous
> pile of empty or nearly-empty relations. People who have large
> databases for the normal reason - i.e. a reasonable number of tables
> that hold a lot of data - will have manifests of very manageable size.
Given that that's a pre-existing issue - at a significantly larger scale
imo - e.g. for pg_dump (even in the --schema-only case), and that there
are tons of backend side issues with lots of relations too, I think
that's fine.
You could of course implement something merge-join like, and implement
the sorted input via a disk base sort. But that's a lot of work (good
luck making tuplesort work in the frontend...). So I'd not go there
unless there's a lot of evidence this is a serious practical issue.
If we find this use too much memory, I think we'd be better off
condensing pathnames into either fewer allocations, or a RelFileNode as
part of the struct (with a fallback to string for other types of
files). But I'd also not go there for now.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-30 01:23 ` David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: David Steele @ 2020-03-30 01:23 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Robert Haas <[email protected]>; +Cc: Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/29/20 9:07 PM, Andres Freund wrote:
> On 2020-03-29 20:42:35 -0400, Robert Haas wrote:
>>> What do you think of having the verification process also call pg_waldump to
>>> validate the WAL CRCs (shown upthread)? That looked helpful and simple.
>>
>> I don't love calls to external binaries, but I think the thing that
>> really bothers me is that pg_waldump is practically bound to terminate
>> with an error, because the last WAL segment will end with a partial
>> record.
>
> I don't think that's the case here. You should know the last required
> record, which should allow to specify the precise end for pg_waldump. If
> it errors out reading to that point, we'd be in trouble.
Exactly. All WAL generated during the backup should read fine with
pg_waldump or there is a problem.
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
@ 2020-03-30 02:08 ` Andres Freund <[email protected]>
2020-03-30 18:35 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-30 02:08 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Robert Haas <[email protected]>; Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-29 21:23:06 -0400, David Steele wrote:
> On 3/29/20 9:07 PM, Andres Freund wrote:
> > On 2020-03-29 20:42:35 -0400, Robert Haas wrote:
> > > > What do you think of having the verification process also call pg_waldump to
> > > > validate the WAL CRCs (shown upthread)? That looked helpful and simple.
> > >
> > > I don't love calls to external binaries, but I think the thing that
> > > really bothers me is that pg_waldump is practically bound to terminate
> > > with an error, because the last WAL segment will end with a partial
> > > record.
> >
> > I don't think that's the case here. You should know the last required
> > record, which should allow to specify the precise end for pg_waldump. If
> > it errors out reading to that point, we'd be in trouble.
>
> Exactly. All WAL generated during the backup should read fine with
> pg_waldump or there is a problem.
See the attached minimal prototype for what I am thinking of.
This would not correctly handle the case where the timeline changes
while taking a base backup. But I'm not sure that'd be all that serious
a limitation for now?
I'd personally not want to use a base backup that included a timeline
switch...
Greetings,
Andres Freund
Attachments:
[text/x-diff] waldump-verify.diff (2.3K, ../../[email protected]/2-waldump-verify.diff)
download | inline diff:
diff --git c/src/bin/pg_basebackup/pg_basebackup.c i/src/bin/pg_basebackup/pg_basebackup.c
index c5d95958b29..b61492eba4c 100644
--- c/src/bin/pg_basebackup/pg_basebackup.c
+++ i/src/bin/pg_basebackup/pg_basebackup.c
@@ -2033,6 +2033,11 @@ BaseBackup(void)
if (verbose)
pg_log_info("base backup completed");
+
+ if (format == 'p')
+ pg_log_info("pg_waldump --just-parse -p \"%s\" -t %u -s %s -e %s",
+ basedir, starttli,
+ xlogstart, xlogend);
}
diff --git c/src/bin/pg_waldump/pg_waldump.c i/src/bin/pg_waldump/pg_waldump.c
index 279acfa0440..63f4a9cba1f 100644
--- c/src/bin/pg_waldump/pg_waldump.c
+++ i/src/bin/pg_waldump/pg_waldump.c
@@ -40,6 +40,7 @@ typedef struct XLogDumpPrivate
typedef struct XLogDumpConfig
{
/* display options */
+ bool just_parse;
bool bkp_details;
int stop_after_records;
int already_displayed_records;
@@ -749,6 +750,7 @@ main(int argc, char **argv)
char *errormsg;
static struct option long_options[] = {
+ {"just-parse", no_argument, NULL, 'j'},
{"bkp-details", no_argument, NULL, 'b'},
{"end", required_argument, NULL, 'e'},
{"follow", no_argument, NULL, 'f'},
@@ -794,6 +796,7 @@ main(int argc, char **argv)
private.endptr = InvalidXLogRecPtr;
private.endptr_reached = false;
+ config.just_parse = false;
config.bkp_details = false;
config.stop_after_records = -1;
config.already_displayed_records = 0;
@@ -810,11 +813,14 @@ main(int argc, char **argv)
goto bad_argument;
}
- while ((option = getopt_long(argc, argv, "be:fn:p:r:s:t:x:z",
+ while ((option = getopt_long(argc, argv, "be:fjn:p:r:s:t:x:z",
long_options, &optindex)) != -1)
{
switch (option)
{
+ case 'j':
+ config.just_parse = true;
+ break;
case 'b':
config.bkp_details = true;
break;
@@ -1076,10 +1082,13 @@ main(int argc, char **argv)
continue;
/* process the record */
- if (config.stats == true)
- XLogDumpCountRecord(&config, &stats, xlogreader_state);
- else
- XLogDumpDisplayRecord(&config, xlogreader_state);
+ if (!config.just_parse)
+ {
+ if (config.stats == true)
+ XLogDumpCountRecord(&config, &stats, xlogreader_state);
+ else
+ XLogDumpDisplayRecord(&config, xlogreader_state);
+ }
/* check whether we printed enough */
config.already_displayed_records++;
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-30 18:35 ` Robert Haas <[email protected]>
2020-03-30 18:59 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-30 18:35 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: David Steele <[email protected]>; Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Sun, Mar 29, 2020 at 10:08 PM Andres Freund <[email protected]> wrote:
> See the attached minimal prototype for what I am thinking of.
>
> This would not correctly handle the case where the timeline changes
> while taking a base backup. But I'm not sure that'd be all that serious
> a limitation for now?
>
> I'd personally not want to use a base backup that included a timeline
> switch...
Interesting concept. I've never (or almost never) used the -s and -e
options to pg_waldump, so I didn't think about using those. I think
having a --just-parse option to pg_waldump is a good idea, though
maybe not with that name e.g. we could call it --quiet.
It is less obvious to me what to do about all that as it pertains to
the current patch. If we want pg_validatebackup to run pg_waldump in
that mode or print out a hint about how to run pg_waldump in that
mode, it would need to obtain the relevant LSNs. I guess that would
require reading the backup_label file. It's not clear to me what we
would do if the backup crosses a timeline switch, assuming that's even
a case pg_basebackup allows. If we don't want to do anything in
pg_validatebackup automatically but just want to document this as a a
possible technique, we could finesse that problem with some
weasel-wording.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 18:35 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 18:59 ` Andres Freund <[email protected]>
2020-03-30 19:23 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-31 18:10 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-30 18:59 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-30 14:35:40 -0400, Robert Haas wrote:
> On Sun, Mar 29, 2020 at 10:08 PM Andres Freund <[email protected]> wrote:
> > See the attached minimal prototype for what I am thinking of.
> >
> > This would not correctly handle the case where the timeline changes
> > while taking a base backup. But I'm not sure that'd be all that serious
> > a limitation for now?
> >
> > I'd personally not want to use a base backup that included a timeline
> > switch...
>
> Interesting concept. I've never (or almost never) used the -s and -e
> options to pg_waldump, so I didn't think about using those.
Oh - it's how I use it most of the time when investigating a specific
problem. I just about always use -s, and often -e. Besides just reducing
the logging output, and avoiding spurious errors, it makes it a lot
easier to iteratively expand the logging for records that are
problematic for the case at hand.
> I think
> having a --just-parse option to pg_waldump is a good idea, though
> maybe not with that name e.g. we could call it --quiet.
Yea, I didn't like the option's name. It's just the first thing that
came to mind.
> It is less obvious to me what to do about all that as it pertains to
> the current patch.
FWIW, I personally think we can live with this not validating WAL in the
first release. But I also think it'd be within reach to do better and
allow for WAL verification.
> If we want pg_validatebackup to run pg_waldump in that mode or print
> out a hint about how to run pg_waldump in that mode, it would need to
> obtain the relevant LSNs.
We could just include those in the manifest. Seems like good information
to have in there to me, as it allows to build the complete list of files
needed for a restore.
> It's not clear to me what we would do if the backup crosses a timeline
> switch, assuming that's even a case pg_basebackup allows.
I've not tested it, but it sure looks like it's possible. Both by having
a standby replaying from a node that promotes (multiple timeline
switches possible too, I think, if the WAL source follows timelines),
and by backing up from a standby that's being promoted.
> If we don't want to do anything in pg_validatebackup automatically but
> just want to document this as a a possible technique, we could finesse
> that problem with some weasel-wording.
It'd probably not be too hard to simply emit multiple commands, one for
each timeline "segment".
I wonder if it'd not be best, independent of whether we build in this
verification, to include that metadata in the manifest file. That's for
sure better than having to build a separate tool to parse timeline
history files.
I think it wouldn't be too hard to compute that information while taking
the base backup. We know the end timeline (ThisTimeLineID), so we can
just call readTimeLineHistory(ThisTimeLineID). Which should then allow
for something pretty trivial along the lines of
timelines = readTimeLineHistory(ThisTimeLineID);
last_start = InvalidXLogRecPtr;
foreach(lc, timelines)
{
TimeLineHistoryEntry *he = lfirst(lc);
if (he->end < startptr)
continue;
//
manifest_emit_wal_range(Min(he->begin, startptr), he->end);
last_start = he->end;
}
if (last_start == InvalidXlogRecPtr)
start = startptr;
else
start = last_start;
manifest_emit_wal_range(start, entptr);
Btw, just in case somebody suggests it: I don't think it's possible to
compute the WAL checksums at this point. In stream mode WAL very well
might already have been removed.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 18:35 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 18:59 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-30 19:23 ` Robert Haas <[email protected]>
2020-03-30 21:08 ` Re: backup manifests Andres Freund <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-30 19:23 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: David Steele <[email protected]>; Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 30, 2020 at 2:59 PM Andres Freund <[email protected]> wrote:
> I wonder if it'd not be best, independent of whether we build in this
> verification, to include that metadata in the manifest file. That's for
> sure better than having to build a separate tool to parse timeline
> history files.
I don't think that's better, or at least not "for sure better". The
backup_label going to include the START TIMELINE, and if -Xfetch is
used, we're also going to have all the timeline history files. If the
backup manifest includes those same pieces of information, then we've
got two sources of truth: one copy in the files the server's actually
going to read, and another copy in the backup_manifest which we're
going to potentially use for validation but ignore at runtime. That
seems not great.
> Btw, just in case somebody suggests it: I don't think it's possible to
> compute the WAL checksums at this point. In stream mode WAL very well
> might already have been removed.
Right.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 18:35 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 18:59 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 19:23 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 21:08 ` Andres Freund <[email protected]>
2020-03-30 22:56 ` Re: backup manifests David Steele <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-30 21:08 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-30 15:23:08 -0400, Robert Haas wrote:
> On Mon, Mar 30, 2020 at 2:59 PM Andres Freund <[email protected]> wrote:
> > I wonder if it'd not be best, independent of whether we build in this
> > verification, to include that metadata in the manifest file. That's for
> > sure better than having to build a separate tool to parse timeline
> > history files.
>
> I don't think that's better, or at least not "for sure better". The
> backup_label going to include the START TIMELINE, and if -Xfetch is
> used, we're also going to have all the timeline history files. If the
> backup manifest includes those same pieces of information, then we've
> got two sources of truth: one copy in the files the server's actually
> going to read, and another copy in the backup_manifest which we're
> going to potentially use for validation but ignore at runtime. That
> seems not great.
The data in the backup label isn't sufficient though. Without having
parsed the timeline file there's no way to verify that the correct WAL
is present. I guess we can also add client side tools to parse
timelines, add command the fetch all of the required files, and then
interpret that somehow.
But that seems much more complicated.
Imo it makes sense to want to be able verify that WAL looks correct even
transporting WAL using another method (say archiving) and thus using
pg_basebackup's -Xnone.
For the manifest to actually list what's required for the base backup
doesn't seem redundant to me. Imo it makes the manifest file make a good
bit more sense, since afterwards it actually describes the whole base
backup.
Taking the redundancy agreement a bit further you can argue that we
don't need a list of relation files at all, since they're in the catalog
:P. Obviously going to that extreme doesn't make all that much
sense... But I do think it's a second source of truth that's independent
of what the backends actually are going to read.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 18:35 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 18:59 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 19:23 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 21:08 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-30 22:56 ` David Steele <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: David Steele @ 2020-03-30 22:56 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Robert Haas <[email protected]>; +Cc: Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/30/20 5:08 PM, Andres Freund wrote:
>
> The data in the backup label isn't sufficient though. Without having
> parsed the timeline file there's no way to verify that the correct WAL
> is present. I guess we can also add client side tools to parse
> timelines, add command the fetch all of the required files, and then
> interpret that somehow.
>
> But that seems much more complicated.
>
> Imo it makes sense to want to be able verify that WAL looks correct even
> transporting WAL using another method (say archiving) and thus using
> pg_basebackup's -Xnone.
>
> For the manifest to actually list what's required for the base backup
> doesn't seem redundant to me. Imo it makes the manifest file make a good
> bit more sense, since afterwards it actually describes the whole base
> backup.
FWIW, pgBackRest stores the backup WAL stop/start in the manifest. To
get this information after the backup is complete requires parsing the
.backup file which doesn't get stored in the backup directory by
pg_basebackup. As far as I know, this is only accessibly to solutions
that implement archive_command. So, pgBackRest could do that but it
seems far more trouble than it is worth.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 18:35 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 18:59 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-31 18:10 ` Robert Haas <[email protected]>
2020-03-31 22:50 ` Re: backup manifests Andres Freund <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-31 18:10 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: David Steele <[email protected]>; Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 30, 2020 at 2:59 PM Andres Freund <[email protected]> wrote:
> I think it wouldn't be too hard to compute that information while taking
> the base backup. We know the end timeline (ThisTimeLineID), so we can
> just call readTimeLineHistory(ThisTimeLineID). Which should then allow
> for something pretty trivial along the lines of
>
> timelines = readTimeLineHistory(ThisTimeLineID);
> last_start = InvalidXLogRecPtr;
> foreach(lc, timelines)
> {
> TimeLineHistoryEntry *he = lfirst(lc);
>
> if (he->end < startptr)
> continue;
>
> //
> manifest_emit_wal_range(Min(he->begin, startptr), he->end);
> last_start = he->end;
> }
>
> if (last_start == InvalidXlogRecPtr)
> start = startptr;
> else
> start = last_start;
>
> manifest_emit_wal_range(start, entptr);
I made an attempt to implement this. In the attached patch set, 0001
and 0002 are (I think) unmodified from the last version. 0003 is a
slightly-rejiggered version of your new pg_waldump option. 0004 whacks
0002 around so that the WAL ranges are included in the manifest and
pg_validatebackup tries to run pg_waldump for each WAL range. It
appears to work in light testing, but I haven't yet (1) tested it
extensively, (2) written good regression tests for it above and beyond
what pg_validatebackup had already, or (3) updated the documentation.
I'm going to work on those things. I would appreciate *very timely*
feedback on anything people do or do not like about this, because I
want to commit this patch set by the end of the work week and that
isn't very far away. I would also appreciate if people would bear in
mind the principle that half a loaf is better than none, and further
improvements can be made in future releases.
As part of my light testing, I tried promoting a standby that was
running pg_basebackup, and found that pg_basebackup failed like this:
pg_basebackup: error: could not get COPY data stream: ERROR: the
standby was promoted during online backup
HINT: This means that the backup being taken is corrupt and should
not be used. Try taking another online backup.
pg_basebackup: removing data directory "/Users/rhaas/pgslave2"
My first thought was that this error message is hard to reconcile with
this comment:
/*
* Send timeline history files too. Only the latest timeline history
* file is required for recovery, and even that only if there happens
* to be a timeline switch in the first WAL segment that contains the
* checkpoint record, or if we're taking a base backup from a standby
* server and the target timeline changes while the backup is taken.
* But they are small and highly useful for debugging purposes, so
* better include them all, always.
*/
But then it occurred to me that this might be a cascading standby.
Maybe the original master died and this machine's master got promoted,
so it has to follow a timeline switch but doesn't itself get promoted.
I think I might try to test out that scenario and see what happens,
but I haven't done so as of this writing. Regardless, it seems like a
really good idea to store a list of WAL ranges rather than a single
start/end/timeline, because even if it's impossible today it might
become possible in the future. Still, unless there's an easy way to
set up a test scenario where multiple WAL ranges need to be verified,
it may be hard to test that this code actually behaves properly.
Thoughts?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/octet-stream] v16-0001-Add-checksum-helper-functions.patch (9.5K, ../../CA+TgmoawEeE5qpFgj5Vy2zZGKzd3ZSEhGrD_JdPqPd2GB8u1Cw@mail.gmail.com/2-v16-0001-Add-checksum-helper-functions.patch)
download | inline diff:
From 9f226ba3b596fb45f16a33100bca4fd1554ad9cc Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Fri, 20 Mar 2020 14:48:33 -0400
Subject: [PATCH v16 1/4] Add checksum helper functions.
Patch written from scratch by me, but it is similar to previous work
by Rushabh Lathia and Suraj Kharage. Suraj also reviewed this version
off-list. Advice on how not to break Windows from Davinder Singh.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
src/common/Makefile | 1 +
src/common/checksum_helper.c | 190 +++++++++++++++++++++++++++
src/include/common/checksum_helper.h | 74 +++++++++++
src/tools/msvc/Mkvcbuild.pm | 4 +-
4 files changed, 267 insertions(+), 2 deletions(-)
create mode 100644 src/common/checksum_helper.c
create mode 100644 src/include/common/checksum_helper.h
diff --git a/src/common/Makefile b/src/common/Makefile
index 6939b9d087..16619e4ba8 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -48,6 +48,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
archive.o \
base64.o \
+ checksum_helper.o \
config_info.o \
controldata_utils.o \
d2s.o \
diff --git a/src/common/checksum_helper.c b/src/common/checksum_helper.c
new file mode 100644
index 0000000000..79a9a7447b
--- /dev/null
+++ b/src/common/checksum_helper.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.c
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/checksum_helper.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/checksum_helper.h"
+
+/*
+ * If 'name' is a recognized checksum type, set *type to the corresponding
+ * constant and return true. Otherwise, set *type to CHECKSUM_TYPE_NONE and
+ * return false.
+ */
+bool
+pg_checksum_parse_type(char *name, pg_checksum_type *type)
+{
+ pg_checksum_type result_type = CHECKSUM_TYPE_NONE;
+ bool result = true;
+
+ if (pg_strcasecmp(name, "none") == 0)
+ result_type = CHECKSUM_TYPE_NONE;
+ else if (pg_strcasecmp(name, "crc32c") == 0)
+ result_type = CHECKSUM_TYPE_CRC32C;
+ else if (pg_strcasecmp(name, "sha224") == 0)
+ result_type = CHECKSUM_TYPE_SHA224;
+ else if (pg_strcasecmp(name, "sha256") == 0)
+ result_type = CHECKSUM_TYPE_SHA256;
+ else if (pg_strcasecmp(name, "sha384") == 0)
+ result_type = CHECKSUM_TYPE_SHA384;
+ else if (pg_strcasecmp(name, "sha512") == 0)
+ result_type = CHECKSUM_TYPE_SHA512;
+ else
+ result = false;
+
+ *type = result_type;
+ return result;
+}
+
+/*
+ * Get the canonical human-readable name corresponding to a checksum type.
+ */
+char *
+pg_checksum_type_name(pg_checksum_type type)
+{
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ return "NONE";
+ case CHECKSUM_TYPE_CRC32C:
+ return "CRC32C";
+ case CHECKSUM_TYPE_SHA224:
+ return "SHA224";
+ case CHECKSUM_TYPE_SHA256:
+ return "SHA256";
+ case CHECKSUM_TYPE_SHA384:
+ return "SHA384";
+ case CHECKSUM_TYPE_SHA512:
+ return "SHA512";
+ }
+
+ Assert(false);
+ return "???";
+}
+
+/*
+ * Initialize a checksum context for checksums of the given type.
+ */
+void
+pg_checksum_init(pg_checksum_context *context, pg_checksum_type type)
+{
+ context->type = type;
+
+ switch (type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ INIT_CRC32C(context->raw_context.c_crc32c);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_init(&context->raw_context.c_sha224);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_init(&context->raw_context.c_sha256);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_init(&context->raw_context.c_sha384);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_init(&context->raw_context.c_sha512);
+ break;
+ }
+}
+
+/*
+ * Update a checksum context with new data.
+ */
+void
+pg_checksum_update(pg_checksum_context *context, const uint8 *input,
+ size_t len)
+{
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ /* do nothing */
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ COMP_CRC32C(context->raw_context.c_crc32c, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_update(&context->raw_context.c_sha224, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_update(&context->raw_context.c_sha256, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_update(&context->raw_context.c_sha384, input, len);
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_update(&context->raw_context.c_sha512, input, len);
+ break;
+ }
+}
+
+/*
+ * Finalize a checksum computation and write the result to an output buffer.
+ *
+ * The caller must ensure that the buffer is at least PG_CHECKSUM_MAX_LENGTH
+ * bytes in length. The return value is the number of bytes actually written.
+ */
+int
+pg_checksum_final(pg_checksum_context *context, uint8 *output)
+{
+ int retval = 0;
+
+ StaticAssertStmt(sizeof(pg_crc32c) <= PG_CHECKSUM_MAX_LENGTH,
+ "CRC-32C digest too big for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA224_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA224 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA256_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA256 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA384_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA384 digest too for PG_CHECKSUM_MAX_LENGTH");
+ StaticAssertStmt(PG_SHA512_DIGEST_LENGTH <= PG_CHECKSUM_MAX_LENGTH,
+ "SHA512 digest too for PG_CHECKSUM_MAX_LENGTH");
+
+ switch (context->type)
+ {
+ case CHECKSUM_TYPE_NONE:
+ break;
+ case CHECKSUM_TYPE_CRC32C:
+ FIN_CRC32C(context->raw_context.c_crc32c);
+ retval = sizeof(pg_crc32c);
+ memcpy(output, &context->raw_context.c_crc32c, retval);
+ break;
+ case CHECKSUM_TYPE_SHA224:
+ pg_sha224_final(&context->raw_context.c_sha224, output);
+ retval = PG_SHA224_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA256:
+ pg_sha256_final(&context->raw_context.c_sha256, output);
+ retval = PG_SHA256_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA384:
+ pg_sha384_final(&context->raw_context.c_sha384, output);
+ retval = PG_SHA384_DIGEST_LENGTH;
+ break;
+ case CHECKSUM_TYPE_SHA512:
+ pg_sha512_final(&context->raw_context.c_sha512, output);
+ retval = PG_SHA512_DIGEST_LENGTH;
+ break;
+ }
+
+ Assert(retval <= PG_CHECKSUM_MAX_LENGTH);
+ return retval;
+}
diff --git a/src/include/common/checksum_helper.h b/src/include/common/checksum_helper.h
new file mode 100644
index 0000000000..48b0745dad
--- /dev/null
+++ b/src/include/common/checksum_helper.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * checksum_helper.h
+ * Compute a checksum of any of various types using common routines
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/common/checksum_helper.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CHECKSUM_HELPER_H
+#define CHECKSUM_HELPER_H
+
+#include "common/sha2.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Supported checksum types. It's not necessarily the case that code using
+ * these functions needs a cryptographically strong checksum; it may only
+ * need to detect accidental modification. That's why we include CRC-32C: it's
+ * much faster than any of the other algorithms. On the other hand, we omit
+ * MD5 here because any new that does need a cryptographically strong checksum
+ * should use something better.
+ */
+typedef enum pg_checksum_type
+{
+ CHECKSUM_TYPE_NONE,
+ CHECKSUM_TYPE_CRC32C,
+ CHECKSUM_TYPE_SHA224,
+ CHECKSUM_TYPE_SHA256,
+ CHECKSUM_TYPE_SHA384,
+ CHECKSUM_TYPE_SHA512
+} pg_checksum_type;
+
+/*
+ * This is just a union of all applicable context types.
+ */
+typedef union pg_checksum_raw_context
+{
+ pg_crc32c c_crc32c;
+ pg_sha224_ctx c_sha224;
+ pg_sha256_ctx c_sha256;
+ pg_sha384_ctx c_sha384;
+ pg_sha512_ctx c_sha512;
+} pg_checksum_raw_context;
+
+/*
+ * This structure provides a convenient way to pass the checksum type and the
+ * checksum context around together.
+ */
+typedef struct pg_checksum_context
+{
+ pg_checksum_type type;
+ pg_checksum_raw_context raw_context;
+} pg_checksum_context;
+
+/*
+ * This is the longest possible output for any checksum algorithm supported
+ * by this file.
+ */
+#define PG_CHECKSUM_MAX_LENGTH PG_SHA512_DIGEST_LENGTH
+
+extern bool pg_checksum_parse_type(char *name, pg_checksum_type *);
+extern char *pg_checksum_type_name(pg_checksum_type);
+
+extern void pg_checksum_init(pg_checksum_context *, pg_checksum_type);
+extern void pg_checksum_update(pg_checksum_context *, const uint8 *input,
+ size_t len);
+extern int pg_checksum_final(pg_checksum_context *, uint8 *output);
+
+#endif
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 39709f20e6..41e7678689 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -120,8 +120,8 @@ sub mkvcbuild
}
our @pgcommonallfiles = qw(
- archive.c
- base64.c config_info.c controldata_utils.c d2s.c encnames.c exec.c
+ archive.c base64.c checksum_helper.c
+ config_info.c controldata_utils.c d2s.c encnames.c exec.c
f2s.c file_perm.c hashfn.c ip.c jsonapi.c
keywords.c kwlookup.c link-canary.c md5.c
pg_lzcompress.c pgfnames.c psprintf.c relpath.c rmtree.c
--
2.17.2 (Apple Git-113)
[application/octet-stream] v16-0002-Generate-backup-manifests-for-base-backups-and-v.patch (117.8K, ../../CA+TgmoawEeE5qpFgj5Vy2zZGKzd3ZSEhGrD_JdPqPd2GB8u1Cw@mail.gmail.com/3-v16-0002-Generate-backup-manifests-for-base-backups-and-v.patch)
download | inline diff:
From b790264fe083919c858d12030075e230b66507ea Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Mon, 30 Mar 2020 16:14:00 -0400
Subject: [PATCH v16 2/4] Generate backup manifests for base backups, and
validate them.
A manifest is a JSON document which includes the file name, size, last
modification time, and a checksum for each file backed up, as well as
a checksum for the manifest itself. By default, we use CRC-32C for the
checksum algorithm, because we are trying to detect corruption and
user error, not foil an adversary. However, pg_basebackup and the
server-side BASE_BACKUP command now have options to select the
checksum algorithm, so users wanting a cryptographic hash function can
select SHA-224, SHA-256, SHA-384, or SHA-512. Users not wanting any
checksums at all can disable them, or disable generating of the backup
manifest altogether. Using a cryptographic hash function in place of
CRC-32C consumes significantly more CPU cycles, which may slow down
backups in some cases.
A new tool called pg_validatebackup can validate a backup against the
manifest. If no checksums are present, it can still check that the
right files exist and that they have the expected sizes. If checksums
are present, it can also verify that each file has the expected
checksum. Only plain format backups can be validated directly, but tar
format backups can be validated after extracting them.
Robert Haas, with help, ideas, review, and testing from David Steele,
Stephen Frost, Andrew Dunstan, Rushabh Lathia, Suraj Kharage, Tushar
Ahuja, Rajkumar Raghuwanshi, Mark Dilger, Davinder Singh, Jeevan
Chalke, Amit Kapila, and Andres Freund.
Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
---
doc/src/sgml/protocol.sgml | 37 +-
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_basebackup.sgml | 64 ++
doc/src/sgml/ref/pg_validatebackup.sgml | 240 ++++++
doc/src/sgml/reference.sgml | 1 +
src/backend/access/transam/xlog.c | 3 +-
src/backend/replication/basebackup.c | 430 +++++++++-
src/backend/replication/repl_gram.y | 13 +
src/backend/replication/repl_scanner.l | 2 +
src/backend/replication/walsender.c | 30 +
src/bin/Makefile | 1 +
src/bin/pg_basebackup/pg_basebackup.c | 185 ++++-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 8 +-
src/bin/pg_validatebackup/.gitignore | 2 +
src/bin/pg_validatebackup/Makefile | 39 +
src/bin/pg_validatebackup/parse_manifest.c | 576 ++++++++++++++
src/bin/pg_validatebackup/parse_manifest.h | 40 +
src/bin/pg_validatebackup/pg_validatebackup.c | 734 ++++++++++++++++++
src/bin/pg_validatebackup/t/001_basic.pl | 30 +
src/bin/pg_validatebackup/t/002_algorithm.pl | 58 ++
src/bin/pg_validatebackup/t/003_corruption.pl | 251 ++++++
src/bin/pg_validatebackup/t/004_options.pl | 89 +++
.../pg_validatebackup/t/005_bad_manifest.pl | 158 ++++
src/bin/pg_validatebackup/t/006_encoding.pl | 27 +
src/include/replication/basebackup.h | 7 +-
src/include/replication/walsender.h | 1 +
26 files changed, 2995 insertions(+), 32 deletions(-)
create mode 100644 doc/src/sgml/ref/pg_validatebackup.sgml
create mode 100644 src/bin/pg_validatebackup/.gitignore
create mode 100644 src/bin/pg_validatebackup/Makefile
create mode 100644 src/bin/pg_validatebackup/parse_manifest.c
create mode 100644 src/bin/pg_validatebackup/parse_manifest.h
create mode 100644 src/bin/pg_validatebackup/pg_validatebackup.c
create mode 100644 src/bin/pg_validatebackup/t/001_basic.pl
create mode 100644 src/bin/pg_validatebackup/t/002_algorithm.pl
create mode 100644 src/bin/pg_validatebackup/t/003_corruption.pl
create mode 100644 src/bin/pg_validatebackup/t/004_options.pl
create mode 100644 src/bin/pg_validatebackup/t/005_bad_manifest.pl
create mode 100644 src/bin/pg_validatebackup/t/006_encoding.pl
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index f139ba0231..536de9a698 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2466,7 +2466,7 @@ The commands accepted in replication mode are:
</varlistentry>
<varlistentry id="protocol-replication-base-backup" xreflabel="BASE_BACKUP">
- <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ]
+ <term><literal>BASE_BACKUP</literal> [ <literal>LABEL</literal> <replaceable>'label'</replaceable> ] [ <literal>PROGRESS</literal> ] [ <literal>FAST</literal> ] [ <literal>WAL</literal> ] [ <literal>NOWAIT</literal> ] [ <literal>MAX_RATE</literal> <replaceable>rate</replaceable> ] [ <literal>TABLESPACE_MAP</literal> ] [ <literal>NOVERIFY_CHECKSUMS</literal> ] [ <literal>MANIFEST</literal> <replaceable>manifest_option</replaceable> ] [ <literal>MANIFEST_CHECKSUMS</literal> <replaceable>checksum_algorithm</replaceable> ]
<indexterm><primary>BASE_BACKUP</primary></indexterm>
</term>
<listitem>
@@ -2576,6 +2576,41 @@ The commands accepted in replication mode are:
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST</literal></term>
+ <listitem>
+ <para>
+ When this option is specified with a value of <literal>yes</literal>
+ or <literal>force-escape</literal>, a backup manifest is created
+ and sent along with the backup. The manifest is a list of every
+ file present in the backup with the exception of any WAL files that
+ may be included. It also stores the size, last modification time, and
+ an optional checksum for each file.
+ A value of <literal>force-escape</literal> forces all filenames
+ to be hex-encoded; otherwise, this type of encoding is performed only
+ for files whose names are non-UTF8 octet sequences.
+ <literal>force-escape</literal> is intended primarily for testing
+ purposes, to be sure that clients which read the backup manifest
+ can handle this case. For compatibility with previous releases,
+ the default is <literal>MANIFEST 'no'</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>MANIFEST_CHECKSUMS</literal></term>
+ <listitem>
+ <para>
+ Specifies the algorithm that should be applied to each file included
+ in the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
<para>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..ab71176cdf 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -211,6 +211,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
<!ENTITY pgRestore SYSTEM "pg_restore.sgml">
<!ENTITY pgRewind SYSTEM "pg_rewind.sgml">
+<!ENTITY pgValidateBackup SYSTEM "pg_validatebackup.sgml">
<!ENTITY pgtestfsync SYSTEM "pgtestfsync.sgml">
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index c8e040bacf..f5dd4799bc 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -561,6 +561,70 @@ PostgreSQL documentation
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>--no-manifest</option></term>
+ <listitem>
+ <para>
+ Disables generation of a backup manifest. If this option is not
+ specified, the server will generate and send a backup manifest
+ which can be verified using <xref linkend="app-pgvalidatebackup" />.
+ The manifest is a list of every file present in the backup with the
+ exception of any WAL files that may be included. It also stores the
+ size, last modification time, and an optional checksum for each file.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-force-encode</option></term>
+ <listitem>
+ <para>
+ Forces all filenames in the backup manifest to be hex-encoded.
+ If this option is not specified, only non-UTF8 filenames are
+ hex-encoded. This option is mostly intended to test that tools which
+ read a backup manifest file properly handle this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--manifest-checksums=<replaceable class="parameter">algorithm</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the checksum algorithm that should be applied to each file
+ included in the backup manifest. Currently, the available
+ algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
+ <literal>SHA224</literal>, <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and <literal>SHA512</literal>.
+ The default is <literal>CRC32C</literal>.
+ </para>
+ <para>
+ If <literal>NONE</literal> is selected, the backup manifest will
+ not contain any checksums. Otherwise, it will contain a checksum
+ of each file in the backup using the specified algorithm. In addition,
+ the manifest will always contain a <literal>SHA256</literal>
+ checksum of its own contents. The <literal>SHA</literal> algorithms
+ are significantly more CPU-intensive than <literal>CRC32C</literal>,
+ so selecting one of them may increase the time required to complete
+ the backup.
+ </para>
+ <para>
+ Using a SHA hash function provides a cryptographically secure digest
+ of each file for users who wish to verify that the backup has not been
+ tampered with, while the CRC32C algorithm provides a checksum which is
+ much faster to calculate and good at catching errors due to accidental
+ changes but is not resistent to targeted modifications. Note that, to
+ be useful against an adversary who has access to the backup, the backup
+ manifest would need to be stored securely elsewhere or otherwise
+ verified not to have been modified since the backup was taken.
+ </para>
+ <para>
+ <xref linkend="app-pgvalidatebackup" /> can be used to check the
+ integrity of a backup against the backup manifest.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
new file mode 100644
index 0000000000..32547e614a
--- /dev/null
+++ b/doc/src/sgml/ref/pg_validatebackup.sgml
@@ -0,0 +1,240 @@
+<!--
+doc/src/sgml/ref/pg_validatebackup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgvalidatebackup">
+ <indexterm zone="app-pgvalidatebackup">
+ <primary>pg_validatebackup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>pg_validatebackup</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_validatebackup</refname>
+ <refpurpose>verify the integrity of a base backup of a
+ <productname>PostgreSQL</productname> cluster</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_validatebackup</command>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>
+ Description
+ </title>
+ <para>
+ <application>pg_validatebackup</application> is used to check the integrity
+ of a database cluster backup taken using <command>pg_basebackup</command>.
+ The backup must be stored in the "plain" format; a "tar" format backup can
+ be checked after extracting it. Backup manifests are created by the server
+ beginning with <productname>PostgreSQL</productname> version 13, so older
+ backups cannot be validated using this tool.
+ </para>
+
+ <para>
+ <application>pg_validatebackup</application> reads the manifest file of a
+ backup, verifies the manifest against its own internal checksum, and then
+ verifies that the same files are present in the target directory as in the
+ manifest itself. Note that this process checks both for files which are present
+ in the manifest but not in the backup and also for files which are present
+ in the backup but not mentioned in the manifest. After checking that the right
+ files are present, it then verifies that each file has the expected checksum,
+ unless the backup was taken with the checksum algorithm set to
+ <literal>none</literal>, in which case checksum verification is not
+ performed. The presence or absence of directories is not checked, except
+ indirectly: if a directory is missing, any files it should have contained
+ will necessarily also be missing.
+ </para>
+
+ <para>
+ When pg_basebackup compares the files and directories in the manifest
+ to those which are present on disk, it will ignore the presence of, or
+ changes to, certain files:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>backup_manifest</literal> will not be present in the
+ manifest, and is therefore ignored. Note that the manifest
+ is still verified internally, but no error will be issued about the
+ presence of a backup_manifest file in the backup directory even though
+ it is not listed in the manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>pg_wal</literal> is ignored because WAL files are sent
+ separately from the backup, and are therefore not described by the
+ backup manifest.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>postgesql.auto.conf</literal>,
+ <literal>standby.signal</literal>,
+ and <literal>recovery.signal</literal> are ignored because they may
+ sometimes be created or modified by the backup client itself.
+ (For example, <literal>pg_basebackup -R</literal> will modify
+ <literal>postgresql.auto.conf</literal> and create
+ <literal>standby.signal</literal>.)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ The following command-line options control the behavior.
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-e</option></term>
+ <term><option>--exit-on-error</option></term>
+ <listitem>
+ <para>
+ Exit as soon as a problem with the backup is detected. If this option
+ is not specified, <literal>pg_basebackup</literal> will continue
+ checking the backup even after a problem has been detected, and will
+ report all problems detected as errors.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-i <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--ignore=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Ignore the specified file or directory, which should be expressed
+ as a relative pathname. If the backup contains extra files, is
+ missing files, or has files that have been modified as compared with
+ what is described in the manifest, this option can be used to suppress
+ the errors that would otherwise occur. If a directory is specified,
+ this option affects the entire subtree rooted at that location.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-m <replaceable class="parameter">path</replaceable></option></term>
+ <term><option>--manifest-path=<replaceable class="parameter">path</replaceable></option></term>
+ <listitem>
+ <para>
+ Use the manifest file at the specified path, rather than one located
+ in the root of the backup directory.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-q</option></term>
+ <term><option>--quiet</option></term>
+ <listitem>
+ <para>
+ Don't print anything when a backup is successfully validated.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-s</option></term>
+ <term><option>--skip-checksums</option></term>
+ <listitem>
+ <para>
+ Do not validate checksums. The presence or absence of files and the
+ sizes of those files will still be checked. This is much faster,
+ because the files themselves do not need to be read.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other options are also available:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_validatebackup</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_validatebackup</application> command
+ line arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal> and
+ validate the integrity of the backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ <para>
+ To create a base backup of the server at <literal>mydbserver</literal>, move
+ the manifest somewhere outside the backup directory, and validate the
+ backup:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234</userinput>
+<prompt>$</prompt> <userinput>mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234</userinput>
+</screen>
+ </para>
+
+ <para>
+ To validate a backup while ignoring a file that was added manually to the
+ backup directory, and also skipping checksum verification:
+<screen>
+<prompt>$</prompt> <userinput>pg_basebackup -h mydbserver -D /usr/local/pgsql/data</userinput>
+<prompt>$</prompt> <userinput>edit /usr/local/pgsql/data/note.to.self</userinput>
+<prompt>$</prompt> <userinput>pg_validatebackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data</userinput>
+</screen>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="app-pgbasebackup"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..d25a77b13c 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -255,6 +255,7 @@
&pgReceivewal;
&pgRecvlogical;
&pgRestore;
+ &pgValidateBackup;
&psqlRef;
&reindexdb;
&vacuumdb;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 1951103b26..5ba3debf82 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -10632,7 +10632,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
ti->oid = pstrdup(de->d_name);
ti->path = pstrdup(buflinkpath.data);
ti->rpath = relpath ? pstrdup(relpath) : NULL;
- ti->size = infotbssize ? sendTablespace(fullpath, true) : -1;
+ ti->size = infotbssize ?
+ sendTablespace(fullpath, ti->oid, true, NULL) : -1;
if (tablespaces)
*tablespaces = lappend(*tablespaces, ti);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index a2e28b064c..deaa4f1c34 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -18,6 +18,7 @@
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+#include "common/checksum_helper.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
@@ -32,6 +33,7 @@
#include "replication/basebackup.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
+#include "storage/buffile.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/dsm_impl.h"
@@ -39,10 +41,19 @@
#include "storage/ipc.h"
#include "storage/reinit.h"
#include "utils/builtins.h"
+#include "utils/json.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
#include "utils/timestamp.h"
+typedef enum manifest_option
+{
+ MANIFEST_OPTION_YES,
+ MANIFEST_OPTION_NO,
+ MANIFEST_OPTION_FORCE_ENCODE
+} manifest_option;
+
typedef struct
{
const char *label;
@@ -52,20 +63,43 @@ typedef struct
bool includewal;
uint32 maxrate;
bool sendtblspcmapfile;
+ manifest_option manifest;
+ pg_checksum_type manifest_checksum_type;
} basebackup_options;
+struct manifest_info
+{
+ BufFile *buffile;
+ pg_checksum_type checksum_type;
+ pg_sha256_ctx manifest_ctx;
+ uint64 manifest_size;
+ bool force_encode;
+ bool first_file;
+ bool still_checksumming;
+};
+
static int64 sendDir(const char *path, int basepathlen, bool sizeonly,
- List *tablespaces, bool sendtblspclinks);
+ List *tablespaces, bool sendtblspclinks,
+ manifest_info *manifest, const char *spcoid);
static bool sendFile(const char *readfilename, const char *tarfilename,
- struct stat *statbuf, bool missing_ok, Oid dboid);
-static void sendFileWithContent(const char *filename, const char *content);
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid);
+static void sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest);
static int64 _tarWriteHeader(const char *filename, const char *linktarget,
struct stat *statbuf, bool sizeonly);
static int64 _tarWriteDir(const char *pathbuf, int basepathlen, struct stat *statbuf,
bool sizeonly);
static void send_int8_string(StringInfoData *buf, int64 intval);
static void SendBackupHeader(List *tablespaces);
+static void InitializeManifest(manifest_info *manifest,
+ basebackup_options *opt);
+static void AppendStringToManifest(manifest_info *manifest, char *s);
+static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx);
+static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
static void SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli);
@@ -102,6 +136,16 @@ do { \
(errmsg("could not read from file \"%s\"", filename))); \
} while (0)
+/*
+ * Convenience macro for appending data to the backup manifest.
+ */
+#define AppendToManifest(manifest, ...) \
+ { \
+ char *_manifest_s = psprintf(__VA_ARGS__); \
+ AppendStringToManifest(manifest, _manifest_s); \
+ pfree(_manifest_s); \
+ }
+
/* The actual number of bytes, transfer of which may cause sleep. */
static uint64 throttling_sample;
@@ -254,6 +298,7 @@ perform_base_backup(basebackup_options *opt)
TimeLineID endtli;
StringInfo labelfile;
StringInfo tblspc_map_file = NULL;
+ manifest_info manifest;
int datadirpathlen;
List *tablespaces = NIL;
@@ -273,12 +318,17 @@ perform_base_backup(basebackup_options *opt)
backup_total);
}
+ /* we're going to use a BufFile, so we need a ResourceOwner */
+ Assert(CurrentResourceOwner == NULL);
+ CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
+
datadirpathlen = strlen(DataDir);
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
tblspc_map_file = makeStringInfo();
+ InitializeManifest(&manifest, opt);
total_checksum_failures = 0;
@@ -316,7 +366,10 @@ perform_base_backup(basebackup_options *opt)
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
- ti->size = opt->progress ? sendDir(".", 1, true, tablespaces, true) : -1;
+ if (opt->progress)
+ ti->size = sendDir(".", 1, true, tablespaces, true, NULL, NULL);
+ else
+ ti->size = -1;
tablespaces = lappend(tablespaces, ti);
/*
@@ -395,7 +448,8 @@ perform_base_backup(basebackup_options *opt)
struct stat statbuf;
/* In the main tar, include the backup_label first... */
- sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data);
+ sendFileWithContent(BACKUP_LABEL_FILE, labelfile->data,
+ &manifest);
/*
* Send tablespace_map file if required and then the bulk of
@@ -403,11 +457,14 @@ perform_base_backup(basebackup_options *opt)
*/
if (tblspc_map_file && opt->sendtblspcmapfile)
{
- sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data);
- sendDir(".", 1, false, tablespaces, false);
+ sendFileWithContent(TABLESPACE_MAP, tblspc_map_file->data,
+ &manifest);
+ sendDir(".", 1, false, tablespaces, false,
+ &manifest, NULL);
}
else
- sendDir(".", 1, false, tablespaces, true);
+ sendDir(".", 1, false, tablespaces, true,
+ &manifest, NULL);
/* ... and pg_control after everything else. */
if (lstat(XLOG_CONTROL_FILE, &statbuf) != 0)
@@ -415,10 +472,11 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
XLOG_CONTROL_FILE)));
- sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf, false, InvalidOid);
+ sendFile(XLOG_CONTROL_FILE, XLOG_CONTROL_FILE, &statbuf,
+ false, InvalidOid, &manifest, NULL);
}
else
- sendTablespace(ti->path, false);
+ sendTablespace(ti->path, ti->oid, false, &manifest);
/*
* If we're including WAL, and this is the main data directory we
@@ -647,7 +705,7 @@ perform_base_backup(basebackup_options *opt)
* complete segment.
*/
StatusFilePath(pathbuf, walFileName, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/*
@@ -670,16 +728,20 @@ perform_base_backup(basebackup_options *opt)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", pathbuf)));
- sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid);
+ sendFile(pathbuf, pathbuf, &statbuf, false, InvalidOid,
+ &manifest, NULL);
/* unconditionally mark file as archived */
StatusFilePath(pathbuf, fname, ".done");
- sendFileWithContent(pathbuf, "");
+ sendFileWithContent(pathbuf, "", &manifest);
}
/* Send CopyDone message for the last tar file */
pq_putemptymessage('c');
}
+
+ SendBackupManifest(&manifest);
+
SendXlogRecPtrResult(endptr, endtli);
if (total_checksum_failures)
@@ -693,6 +755,9 @@ perform_base_backup(basebackup_options *opt)
errmsg("checksum verification failure during base backup")));
}
+ /* clean up the resource owner we created */
+ WalSndResourceCleanup(true);
+
pgstat_progress_end_command();
}
@@ -724,8 +789,13 @@ parse_basebackup_options(List *options, basebackup_options *opt)
bool o_maxrate = false;
bool o_tablespace_map = false;
bool o_noverify_checksums = false;
+ bool o_manifest = false;
+ bool o_manifest_checksums = false;
MemSet(opt, 0, sizeof(*opt));
+ opt->manifest = MANIFEST_OPTION_NO;
+ opt->manifest_checksum_type = CHECKSUM_TYPE_CRC32C;
+
foreach(lopt, options)
{
DefElem *defel = (DefElem *) lfirst(lopt);
@@ -812,12 +882,61 @@ parse_basebackup_options(List *options, basebackup_options *opt)
noverify_checksums = true;
o_noverify_checksums = true;
}
+ else if (strcmp(defel->defname, "manifest") == 0)
+ {
+ char *optval = strVal(defel->arg);
+ bool manifest_bool;
+
+ if (o_manifest)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (parse_bool(optval, &manifest_bool))
+ {
+ if (manifest_bool)
+ opt->manifest = MANIFEST_OPTION_YES;
+ else
+ opt->manifest = MANIFEST_OPTION_NO;
+ }
+ else if (pg_strcasecmp(optval, "force-encode") == 0)
+ opt->manifest = MANIFEST_OPTION_FORCE_ENCODE;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized manifest option: \"%s\"",
+ optval)));
+ o_manifest = true;
+ }
+ else if (strcmp(defel->defname, "manifest_checksums") == 0)
+ {
+ char *optval = strVal(defel->arg);
+
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("duplicate option \"%s\"", defel->defname)));
+ if (!pg_checksum_parse_type(optval,
+ &opt->manifest_checksum_type))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized checksum algorithm: \"%s\"",
+ optval)));
+ o_manifest_checksums = true;
+ }
else
elog(ERROR, "option \"%s\" not recognized",
defel->defname);
}
if (opt->label == NULL)
opt->label = "base backup";
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ {
+ if (o_manifest_checksums)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("manifest checksums require a backup manifest")));
+ opt->manifest_checksum_type = CHECKSUM_TYPE_NONE;
+ }
}
@@ -933,6 +1052,249 @@ SendBackupHeader(List *tablespaces)
pq_puttextmessage('C', "SELECT");
}
+/*
+ * Initialize state so that we can construct a backup manifest.
+ *
+ * NB: Although the checksum type for the data files is configurable, the
+ * checksum for the manifest itself always uses SHA-256. See comments in
+ * SendBackupManifest.
+ */
+static void
+InitializeManifest(manifest_info *manifest, basebackup_options *opt)
+{
+ if (opt->manifest == MANIFEST_OPTION_NO)
+ manifest->buffile = NULL;
+ else
+ manifest->buffile = BufFileCreateTemp(false);
+ manifest->checksum_type = opt->manifest_checksum_type;
+ pg_sha256_init(&manifest->manifest_ctx);
+ manifest->manifest_size = UINT64CONST(0);
+ manifest->force_encode = (opt->manifest == MANIFEST_OPTION_FORCE_ENCODE);
+ manifest->first_file = true;
+ manifest->still_checksumming = true;
+
+ if (opt->manifest != MANIFEST_OPTION_NO)
+ AppendToManifest(manifest,
+ "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n"
+ "\"Files\": [");
+}
+
+/*
+ * Append a cstring to the manifest.
+ */
+static void
+AppendStringToManifest(manifest_info *manifest, char *s)
+{
+ int len = strlen(s);
+ size_t written;
+
+ Assert(manifest != NULL);
+ if (manifest->still_checksumming)
+ pg_sha256_update(&manifest->manifest_ctx, (uint8 *) s, len);
+ written = BufFileWrite(manifest->buffile, s, len);
+ if (written != len)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write to temporary file: %m")));
+ manifest->manifest_size += len;
+}
+
+/*
+ * Add an entry to the backup manifest for a file.
+ */
+static void
+AddFileToManifest(manifest_info *manifest, const char *spcoid,
+ const char *pathname, size_t size, time_t mtime,
+ pg_checksum_context *checksum_ctx)
+{
+ char pathbuf[MAXPGPATH];
+ int pathlen;
+ StringInfoData buf;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /*
+ * If this file is part of a tablespace, the pathname passed to this
+ * function will be relative to the tar file that contains it. We want the
+ * pathname relative to the data directory (ignoring the intermediate
+ * symlink traversal).
+ */
+ if (spcoid != NULL)
+ {
+ snprintf(pathbuf, sizeof(pathbuf), "pg_tblspc/%s/%s", spcoid,
+ pathname);
+ pathname = pathbuf;
+ }
+
+ /*
+ * Each file's entry need to be separated from any entry that follows
+ * by a comma, but there's no comma before the first one or after the
+ * last one. To make that work, adding a file to the manifest starts
+ * by terminating the most recently added line, with a comma if
+ * appropriate, but does not terminate the line inserted for this file.
+ */
+ initStringInfo(&buf);
+ if (manifest->first_file)
+ {
+ appendStringInfoString(&buf, "\n");
+ manifest->first_file = false;
+ }
+ else
+ appendStringInfoString(&buf, ",\n");
+
+ /*
+ * Write the relative pathname to this file out to the manifest. The
+ * manifest is always stored in UTF-8, so we have to encode paths that
+ * are not valid in that encoding.
+ */
+ pathlen = strlen(pathname);
+ if (!manifest->force_encode &&
+ pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
+ {
+ appendStringInfoString(&buf, "{ \"Path\": ");
+ escape_json(&buf, pathname);
+ appendStringInfoString(&buf, ", ");
+ }
+ else
+ {
+ appendStringInfoString(&buf, "{ \"Encoded-Path\": \"");
+ enlargeStringInfo(&buf, 2 * pathlen);
+ buf.len += hex_encode((char *) pathname, pathlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\", ");
+ }
+
+ appendStringInfo(&buf, "\"Size\": %zu, ", size);
+
+ /*
+ * Convert last modification time to a string and append it to the
+ * manifest. Since it's not clear what time zone to use and since time
+ * zone definitions can change, possibly causing confusion, use GMT always.
+ */
+ appendStringInfoString(&buf, "\"Last-Modified\": \"");
+ enlargeStringInfo(&buf, 128);
+ buf.len += pg_strftime(&buf.data[buf.len], 128, "%Y-%m-%d %H:%M:%S %Z",
+ pg_gmtime(&mtime));
+ appendStringInfoString(&buf, "\"");
+
+ /* Add checksum information. */
+ if (checksum_ctx->type != CHECKSUM_TYPE_NONE)
+ {
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ checksumlen = pg_checksum_final(checksum_ctx, checksumbuf);
+
+ appendStringInfo(&buf,
+ ", \"Checksum-Algorithm\": \"%s\", \"Checksum\": \"",
+ pg_checksum_type_name(checksum_ctx->type));
+ enlargeStringInfo(&buf, 2 * checksumlen);
+ buf.len += hex_encode((char *) checksumbuf, checksumlen,
+ &buf.data[buf.len]);
+ appendStringInfoString(&buf, "\"");
+ }
+
+ /* Close out the object. */
+ appendStringInfoString(&buf, " }");
+
+ /* OK, add it to the manifest. */
+ AppendStringToManifest(manifest, buf.data);
+
+ /* Avoid leaking memory. */
+ pfree(buf.data);
+}
+
+/*
+ * Finalize the backup manifest, and send it to the client.
+ */
+static void
+SendBackupManifest(manifest_info *manifest)
+{
+ StringInfoData protobuf;
+ uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
+ char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
+ size_t manifest_bytes_done = 0;
+
+ /*
+ * If there is no buffile, then the user doesn't want a manifest, so
+ * don't waste any time generating one.
+ */
+ if (manifest->buffile == NULL)
+ return;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "],\n");
+
+ /*
+ * Append manifest checksum, so that the problems with the manifest itself
+ * can be detected.
+ *
+ * We always use SHA-256 for this, regardless of what algorithm is chosen
+ * for checksumming the files. If we ever want to make the checksum
+ * algorithm used for the manifest file variable, the client will need a
+ * way to figure out which algorithm to use as close to the beginning of
+ * the manifest file as possible, to avoid having to read the whole thing
+ * twice.
+ */
+ manifest->still_checksumming = false;
+ pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
+ AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
+ hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
+ checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
+ AppendStringToManifest(manifest, checksumstringbuf);
+ AppendStringToManifest(manifest, "\"}\n");
+
+ /*
+ * We've written all the data to the manifest file. Rewind the file so
+ * that we can read it all back.
+ */
+ if (BufFileSeek(manifest->buffile, 0, 0L, SEEK_SET))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not rewind temporary file: %m")));
+
+ /* Send CopyOutResponse message */
+ pq_beginmessage(&protobuf, 'H');
+ pq_sendbyte(&protobuf, 0); /* overall format */
+ pq_sendint16(&protobuf, 0); /* natts */
+ pq_endmessage(&protobuf);
+
+ /*
+ * Send CopyData messages.
+ *
+ * We choose to read back the data from the temporary file in chunks of
+ * size BLCKSZ; this isn't necessary, but buffile.c uses that as the I/O
+ * size, so it seems to make sense to match that value here.
+ */
+ while (manifest_bytes_done < manifest->manifest_size)
+ {
+ char manifestbuf[BLCKSZ];
+ size_t bytes_to_read;
+ size_t rc;
+
+ bytes_to_read = Min(sizeof(manifestbuf),
+ manifest->manifest_size - manifest_bytes_done);
+ rc = BufFileRead(manifest->buffile, manifestbuf, bytes_to_read);
+ if (rc != bytes_to_read)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from temporary file: %m")));
+ pq_putmessage('d', manifestbuf, bytes_to_read);
+ manifest_bytes_done += bytes_to_read;
+ }
+
+ /* No more data, so send CopyDone message */
+ pq_putemptymessage('c');
+
+ /* Release resources */
+ BufFileClose(manifest->buffile);
+}
+
/*
* Send a single resultset containing just a single
* XLogRecPtr record (in text format)
@@ -993,11 +1355,15 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
* Inject a file with given name and content in the output tar stream.
*/
static void
-sendFileWithContent(const char *filename, const char *content)
+sendFileWithContent(const char *filename, const char *content,
+ manifest_info *manifest)
{
struct stat statbuf;
int pad,
len;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
len = strlen(content);
@@ -1032,6 +1398,10 @@ sendFileWithContent(const char *filename, const char *content)
pq_putmessage('d', buf, pad);
update_basebackup_progress(pad);
}
+
+ pg_checksum_update(&checksum_ctx, (uint8 *) content, len);
+ AddFileToManifest(manifest, NULL, filename, len, statbuf.st_mtime,
+ &checksum_ctx);
}
/*
@@ -1042,7 +1412,8 @@ sendFileWithContent(const char *filename, const char *content)
* Only used to send auxiliary tablespaces, not PGDATA.
*/
int64
-sendTablespace(char *path, bool sizeonly)
+sendTablespace(char *path, char *spcoid, bool sizeonly,
+ manifest_info *manifest)
{
int64 size;
char pathbuf[MAXPGPATH];
@@ -1075,7 +1446,8 @@ sendTablespace(char *path, bool sizeonly)
sizeonly);
/* Send all the files in the tablespace version directory */
- size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true);
+ size += sendDir(pathbuf, strlen(path), sizeonly, NIL, true, manifest,
+ spcoid);
return size;
}
@@ -1094,7 +1466,7 @@ sendTablespace(char *path, bool sizeonly)
*/
static int64
sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
- bool sendtblspclinks)
+ bool sendtblspclinks, manifest_info *manifest, const char *spcoid)
{
DIR *dir;
struct dirent *de;
@@ -1374,7 +1746,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
skip_this_dir = true;
if (!skip_this_dir)
- size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces, sendtblspclinks);
+ size += sendDir(pathbuf, basepathlen, sizeonly, tablespaces,
+ sendtblspclinks, manifest, spcoid);
}
else if (S_ISREG(statbuf.st_mode))
{
@@ -1382,7 +1755,8 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
if (!sizeonly)
sent = sendFile(pathbuf, pathbuf + basepathlen + 1, &statbuf,
- true, isDbDir ? atooid(lastDir + 1) : InvalidOid);
+ true, isDbDir ? atooid(lastDir + 1) : InvalidOid,
+ manifest, spcoid);
if (sent || sizeonly)
{
@@ -1452,8 +1826,9 @@ is_checksummed_file(const char *fullpath, const char *filename)
* and the file did not exist.
*/
static bool
-sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf,
- bool missing_ok, Oid dboid)
+sendFile(const char *readfilename, const char *tarfilename,
+ struct stat *statbuf, bool missing_ok, Oid dboid,
+ manifest_info *manifest, const char *spcoid)
{
FILE *fp;
BlockNumber blkno = 0;
@@ -1470,6 +1845,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
int segmentno = 0;
char *segmentpath;
bool verify_checksum = false;
+ pg_checksum_context checksum_ctx;
+
+ pg_checksum_init(&checksum_ctx, manifest->checksum_type);
fp = AllocateFile(readfilename, "rb");
if (fp == NULL)
@@ -1640,6 +2018,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
(errmsg("base backup could not send data, aborting backup")));
update_basebackup_progress(cnt);
+ /* Also feed it to the checksum machinery. */
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
+
len += cnt;
throttle(cnt);
@@ -1664,6 +2045,7 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
{
cnt = Min(sizeof(buf), statbuf->st_size - len);
pq_putmessage('d', buf, cnt);
+ pg_checksum_update(&checksum_ctx, (uint8 *) buf, cnt);
update_basebackup_progress(cnt);
len += cnt;
throttle(cnt);
@@ -1672,7 +2054,8 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
/*
* Pad to 512 byte boundary, per tar format requirements. (This small
- * piece of data is probably not worth throttling.)
+ * piece of data is probably not worth throttling, and is not checksummed
+ * because it's not actually part of the file.)
*/
pad = ((len + 511) & ~511) - len;
if (pad > 0)
@@ -1697,6 +2080,9 @@ sendFile(const char *readfilename, const char *tarfilename, struct stat *statbuf
total_checksum_failures += checksum_failures;
+ AddFileToManifest(manifest, spcoid, tarfilename, statbuf->st_size,
+ statbuf->st_mtime, &checksum_ctx);
+
return true;
}
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 14fcd53221..f93a0de218 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -87,6 +87,8 @@ static SQLCmd *make_sqlcmd(void);
%token K_EXPORT_SNAPSHOT
%token K_NOEXPORT_SNAPSHOT
%token K_USE_SNAPSHOT
+%token K_MANIFEST
+%token K_MANIFEST_CHECKSUMS
%type <node> command
%type <node> base_backup start_replication start_logical_replication
@@ -156,6 +158,7 @@ var_name: IDENT { $$ = $1; }
/*
* BASE_BACKUP [LABEL '<label>'] [PROGRESS] [FAST] [WAL] [NOWAIT]
* [MAX_RATE %d] [TABLESPACE_MAP] [NOVERIFY_CHECKSUMS]
+ * [MANIFEST %s] [MANIFEST_CHECKSUMS %s]
*/
base_backup:
K_BASE_BACKUP base_backup_opt_list
@@ -214,6 +217,16 @@ base_backup_opt:
$$ = makeDefElem("noverify_checksums",
(Node *)makeInteger(true), -1);
}
+ | K_MANIFEST SCONST
+ {
+ $$ = makeDefElem("manifest",
+ (Node *)makeString($2), -1);
+ }
+ | K_MANIFEST_CHECKSUMS SCONST
+ {
+ $$ = makeDefElem("manifest_checksums",
+ (Node *)makeString($2), -1);
+ }
;
create_replication_slot:
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 14c9a1e798..452ad9fc27 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -107,6 +107,8 @@ EXPORT_SNAPSHOT { return K_EXPORT_SNAPSHOT; }
NOEXPORT_SNAPSHOT { return K_NOEXPORT_SNAPSHOT; }
USE_SNAPSHOT { return K_USE_SNAPSHOT; }
WAIT { return K_WAIT; }
+MANIFEST { return K_MANIFEST; }
+MANIFEST_CHECKSUMS { return K_MANIFEST_CHECKSUMS; }
"," { return ','; }
";" { return ';'; }
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 76ec3c7dd0..3b117d8367 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -315,6 +315,8 @@ WalSndErrorCleanup(void)
replication_active = false;
+ WalSndResourceCleanup(false);
+
if (got_STOPPING || got_SIGUSR2)
proc_exit(0);
@@ -322,6 +324,34 @@ WalSndErrorCleanup(void)
WalSndSetState(WALSNDSTATE_STARTUP);
}
+/*
+ * Clean up any ResourceOwner we created.
+ */
+void
+WalSndResourceCleanup(bool isCommit)
+{
+ ResourceOwner resowner;
+
+ if (CurrentResourceOwner == NULL)
+ return;
+
+ /*
+ * Deleting CurrentResourceOwner is not allowed, so we must save a
+ * pointer in a local variable and clear it first.
+ */
+ resowner = CurrentResourceOwner;
+ CurrentResourceOwner = NULL;
+
+ /* Now we can release resources and delete it. */
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_BEFORE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_LOCKS, isCommit, true);
+ ResourceOwnerRelease(resowner,
+ RESOURCE_RELEASE_AFTER_LOCKS, isCommit, true);
+ ResourceOwnerDelete(resowner);
+}
+
/*
* Handle a client's connection abort in an orderly manner.
*/
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f4120a34f..77bceea4fe 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pg_test_fsync \
pg_test_timing \
pg_upgrade \
+ pg_validatebackup \
pg_waldump \
pgbench \
psql \
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index c5d95958b2..058125be67 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -88,6 +88,12 @@ typedef struct UnpackTarState
FILE *file;
} UnpackTarState;
+typedef struct WriteManifestState
+{
+ char filename[MAXPGPATH];
+ FILE *file;
+} WriteManifestState;
+
typedef void (*WriteDataCallback) (size_t nbytes, char *buf,
void *callback_data);
@@ -136,6 +142,9 @@ static bool temp_replication_slot = true;
static bool create_slot = false;
static bool no_slot = false;
static bool verify_checksums = true;
+static bool manifest = true;
+static bool manifest_force_encode = false;
+static char *manifest_checksums = NULL;
static bool success = false;
static bool made_new_pgdata = false;
@@ -181,6 +190,12 @@ static void ReceiveTarCopyChunk(size_t r, char *copybuf, void *callback_data);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf,
void *callback_data);
+static void ReceiveBackupManifest(PGconn *conn);
+static void ReceiveBackupManifestChunk(size_t r, char *copybuf,
+ void *callback_data);
+static void ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf);
+static void ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -388,6 +403,11 @@ usage(void)
printf(_(" --no-verify-checksums\n"
" do not verify checksums\n"));
printf(_(" --no-estimate-size do not estimate backup size in server side\n"));
+ printf(_(" --no-manifest suppress generation of backup manifest\n"));
+ printf(_(" --manifest-force-encode\n"
+ " hex encode all filenames in manifest\n"));
+ printf(_(" --manifest-checksums=SHA{224,256,384,512}|CRC32C|NONE\n"
+ " use algorithm for manifest checksums\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=CONNSTR connection string\n"));
@@ -1186,6 +1206,31 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
}
+ /*
+ * Normally, we emit the backup manifest as a separate file, but when
+ * we're writing a tarfile to stdout, we don't have that option, so
+ * include it in the one tarfile we've got.
+ */
+ if (strcmp(basedir, "-") == 0)
+ {
+ char header[512];
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+ ReceiveBackupManifestInMemory(conn, &buf);
+ if (PQExpBufferDataBroken(buf))
+ {
+ pg_log_error("out of memory");
+ exit(1);
+ }
+ tarCreateHeader(header, "backup_manifest", NULL, buf.len,
+ pg_file_create_mode, 04000, 02000,
+ time(NULL));
+ writeTarData(&state, header, sizeof(header));
+ writeTarData(&state, buf.data, buf.len);
+ termPQExpBuffer(&buf);
+ }
+
/* 2 * 512 bytes empty data at end of file */
writeTarData(&state, zerobuf, sizeof(zerobuf));
@@ -1657,6 +1702,64 @@ ReceiveTarAndUnpackCopyChunk(size_t r, char *copybuf, void *callback_data)
} /* continuing data in existing file */
}
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifest(PGconn *conn)
+{
+ WriteManifestState state;
+
+ snprintf(state.filename, sizeof(state.filename),
+ "%s/backup_manifest", basedir);
+ state.file = fopen(state.filename, "wb");
+ if (state.file == NULL)
+ {
+ pg_log_error("could not create file \"%s\": %m", state.filename);
+ exit(1);
+ }
+
+ ReceiveCopyData(conn, ReceiveBackupManifestChunk, &state);
+
+ fclose(state.file);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestChunk(size_t r, char *copybuf, void *callback_data)
+{
+ WriteManifestState *state = callback_data;
+
+ if (fwrite(copybuf, r, 1, state->file) != 1)
+ {
+ pg_log_error("could not write to file \"%s\": %m", state->filename);
+ exit(1);
+ }
+}
+
+/*
+ * Receive the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemory(PGconn *conn, PQExpBuffer buf)
+{
+ ReceiveCopyData(conn, ReceiveBackupManifestInMemoryChunk, buf);
+}
+
+/*
+ * Receive one chunk of the backup manifest file and write it out to a file.
+ */
+static void
+ReceiveBackupManifestInMemoryChunk(size_t r, char *copybuf,
+ void *callback_data)
+{
+ PQExpBuffer buf = callback_data;
+
+ appendPQExpBuffer(buf, copybuf, r);
+}
+
static void
BaseBackup(void)
{
@@ -1667,6 +1770,8 @@ BaseBackup(void)
char *basebkp;
char escaped_label[MAXPGPATH];
char *maxrate_clause = NULL;
+ char *manifest_clause;
+ char *manifest_checksums_clause = "";
int i;
char xlogstart[64];
char xlogend[64];
@@ -1674,6 +1779,7 @@ BaseBackup(void)
maxServerMajor;
int serverVersion,
serverMajor;
+ int writing_to_stdout;
Assert(conn != NULL);
@@ -1728,6 +1834,33 @@ BaseBackup(void)
if (maxrate > 0)
maxrate_clause = psprintf("MAX_RATE %u", maxrate);
+ if (manifest)
+ {
+ if (serverMajor < 1300)
+ {
+ const char *serverver = PQparameterStatus(conn, "server_version");
+
+ pg_log_error("backup manifests are not supported by server version %s",
+ serverver ? serverver : "'unknown'");
+ exit(1);
+ }
+
+ if (manifest_force_encode)
+ manifest_clause = "MANIFEST 'force-encode'";
+ else
+ manifest_clause = "MANIFEST 'yes'";
+ if (manifest_checksums != NULL)
+ manifest_checksums_clause = psprintf("MANIFEST_CHECKSUMS '%s'",
+ manifest_checksums);
+ }
+ else
+ {
+ if (serverMajor < 1300)
+ manifest_clause = "";
+ else
+ manifest_clause = "MANIFEST 'no'";
+ }
+
if (verbose)
pg_log_info("initiating base backup, waiting for checkpoint to complete");
@@ -1741,7 +1874,7 @@ BaseBackup(void)
}
basebkp =
- psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s",
+ psprintf("BASE_BACKUP LABEL '%s' %s %s %s %s %s %s %s %s %s",
escaped_label,
estimatesize ? "PROGRESS" : "",
includewal == FETCH_WAL ? "WAL" : "",
@@ -1749,7 +1882,9 @@ BaseBackup(void)
includewal == NO_WAL ? "" : "NOWAIT",
maxrate_clause ? maxrate_clause : "",
format == 't' ? "TABLESPACE_MAP" : "",
- verify_checksums ? "" : "NOVERIFY_CHECKSUMS");
+ verify_checksums ? "" : "NOVERIFY_CHECKSUMS",
+ manifest_clause,
+ manifest_checksums_clause);
if (PQsendQuery(conn, basebkp) == 0)
{
@@ -1837,7 +1972,8 @@ BaseBackup(void)
/*
* When writing to stdout, require a single tablespace
*/
- if (format == 't' && strcmp(basedir, "-") == 0 && PQntuples(res) > 1)
+ writing_to_stdout = format == 't' && strcmp(basedir, "-") == 0;
+ if (writing_to_stdout && PQntuples(res) > 1)
{
pg_log_error("can only write single tablespace to stdout, database has %d",
PQntuples(res));
@@ -1866,6 +2002,19 @@ BaseBackup(void)
ReceiveAndUnpackTarFile(conn, res, i);
} /* Loop over all tablespaces */
+ /*
+ * Now receive backup manifest, if appropriate.
+ *
+ * If we're writing a tarfile to stdout, ReceiveTarFile will have already
+ * processed the backup manifest and included it in the output tarfile.
+ * Such a configuration doesn't allow for writing multiple files.
+ *
+ * If we're talking to an older server, it won't send a backup manifest,
+ * so don't try to receive one.
+ */
+ if (!writing_to_stdout && manifest)
+ ReceiveBackupManifest(conn);
+
if (showprogress)
{
progress_report(PQntuples(res), NULL, true);
@@ -2069,6 +2218,9 @@ main(int argc, char **argv)
{"no-slot", no_argument, NULL, 2},
{"no-verify-checksums", no_argument, NULL, 3},
{"no-estimate-size", no_argument, NULL, 4},
+ {"no-manifest", no_argument, NULL, 5},
+ {"manifest-force-encode", no_argument, NULL, 6},
+ {"manifest-checksums", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
int c;
@@ -2096,7 +2248,7 @@ main(int argc, char **argv)
atexit(cleanup_directories_atexit);
- while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvP",
+ while ((c = getopt_long(argc, argv, "CD:F:r:RS:T:X:l:nNzZ:d:c:h:p:U:s:wWkvPm:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2240,6 +2392,15 @@ main(int argc, char **argv)
case 4:
estimatesize = false;
break;
+ case 5:
+ manifest = false;
+ break;
+ case 6:
+ manifest_force_encode = true;
+ break;
+ case 7:
+ manifest_checksums = pg_strdup(optarg);
+ break;
default:
/*
@@ -2370,6 +2531,22 @@ main(int argc, char **argv)
exit(1);
}
+ if (!manifest && manifest_checksums != NULL)
+ {
+ pg_log_error("--no-manifest and --manifest-checksums are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (!manifest && manifest_force_encode)
+ {
+ pg_log_error("--no-manifest and --manifest-force-encode are incompatible options");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
/* connection in replication mode to server */
conn = GetConnection();
if (!conn)
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..63381764e9 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 109;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -104,6 +104,7 @@ foreach my $filename (@tempRelationFiles)
$node->command_ok([ 'pg_basebackup', '-D', "$tempdir/backup", '-X', 'none' ],
'pg_basebackup runs');
ok(-f "$tempdir/backup/PG_VERSION", 'backup was created');
+ok(-f "$tempdir/backup/backup_manifest", 'backup manifest included');
# Permissions on backup should be default
SKIP:
@@ -160,11 +161,12 @@ rmtree("$tempdir/backup");
$node->command_ok(
[
- 'pg_basebackup', '-D', "$tempdir/backup2", '--waldir',
- "$tempdir/xlog2"
+ 'pg_basebackup', '-D', "$tempdir/backup2", '--no-manifest',
+ '--waldir', "$tempdir/xlog2"
],
'separate xlog directory');
ok(-f "$tempdir/backup2/PG_VERSION", 'backup was created');
+ok(! -f "$tempdir/backup2/backup_manifest", 'manifest was suppressed');
ok(-d "$tempdir/xlog2/", 'xlog directory was created');
rmtree("$tempdir/backup2");
rmtree("$tempdir/xlog2");
diff --git a/src/bin/pg_validatebackup/.gitignore b/src/bin/pg_validatebackup/.gitignore
new file mode 100644
index 0000000000..21e0a92429
--- /dev/null
+++ b/src/bin/pg_validatebackup/.gitignore
@@ -0,0 +1,2 @@
+/pg_validatebackup
+/tmp_check/
diff --git a/src/bin/pg_validatebackup/Makefile b/src/bin/pg_validatebackup/Makefile
new file mode 100644
index 0000000000..04ef7d3051
--- /dev/null
+++ b/src/bin/pg_validatebackup/Makefile
@@ -0,0 +1,39 @@
+# src/bin/pg_validatebackup/Makefile
+
+PGFILEDESC = "pg_validatebackup - validate a backup against a backup manifest"
+PGAPPICON = win32
+
+subdir = src/bin/pg_validatebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+# We need libpq only because fe_utils does.
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS = \
+ $(WIN32RES) \
+ parse_manifest.o \
+ pg_validatebackup.o
+
+all: pg_validatebackup
+
+pg_validatebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_validatebackup$(X) '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_validatebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_validatebackup$(X) $(OBJS)
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
new file mode 100644
index 0000000000..e6b42adfda
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -0,0 +1,576 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.c
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "parse_manifest.h"
+#include "common/jsonapi.h"
+
+/*
+ * Semantic states for JSON manifest parsing.
+ */
+typedef enum
+{
+ JM_EXPECT_TOPLEVEL_START,
+ JM_EXPECT_TOPLEVEL_END,
+ JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_VERSION_VALUE,
+ JM_EXPECT_FILES_FIELD,
+ JM_EXPECT_FILES_ARRAY_START,
+ JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_THIS_FILE_FIELD,
+ JM_EXPECT_THIS_FILE_VALUE,
+ JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
+ JM_EXPECT_EOF
+} JsonManifestSemanticState;
+
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMFF_PATH,
+ JMFF_ENCODED_PATH,
+ JMFF_SIZE,
+ JMFF_LAST_MODIFIED,
+ JMFF_CHECKSUM_ALGORITHM,
+ JMFF_CHECKSUM
+} JsonManifestFileField;
+
+/*
+ * Internal state used while decoding the JSON-format backup manifest.
+ */
+typedef struct
+{
+ JsonManifestParseContext *context;
+ JsonManifestSemanticState state;
+ JsonManifestFileField field;
+ char *pathname;
+ char *encoded_pathname;
+ char *size;
+ char *algorithm;
+ pg_checksum_type checksum_algorithm;
+ char *checksum;
+ char *manifest_checksum;
+} JsonManifestParseState;
+
+static void json_manifest_object_start(void *state);
+static void json_manifest_object_end(void *state);
+static void json_manifest_array_start(void *state);
+static void json_manifest_array_end(void *state);
+static void json_manifest_object_field_start(void *state, char *fname,
+ bool isnull);
+static void json_manifest_scalar(void *state, char *token,
+ JsonTokenType tokentype);
+static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void verify_manifest_checksum(JsonManifestParseState *parse,
+ char *buffer, size_t size);
+static void json_manifest_parse_failure(JsonManifestParseContext *context,
+ char *msg);
+
+static int hexdecode_char(char c);
+static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+
+/*
+ * Main entrypoint to parse a JSON-format backup manifest.
+ *
+ * Caller should set up the parsing context and then invoke this function.
+ * For each file whose information is extracted from the manifest,
+ * context->perfile_cb is invoked. In case of trouble, context->error_cb is
+ * invoked and is expected not to return.
+ */
+void
+json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ size_t size)
+{
+ JsonLexContext *lex;
+ JsonParseErrorType json_error;
+ JsonSemAction sem;
+ JsonManifestParseState parse;
+
+ /* Set up our private parsing context. */
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.context = context;
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+ sem.object_start = json_manifest_object_start;
+ sem.object_end = json_manifest_object_end;
+ sem.array_start = json_manifest_array_start;
+ sem.array_end = json_manifest_array_end;
+ sem.object_field_start = json_manifest_object_field_start;
+ sem.object_field_end = NULL;
+ sem.array_element_start = NULL;
+ sem.array_element_end = NULL;
+ sem.scalar = json_manifest_scalar;
+
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+ /* Validate the checksum. */
+ verify_manifest_checksum(&parse, buffer, size);
+}
+
+/*
+ * Invoked at the start of each object in the JSON document.
+ *
+ * The document as a whole is expected to be an object with three keys
+ * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
+ * file is expected to be an object with various keys (Path, Size, etc.).
+ * If we're not at the beginning of either the toplevel object or the object
+ * for a particular file, it's an error.
+ */
+static void
+json_manifest_object_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_START:
+ parse->state = JM_EXPECT_VERSION_FIELD;
+ break;
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ parse->pathname = NULL;
+ parse->encoded_pathname = NULL;
+ parse->size = NULL;
+ parse->algorithm = NULL;
+ parse->checksum = NULL;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each object in the JSON document.
+ *
+ * The possible cases here are the same as for json_manifest_object_start.
+ * There's nothing special to do at the end of the document, but when we
+ * reach the end of an object representing a particular file, we must call
+ * json_manifest_finalize_file() to save the associated details.
+ */
+static void
+json_manifest_object_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_TOPLEVEL_END:
+ parse->state = JM_EXPECT_EOF;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ json_manifest_finalize_file(parse);
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each array in the JSON document.
+ *
+ * Within the toplevel object, the value associated with the "Files" key
+ * should be an array. No other arrays are expected.
+ */
+static void
+json_manifest_array_start(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_START:
+ parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array start");
+ break;
+ }
+}
+
+/*
+ * Invoked at the end of each array in the JSON document.
+ *
+ * Just like json_manifest_array_start, there's only one expected case
+ * here.
+ */
+static void
+json_manifest_array_end(void *state)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_FILES_ARRAY_NEXT:
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected array end");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each object field in the JSON document.
+ */
+static void
+json_manifest_object_field_start(void *state, char *fname, bool isnull)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_FIELD:
+ /* Inside toplevel object, expecting version indicator. */
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ break;
+ case JM_EXPECT_FILES_FIELD:
+ /* Inside toplevel object, expecting "Files" next. */
+ if (strcmp(fname, "Files") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected file list");
+ parse->state = JM_EXPECT_FILES_ARRAY_START;
+ break;
+ case JM_EXPECT_THIS_FILE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Path") == 0)
+ parse->field = JMFF_PATH;
+ else if (strcmp(fname, "Encoded-Path") == 0)
+ parse->field = JMFF_ENCODED_PATH;
+ else if (strcmp(fname, "Size") == 0)
+ parse->field = JMFF_SIZE;
+ else if (strcmp(fname, "Last-Modified") == 0)
+ parse->field = JMFF_LAST_MODIFIED;
+ else if (strcmp(fname, "Checksum-Algorithm") == 0)
+ parse->field = JMFF_CHECKSUM_ALGORITHM;
+ else if (strcmp(fname, "Checksum") == 0)
+ parse->field = JMFF_CHECKSUM;
+ else
+ json_manifest_parse_failure(parse->context,
+ "unexpected file field");
+ parse->state = JM_EXPECT_THIS_FILE_VALUE;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
+ /* Inside toplevel object, expecting "Manifest-Checksum" next. */
+ if (strcmp(fname, "Manifest-Checksum") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected manifest checksum");
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context,
+ "unexpected object field");
+ break;
+ }
+}
+
+/*
+ * Invoked at the start of each scalar in the JSON document.
+ *
+ * Object field names don't reach this code; those are handled by
+ * json_manifest_object_field_start. When we're inside of the object for
+ * a particular file, that function will have noticed the name of the field,
+ * and we'll get the corresponding value here. When we're in the toplevel
+ * object, the parse state itself tells us which field this is.
+ *
+ * In all cases except for PostgreSQL-Backup-Manifest-Version, which we
+ * can just check on the spot, the goal here is just to save the value in
+ * the parse state for later use. We don't actually do anything until we
+ * reach either the end of the object representing this file, or the end
+ * of the manifest, as the case may be.
+ */
+static void
+json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
+{
+ JsonManifestParseState *parse = state;
+
+ switch (parse->state)
+ {
+ case JM_EXPECT_VERSION_VALUE:
+ if (strcmp(token, "1") != 0)
+ json_manifest_parse_failure(parse->context,
+ "unexpected manifest version");
+ parse->state = JM_EXPECT_FILES_FIELD;
+ break;
+ case JM_EXPECT_THIS_FILE_VALUE:
+ switch (parse->field)
+ {
+ case JMFF_PATH:
+ parse->pathname = token;
+ break;
+ case JMFF_ENCODED_PATH:
+ parse->encoded_pathname = token;
+ break;
+ case JMFF_SIZE:
+ parse->size = token;
+ break;
+ case JMFF_LAST_MODIFIED:
+ pfree(token); /* unused */
+ break;
+ case JMFF_CHECKSUM_ALGORITHM:
+ parse->algorithm = token;
+ break;
+ case JMFF_CHECKSUM:
+ parse->checksum = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_FILE_FIELD;
+ break;
+ case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
+ parse->state = JM_EXPECT_TOPLEVEL_END;
+ parse->manifest_checksum = token;
+ break;
+ default:
+ json_manifest_parse_failure(parse->context, "unexpected scalar");
+ break;
+ }
+}
+
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * file, and invoke the per-file callback so that the caller gets those
+ * details. This happens for each file when the corresponding JSON object is
+ * completely parsed.
+ */
+static void
+json_manifest_finalize_file(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t size;
+ char *ep;
+ int checksum_string_length;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+
+ /* Pathname and size are required. */
+ if (parse->pathname == NULL && parse->encoded_pathname == NULL)
+ json_manifest_parse_failure(parse->context, "missing pathname");
+ if (parse->pathname != NULL && parse->encoded_pathname != NULL)
+ json_manifest_parse_failure(parse->context,
+ "both pathname and encoded pathname");
+ if (parse->size == NULL)
+ json_manifest_parse_failure(parse->context, "missing size");
+ if (parse->algorithm == NULL && parse->checksum != NULL)
+ json_manifest_parse_failure(parse->context,
+ "checksum without algorithm");
+
+ /* Decode encoded pathname, if that's what we have. */
+ if (parse->encoded_pathname != NULL)
+ {
+ int encoded_length = strlen(parse->encoded_pathname);
+ int raw_length = encoded_length / 2;
+
+ parse->pathname = palloc(raw_length + 1);
+ if (encoded_length % 2 != 0 ||
+ !hexdecode_string((uint8 *) parse->pathname,
+ parse->encoded_pathname,
+ raw_length))
+ json_manifest_parse_failure(parse->context,
+ "unable to decode filename");
+ parse->pathname[raw_length] = '\0';
+ pfree(parse->encoded_pathname);
+ parse->encoded_pathname = NULL;
+ }
+
+ /* Parse size. */
+ size = strtoul(parse->size, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "file size is not an integer");
+
+ /* Parse the checksum algorithm, if it's present. */
+ if (parse->algorithm == NULL)
+ checksum_type = CHECKSUM_TYPE_NONE;
+ else if (!pg_checksum_parse_type(parse->algorithm, &checksum_type))
+ context->error_cb(context, "unrecognized checksum algorithm: \"%s\"",
+ parse->algorithm);
+
+ /* Parse the checksum payload, if it's present. */
+ checksum_string_length = parse->checksum == NULL ? 0
+ : strlen(parse->checksum);
+ if (checksum_string_length == 0)
+ {
+ checksum_length = 0;
+ checksum_payload = NULL;
+ }
+ else
+ {
+ checksum_length = checksum_string_length / 2;
+ checksum_payload = palloc(checksum_length);
+ if (checksum_string_length % 2 != 0 ||
+ !hexdecode_string(checksum_payload, parse->checksum,
+ checksum_length))
+ context->error_cb(context,
+ "invalid checksum for file \"%s\": \"%s\"",
+ parse->pathname, parse->checksum);
+ }
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perfile_cb(context, parse->pathname, size,
+ checksum_type, checksum_length, checksum_payload);
+
+ /* Free memory we no longer need. */
+ if (parse->size != NULL)
+ {
+ pfree(parse->size);
+ parse->size = NULL;
+ }
+ if (parse->algorithm != NULL)
+ {
+ pfree(parse->algorithm);
+ parse->algorithm = NULL;
+ }
+ if (parse->checksum != NULL)
+ {
+ pfree(parse->checksum);
+ parse->checksum = NULL;
+ }
+}
+
+/*
+ * Verify that the manifest checksum is correct.
+ *
+ * The last line of the manifest file is excluded from the manifest checksum,
+ * because the last line is expected to contain the checksum that covers
+ * the rest of the file.
+ */
+static void
+verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
+ size_t size)
+{
+ JsonManifestParseContext *context = parse->context;
+ size_t i;
+ size_t number_of_newlines = 0;
+ size_t ultimate_newline = 0;
+ size_t penultimate_newline = 0;
+ pg_sha256_ctx manifest_ctx;
+ uint8 manifest_checksum_actual[PG_SHA256_DIGEST_LENGTH];
+ uint8 manifest_checksum_expected[PG_SHA256_DIGEST_LENGTH];
+
+ /* Find the last two newlines in the file. */
+ for (i = 0; i < size; ++i)
+ {
+ if (buffer[i] == '\n')
+ {
+ ++number_of_newlines;
+ penultimate_newline = ultimate_newline;
+ ultimate_newline = i;
+ }
+ }
+
+ /*
+ * Make sure that the last newline is right at the end, and that there are
+ * at least two lines total. We need this to be true in order for the
+ * following code, which computes the manifest checksum, to work properly.
+ */
+ if (number_of_newlines < 2)
+ json_manifest_parse_failure(parse->context,
+ "expected at least 2 lines");
+ if (ultimate_newline != size - 1)
+ json_manifest_parse_failure(parse->context,
+ "last line not newline-terminated");
+
+ /* Checksum the rest. */
+ pg_sha256_init(&manifest_ctx);
+ pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
+ pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
+
+ /* Now verify it. */
+ if (parse->manifest_checksum == NULL)
+ context->error_cb(parse->context, "manifest has no checksum");
+ if (strlen(parse->manifest_checksum) != PG_SHA256_DIGEST_LENGTH * 2 ||
+ !hexdecode_string(manifest_checksum_expected, parse->manifest_checksum,
+ PG_SHA256_DIGEST_LENGTH))
+ context->error_cb(context, "invalid manifest checksum: \"%s\"",
+ parse->manifest_checksum);
+ if (memcmp(manifest_checksum_actual, manifest_checksum_expected,
+ PG_SHA256_DIGEST_LENGTH) != 0)
+ context->error_cb(context, "manifest checksum mismatch");
+}
+
+/*
+ * Report a parse error.
+ *
+ * This is intended to be used for fairly low-level failures that probably
+ * shouldn't occur unless somebody has deliberately constructed a bad manifest,
+ * or unless the server is generating bad manifests due to some bug. msg should
+ * be a short string giving some hint as to what the problem is.
+ */
+static void
+json_manifest_parse_failure(JsonManifestParseContext *context, char *msg)
+{
+ context->error_cb(context, "could not parse backup manifest: %s", msg);
+}
+
+/*
+ * Convert a character which represents a hexadecimal digit to an integer.
+ *
+ * Returns -1 if the character is not a hexadecimal digit.
+ */
+static int
+hexdecode_char(char c)
+{
+ if (c >= '0' && c <= '9')
+ return c - '0';
+ if (c >= 'a' && c <= 'f')
+ return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F')
+ return c - 'A' + 10;
+
+ return -1;
+}
+
+/*
+ * Decode a hex string into a byte string, 2 hex chars per byte.
+ *
+ * Returns false if invalid characters are encountered; otherwise true.
+ */
+static bool
+hexdecode_string(uint8 *result, char *input, int nbytes)
+{
+ int i;
+
+ for (i = 0; i < nbytes; ++i)
+ {
+ int n1 = hexdecode_char(input[i * 2]);
+ int n2 = hexdecode_char(input[i * 2 + 1]);
+
+ if (n1 < 0 || n2 < 0)
+ return false;
+ result[i] = n1 * 16 + n2;
+ }
+
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
new file mode 100644
index 0000000000..25d140f72f
--- /dev/null
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -0,0 +1,40 @@
+/*-------------------------------------------------------------------------
+ *
+ * parse_manifest.h
+ * Parse a backup manifest in JSON format.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/parse_manifest.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PARSE_MANIFEST_H
+#define PARSE_MANIFEST_H
+
+#include "common/checksum_helper.h"
+#include "mb/pg_wchar.h"
+
+struct JsonManifestParseContext;
+typedef struct JsonManifestParseContext JsonManifestParseContext;
+
+typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
+ char *pathname,
+ size_t size, pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
+ char *fmt, ...) pg_attribute_printf(2, 3);
+
+struct JsonManifestParseContext
+{
+ void *private_data;
+ json_manifest_perfile_callback perfile_cb;
+ json_manifest_error_callback error_cb;
+};
+
+extern void json_parse_manifest(JsonManifestParseContext *context,
+ char *buffer, size_t size);
+
+#endif
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
new file mode 100644
index 0000000000..eb1473d9d0
--- /dev/null
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -0,0 +1,734 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_validatebackup.c
+ * Validate a backup against a backup manifest.
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_validatebackup/pg_validatebackup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+
+#include "common/hashfn.h"
+#include "common/logging.h"
+#include "fe_utils/simple_list.h"
+#include "getopt_long.h"
+#include "parse_manifest.h"
+
+/*
+ * For efficiency, we'd like our hash table containing information about the
+ * manifest to start out with approximately the correct number of entries.
+ * There's no way to know the exact number of entries without reading the whole
+ * file, but we can get an estimate by dividing the file size by the estimated
+ * number of bytes per line.
+ *
+ * This could be off by about a factor of two in either direction, because the
+ * checksum algorithm has a big impact on the line lengths; e.g. a SHA512
+ * checksum is 128 hex bytes, whereas a CRC-32C value is only 8, and there
+ * might be no checksum at all.
+ */
+#define ESTIMATED_BYTES_PER_MANIFEST_LINE 100
+
+/*
+ * How many bytes should we try to read from a file at once?
+ */
+#define READ_CHUNK_SIZE 4096
+
+/*
+ * Information about each file described by the manifest file is parsed to
+ * produce an object like this.
+ */
+typedef struct manifestfile
+{
+ uint32 status; /* hash status */
+ char *pathname;
+ size_t size;
+ pg_checksum_type checksum_type;
+ int checksum_length;
+ uint8 *checksum_payload;
+ bool matched;
+ bool bad;
+} manifestfile;
+
+/*
+ * Define a hash table which we can use to store information about the files
+ * mentioned in the backup manifest.
+ */
+static uint32 hash_string_pointer(char *s);
+#define SH_PREFIX manifestfiles
+#define SH_ELEMENT_TYPE manifestfile
+#define SH_KEY_TYPE char *
+#define SH_KEY pathname
+#define SH_HASH_KEY(tb, key) hash_string_pointer(key)
+#define SH_EQUAL(tb, a, b) (strcmp(a, b) == 0)
+#define SH_SCOPE static inline
+#define SH_RAW_ALLOCATOR pg_malloc0
+#define SH_DECLARE
+#define SH_DEFINE
+#include "lib/simplehash.h"
+
+/*
+ * All of the context information we need while checking a backup manifest.
+ */
+typedef struct validator_context
+{
+ manifestfiles_hash *ht;
+ char *backup_directory;
+ SimpleStringList ignore_list;
+ bool exit_on_error;
+ bool saw_any_error;
+} validator_context;
+
+static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+
+static void record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length,
+ uint8 *checksum_payload);
+static void report_manifest_error(JsonManifestParseContext *context,
+ char *fmt, ...)
+ pg_attribute_printf(2, 3) pg_attribute_noreturn();
+
+static void validate_backup_directory(validator_context *context,
+ char *relpath, char *fullpath);
+static void validate_backup_file(validator_context *context,
+ char *relpath, char *fullpath);
+static void report_extra_backup_files(validator_context *context);
+static void validate_backup_checksums(validator_context *context);
+static void validate_file_checksum(validator_context *context,
+ manifestfile *tabent, char *pathname);
+
+static void report_backup_error(validator_context *context,
+ const char *pg_restrict fmt,...)
+ pg_attribute_printf(2, 3);
+static void report_fatal_error(const char *pg_restrict fmt,...)
+ pg_attribute_printf(1, 2) pg_attribute_noreturn();
+static bool should_ignore_relpath(validator_context *context, char *relpath);
+
+static void usage(void);
+
+static const char *progname;
+
+/*
+ * Main entry point.
+ */
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"exit-on-error", no_argument, NULL, 'e'},
+ {"ignore", required_argument, NULL, 'i'},
+ {"manifest-path", required_argument, NULL, 'm'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"skip-checksums", no_argument, NULL, 's'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+ validator_context context;
+ char *manifest_path = NULL;
+ bool quiet = false;
+ bool skip_checksums = false;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
+ progname = get_progname(argv[0]);
+
+ memset(&context, 0, sizeof(context));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_validatebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /*
+ * Skip certain files in the toplevel directory.
+ *
+ * Ignore the backup_manifest file, because it's not included in the
+ * backup manifest.
+ *
+ * Ignore the pg_wal directory, because those files are not included in
+ * the backup manifest either, since they are fetched separately from the
+ * backup itself.
+ *
+ * Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
+ * because we expect that those files may sometimes be created or changed
+ * as part of the backup process. For example, pg_basebackup -R will
+ * modify postgresql.auto.conf and create standby.signal.
+ */
+ simple_string_list_append(&context.ignore_list, "backup_manifest");
+ simple_string_list_append(&context.ignore_list, "pg_wal");
+ simple_string_list_append(&context.ignore_list, "postgresql.auto.conf");
+ simple_string_list_append(&context.ignore_list, "recovery.signal");
+ simple_string_list_append(&context.ignore_list, "standby.signal");
+
+ while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'e':
+ context.exit_on_error = true;
+ break;
+ case 'i':
+ {
+ char *arg = pstrdup(optarg);
+
+ canonicalize_path(arg);
+ simple_string_list_append(&context.ignore_list, arg);
+ break;
+ }
+ case 'm':
+ manifest_path = pstrdup(optarg);
+ canonicalize_path(manifest_path);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 's':
+ skip_checksums = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /* Get backup directory name */
+ if (optind >= argc)
+ {
+ pg_log_fatal("no backup directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ context.backup_directory = pstrdup(argv[optind++]);
+ canonicalize_path(context.backup_directory);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_fatal("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /* By default, look for the manifest in the backup directory. */
+ if (manifest_path == NULL)
+ manifest_path = psprintf("%s/backup_manifest",
+ context.backup_directory);
+
+ /*
+ * Try to read the manifest. We treat any errors encountered while parsing
+ * the manifest as fatal; there doesn't seem to be much point in trying to
+ * validate the backup directory against a corrupted manifest.
+ */
+ context.ht = parse_manifest_file(manifest_path);
+
+ /*
+ * Now scan the files in the backup directory. At this stage, we verify
+ * that every file on disk is present in the manifest and that the sizes
+ * match. We also set the "matched" flag on every manifest entry that
+ * corresponds to a file on disk.
+ */
+ validate_backup_directory(&context, NULL, context.backup_directory);
+
+ /*
+ * The "matched" flag should now be set on every entry in the hash table.
+ * Any entries for which the bit is not set are files mentioned in the
+ * manifest that don't exist on disk.
+ */
+ report_extra_backup_files(&context);
+
+ /*
+ * Finally, do the expensive work of verifying file checksums, unless we
+ * were told to skip it.
+ */
+ if (!skip_checksums)
+ validate_backup_checksums(&context);
+
+ /*
+ * If everything looks OK, tell the user this, unless we were asked to
+ * work quietly.
+ */
+ if (!context.saw_any_error && !quiet)
+ printf("backup successfully verified\n");
+
+ return context.saw_any_error ? 1 : 0;
+}
+
+/*
+ * Parse a manifest file and construct a hash table with information about
+ * all the files it mentions.
+ */
+static manifestfiles_hash *
+parse_manifest_file(char *manifest_path)
+{
+ int fd;
+ struct stat statbuf;
+ off_t estimate;
+ uint32 initial_size;
+ manifestfiles_hash *ht;
+ char *buffer;
+ int rc;
+ JsonManifestParseContext context;
+
+ /* Open the manifest file. */
+ if ((fd = open(manifest_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ report_fatal_error("could not open file \"%s\": %m", manifest_path);
+
+ /* Figure out how big the manifest is. */
+ if (fstat(fd, &statbuf) != 0)
+ report_fatal_error("could not stat file \"%s\": %m", manifest_path);
+
+ /* Guess how large to make the hash table based on the manifest size. */
+ estimate = statbuf.st_size / ESTIMATED_BYTES_PER_MANIFEST_LINE;
+ initial_size = Min(PG_UINT32_MAX, Max(estimate, 256));
+
+ /* Create the hash table. */
+ ht = manifestfiles_create(initial_size, NULL);
+
+ /*
+ * Slurp in the whole file.
+ *
+ * This is not ideal, but there's currently no easy way to get
+ * pg_parse_json() to perform incremental parsing.
+ */
+ buffer = pg_malloc(statbuf.st_size);
+ rc = read(fd, buffer, statbuf.st_size);
+ if (rc != statbuf.st_size)
+ {
+ if (rc < 0)
+ report_fatal_error("could not read file \"%s\": %m",
+ manifest_path);
+ else
+ report_fatal_error("could not read file \"%s\": read %d of %zu",
+ manifest_path, rc, (size_t) statbuf.st_size);
+ }
+
+ /* Close the manifest file. */
+ close(fd);
+
+ /* Parse the manifest as JSON. */
+ context.private_data = ht;
+ context.perfile_cb = record_manifest_details_for_file;
+ context.error_cb = report_manifest_error;
+ json_parse_manifest(&context, buffer, statbuf.st_size);
+
+ /* Done with the buffer. */
+ pfree(buffer);
+
+ /* Return the hash table we constructed. */
+ return ht;
+}
+
+/*
+ * Report an error while parsing the manifest.
+ *
+ * We consider all such errors to be fatal errors. The manifest parser
+ * expects this function not to return.
+ */
+static void
+report_manifest_error(JsonManifestParseContext *context, char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Record details extracted from the backup manifest for one file.
+ */
+static void
+record_manifest_details_for_file(JsonManifestParseContext *context,
+ char *pathname, size_t size,
+ pg_checksum_type checksum_type,
+ int checksum_length, uint8 *checksum_payload)
+{
+ manifestfiles_hash *ht = context->private_data;
+ manifestfile *tabent;
+ bool found;
+
+ /* Make a new entry in the hash table for this file. */
+ tabent = manifestfiles_insert(ht, pathname, &found);
+ if (found)
+ report_fatal_error("duplicate pathname in backup manifest: \"%s\"",
+ pathname);
+
+ /* Initialize the entry. */
+ tabent->size = size;
+ tabent->checksum_type = checksum_type;
+ tabent->checksum_length = checksum_length;
+ tabent->checksum_payload = checksum_payload;
+ tabent->matched = false;
+ tabent->bad = false;
+}
+
+/*
+ * Validate one directory.
+ *
+ * 'relpath' is NULL if we are to validate the top-level backup directory,
+ * and otherwise the relative path to the directory that is to be validated.
+ *
+ * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
+ * filesystem path at which it can be found.
+ */
+static void
+validate_backup_directory(validator_context *context, char *relpath,
+ char *fullpath)
+{
+ DIR *dir;
+ struct dirent *dirent;
+
+ dir = opendir(fullpath);
+ if (dir == NULL)
+ {
+ /*
+ * If even the toplevel backup directory cannot be found, treat this
+ * as a fatal error.
+ */
+ if (relpath == NULL)
+ report_fatal_error("could not open directory \"%s\": %m", fullpath);
+
+ /*
+ * Otherwise, treat this as a non-fatal error, but ignore any further
+ * errors related to this path and anything beneath it.
+ */
+ report_backup_error(context,
+ "could not open directory \"%s\": %m", fullpath);
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ while (errno = 0, (dirent = readdir(dir)) != NULL)
+ {
+ char *filename = dirent->d_name;
+ char *newfullpath = psprintf("%s/%s", fullpath, filename);
+ char *newrelpath;
+
+ /* Skip "." and ".." */
+ if (filename[0] == '.' && (filename[1] == '\0'
+ || strcmp(filename, "..") == 0))
+ continue;
+
+ if (relpath == NULL)
+ newrelpath = pstrdup(filename);
+ else
+ newrelpath = psprintf("%s/%s", relpath, filename);
+
+ if (!should_ignore_relpath(context, newrelpath))
+ validate_backup_file(context, newrelpath, newfullpath);
+
+ pfree(newfullpath);
+ pfree(newrelpath);
+ }
+
+ if (closedir(dir))
+ {
+ report_backup_error(context,
+ "could not close directory \"%s\": %m", fullpath);
+ return;
+ }
+}
+
+/*
+ * Validate one file (which might actually be a directory or a symlink).
+ *
+ * The arguments to this function have the same meaning as the arguments to
+ * validate_backup_directory.
+ */
+static void
+validate_backup_file(validator_context *context, char *relpath, char *fullpath)
+{
+ struct stat sb;
+ manifestfile *tabent;
+
+ if (stat(fullpath, &sb) != 0)
+ {
+ report_backup_error(context,
+ "could not stat file or directory \"%s\": %m",
+ relpath);
+
+ /*
+ * Suppress further errors related to this path name and, if it's a
+ * directory, anything underneath it.
+ */
+ simple_string_list_append(&context->ignore_list, relpath);
+
+ return;
+ }
+
+ /* If it's a directory, just recurse. */
+ if (S_ISDIR(sb.st_mode))
+ {
+ validate_backup_directory(context, relpath, fullpath);
+ return;
+ }
+
+ /* If it's not a directory, it should be a plain file. */
+ if (!S_ISREG(sb.st_mode))
+ {
+ report_backup_error(context,
+ "\"%s\" is not a file or directory",
+ relpath);
+ return;
+ }
+
+ /* Check whether there's an entry in the manifest hash. */
+ tabent = manifestfiles_lookup(context->ht, relpath);
+ if (tabent == NULL)
+ {
+ report_backup_error(context,
+ "\"%s\" is present on disk but not in the manifest",
+ relpath);
+ return;
+ }
+
+ /* Flag this entry as having been encountered in the filesystem. */
+ tabent->matched = true;
+
+ /* Check that the size matches. */
+ if (tabent->size != sb.st_size)
+ {
+ report_backup_error(context,
+ "\"%s\" has size %zu on disk but size %zu in the manifest",
+ relpath, (size_t) sb.st_size, tabent->size);
+ tabent->bad = true;
+ }
+
+ /*
+ * We don't validate checksums at this stage. We first finish validating
+ * that we have the expected set of files with the expected sizes, and
+ * only afterwards verify the checksums. That's because computing
+ * checksums may take a while, and we'd like to report more obvious
+ * problems quickly.
+ */
+}
+
+/*
+ * Scan the hash table for entries where the 'matched' flag is not set; report
+ * that such files are present in the manifest but not on disk.
+ */
+static void
+report_extra_backup_files(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ if (!tabent->matched &&
+ !should_ignore_relpath(context, tabent->pathname))
+ report_backup_error(context,
+ "\"%s\" is present in the manifest but not on disk",
+ tabent->pathname);
+}
+
+/*
+ * Validate checksums for hash table entries that are otherwise unproblematic.
+ * If we've already reported some problem related to a hash table entry, or
+ * if it has no checksum, just skip it.
+ */
+static void
+validate_backup_checksums(validator_context *context)
+{
+ manifestfiles_iterator it;
+ manifestfile *tabent;
+
+ manifestfiles_start_iterate(context->ht, &it);
+ while ((tabent = manifestfiles_iterate(context->ht, &it)) != NULL)
+ {
+ if (tabent->matched && !tabent->bad &&
+ tabent->checksum_type != CHECKSUM_TYPE_NONE &&
+ !should_ignore_relpath(context, tabent->pathname))
+ {
+ char *fullpath;
+
+ /* Compute the full pathname to the target file. */
+ fullpath = psprintf("%s/%s", context->backup_directory,
+ tabent->pathname);
+
+ /* Do the actual checksum validation. */
+ validate_file_checksum(context, tabent, fullpath);
+
+ /* Avoid leaking memory. */
+ pfree(fullpath);
+ }
+ }
+}
+
+/*
+ * Validate the checksum of a single file.
+ */
+static void
+validate_file_checksum(validator_context *context, manifestfile *tabent,
+ char *fullpath)
+{
+ pg_checksum_context checksum_ctx;
+ char *relpath = tabent->pathname;
+ int fd;
+ int rc;
+ uint8 buffer[READ_CHUNK_SIZE];
+ uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
+ int checksumlen;
+
+ /* Open the target file. */
+ if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ report_backup_error(context, "could not open file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* Initialize checksum context. */
+ pg_checksum_init(&checksum_ctx, tabent->checksum_type);
+
+ /* Read the file chunk by chunk, updating the checksum as we go. */
+ while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
+ pg_checksum_update(&checksum_ctx, buffer, rc);
+ if (rc < 0)
+ report_backup_error(context, "could not read file \"%s\": %m",
+ relpath);
+
+ /* Close the file. */
+ if (close(fd) != 0)
+ {
+ report_backup_error(context, "could not close file \"%s\": %m",
+ relpath);
+ return;
+ }
+
+ /* If we didn't manage to read the whole file, bail out now. */
+ if (rc < 0)
+ return;
+
+ /* Get the final checksum. */
+ checksumlen = pg_checksum_final(&checksum_ctx, checksumbuf);
+
+ /* And check it against the manifest. */
+ if (checksumlen != tabent->checksum_length)
+ report_backup_error(context,
+ "file \"%s\" has checksum of length %d, but expected %d",
+ relpath, tabent->checksum_length, checksumlen);
+ else if (memcmp(checksumbuf, tabent->checksum_payload, checksumlen) != 0)
+ report_backup_error(context,
+ "checksum mismatch for file \"%s\"",
+ relpath);
+}
+
+/*
+ * Report a problem with the backup.
+ *
+ * Update the context to indicate that we saw an error, and exit if the
+ * context says we should.
+ */
+static void
+report_backup_error(validator_context *context, const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_ERROR, fmt, ap);
+ va_end(ap);
+
+ context->saw_any_error = true;
+ if (context->exit_on_error)
+ exit(1);
+}
+
+/*
+ * Report a fatal error and exit
+ */
+static void
+report_fatal_error(const char *pg_restrict fmt,...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ pg_log_generic_v(PG_LOG_FATAL, fmt, ap);
+ va_end(ap);
+
+ exit(1);
+}
+
+/*
+ * Is the specified relative path, or some prefix of it, listed in the set
+ * of paths to ignore?
+ *
+ * Note that by "prefix" we mean a parent directory; for this purpose,
+ * "aa/bb" is not a prefix of "aa/bbb", but it is a prefix of "aa/bb/cc".
+ */
+static bool
+should_ignore_relpath(validator_context *context, char *relpath)
+{
+ SimpleStringListCell *cell;
+
+ for (cell = context->ignore_list.head; cell != NULL; cell = cell->next)
+ {
+ char *r = relpath;
+ char *v = cell->val;
+
+ while (*v != '\0' && *r == *v)
+ ++r, ++v;
+
+ if (*v == '\0' && (*r == '\0' || *r == '/'))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Helper function for manifestfiles hash table.
+ */
+static uint32
+hash_string_pointer(char *s)
+{
+ unsigned char *ss = (unsigned char *) s;
+
+ return hash_bytes(ss, strlen(s));
+}
+
+/*
+ * Print out usage information and exit.
+ */
+static void
+usage(void)
+{
+ printf(_("%s validates a backup against the backup manifest.\n\n"), progname);
+ printf(_("Usage:\n %s [OPTION]... BACKUPDIR\n\n"), progname);
+ printf(_("Options:\n"));
+ printf(_(" -e, --exit-on-error exit immediately on error\n"));
+ printf(_(" -i, --ignore=RELATIVE_PATH ignore indicated path\n"));
+ printf(_(" -m, --manifest=PATH use specified path for manifest\n"));
+ printf(_(" -s, --skip-checksums skip checksum verification\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+ printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
+}
diff --git a/src/bin/pg_validatebackup/t/001_basic.pl b/src/bin/pg_validatebackup/t/001_basic.pl
new file mode 100644
index 0000000000..6d4b8ea01a
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/001_basic.pl
@@ -0,0 +1,30 @@
+use strict;
+use warnings;
+use TestLib;
+use Test::More tests => 16;
+
+my $tempdir = TestLib::tempdir;
+
+program_help_ok('pg_validatebackup');
+program_version_ok('pg_validatebackup');
+program_options_handling_ok('pg_validatebackup');
+
+command_fails_like(['pg_validatebackup'],
+ qr/no backup directory specified/,
+ 'target directory must be specified');
+command_fails_like(['pg_validatebackup', $tempdir],
+ qr/could not open file.*\/backup_manifest\"/,
+ 'pg_validatebackup requires a manifest');
+command_fails_like(['pg_validatebackup', $tempdir, $tempdir],
+ qr/too many command-line arguments/,
+ 'multiple target directories not allowed');
+
+# create fake manifest file
+open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+close($fh);
+
+# but then try to use an alternate, nonexisting manifest
+command_fails_like(['pg_validatebackup', '-m', "$tempdir/not_the_manifest",
+ $tempdir],
+ qr/could not open file.*\/not_the_manifest\"/,
+ 'pg_validatebackup respects -m flag');
diff --git a/src/bin/pg_validatebackup/t/002_algorithm.pl b/src/bin/pg_validatebackup/t/002_algorithm.pl
new file mode 100644
index 0000000000..98871e12a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/002_algorithm.pl
@@ -0,0 +1,58 @@
+# Verify that we can take and validate backups with various checksum types.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 19;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+for my $algorithm (qw(bogus none crc32c sha224 sha256 sha384 sha512))
+{
+ my $backup_path = $master->backup_dir . '/' . $algorithm;
+ my @backup = ('pg_basebackup', '-D', $backup_path,
+ '--manifest-checksums', $algorithm,
+ '--no-sync');
+ my @validate = ('pg_validatebackup', '-e', $backup_path);
+
+ # A backup with a bogus algorithm should fail.
+ if ($algorithm eq 'bogus')
+ {
+ $master->command_fails(\@backup,
+ "backup fails with algorithm \"$algorithm\"");
+ next;
+ }
+
+ # A backup with a valid algorithm should work.
+ $master->command_ok(\@backup, "backup ok with algorithm \"$algorithm\"");
+
+ # We expect each real checksum algorithm to be mentioned on every line of
+ # the backup manifest file except the first and last; for simplicity, we
+ # just check that it shows up lots of times. When the checksum algorithm
+ # is none, we just check that the manifest exists.
+ if ($algorithm eq 'none')
+ {
+ ok(-f "$backup_path/backup_manifest", "backup manifest exists");
+ }
+ else
+ {
+ my $manifest = slurp_file("$backup_path/backup_manifest");
+ my $count_of_algorithm_in_manifest =
+ (() = $manifest =~ /$algorithm/mig);
+ cmp_ok($count_of_algorithm_in_manifest, '>', 100,
+ "$algorithm is mentioned many times in the manifest");
+ }
+
+ # Make sure that it validates OK.
+ $master->command_ok(\@validate,
+ "validate backup with algorithm \"$algorithm\"");
+
+ # Remove backup immediately to save disk space.
+ rmtree($backup_path);
+}
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
new file mode 100644
index 0000000000..6ad29a031f
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -0,0 +1,251 @@
+# Verify that various forms of corruption are detected by pg_validatebackup.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+
+# Include a user-defined tablespace in the hopes of detecting problems in that
+# area.
+my $source_ts_path = TestLib::tempdir;
+$master->safe_psql('postgres', <<EOM);
+CREATE TABLE x1 (a int);
+INSERT INTO x1 VALUES (111);
+CREATE TABLESPACE ts1 LOCATION '$source_ts_path';
+CREATE TABLE x2 (a int) TABLESPACE ts1;
+INSERT INTO x1 VALUES (222);
+EOM
+
+my @scenario = (
+ {
+ 'name' => 'extra_file',
+ 'mutilate' => \&mutilate_extra_file,
+ 'fails_like' =>
+ qr/extra_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'extra_tablespace_file',
+ 'mutilate' => \&mutilate_extra_tablespace_file,
+ 'fails_like' =>
+ qr/extra_ts_file.*present on disk but not in the manifest/
+ },
+ {
+ 'name' => 'missing_file',
+ 'mutilate' => \&mutilate_missing_file,
+ 'fails_like' =>
+ qr/pg_xact\/0000.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'missing_tablespace',
+ 'mutilate' => \&mutilate_missing_tablespace,
+ 'fails_like' =>
+ qr/pg_tblspc.*present in the manifest but not on disk/
+ },
+ {
+ 'name' => 'append_to_file',
+ 'mutilate' => \&mutilate_append_to_file,
+ 'fails_like' =>
+ qr/has size \d+ on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'truncate_file',
+ 'mutilate' => \&mutilate_truncate_file,
+ 'fails_like' =>
+ qr/has size 0 on disk but size \d+ in the manifest/
+ },
+ {
+ 'name' => 'replace_file',
+ 'mutilate' => \&mutilate_replace_file,
+ 'fails_like' => qr/checksum mismatch for file/
+ },
+ {
+ 'name' => 'bad_manifest',
+ 'mutilate' => \&mutilate_bad_manifest,
+ 'fails_like' => qr/manifest checksum mismatch/
+ },
+ {
+ 'name' => 'open_file_fails',
+ 'mutilate' => \&mutilate_open_file_fails,
+ 'fails_like' => qr/could not open file/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'open_directory_fails',
+ 'mutilate' => \&mutilate_open_directory_fails,
+ 'fails_like' => qr/could not open directory/,
+ 'skip_on_windows' => 1
+ },
+ {
+ 'name' => 'search_directory_fails',
+ 'mutilate' => \&mutilate_search_directory_fails,
+ 'cleanup' => \&cleanup_search_directory_fails,
+ 'fails_like' => qr/could not stat file or directory/,
+ 'skip_on_windows' => 1
+ }
+);
+
+for my $scenario (@scenario)
+{
+ my $name = $scenario->{'name'};
+
+ SKIP:
+ {
+ skip "unix-style permissions not supported on Windows", 4
+ if $scenario->{'skip_on_windows'} && $windows_os;
+
+ # Take a backup and check that it validates OK.
+ my $backup_path = $master->backup_dir . '/' . $name;
+ my $backup_ts_path = TestLib::tempdir;
+ $master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '-T', "${source_ts_path}=${backup_ts_path}"],
+ "base backup ok");
+ command_ok(['pg_validatebackup', $backup_path ],
+ "intact backup validated");
+
+ # Mutilate the backup in some way.
+ $scenario->{'mutilate'}->($backup_path);
+
+ # Now check that the backup no longer validates.
+ command_fails_like(['pg_validatebackup', $backup_path ],
+ $scenario->{'fails_like'},
+ "corrupt backup fails validation: $name");
+
+ # Run cleanup hook, if provided.
+ $scenario->{'cleanup'}->($backup_path)
+ if exists $scenario->{'cleanup'};
+
+ # Finally, use rmtree to reclaim space.
+ rmtree($backup_path);
+ }
+}
+
+sub create_extra_file
+{
+ my ($backup_path, $relative_path) = @_;
+ my $pathname = "$backup_path/$relative_path";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh "This is an extra file.\n";
+ close($fh);
+}
+
+# Add a file into the root directory of the backup.
+sub mutilate_extra_file
+{
+ my ($backup_path) = @_;
+ create_extra_file($backup_path, "extra_file");
+}
+
+# Add a file inside the user-defined tablespace.
+sub mutilate_extra_tablespace_file
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my ($catvdir) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid");
+ my ($tsdboid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc/$tsoid/$catvdir");
+ create_extra_file($backup_path,
+ "pg_tblspc/$tsoid/$catvdir/$tsdboid/extra_ts_file");
+}
+
+# Remove a file.
+sub mutilate_missing_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_xact/0000";
+ unlink($pathname) || die "$pathname: $!";
+}
+
+# Remove the symlink to the user-defined tablespace.
+sub mutilate_missing_tablespace
+{
+ my ($backup_path) = @_;
+ my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
+ slurp_dir("$backup_path/pg_tblspc");
+ my $pathname = "$backup_path/pg_tblspc/$tsoid";
+ if ($windows_os)
+ {
+ rmdir($pathname) || die "$pathname: $!";
+ }
+ else
+ {
+ unlink($pathname) || die "$pathname: $!";
+ }
+}
+
+# Append an additional bytes to a file.
+sub mutilate_append_to_file
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/global/pg_control", 'x';
+}
+
+# Truncate a file to zero length.
+sub mutilate_truncate_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/global/pg_control";
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ close($fh);
+}
+
+# Replace a file's contents without changing the length of the file. This is
+# not a particularly efficient way to do this, so we pick a file that's
+# expected to be short.
+sub mutilate_replace_file
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ my $contents = slurp_file($pathname);
+ open(my $fh, '>', $pathname) || die "open $pathname: $!";
+ print $fh 'q' x length($contents);
+ close($fh);
+}
+
+# Corrupt the backup manifest.
+sub mutilate_bad_manifest
+{
+ my ($backup_path) = @_;
+ append_to_file "$backup_path/backup_manifest", "\n";
+}
+
+# Create a file that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_file_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/PG_VERSION";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be opened. (This is skipped on Windows.)
+sub mutilate_open_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/pg_subtrans";
+ chmod(0, $pathname) || die "chmod $pathname: $!";
+}
+
+# Create a directory that can't be searched. (This is skipped on Windows.)
+sub mutilate_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0400, $pathname) || die "chmod $pathname: $!";
+}
+
+# rmtree can't cope with a mode 400 directory, so change back to 700.
+sub cleanup_search_directory_fails
+{
+ my ($backup_path) = @_;
+ my $pathname = "$backup_path/base";
+ chmod(0700, $pathname) || die "chmod $pathname: $!";
+}
diff --git a/src/bin/pg_validatebackup/t/004_options.pl b/src/bin/pg_validatebackup/t/004_options.pl
new file mode 100644
index 0000000000..8f185626ed
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/004_options.pl
@@ -0,0 +1,89 @@
+# Verify the behavior of assorted pg_validatebackup options.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use File::Path qw(rmtree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 25;
+
+# Start up the server and take a backup.
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_options';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync' ],
+ "base backup ok");
+
+# Verify that pg_validatebackup -q succeeds and produces no output.
+my $stdout;
+my $stderr;
+my $result = IPC::Run::run ['pg_validatebackup', '-q', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok($result, "-q succeeds: exit code 0");
+is($stdout, '', "-q succeeds: no stdout");
+is($stderr, '', "-q succeeds: no stderr");
+
+# Corrupt the PG_VERSION file.
+my $version_pathname = "$backup_path/PG_VERSION";
+my $version_contents = slurp_file($version_pathname);
+open(my $fh, '>', $version_pathname) || die "open $version_pathname: $!";
+print $fh 'q' x length($version_contents);
+close($fh);
+
+# Verify that pg_validatebackup -q now fails.
+command_fails_like(['pg_validatebackup', '-q', $backup_path ],
+ qr/checksum mismatch for file \"PG_VERSION\"/,
+ '-q checksum mismatch');
+
+# Since we didn't change the length of the file, validation should succeed
+# if we ignore checksums. Check that we get the right message, too.
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ '-s skips checksumming');
+
+# Validation should succeed if we ignore the problem file.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/backup successfully verified/,
+ '-i ignores problem file');
+
+# PG_VERSION is already corrupt; let's try also removing all of pg_xact.
+rmtree($backup_path . "/pg_xact");
+
+# We're ignoring the problem with PG_VERSION, but not the problem with
+# pg_xact, so validation should fail here.
+command_fails_like(['pg_validatebackup', '-i', 'PG_VERSION', $backup_path ],
+ qr/pg_xact.*is present in the manifest but not on disk/,
+ '-i does not ignore all problems');
+
+# If we use -i twice, we should be able to ignore all of the problems.
+command_like(['pg_validatebackup', '-i', 'PG_VERSION', '-i', 'pg_xact',
+ $backup_path ],
+ qr/backup successfully verified/,
+ 'multiple -i options work');
+
+# Verify that when -i is not used, both problems are reported.
+$result = IPC::Run::run ['pg_validatebackup', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "multiple problems: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "multiple problems: missing files reported");
+like($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "multiple problems: checksum mismatch reported");
+
+# Verify that when -e is used, only the problem detected first is reported.
+$result = IPC::Run::run ['pg_validatebackup', '-e', $backup_path ],
+ '>', \$stdout, '2>', \$stderr;
+ok(!$result, "-e reports 1 error: fails");
+like($stderr, qr/pg_xact.*is present in the manifest but not on disk/,
+ "-e reports 1 error: missing files reported");
+unlike($stderr, qr/checksum mismatch for file \"PG_VERSION\"/,
+ "-e reports 1 error: checksum mismatch not reported");
+
+# Test valid manifest with nonexistent backup directory.
+command_fails_like(['pg_validatebackup', '-m', "$backup_path/backup_manifest",
+ "$backup_path/fake" ],
+ qr/could not open directory/,
+ 'nonexistent backup directory');
diff --git a/src/bin/pg_validatebackup/t/005_bad_manifest.pl b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
new file mode 100644
index 0000000000..9c503600d2
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
@@ -0,0 +1,158 @@
+# Test the behavior of pg_validatebackup when the backup manifest has
+# problems.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 44;
+
+my $tempdir = TestLib::tempdir;
+
+test_bad_manifest('input string ended unexpectedly',
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
+{
+EOM
+
+test_parse_error('unexpected object end', <<EOM);
+{}
+EOM
+
+test_parse_error('unexpected array start', <<EOM);
+[]
+EOM
+
+test_parse_error('expected version indicator', <<EOM);
+{"not-expected": 1}
+EOM
+
+test_parse_error('unexpected manifest version', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": "phooey"}
+EOM
+
+test_parse_error('unexpected scalar', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": true}
+EOM
+
+test_parse_error('expected file list', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Oops": 1}
+EOM
+
+test_parse_error('unexpected object start', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": {}}
+EOM
+
+test_parse_error('missing pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [{}]}
+EOM
+
+test_parse_error('both pathname and encoded pathname', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Encoded-Path": "1234"}
+]}
+EOM
+
+test_parse_error('unexpected file field', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Oops": 1}
+]}
+EOM
+
+test_parse_error('missing size', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x"}
+]}
+EOM
+
+test_parse_error('file size is not an integer', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": "Oops"}
+]}
+EOM
+
+test_parse_error('unable to decode filename', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Encoded-Path": "123", "Size": 0}
+]}
+EOM
+
+test_fatal_error('duplicate pathname in backup manifest', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 0},
+ {"Path": "x", "Size": 0}
+]}
+EOM
+
+test_parse_error('checksum without algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum": "Oops"}
+]}
+EOM
+
+test_fatal_error('unrecognized checksum algorithm', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "Oops", "Checksum": "00"}
+]}
+EOM
+
+test_fatal_error('invalid checksum for file', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [
+ {"Path": "x", "Size": 100, "Checksum-Algorithm": "CRC32C", "Checksum": "0"}
+]}
+EOM
+
+test_parse_error('expected manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Oops": 1}
+EOM
+
+test_parse_error('expected at least 2 lines', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [], "Manifest-Checksum": null}
+EOM
+
+my $manifest_without_newline = <<EOM;
+{"PostgreSQL-Backup-Manifest-Version": 1,
+ "Files": [],
+ "Manifest-Checksum": null}
+EOM
+chomp($manifest_without_newline);
+test_parse_error('last line not newline-terminated',
+ $manifest_without_newline);
+
+test_fatal_error('invalid manifest checksum', <<EOM);
+{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
+ "Manifest-Checksum": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890-"}
+EOM
+
+sub test_parse_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/could not parse backup manifest: $test_name/,
+ $manifest_contents);
+}
+
+sub test_fatal_error
+{
+ my ($test_name, $manifest_contents) = @_;
+
+ test_bad_manifest($test_name,
+ qr/fatal: $test_name/,
+ $manifest_contents);
+}
+
+sub test_bad_manifest
+{
+ my ($test_name, $regexp, $manifest_contents) = @_;
+
+ open(my $fh, '>', "$tempdir/backup_manifest") || die "open: $!";
+ print $fh $manifest_contents;
+ close($fh);
+
+ command_fails_like(['pg_validatebackup', $tempdir], $regexp,
+ $test_name);
+}
diff --git a/src/bin/pg_validatebackup/t/006_encoding.pl b/src/bin/pg_validatebackup/t/006_encoding.pl
new file mode 100644
index 0000000000..5e3e7152a5
--- /dev/null
+++ b/src/bin/pg_validatebackup/t/006_encoding.pl
@@ -0,0 +1,27 @@
+# Verify that pg_validatebackup handles hex-encoded filenames correctly.
+
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 5;
+
+my $master = get_new_node('master');
+$master->init(allows_streaming => 1);
+$master->start;
+my $backup_path = $master->backup_dir . '/test_encoding';
+$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
+ '--manifest-force-encode' ],
+ "backup ok with forced hex encoding");
+
+my $manifest = slurp_file("$backup_path/backup_manifest");
+my $count_of_encoded_path_in_manifest =
+ (() = $manifest =~ /Encoded-Path/mig);
+cmp_ok($count_of_encoded_path_in_manifest, '>', 100,
+ "many paths are encoded in the manifest");
+
+command_like(['pg_validatebackup', '-s', $backup_path ],
+ qr/backup successfully verified/,
+ 'backup with forced encoding validated');
diff --git a/src/include/replication/basebackup.h b/src/include/replication/basebackup.h
index 07ed281bd6..d5b594c928 100644
--- a/src/include/replication/basebackup.h
+++ b/src/include/replication/basebackup.h
@@ -12,6 +12,7 @@
#ifndef _BASEBACKUP_H
#define _BASEBACKUP_H
+#include "lib/stringinfo.h"
#include "nodes/replnodes.h"
/*
@@ -29,8 +30,12 @@ typedef struct
int64 size;
} tablespaceinfo;
+struct manifest_info;
+typedef struct manifest_info manifest_info;
+
extern void SendBaseBackup(BaseBackupCmd *cmd);
-extern int64 sendTablespace(char *path, bool sizeonly);
+extern int64 sendTablespace(char *path, char *oid, bool sizeonly,
+ manifest_info *manifest);
#endif /* _BASEBACKUP_H */
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index fd4305e53f..40d81b87f0 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -38,6 +38,7 @@ extern bool log_replication_commands;
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
+extern void WalSndResourceCleanup(bool isCommit);
extern void WalSndSignals(void);
extern Size WalSndShmemSize(void);
extern void WalSndShmemInit(void);
--
2.17.2 (Apple Git-113)
[application/octet-stream] v16-0004-WIP-Store-WAL-ranges-in-manifest-and-validate-th.patch (33.4K, ../../CA+TgmoawEeE5qpFgj5Vy2zZGKzd3ZSEhGrD_JdPqPd2GB8u1Cw@mail.gmail.com/4-v16-0004-WIP-Store-WAL-ranges-in-manifest-and-validate-th.patch)
download | inline diff:
From a5f48cabe5e2de4b38b099c9ccbeb35c093ba4f8 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Tue, 31 Mar 2020 11:50:05 -0400
Subject: [PATCH v16 4/4] WIP: Store WAL ranges in manifest and validate them
w/pg_waldump.
---
src/backend/replication/basebackup.c | 109 ++++++-
src/bin/pg_validatebackup/parse_manifest.c | 270 ++++++++++++++----
src/bin/pg_validatebackup/parse_manifest.h | 5 +
src/bin/pg_validatebackup/pg_validatebackup.c | 183 ++++++++++--
.../pg_validatebackup/t/005_bad_manifest.pl | 9 +-
5 files changed, 492 insertions(+), 84 deletions(-)
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index deaa4f1c34..f56d2c97b5 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -16,6 +16,7 @@
#include <unistd.h>
#include <time.h>
+#include "access/timeline.h"
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
#include "common/checksum_helper.h"
@@ -99,6 +100,9 @@ static void AppendStringToManifest(manifest_info *manifest, char *s);
static void AddFileToManifest(manifest_info *manifest, const char *spcoid,
const char *pathname, size_t size, time_t mtime,
pg_checksum_context *checksum_ctx);
+static void AddWALInfoToManifest(manifest_info *manifest, XLogRecPtr startptr,
+ TimeLineID starttli, XLogRecPtr endptr,
+ TimeLineID endtli);
static void SendBackupManifest(manifest_info *manifest);
static void perform_base_backup(basebackup_options *opt);
static void parse_basebackup_options(List *options, basebackup_options *opt);
@@ -740,6 +744,8 @@ perform_base_backup(basebackup_options *opt)
pq_putemptymessage('c');
}
+ AddWALInfoToManifest(&manifest, startptr, starttli, endptr, endtli);
+
SendBackupManifest(&manifest);
SendXlogRecPtrResult(endptr, endtli);
@@ -1209,6 +1215,101 @@ AddFileToManifest(manifest_info *manifest, const char *spcoid,
pfree(buf.data);
}
+/*
+ * Add information about the WAL that will need to be replayed when restoring
+ * this backup to the manifest.
+ */
+static void
+AddWALInfoToManifest(manifest_info *manifest, XLogRecPtr startptr,
+ TimeLineID starttli, XLogRecPtr endptr, TimeLineID endtli)
+{
+ List *timelines = readTimeLineHistory(endtli);
+ ListCell *lc;
+ bool first_wal_range = true;
+ bool found_ending_tli = false;
+
+ /* If there is no buffile, then the user doesn't want a manifest. */
+ if (manifest->buffile == NULL)
+ return;
+
+ /* Terminate the list of files. */
+ AppendStringToManifest(manifest, "\n],\n");
+
+ /* Start a list of LSN ranges. */
+ AppendStringToManifest(manifest, "\"WAL-Ranges\": [\n");
+
+ foreach (lc, timelines)
+ {
+ TimeLineHistoryEntry *entry = lfirst(lc);
+ XLogRecPtr tl_endptr;
+
+ /*
+ * We only care about timelines that were active during the backup.
+ * Skip any that ended before the backup started. (Note that if
+ * entry->end is InvalidXLogRecPtr, it means that the timeline has not
+ * yet ended.)
+ */
+ if (!XLogRecPtrIsInvalid(entry->end) && entry->end < startptr)
+ continue;
+
+ /*
+ * Because the timeline history file lists older timelines before
+ * newer ones, the first timeline we encounter that is new enough to
+ * matter ought to match the starting timeline of the backup.
+ */
+ if (first_wal_range && starttli != entry->tli)
+ ereport(ERROR,
+ errmsg("start timeline %u does not match timeline history",
+ starttli));
+
+ if (!XLogRecPtrIsInvalid(entry->end))
+ tl_endptr = entry->end;
+ else
+ {
+ tl_endptr = endptr;
+
+ /*
+ * If we reach a TLI that has no end LSN, there can't be any more
+ * timelines in the history after this point, so we'd better have
+ * arrived at the expected ending TLI. If not, something's gone
+ * horribly wrong.
+ */
+ if (endtli != entry->tli)
+ ereport(ERROR,
+ errmsg("end timeline %u does not match timeline history",
+ endtli));
+ }
+
+ AppendToManifest(manifest,
+ "%s{ \"Timeline\": %u, \"Start-LSN\": \"%X/%X\", \"End-LSN\": \"%X/%X\" }",
+ first_wal_range ? "" : ",\n",
+ entry->tli,
+ (uint32) (startptr >> 32), (uint32) startptr,
+ (uint32) (tl_endptr >> 32), (uint32) tl_endptr);
+
+ startptr = entry->end;
+ if (endtli == entry->tli)
+ {
+ found_ending_tli = true;
+ break;
+ }
+
+ first_wal_range = false;
+ }
+
+ /*
+ * The last entry in the timeline history for the ending timeline should
+ * be the ending timeline itself. Verify that this is what we observed.
+ */
+ if (!found_ending_tli)
+ ereport(ERROR,
+ errmsg("ending timeline %u not found in timeline history",
+ endtli));
+
+ /* Terminate the list of WAL ranges. */
+ AppendStringToManifest(manifest, "\n],\n");
+}
+
/*
* Finalize the backup manifest, and send it to the client.
*/
@@ -1220,16 +1321,10 @@ SendBackupManifest(manifest_info *manifest)
char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
size_t manifest_bytes_done = 0;
- /*
- * If there is no buffile, then the user doesn't want a manifest, so
- * don't waste any time generating one.
- */
+ /* If there is no buffile, then the user doesn't want a manifest. */
if (manifest->buffile == NULL)
return;
- /* Terminate the list of files. */
- AppendStringToManifest(manifest, "],\n");
-
/*
* Append manifest checksum, so that the problems with the manifest itself
* can be detected.
diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
index e6b42adfda..461ac36b76 100644
--- a/src/bin/pg_validatebackup/parse_manifest.c
+++ b/src/bin/pg_validatebackup/parse_manifest.c
@@ -23,14 +23,16 @@ typedef enum
{
JM_EXPECT_TOPLEVEL_START,
JM_EXPECT_TOPLEVEL_END,
- JM_EXPECT_VERSION_FIELD,
+ JM_EXPECT_TOPLEVEL_FIELD,
JM_EXPECT_VERSION_VALUE,
- JM_EXPECT_FILES_FIELD,
- JM_EXPECT_FILES_ARRAY_START,
- JM_EXPECT_FILES_ARRAY_NEXT,
+ JM_EXPECT_FILES_START,
+ JM_EXPECT_FILES_NEXT,
JM_EXPECT_THIS_FILE_FIELD,
JM_EXPECT_THIS_FILE_VALUE,
- JM_EXPECT_MANIFEST_CHECKSUM_FIELD,
+ JM_EXPECT_WAL_RANGES_START,
+ JM_EXPECT_WAL_RANGES_NEXT,
+ JM_EXPECT_THIS_WAL_RANGE_FIELD,
+ JM_EXPECT_THIS_WAL_RANGE_VALUE,
JM_EXPECT_MANIFEST_CHECKSUM_VALUE,
JM_EXPECT_EOF
} JsonManifestSemanticState;
@@ -48,6 +50,16 @@ typedef enum
JMFF_CHECKSUM
} JsonManifestFileField;
+/*
+ * Possible fields for one file as described by the manifest.
+ */
+typedef enum
+{
+ JMWRF_TIMELINE,
+ JMWRF_START_LSN,
+ JMWRF_END_LSN
+} JsonManifestWALRangeField;
+
/*
* Internal state used while decoding the JSON-format backup manifest.
*/
@@ -55,13 +67,24 @@ typedef struct
{
JsonManifestParseContext *context;
JsonManifestSemanticState state;
- JsonManifestFileField field;
+
+ /* These fields are used for parsing objects in the list of files. */
+ JsonManifestFileField file_field;
char *pathname;
char *encoded_pathname;
char *size;
char *algorithm;
pg_checksum_type checksum_algorithm;
char *checksum;
+
+ /* These fields are used for parsing objects in the list of WAL ranges. */
+ JsonManifestWALRangeField wal_range_field;
+ char *timeline;
+ char *start_lsn;
+ char *end_lsn;
+
+ /* Miscellaneous other stuff. */
+ bool saw_version_field;
char *manifest_checksum;
} JsonManifestParseState;
@@ -74,6 +97,7 @@ static void json_manifest_object_field_start(void *state, char *fname,
static void json_manifest_scalar(void *state, char *token,
JsonTokenType tokentype);
static void json_manifest_finalize_file(JsonManifestParseState *parse);
+static void json_manifest_finalize_wal_range(JsonManifestParseState *parse);
static void verify_manifest_checksum(JsonManifestParseState *parse,
char *buffer, size_t size);
static void json_manifest_parse_failure(JsonManifestParseContext *context,
@@ -81,6 +105,7 @@ static void json_manifest_parse_failure(JsonManifestParseContext *context,
static int hexdecode_char(char c);
static bool hexdecode_string(uint8 *result, char *input, int nbytes);
+static bool parse_xlogrecptr(XLogRecPtr *result, char *input);
/*
* Main entrypoint to parse a JSON-format backup manifest.
@@ -100,8 +125,9 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
JsonManifestParseState parse;
/* Set up our private parsing context. */
- parse.state = JM_EXPECT_TOPLEVEL_START;
parse.context = context;
+ parse.state = JM_EXPECT_TOPLEVEL_START;
+ parse.saw_version_field = false;
/* Create a JSON lexing context. */
lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
@@ -132,11 +158,9 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
/*
* Invoked at the start of each object in the JSON document.
*
- * The document as a whole is expected to be an object with three keys
- * (PostgreSQL-Backup-Manifest-Version, Files, Manifest-Checksum) and each
- * file is expected to be an object with various keys (Path, Size, etc.).
- * If we're not at the beginning of either the toplevel object or the object
- * for a particular file, it's an error.
+ * The document as a whole is expected to be an object; each file and each
+ * WAL range is also expected to be an object. If we're anywhere else in the
+ * document, it's an error.
*/
static void
json_manifest_object_start(void *state)
@@ -146,9 +170,9 @@ json_manifest_object_start(void *state)
switch (parse->state)
{
case JM_EXPECT_TOPLEVEL_START:
- parse->state = JM_EXPECT_VERSION_FIELD;
+ parse->state = JM_EXPECT_TOPLEVEL_FIELD;
break;
- case JM_EXPECT_FILES_ARRAY_NEXT:
+ case JM_EXPECT_FILES_NEXT:
parse->state = JM_EXPECT_THIS_FILE_FIELD;
parse->pathname = NULL;
parse->encoded_pathname = NULL;
@@ -156,6 +180,12 @@ json_manifest_object_start(void *state)
parse->algorithm = NULL;
parse->checksum = NULL;
break;
+ case JM_EXPECT_WAL_RANGES_NEXT:
+ parse->state = JM_EXPECT_THIS_WAL_RANGE_FIELD;
+ parse->timeline = NULL;
+ parse->start_lsn = NULL;
+ parse->end_lsn = NULL;
+ break;
default:
json_manifest_parse_failure(parse->context,
"unexpected object start");
@@ -168,8 +198,8 @@ json_manifest_object_start(void *state)
*
* The possible cases here are the same as for json_manifest_object_start.
* There's nothing special to do at the end of the document, but when we
- * reach the end of an object representing a particular file, we must call
- * json_manifest_finalize_file() to save the associated details.
+ * reach the end of an object representing a particular file or WAL range,
+ * we must call json_manifest_finalize_file() to save the associated details.
*/
static void
json_manifest_object_end(void *state)
@@ -183,7 +213,11 @@ json_manifest_object_end(void *state)
break;
case JM_EXPECT_THIS_FILE_FIELD:
json_manifest_finalize_file(parse);
- parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ parse->state = JM_EXPECT_FILES_NEXT;
+ break;
+ case JM_EXPECT_THIS_WAL_RANGE_FIELD:
+ json_manifest_finalize_wal_range(parse);
+ parse->state = JM_EXPECT_WAL_RANGES_NEXT;
break;
default:
json_manifest_parse_failure(parse->context,
@@ -196,7 +230,8 @@ json_manifest_object_end(void *state)
* Invoked at the start of each array in the JSON document.
*
* Within the toplevel object, the value associated with the "Files" key
- * should be an array. No other arrays are expected.
+ * should be an array. Similarly for the "WAL-Ranges" key. No other arrays
+ * are expected.
*/
static void
json_manifest_array_start(void *state)
@@ -205,8 +240,11 @@ json_manifest_array_start(void *state)
switch (parse->state)
{
- case JM_EXPECT_FILES_ARRAY_START:
- parse->state = JM_EXPECT_FILES_ARRAY_NEXT;
+ case JM_EXPECT_FILES_START:
+ parse->state = JM_EXPECT_FILES_NEXT;
+ break;
+ case JM_EXPECT_WAL_RANGES_START:
+ parse->state = JM_EXPECT_WAL_RANGES_NEXT;
break;
default:
json_manifest_parse_failure(parse->context,
@@ -218,8 +256,7 @@ json_manifest_array_start(void *state)
/*
* Invoked at the end of each array in the JSON document.
*
- * Just like json_manifest_array_start, there's only one expected case
- * here.
+ * The cases here are analogous to those in json_manifest_array_start.
*/
static void
json_manifest_array_end(void *state)
@@ -228,8 +265,9 @@ json_manifest_array_end(void *state)
switch (parse->state)
{
- case JM_EXPECT_FILES_ARRAY_NEXT:
- parse->state = JM_EXPECT_MANIFEST_CHECKSUM_FIELD;
+ case JM_EXPECT_FILES_NEXT:
+ case JM_EXPECT_WAL_RANGES_NEXT:
+ parse->state = JM_EXPECT_TOPLEVEL_FIELD;
break;
default:
json_manifest_parse_failure(parse->context,
@@ -248,46 +286,82 @@ json_manifest_object_field_start(void *state, char *fname, bool isnull)
switch (parse->state)
{
- case JM_EXPECT_VERSION_FIELD:
- /* Inside toplevel object, expecting version indicator. */
- if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
- json_manifest_parse_failure(parse->context,
- "expected version indicator");
- parse->state = JM_EXPECT_VERSION_VALUE;
- break;
- case JM_EXPECT_FILES_FIELD:
- /* Inside toplevel object, expecting "Files" next. */
- if (strcmp(fname, "Files") != 0)
- json_manifest_parse_failure(parse->context,
- "expected file list");
- parse->state = JM_EXPECT_FILES_ARRAY_START;
+ case JM_EXPECT_TOPLEVEL_FIELD:
+ /*
+ * Inside toplevel object. The version indicator should always
+ * be the first field.
+ */
+ if (!parse->saw_version_field)
+ {
+ if (strcmp(fname, "PostgreSQL-Backup-Manifest-Version") != 0)
+ json_manifest_parse_failure(parse->context,
+ "expected version indicator");
+ parse->state = JM_EXPECT_VERSION_VALUE;
+ parse->saw_version_field = true;
+ break;
+ }
+
+ /* Is this the list of files? */
+ if (strcmp(fname, "Files") == 0)
+ {
+ parse->state = JM_EXPECT_FILES_START;
+ break;
+ }
+
+ /* Is this the list of WAL ranges? */
+ if (strcmp(fname, "WAL-Ranges") == 0)
+ {
+ parse->state = JM_EXPECT_WAL_RANGES_START;
+ break;
+ }
+
+ /* Is this the manifest checksum? */
+ if (strcmp(fname, "Manifest-Checksum") == 0)
+ {
+ parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ break;
+ }
+
+ /* It's not a field we recognize. */
+ fprintf(stderr, "fname = %s\n", fname);
+ json_manifest_parse_failure(parse->context,
+ "unknown toplevel field");
break;
+
case JM_EXPECT_THIS_FILE_FIELD:
/* Inside object for one file; which key have we got? */
if (strcmp(fname, "Path") == 0)
- parse->field = JMFF_PATH;
+ parse->file_field = JMFF_PATH;
else if (strcmp(fname, "Encoded-Path") == 0)
- parse->field = JMFF_ENCODED_PATH;
+ parse->file_field = JMFF_ENCODED_PATH;
else if (strcmp(fname, "Size") == 0)
- parse->field = JMFF_SIZE;
+ parse->file_field = JMFF_SIZE;
else if (strcmp(fname, "Last-Modified") == 0)
- parse->field = JMFF_LAST_MODIFIED;
+ parse->file_field = JMFF_LAST_MODIFIED;
else if (strcmp(fname, "Checksum-Algorithm") == 0)
- parse->field = JMFF_CHECKSUM_ALGORITHM;
+ parse->file_field = JMFF_CHECKSUM_ALGORITHM;
else if (strcmp(fname, "Checksum") == 0)
- parse->field = JMFF_CHECKSUM;
+ parse->file_field = JMFF_CHECKSUM;
else
json_manifest_parse_failure(parse->context,
"unexpected file field");
parse->state = JM_EXPECT_THIS_FILE_VALUE;
break;
- case JM_EXPECT_MANIFEST_CHECKSUM_FIELD:
- /* Inside toplevel object, expecting "Manifest-Checksum" next. */
- if (strcmp(fname, "Manifest-Checksum") != 0)
+
+ case JM_EXPECT_THIS_WAL_RANGE_FIELD:
+ /* Inside object for one file; which key have we got? */
+ if (strcmp(fname, "Timeline") == 0)
+ parse->wal_range_field = JMWRF_TIMELINE;
+ else if (strcmp(fname, "Start-LSN") == 0)
+ parse->wal_range_field = JMWRF_START_LSN;
+ else if (strcmp(fname, "End-LSN") == 0)
+ parse->wal_range_field = JMWRF_END_LSN;
+ else
json_manifest_parse_failure(parse->context,
- "expected manifest checksum");
- parse->state = JM_EXPECT_MANIFEST_CHECKSUM_VALUE;
+ "unexpected wal range field");
+ parse->state = JM_EXPECT_THIS_WAL_RANGE_VALUE;
break;
+
default:
json_manifest_parse_failure(parse->context,
"unexpected object field");
@@ -300,9 +374,9 @@ json_manifest_object_field_start(void *state, char *fname, bool isnull)
*
* Object field names don't reach this code; those are handled by
* json_manifest_object_field_start. When we're inside of the object for
- * a particular file, that function will have noticed the name of the field,
- * and we'll get the corresponding value here. When we're in the toplevel
- * object, the parse state itself tells us which field this is.
+ * a particular file or WAL range, that function will have noticed the name
+ * of the field, and we'll get the corresponding value here. When we're in
+ * the toplevel object, the parse state itself tells us which field this is.
*
* In all cases except for PostgreSQL-Backup-Manifest-Version, which we
* can just check on the spot, the goal here is just to save the value in
@@ -321,10 +395,11 @@ json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
if (strcmp(token, "1") != 0)
json_manifest_parse_failure(parse->context,
"unexpected manifest version");
- parse->state = JM_EXPECT_FILES_FIELD;
+ parse->state = JM_EXPECT_TOPLEVEL_FIELD;
break;
+
case JM_EXPECT_THIS_FILE_VALUE:
- switch (parse->field)
+ switch (parse->file_field)
{
case JMFF_PATH:
parse->pathname = token;
@@ -347,10 +422,28 @@ json_manifest_scalar(void *state, char *token, JsonTokenType tokentype)
}
parse->state = JM_EXPECT_THIS_FILE_FIELD;
break;
+
+ case JM_EXPECT_THIS_WAL_RANGE_VALUE:
+ switch (parse->wal_range_field)
+ {
+ case JMWRF_TIMELINE:
+ parse->timeline = token;
+ break;
+ case JMWRF_START_LSN:
+ parse->start_lsn = token;
+ break;
+ case JMWRF_END_LSN:
+ parse->end_lsn = token;
+ break;
+ }
+ parse->state = JM_EXPECT_THIS_WAL_RANGE_FIELD;
+ break;
+
case JM_EXPECT_MANIFEST_CHECKSUM_VALUE:
parse->state = JM_EXPECT_TOPLEVEL_END;
parse->manifest_checksum = token;
break;
+
default:
json_manifest_parse_failure(parse->context, "unexpected scalar");
break;
@@ -459,6 +552,62 @@ json_manifest_finalize_file(JsonManifestParseState *parse)
}
}
+/*
+ * Do additional parsing and sanity-checking of the details gathered for one
+ * WAL range, and invoke the per-WAL-range callback so that the caller gets
+ * those details. This happens for each WAL range when the corresponding JSON
+ * object is completely parsed.
+ */
+static void
+json_manifest_finalize_wal_range(JsonManifestParseState *parse)
+{
+ JsonManifestParseContext *context = parse->context;
+ TimeLineID tli;
+ XLogRecPtr start_lsn,
+ end_lsn;
+ char *ep;
+
+ /* Make sure all fields are present. */
+ if (parse->timeline == NULL)
+ json_manifest_parse_failure(parse->context, "missing timeline");
+ if (parse->start_lsn == NULL)
+ json_manifest_parse_failure(parse->context, "missing start LSN");
+ if (parse->end_lsn == NULL)
+ json_manifest_parse_failure(parse->context, "missing end LSN");
+
+ /* Parse timeline. */
+ tli = strtoul(parse->timeline, &ep, 10);
+ if (*ep)
+ json_manifest_parse_failure(parse->context,
+ "timeline is not an integer");
+ if (!parse_xlogrecptr(&start_lsn, parse->start_lsn))
+ json_manifest_parse_failure(parse->context,
+ "unable to parse start LSN");
+ if (!parse_xlogrecptr(&end_lsn, parse->end_lsn))
+ json_manifest_parse_failure(parse->context,
+ "unable to parse end LSN");
+
+ /* Invoke the callback with the details we've gathered. */
+ context->perwalrange_cb(context, tli, start_lsn, end_lsn);
+
+ /* Free memory we no longer need. */
+ if (parse->timeline != NULL)
+ {
+ pfree(parse->timeline);
+ parse->timeline = NULL;
+ }
+ if (parse->start_lsn != NULL)
+ {
+ pfree(parse->start_lsn);
+ parse->start_lsn = NULL;
+ }
+ if (parse->end_lsn != NULL)
+ {
+ pfree(parse->end_lsn);
+ parse->end_lsn = NULL;
+ }
+}
+
/*
* Verify that the manifest checksum is correct.
*
@@ -574,3 +723,18 @@ hexdecode_string(uint8 *result, char *input, int nbytes)
return true;
}
+
+/*
+ * Parse an XLogRecPtr expressed using the usual string format.
+ */
+static bool
+parse_xlogrecptr(XLogRecPtr *result, char *input)
+{
+ uint32 hi;
+ uint32 lo;
+
+ if (sscanf(input, "%X/%X", &hi, &lo) != 2)
+ return false;
+ *result = ((uint64) hi) << 32 | lo;
+ return true;
+}
diff --git a/src/bin/pg_validatebackup/parse_manifest.h b/src/bin/pg_validatebackup/parse_manifest.h
index 25d140f72f..f0a4fac36b 100644
--- a/src/bin/pg_validatebackup/parse_manifest.h
+++ b/src/bin/pg_validatebackup/parse_manifest.h
@@ -14,6 +14,7 @@
#ifndef PARSE_MANIFEST_H
#define PARSE_MANIFEST_H
+#include "access/xlogdefs.h"
#include "common/checksum_helper.h"
#include "mb/pg_wchar.h"
@@ -24,6 +25,9 @@ typedef void (*json_manifest_perfile_callback)(JsonManifestParseContext *,
char *pathname,
size_t size, pg_checksum_type checksum_type,
int checksum_length, uint8 *checksum_payload);
+typedef void (*json_manifest_perwalrange_callback)(JsonManifestParseContext *,
+ TimeLineID tli,
+ XLogRecPtr start_lsn, XLogRecPtr end_lsn);
typedef void (*json_manifest_error_callback)(JsonManifestParseContext *,
char *fmt, ...) pg_attribute_printf(2, 3);
@@ -31,6 +35,7 @@ struct JsonManifestParseContext
{
void *private_data;
json_manifest_perfile_callback perfile_cb;
+ json_manifest_perwalrange_callback perwalrange_cb;
json_manifest_error_callback error_cb;
};
diff --git a/src/bin/pg_validatebackup/pg_validatebackup.c b/src/bin/pg_validatebackup/pg_validatebackup.c
index eb1473d9d0..2c9d06a3a1 100644
--- a/src/bin/pg_validatebackup/pg_validatebackup.c
+++ b/src/bin/pg_validatebackup/pg_validatebackup.c
@@ -43,8 +43,8 @@
#define READ_CHUNK_SIZE 4096
/*
- * Information about each file described by the manifest file is parsed to
- * produce an object like this.
+ * Each file described by the manifest file is parsed to produce an object
+ * like this.
*/
typedef struct manifestfile
{
@@ -75,6 +75,29 @@ static uint32 hash_string_pointer(char *s);
#define SH_DEFINE
#include "lib/simplehash.h"
+/*
+ * Each WAL range described by the manifest file is parsed to produce an
+ * object like this.
+ */
+typedef struct manifest_wal_range
+{
+ TimeLineID tli;
+ XLogRecPtr start_lsn;
+ XLogRecPtr end_lsn;
+ struct manifest_wal_range *next;
+ struct manifest_wal_range *prev;
+} manifest_wal_range;
+
+/*
+ * Details we need in callbacks that occur while parsing a backup manifest.
+ */
+typedef struct parser_context
+{
+ manifestfiles_hash *ht;
+ manifest_wal_range *first_wal_range;
+ manifest_wal_range *last_wal_range;
+} parser_context;
+
/*
* All of the context information we need while checking a backup manifest.
*/
@@ -87,13 +110,18 @@ typedef struct validator_context
bool saw_any_error;
} validator_context;
-static manifestfiles_hash *parse_manifest_file(char *manifest_path);
+static void parse_manifest_file(char *manifest_path, manifestfiles_hash **ht_p,
+ manifest_wal_range **first_wal_range_p);
static void record_manifest_details_for_file(JsonManifestParseContext *context,
char *pathname, size_t size,
pg_checksum_type checksum_type,
int checksum_length,
uint8 *checksum_payload);
+static void record_manifest_details_for_wal_range(JsonManifestParseContext *context,
+ TimeLineID tli,
+ XLogRecPtr start_lsn,
+ XLogRecPtr end_lsn);
static void report_manifest_error(JsonManifestParseContext *context,
char *fmt, ...)
pg_attribute_printf(2, 3) pg_attribute_noreturn();
@@ -106,6 +134,10 @@ static void report_extra_backup_files(validator_context *context);
static void validate_backup_checksums(validator_context *context);
static void validate_file_checksum(validator_context *context,
manifestfile *tabent, char *pathname);
+static void parse_required_wal(validator_context *context,
+ char *pg_waldump_path,
+ char *wal_directory,
+ manifest_wal_range *first_wal_range);
static void report_backup_error(validator_context *context,
const char *pg_restrict fmt,...)
@@ -128,16 +160,23 @@ main(int argc, char **argv)
{"exit-on-error", no_argument, NULL, 'e'},
{"ignore", required_argument, NULL, 'i'},
{"manifest-path", required_argument, NULL, 'm'},
+ {"no-parse-wal", no_argument, NULL, 'n'},
+ {"print-parse-wal", no_argument, NULL, 'p'},
{"quiet", no_argument, NULL, 'q'},
{"skip-checksums", no_argument, NULL, 's'},
+ {"wal-directory", required_argument, NULL, 'w'},
{NULL, 0, NULL, 0}
};
int c;
validator_context context;
+ manifest_wal_range *first_wal_range;
char *manifest_path = NULL;
+ bool no_parse_wal = false;
bool quiet = false;
bool skip_checksums = false;
+ char *wal_directory = NULL;
+ char *pg_waldump_path = NULL;
pg_logging_init(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_validatebackup"));
@@ -167,7 +206,7 @@ main(int argc, char **argv)
*
* Ignore the pg_wal directory, because those files are not included in
* the backup manifest either, since they are fetched separately from the
- * backup itself.
+ * backup itself, and validated via a separate mechanism.
*
* Ignore postgresql.auto.conf, recovery.signal, and standby.signal,
* because we expect that those files may sometimes be created or changed
@@ -180,7 +219,7 @@ main(int argc, char **argv)
simple_string_list_append(&context.ignore_list, "recovery.signal");
simple_string_list_append(&context.ignore_list, "standby.signal");
- while ((c = getopt_long(argc, argv, "ei:m:qs", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "ei:m:nqsw", long_options, NULL)) != -1)
{
switch (c)
{
@@ -199,12 +238,19 @@ main(int argc, char **argv)
manifest_path = pstrdup(optarg);
canonicalize_path(manifest_path);
break;
+ case 'n':
+ no_parse_wal = true;
+ break;
case 'q':
quiet = true;
break;
case 's':
skip_checksums = true;
break;
+ case 'w':
+ wal_directory = pstrdup(optarg);
+ canonicalize_path(wal_directory);
+ break;
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
@@ -233,17 +279,49 @@ main(int argc, char **argv)
exit(1);
}
+ /* Unless --no-parse-wal was specified, we will need pg_waldump. */
+ if (!no_parse_wal)
+ {
+ int ret;
+
+ pg_waldump_path = pg_malloc(MAXPGPATH);
+ ret = find_other_exec(argv[0], "pg_waldump",
+ "pg_waldump (PostgreSQL) " PG_VERSION "\n",
+ pg_waldump_path);
+ if (ret < 0)
+ {
+ char full_path[MAXPGPATH];
+
+ if (find_my_exec(argv[0], full_path) < 0)
+ strlcpy(full_path, progname, sizeof(full_path));
+ if (ret == -1)
+ pg_log_fatal("The program \"%s\" is needed by %s but was\n"
+ "not found in the same directory as \"%s\".\n"
+ "Check your installation.",
+ "pg_waldump", "pg_validatebackup", full_path);
+ else
+ pg_log_fatal("The program \"%s\" was found by \"%s\" but was\n"
+ "not the same version as %s.\n"
+ "Check your installation.",
+ "pg_waldump", full_path, "pg_validatebackup");
+ }
+ }
+
/* By default, look for the manifest in the backup directory. */
if (manifest_path == NULL)
manifest_path = psprintf("%s/backup_manifest",
context.backup_directory);
+ /* By default, look for the WAL in the backup directory, too. */
+ if (wal_directory == NULL)
+ wal_directory = psprintf("%s/pg_wal", context.backup_directory);
+
/*
* Try to read the manifest. We treat any errors encountered while parsing
* the manifest as fatal; there doesn't seem to be much point in trying to
* validate the backup directory against a corrupted manifest.
*/
- context.ht = parse_manifest_file(manifest_path);
+ parse_manifest_file(manifest_path, &context.ht, &first_wal_range);
/*
* Now scan the files in the backup directory. At this stage, we verify
@@ -261,12 +339,20 @@ main(int argc, char **argv)
report_extra_backup_files(&context);
/*
- * Finally, do the expensive work of verifying file checksums, unless we
- * were told to skip it.
+ * Now do the expensive work of verifying file checksums, unless we were
+ * told to skip it.
*/
if (!skip_checksums)
validate_backup_checksums(&context);
+ /*
+ * Try to parse the required ranges of WAL records, unless we were told
+ * not to do so.
+ */
+ if (!no_parse_wal)
+ parse_required_wal(&context, pg_waldump_path,
+ wal_directory, first_wal_range);
+
/*
* If everything looks OK, tell the user this, unless we were asked to
* work quietly.
@@ -278,11 +364,13 @@ main(int argc, char **argv)
}
/*
- * Parse a manifest file and construct a hash table with information about
- * all the files it mentions.
+ * Parse a manifest file. Construct a hash table with information about
+ * all the files it mentions, and a linked list of all the WAL ranges it
+ * mentions.
*/
-static manifestfiles_hash *
-parse_manifest_file(char *manifest_path)
+static void
+parse_manifest_file(char *manifest_path, manifestfiles_hash **ht_p,
+ manifest_wal_range **first_wal_range_p)
{
int fd;
struct stat statbuf;
@@ -291,6 +379,7 @@ parse_manifest_file(char *manifest_path)
manifestfiles_hash *ht;
char *buffer;
int rc;
+ parser_context private_context;
JsonManifestParseContext context;
/* Open the manifest file. */
@@ -329,17 +418,22 @@ parse_manifest_file(char *manifest_path)
/* Close the manifest file. */
close(fd);
- /* Parse the manifest as JSON. */
- context.private_data = ht;
+ /* Parse the manifest. */
+ private_context.ht = ht;
+ private_context.first_wal_range = NULL;
+ private_context.last_wal_range = NULL;
+ context.private_data = &private_context;
context.perfile_cb = record_manifest_details_for_file;
+ context.perwalrange_cb = record_manifest_details_for_wal_range;
context.error_cb = report_manifest_error;
json_parse_manifest(&context, buffer, statbuf.st_size);
/* Done with the buffer. */
pfree(buffer);
- /* Return the hash table we constructed. */
- return ht;
+ /* Return the file hash table and WAL range list we constructed. */
+ *ht_p = ht;
+ *first_wal_range_p = private_context.first_wal_range;
}
/*
@@ -369,7 +463,8 @@ record_manifest_details_for_file(JsonManifestParseContext *context,
pg_checksum_type checksum_type,
int checksum_length, uint8 *checksum_payload)
{
- manifestfiles_hash *ht = context->private_data;
+ parser_context *pcxt = context->private_data;
+ manifestfiles_hash *ht = pcxt->ht;
manifestfile *tabent;
bool found;
@@ -388,6 +483,31 @@ record_manifest_details_for_file(JsonManifestParseContext *context,
tabent->bad = false;
}
+/*
+ * Record details extracted from the backup manifest for one WAL range.
+ */
+static void
+record_manifest_details_for_wal_range(JsonManifestParseContext *context,
+ TimeLineID tli,
+ XLogRecPtr start_lsn, XLogRecPtr end_lsn)
+{
+ parser_context *pcxt = context->private_data;
+ manifest_wal_range *range;
+
+ /* Allocate and initialize a struct describing this WAL range. */
+ range = palloc(sizeof(manifest_wal_range));
+ range->tli = tli;
+ range->start_lsn = start_lsn;
+ range->end_lsn = end_lsn;
+ range->prev = pcxt->last_wal_range;
+ range->next = NULL;
+
+ /* Add it to the list. */
+ if (pcxt->first_wal_range == NULL)
+ pcxt->first_wal_range = range;
+ pcxt->last_wal_range = range;
+}
+
/*
* Validate one directory.
*
@@ -641,6 +761,35 @@ validate_file_checksum(validator_context *context, manifestfile *tabent,
relpath);
}
+/*
+ * Attempt to parse the WAL files required to restore from backup using
+ * pg_waldump.
+ */
+static void
+parse_required_wal(validator_context *context, char *pg_waldump_path,
+ char *wal_directory, manifest_wal_range *first_wal_range)
+{
+ manifest_wal_range *this_wal_range = first_wal_range;
+
+ while (this_wal_range != NULL)
+ {
+ char *pg_waldump_cmd;
+
+ pg_waldump_cmd = psprintf("\"%s\" --quiet --path=\"%s\" --timeline=%u --start=%X/%X --end=%X/%X\n",
+ pg_waldump_path, wal_directory, this_wal_range->tli,
+ (uint32) (this_wal_range->start_lsn >> 32),
+ (uint32) this_wal_range->start_lsn,
+ (uint32) (this_wal_range->end_lsn >> 32),
+ (uint32) this_wal_range->end_lsn);
+ if (system(pg_waldump_cmd) != 0)
+ report_backup_error(context,
+ "WAL parsing failed for timeline %u",
+ this_wal_range->tli);
+
+ this_wal_range = this_wal_range->next;
+ }
+}
+
/*
* Report a problem with the backup.
*
diff --git a/src/bin/pg_validatebackup/t/005_bad_manifest.pl b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
index 9c503600d2..23c2f8338c 100644
--- a/src/bin/pg_validatebackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_validatebackup/t/005_bad_manifest.pl
@@ -7,7 +7,7 @@ use Cwd;
use Config;
use PostgresNode;
use TestLib;
-use Test::More tests => 44;
+use Test::More tests => 42;
my $tempdir = TestLib::tempdir;
@@ -37,7 +37,7 @@ test_parse_error('unexpected scalar', <<EOM);
{"PostgreSQL-Backup-Manifest-Version": 1, "Files": true}
EOM
-test_parse_error('expected file list', <<EOM);
+test_parse_error('unknown toplevel field', <<EOM);
{"PostgreSQL-Backup-Manifest-Version": 1, "Oops": 1}
EOM
@@ -104,11 +104,6 @@ test_fatal_error('invalid checksum for file', <<EOM);
]}
EOM
-test_parse_error('expected manifest checksum', <<EOM);
-{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [],
- "Oops": 1}
-EOM
-
test_parse_error('expected at least 2 lines', <<EOM);
{"PostgreSQL-Backup-Manifest-Version": 1, "Files": [], "Manifest-Checksum": null}
EOM
--
2.17.2 (Apple Git-113)
[application/octet-stream] v16-0003-pg_waldump-Add-quiet-option.patch (2.4K, ../../CA+TgmoawEeE5qpFgj5Vy2zZGKzd3ZSEhGrD_JdPqPd2GB8u1Cw@mail.gmail.com/5-v16-0003-pg_waldump-Add-quiet-option.patch)
download | inline diff:
From 41c255a4e57445eb8818d9635f982d665c2c4698 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Tue, 31 Mar 2020 08:13:25 -0400
Subject: [PATCH v16 3/4] pg_waldump: Add --quiet option.
Andres Freund and Robert Haas
---
src/bin/pg_waldump/pg_waldump.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c
index 279acfa044..14dca65beb 100644
--- a/src/bin/pg_waldump/pg_waldump.c
+++ b/src/bin/pg_waldump/pg_waldump.c
@@ -40,6 +40,7 @@ typedef struct XLogDumpPrivate
typedef struct XLogDumpConfig
{
/* display options */
+ bool quiet;
bool bkp_details;
int stop_after_records;
int already_displayed_records;
@@ -755,6 +756,7 @@ main(int argc, char **argv)
{"help", no_argument, NULL, '?'},
{"limit", required_argument, NULL, 'n'},
{"path", required_argument, NULL, 'p'},
+ {"quiet", no_argument, NULL, 'q'},
{"rmgr", required_argument, NULL, 'r'},
{"start", required_argument, NULL, 's'},
{"timeline", required_argument, NULL, 't'},
@@ -794,6 +796,7 @@ main(int argc, char **argv)
private.endptr = InvalidXLogRecPtr;
private.endptr_reached = false;
+ config.quiet = false;
config.bkp_details = false;
config.stop_after_records = -1;
config.already_displayed_records = 0;
@@ -810,7 +813,7 @@ main(int argc, char **argv)
goto bad_argument;
}
- while ((option = getopt_long(argc, argv, "be:fn:p:r:s:t:x:z",
+ while ((option = getopt_long(argc, argv, "be:fn:p:qr:s:t:x:z",
long_options, &optindex)) != -1)
{
switch (option)
@@ -840,6 +843,9 @@ main(int argc, char **argv)
case 'p':
waldir = pg_strdup(optarg);
break;
+ case 'q':
+ config.quiet = true;
+ break;
case 'r':
{
int i;
@@ -1075,11 +1081,14 @@ main(int argc, char **argv)
config.filter_by_xid != record->xl_xid)
continue;
- /* process the record */
- if (config.stats == true)
- XLogDumpCountRecord(&config, &stats, xlogreader_state);
- else
- XLogDumpDisplayRecord(&config, xlogreader_state);
+ /* perform any per-record work */
+ if (!config.quiet)
+ {
+ if (config.stats == true)
+ XLogDumpCountRecord(&config, &stats, xlogreader_state);
+ else
+ XLogDumpDisplayRecord(&config, xlogreader_state);
+ }
/* check whether we printed enough */
config.already_displayed_records++;
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:07 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:23 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 02:08 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 18:35 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 18:59 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-31 18:10 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-31 22:50 ` Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-31 22:50 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Noah Misch <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-31 14:10:34 -0400, Robert Haas wrote:
> I made an attempt to implement this.
Awesome!
> In the attached patch set, 0001 I'm going to work on those things. I
> would appreciate *very timely* feedback on anything people do or do
> not like about this, because I want to commit this patch set by the
> end of the work week and that isn't very far away. I would also
> appreciate if people would bear in mind the principle that half a loaf
> is better than none, and further improvements can be made in future
> releases.
>
> As part of my light testing, I tried promoting a standby that was
> running pg_basebackup, and found that pg_basebackup failed like this:
>
> pg_basebackup: error: could not get COPY data stream: ERROR: the
> standby was promoted during online backup
> HINT: This means that the backup being taken is corrupt and should
> not be used. Try taking another online backup.
> pg_basebackup: removing data directory "/Users/rhaas/pgslave2"
>
> My first thought was that this error message is hard to reconcile with
> this comment:
>
> /*
> * Send timeline history files too. Only the latest timeline history
> * file is required for recovery, and even that only if there happens
> * to be a timeline switch in the first WAL segment that contains the
> * checkpoint record, or if we're taking a base backup from a standby
> * server and the target timeline changes while the backup is taken.
> * But they are small and highly useful for debugging purposes, so
> * better include them all, always.
> */
>
> But then it occurred to me that this might be a cascading standby.
Yea. The check just prevents the walsender's database from being
promoted:
/*
* Check if the postmaster has signaled us to exit, and abort with an
* error in that case. The error handler further up will call
* do_pg_abort_backup() for us. Also check that if the backup was
* started while still in recovery, the server wasn't promoted.
* do_pg_stop_backup() will check that too, but it's better to stop
* the backup early than continue to the end and fail there.
*/
CHECK_FOR_INTERRUPTS();
if (RecoveryInProgress() != backup_started_in_recovery)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("the standby was promoted during online backup"),
errhint("This means that the backup being taken is corrupt "
"and should not be used. "
"Try taking another online backup.")));
and
if (strcmp(backupfrom, "standby") == 0 && !backup_started_in_recovery)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("the standby was promoted during online backup"),
errhint("This means that the backup being taken is corrupt "
"and should not be used. "
"Try taking another online backup.")));
So that just prevents promotions of the current node, afaict.
> Regardless, it seems like a really good idea to store a list of WAL
> ranges rather than a single start/end/timeline, because even if it's
> impossible today it might become possible in the future.
Indeed.
> Still, unless there's an easy way to set up a test scenario where
> multiple WAL ranges need to be verified, it may be hard to test that
> this code actually behaves properly.
I think it'd be possible to test without a fully cascading setup, by
creating an initial base backup, then do some work to create a bunch of
new timelines, and then start the initial base backup. That'd have to
follow all those timelines. Not sure that's better than a cascading
setup though.
> +/*
> + * Add information about the WAL that will need to be replayed when restoring
> + * this backup to the manifest.
> + */
> +static void
> +AddWALInfoToManifest(manifest_info *manifest, XLogRecPtr startptr,
> + TimeLineID starttli, XLogRecPtr endptr, TimeLineID endtli)
> +{
> + List *timelines = readTimeLineHistory(endtli);
should probably happen after the manifest->buffile check.
> + ListCell *lc;
> + bool first_wal_range = true;
> + bool found_ending_tli = false;
> +
> + /* If there is no buffile, then the user doesn't want a manifest. */
> + if (manifest->buffile == NULL)
> + return;
Not really about this patch/function specifically: I wonder if this'd
look better if you added ManifestEnabled() macro instead of repeating
the comment repeatedly.
> + /* Unless --no-parse-wal was specified, we will need pg_waldump. */
> + if (!no_parse_wal)
> + {
> + int ret;
> +
> + pg_waldump_path = pg_malloc(MAXPGPATH);
> + ret = find_other_exec(argv[0], "pg_waldump",
> + "pg_waldump (PostgreSQL) " PG_VERSION "\n",
> + pg_waldump_path);
> + if (ret < 0)
> + {
> + char full_path[MAXPGPATH];
> +
> + if (find_my_exec(argv[0], full_path) < 0)
> + strlcpy(full_path, progname, sizeof(full_path));
> + if (ret == -1)
> + pg_log_fatal("The program \"%s\" is needed by %s but was\n"
> + "not found in the same directory as \"%s\".\n"
> + "Check your installation.",
> + "pg_waldump", "pg_validatebackup", full_path);
> + else
> + pg_log_fatal("The program \"%s\" was found by \"%s\" but was\n"
> + "not the same version as %s.\n"
> + "Check your installation.",
> + "pg_waldump", full_path, "pg_validatebackup");
> + }
> + }
ISTM, and this can definitely wait for another time, that we should have
one wrapper doing all of this, instead of having quite a few copies of
very similar logic to the above.
> +/*
> + * Attempt to parse the WAL files required to restore from backup using
> + * pg_waldump.
> + */
> +static void
> +parse_required_wal(validator_context *context, char *pg_waldump_path,
> + char *wal_directory, manifest_wal_range *first_wal_range)
> +{
> + manifest_wal_range *this_wal_range = first_wal_range;
> +
> + while (this_wal_range != NULL)
> + {
> + char *pg_waldump_cmd;
> +
> + pg_waldump_cmd = psprintf("\"%s\" --quiet --path=\"%s\" --timeline=%u --start=%X/%X --end=%X/%X\n",
> + pg_waldump_path, wal_directory, this_wal_range->tli,
> + (uint32) (this_wal_range->start_lsn >> 32),
> + (uint32) this_wal_range->start_lsn,
> + (uint32) (this_wal_range->end_lsn >> 32),
> + (uint32) this_wal_range->end_lsn);
> + if (system(pg_waldump_cmd) != 0)
> + report_backup_error(context,
> + "WAL parsing failed for timeline %u",
> + this_wal_range->tli);
> +
> + this_wal_range = this_wal_range->next;
> + }
> +}
Should we have a function to properly escape paths in cases like this?
Not that it's likely or really problematic, but the quoting for path
could be "circumvented".
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 05:58 ` Noah Misch <[email protected]>
2020-03-30 06:24 ` Re: backup manifests Amit Kapila <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Noah Misch @ 2020-03-30 05:58 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Sun, Mar 29, 2020 at 08:42:35PM -0400, Robert Haas wrote:
> On Sat, Mar 28, 2020 at 11:40 PM Noah Misch <[email protected]> wrote:
> > I think this functionality doesn't belong in its own program. If you suspect
> > pg_basebackup or pg_restore will eventually gain the ability to merge
> > incremental backups into a recovery-ready base backup, I would put the
> > functionality in that program. Otherwise, I would put it in pg_checksums.
> > For me, part of the friction here is that the program description indicates
> > general verification, but the actual functionality merely checks hashes on a
> > directory tree that happens to represent a PostgreSQL base backup.
>
> Suraj's original patch made this part of pg_basebackup, but I didn't
> really like that, because I wanted it to have its own set of options.
> I still think all the options I've added are pretty useful ones, and I
> can think of other things somebody might want to do. It feels very
> uncomfortable to make pg_basebackup, or pg_checksums, take either
> options from set A and do thing X, or options from set B and do thing
> Y.
pg_checksums does already have that property, for what it's worth. (More
specifically, certain options dictate the mode, and it reports an error if
another option is incompatible with the mode.)
> But it feels clear that the name pg_validatebackup is not going
> over very well with anyone. I think I should rename it to
> pg_validatemanifest.
Between those two, I would use "pg_validatebackup" if there's a fair chance it
will end up doing the pg_waldump check. Otherwise, I would use
"pg_validatemanifest". I still most prefer delivering this as a mode of an
existing program.
> > > + parse->pathname = palloc(raw_length + 1);
> >
> > I don't see this freed anywhere; is it? (It's useful to make peak memory
> > consumption not grow in proportion to the number of files backed up.)
>
> We need the hash table to remain populated for the whole run time of
> the tool, because we're essentially doing a full join of the actual
> directory contents against the manifest contents. That's a bit
> unfortunate but it doesn't seem simple to improve. I think the only
> people who are really going to suffer are people who have an enormous
> pile of empty or nearly-empty relations. People who have large
> databases for the normal reason - i.e. a reasonable number of tables
> that hold a lot of data - will have manifests of very manageable size.
Okay.
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 05:58 ` Re: backup manifests Noah Misch <[email protected]>
@ 2020-03-30 06:24 ` Amit Kapila <[email protected]>
2020-03-30 19:04 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Amit Kapila @ 2020-03-30 06:24 UTC (permalink / raw)
To: Noah Misch <[email protected]>; +Cc: Robert Haas <[email protected]>; David Steele <[email protected]>; Stephen Frost <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 30, 2020 at 11:28 AM Noah Misch <[email protected]> wrote:
>
> On Sun, Mar 29, 2020 at 08:42:35PM -0400, Robert Haas wrote:
>
> > But it feels clear that the name pg_validatebackup is not going
> > over very well with anyone. I think I should rename it to
> > pg_validatemanifest.
>
> Between those two, I would use "pg_validatebackup" if there's a fair chance it
> will end up doing the pg_waldump check. Otherwise, I would use
> "pg_validatemanifest".
>
+1.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 05:58 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 06:24 ` Re: backup manifests Amit Kapila <[email protected]>
@ 2020-03-30 19:04 ` Robert Haas <[email protected]>
2020-03-30 19:16 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-30 19:04 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Noah Misch <[email protected]>; David Steele <[email protected]>; Stephen Frost <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 30, 2020 at 2:24 AM Amit Kapila <[email protected]> wrote:
> > Between those two, I would use "pg_validatebackup" if there's a fair chance it
> > will end up doing the pg_waldump check. Otherwise, I would use
> > "pg_validatemanifest".
>
> +1.
I guess I'd like to be clear here that I have no fundamental
disagreement with taking this tool in any direction that people would
like it to go. For me it's just a question of timing. Feature freeze
is now a week or so away, and nothing complicated is going to get done
in that time. If we can all agree on something simple based on
Andres's recent proposal, cool, but I'm not yet sure that will be the
case, so what's plan B? We could decide that what I have here is just
too little to be a viable facility on its own, but I think Stephen is
the only one taking that position. We could release it as
pg_validatemanifest with a plan to rename it if other backup-related
checks are added later. We could release it as pg_validatebackup with
the idea to avoid having to rename it when more backup-related checks
are added later, but with a greater possibility of confusion in the
meantime and no hard guarantee that anyone will actually develop such
checks. We could put it in to pg_checksums, but I think that's really
backing ourselves into a corner: if backup validation develops other
checks that are not checksum-related, what then? I'd much rather
gamble on keeping things together by topic (backup) than technology
used internally (checksum). Putting it into pg_basebackup is another
option, and would avoid that problem, but it's not my preferred
option, because as I noted before, I think the command-line options
will get confusing.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 05:58 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 06:24 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-30 19:04 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 19:16 ` Andres Freund <[email protected]>
2020-03-31 05:40 ` Re: backup manifests Noah Misch <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-30 19:16 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Noah Misch <[email protected]>; David Steele <[email protected]>; Stephen Frost <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-30 15:04:55 -0400, Robert Haas wrote:
> I guess I'd like to be clear here that I have no fundamental
> disagreement with taking this tool in any direction that people would
> like it to go. For me it's just a question of timing. Feature freeze
> is now a week or so away, and nothing complicated is going to get done
> in that time. If we can all agree on something simple based on
> Andres's recent proposal, cool, but I'm not yet sure that will be the
> case, so what's plan B? We could decide that what I have here is just
> too little to be a viable facility on its own, but I think Stephen is
> the only one taking that position. We could release it as
> pg_validatemanifest with a plan to rename it if other backup-related
> checks are added later. We could release it as pg_validatebackup with
> the idea to avoid having to rename it when more backup-related checks
> are added later, but with a greater possibility of confusion in the
> meantime and no hard guarantee that anyone will actually develop such
> checks. We could put it in to pg_checksums, but I think that's really
> backing ourselves into a corner: if backup validation develops other
> checks that are not checksum-related, what then? I'd much rather
> gamble on keeping things together by topic (backup) than technology
> used internally (checksum). Putting it into pg_basebackup is another
> option, and would avoid that problem, but it's not my preferred
> option, because as I noted before, I think the command-line options
> will get confusing.
I'm mildly inclined to name it pg_validate, pg_validate_dbdir or
such. And eventually (definitely not this release) subsume pg_checksums
in it. That way we can add other checkers too.
I don't really see a point in ending up with lots of different commands
over time. Partially because there's probably plenty checks where the
overall cost can be drastically reduced by combining IO. Partially
because there's probably plenty shareable infrastructure. And partially
because I think it makes discovery for users a lot easier.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 05:58 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 06:24 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-30 19:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 19:16 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-31 05:40 ` Noah Misch <[email protected]>
2020-03-31 09:26 ` Re: backup manifests Amit Kapila <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Noah Misch @ 2020-03-31 05:40 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; David Steele <[email protected]>; Stephen Frost <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 30, 2020 at 12:16:31PM -0700, Andres Freund wrote:
> On 2020-03-30 15:04:55 -0400, Robert Haas wrote:
> > I guess I'd like to be clear here that I have no fundamental
> > disagreement with taking this tool in any direction that people would
> > like it to go. For me it's just a question of timing. Feature freeze
> > is now a week or so away, and nothing complicated is going to get done
> > in that time. If we can all agree on something simple based on
> > Andres's recent proposal, cool, but I'm not yet sure that will be the
> > case, so what's plan B? We could decide that what I have here is just
> > too little to be a viable facility on its own, but I think Stephen is
> > the only one taking that position. We could release it as
> > pg_validatemanifest with a plan to rename it if other backup-related
> > checks are added later. We could release it as pg_validatebackup with
> > the idea to avoid having to rename it when more backup-related checks
> > are added later, but with a greater possibility of confusion in the
> > meantime and no hard guarantee that anyone will actually develop such
> > checks. We could put it in to pg_checksums, but I think that's really
> > backing ourselves into a corner: if backup validation develops other
> > checks that are not checksum-related, what then? I'd much rather
> > gamble on keeping things together by topic (backup) than technology
> > used internally (checksum). Putting it into pg_basebackup is another
> > option, and would avoid that problem, but it's not my preferred
> > option, because as I noted before, I think the command-line options
> > will get confusing.
>
> I'm mildly inclined to name it pg_validate, pg_validate_dbdir or
> such. And eventually (definitely not this release) subsume pg_checksums
> in it. That way we can add other checkers too.
Works for me; of those two, I prefer pg_validate.
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 05:58 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 06:24 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-30 19:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 19:16 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-31 05:40 ` Re: backup manifests Noah Misch <[email protected]>
@ 2020-03-31 09:26 ` Amit Kapila <[email protected]>
2020-03-31 11:58 ` Re: backup manifests Stephen Frost <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Amit Kapila @ 2020-03-31 09:26 UTC (permalink / raw)
To: Noah Misch <[email protected]>; +Cc: Andres Freund <[email protected]>; Robert Haas <[email protected]>; David Steele <[email protected]>; Stephen Frost <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Tue, Mar 31, 2020 at 11:10 AM Noah Misch <[email protected]> wrote:
>
> On Mon, Mar 30, 2020 at 12:16:31PM -0700, Andres Freund wrote:
> > On 2020-03-30 15:04:55 -0400, Robert Haas wrote:
> > > I guess I'd like to be clear here that I have no fundamental
> > > disagreement with taking this tool in any direction that people would
> > > like it to go. For me it's just a question of timing. Feature freeze
> > > is now a week or so away, and nothing complicated is going to get done
> > > in that time. If we can all agree on something simple based on
> > > Andres's recent proposal, cool, but I'm not yet sure that will be the
> > > case, so what's plan B? We could decide that what I have here is just
> > > too little to be a viable facility on its own, but I think Stephen is
> > > the only one taking that position. We could release it as
> > > pg_validatemanifest with a plan to rename it if other backup-related
> > > checks are added later. We could release it as pg_validatebackup with
> > > the idea to avoid having to rename it when more backup-related checks
> > > are added later, but with a greater possibility of confusion in the
> > > meantime and no hard guarantee that anyone will actually develop such
> > > checks. We could put it in to pg_checksums, but I think that's really
> > > backing ourselves into a corner: if backup validation develops other
> > > checks that are not checksum-related, what then? I'd much rather
> > > gamble on keeping things together by topic (backup) than technology
> > > used internally (checksum). Putting it into pg_basebackup is another
> > > option, and would avoid that problem, but it's not my preferred
> > > option, because as I noted before, I think the command-line options
> > > will get confusing.
> >
> > I'm mildly inclined to name it pg_validate, pg_validate_dbdir or
> > such. And eventually (definitely not this release) subsume pg_checksums
> > in it. That way we can add other checkers too.
>
> Works for me; of those two, I prefer pg_validate.
>
pg_validate sounds like a tool with a much bigger purpose. I think
even things like amcheck could also fall under it.
This patch has two parts (a) Generate backup manifests for base
backups, and (b) Validate backup (manifest). It seems to me that
there are not many things pending for (a), can't we commit that first
or is it the case that (a) depends on (b)? This is *not* a suggestion
to leave pg_validatebackup from this release rather just to commit if
something is ready and meaningful on its own.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 17:53 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-29 03:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 00:42 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 05:58 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-30 06:24 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-30 19:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 19:16 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-31 05:40 ` Re: backup manifests Noah Misch <[email protected]>
2020-03-31 09:26 ` Re: backup manifests Amit Kapila <[email protected]>
@ 2020-03-31 11:58 ` Stephen Frost <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Stephen Frost @ 2020-03-31 11:58 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Noah Misch <[email protected]>; Andres Freund <[email protected]>; Robert Haas <[email protected]>; David Steele <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Amit Kapila ([email protected]) wrote:
> On Tue, Mar 31, 2020 at 11:10 AM Noah Misch <[email protected]> wrote:
> > On Mon, Mar 30, 2020 at 12:16:31PM -0700, Andres Freund wrote:
> > > On 2020-03-30 15:04:55 -0400, Robert Haas wrote:
> > > > I guess I'd like to be clear here that I have no fundamental
> > > > disagreement with taking this tool in any direction that people would
> > > > like it to go. For me it's just a question of timing. Feature freeze
> > > > is now a week or so away, and nothing complicated is going to get done
> > > > in that time. If we can all agree on something simple based on
> > > > Andres's recent proposal, cool, but I'm not yet sure that will be the
> > > > case, so what's plan B? We could decide that what I have here is just
> > > > too little to be a viable facility on its own, but I think Stephen is
> > > > the only one taking that position. We could release it as
> > > > pg_validatemanifest with a plan to rename it if other backup-related
> > > > checks are added later. We could release it as pg_validatebackup with
> > > > the idea to avoid having to rename it when more backup-related checks
> > > > are added later, but with a greater possibility of confusion in the
> > > > meantime and no hard guarantee that anyone will actually develop such
> > > > checks. We could put it in to pg_checksums, but I think that's really
> > > > backing ourselves into a corner: if backup validation develops other
> > > > checks that are not checksum-related, what then? I'd much rather
> > > > gamble on keeping things together by topic (backup) than technology
> > > > used internally (checksum). Putting it into pg_basebackup is another
> > > > option, and would avoid that problem, but it's not my preferred
> > > > option, because as I noted before, I think the command-line options
> > > > will get confusing.
> > >
> > > I'm mildly inclined to name it pg_validate, pg_validate_dbdir or
> > > such. And eventually (definitely not this release) subsume pg_checksums
> > > in it. That way we can add other checkers too.
> >
> > Works for me; of those two, I prefer pg_validate.
>
> pg_validate sounds like a tool with a much bigger purpose. I think
> even things like amcheck could also fall under it.
Yeah, I tend to agree with this.
> This patch has two parts (a) Generate backup manifests for base
> backups, and (b) Validate backup (manifest). It seems to me that
> there are not many things pending for (a), can't we commit that first
> or is it the case that (a) depends on (b)? This is *not* a suggestion
> to leave pg_validatebackup from this release rather just to commit if
> something is ready and meaningful on its own.
I suspect the idea here is that we don't really want to commit something
that nothing is actually using, and that's understandable and justified
here- consider that even in this recent discussion there was talk that
maybe we should have included permissions and ownership in the manifest,
or starting and ending WAL positions, so that they'd be able to be
checked by this tool more easily (and because it's just useful to have
all that info in one place... I don't really agree with the concerns
that it's an issue for static information like that to be duplicated).
In other words, while the manifest creation code might be something we
could commit, without a tool to use it (which does all the things that
we think it needs to, to perform some high-level task, such as "validate
a backup") we don't know that the manifest that's actually generated is
really up to snuff and has what it needs to have to perform that task.
I had been hoping that the discussion Andres was leading regarding
leveraging pg_waldump (or maybe just code from it..) would get us to a
point where pg_validatebackup would check that we have all of the WAL
needed for the backup to be consistent and that it would then verify the
internal checksums of the WAL. That would certainly be a good solution
for this time around, in my view, and is already all existing
client-side code. I do think we'd want to have a note about how we
verify pg_wal differently from the other files which are in the
manifest.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
@ 2020-03-27 18:34 ` Robert Haas <[email protected]>
2020-03-27 19:48 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 20:12 ` Re: backup manifests Andres Freund <[email protected]>
1 sibling, 2 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-27 18:34 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Thu, Mar 26, 2020 at 4:37 PM David Steele <[email protected]> wrote:
> I agree with Stephen that this should be done, but I agree with you that
> it can wait for a future commit. However, I do think:
>
> 1) It should be called out rather plainly in the documentation.
> 2) If there are files in pg_wal then pg_validatebackup should inform the
> user that those files have not been validated.
I agree with you about #1, and I suspect that there's a way to improve
what I've got here now, but I think I might be too close to this to
figure out what the best way would be, so suggestions welcome.
I think #2 is an interesting idea and could possibly reduce the danger
of user confusion on this point considerably - because, let's face it,
not everyone is going to read the documentation. However, I'm having a
hard time figuring out exactly what we'd print. Right now on success,
unless you specify -q, you get:
[rhaas ~]$ pg_validatebackup ~/pgslave
backup successfully verified
But it feels strange and possibly confusing to me to print something like:
[rhaas ~]$ pg_validatebackup ~/pgslave
backup successfully verified (except for pg_wal)
...because there are a few other exceptions too, and also because it
might make the user think that we normally check that but for some
reason decided to skip it in this case. Maybe something more verbose
like:
[rhaas ~]$ pg_validatebackup ~/pgslave
backup files successfully verified
your backup contains a pg_wal directory, but this tool can't validate
that, so do it yourself
...but that seems a little obnoxious and a little silly to print out every time.
Ideas?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 18:34 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 19:48 ` Stephen Frost <[email protected]>
1 sibling, 0 replies; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 19:48 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Thu, Mar 26, 2020 at 4:37 PM David Steele <[email protected]> wrote:
> > I agree with Stephen that this should be done, but I agree with you that
> > it can wait for a future commit. However, I do think:
> >
> > 1) It should be called out rather plainly in the documentation.
> > 2) If there are files in pg_wal then pg_validatebackup should inform the
> > user that those files have not been validated.
>
> I agree with you about #1, and I suspect that there's a way to improve
> what I've got here now, but I think I might be too close to this to
> figure out what the best way would be, so suggestions welcome.
>
> I think #2 is an interesting idea and could possibly reduce the danger
> of user confusion on this point considerably - because, let's face it,
> not everyone is going to read the documentation. However, I'm having a
> hard time figuring out exactly what we'd print. Right now on success,
> unless you specify -q, you get:
>
> [rhaas ~]$ pg_validatebackup ~/pgslave
> backup successfully verified
>
> But it feels strange and possibly confusing to me to print something like:
>
> [rhaas ~]$ pg_validatebackup ~/pgslave
> backup successfully verified (except for pg_wal)
>
> ...because there are a few other exceptions too, and also because it
The exceptions you're referring to here are things like the various
signal files, that the user can recreated pretty easily..? I don't
think those really rise to the level of pg_wal.
What I would hope to see (... well, we know what I *really* would hope
to see, but if we really go this route) is something like:
WARNING: pg_wal not empty, WAL files are not validated by this tool
data files successfully verified
and a non-zero exit code.
Basically, if you're doing WAL yourself, then you'd use pg_receivewal
and maybe your own manifest-building code for WAL or something and then
use -X none with pg_basebackup.
Then again, I'd have -X none throw a warning too. I'd be alright with
all of these having override switches to say "ok, I get it, don't
complain about it".
I disagree with the idea of writing "backup successfully verified" when
we aren't doing any checking of the WAL that's essential for the backup
(unlike various signal files and whatnot, which aren't...).
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 18:34 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 20:12 ` Andres Freund <[email protected]>
2020-03-27 21:07 ` Re: backup manifests Stephen Frost <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-27 20:12 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-27 14:34:19 -0400, Robert Haas wrote:
> I think #2 is an interesting idea and could possibly reduce the danger
> of user confusion on this point considerably - because, let's face it,
> not everyone is going to read the documentation. However, I'm having a
> hard time figuring out exactly what we'd print. Right now on success,
> unless you specify -q, you get:
>
> [rhaas ~]$ pg_validatebackup ~/pgslave
> backup successfully verified
>
> But it feels strange and possibly confusing to me to print something like:
>
> [rhaas ~]$ pg_validatebackup ~/pgslave
> backup successfully verified (except for pg_wal)
You could print something like:
WAL necessary to restore this base backup can be validated with:
pg_waldump -p ~/pgslave -t tl -s backup_start_location -e backup_end_loc > /dev/null && echo true
Obviously that specific invocation sucks, but it'd not be hard to add an
option to waldump to not output anything.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 18:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:12 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 21:07 ` Stephen Frost <[email protected]>
2020-03-27 22:00 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 21:07 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; David Steele <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Andres Freund ([email protected]) wrote:
> On 2020-03-27 14:34:19 -0400, Robert Haas wrote:
> > I think #2 is an interesting idea and could possibly reduce the danger
> > of user confusion on this point considerably - because, let's face it,
> > not everyone is going to read the documentation. However, I'm having a
> > hard time figuring out exactly what we'd print. Right now on success,
> > unless you specify -q, you get:
> >
> > [rhaas ~]$ pg_validatebackup ~/pgslave
> > backup successfully verified
> >
> > But it feels strange and possibly confusing to me to print something like:
> >
> > [rhaas ~]$ pg_validatebackup ~/pgslave
> > backup successfully verified (except for pg_wal)
>
> You could print something like:
> WAL necessary to restore this base backup can be validated with:
>
> pg_waldump -p ~/pgslave -t tl -s backup_start_location -e backup_end_loc > /dev/null && echo true
>
> Obviously that specific invocation sucks, but it'd not be hard to add an
> option to waldump to not output anything.
Interesting idea to use pg_waldump.
I had suggested up-thread, and I'm still fine with, having
pg_validatebackup scan the WAL and check the internal checksums. I'd
prefer an option that uses hashes to check when the user has asked for
hashes with SHA256 or something, but at least scanning the WAL and
making sure it validates its internal checksum (and is actually all
there, which is pretty darn critical) would be enough to say that we're
pretty sure the backup is valid.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-26 20:37 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 18:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:12 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 21:07 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 22:00 ` Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-27 22:00 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Robert Haas <[email protected]>; David Steele <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-27 17:07:42 -0400, Stephen Frost wrote:
> I had suggested up-thread, and I'm still fine with, having
> pg_validatebackup scan the WAL and check the internal checksums. I'd
> prefer an option that uses hashes to check when the user has asked for
> hashes with SHA256 or something, but at least scanning the WAL and
> making sure it validates its internal checksum (and is actually all
> there, which is pretty darn critical) would be enough to say that we're
> pretty sure the backup is valid.
I'd say that actually parsing the WAL will give you a lot higher
confidence than verifying a sha256 for each file. There's plenty of ways
to screw up the pg_wal on the source server (I've seen several
restore_commands doing so, particularly when eagerly fetching). Sure,
it'll not help against an attacker, but I'm not sure I see the threat
model.
There's imo a cost argument against doing WAL verification by reading
it, but that'd mostly be a factor when comparing against a faster
whole-file checksum.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 04:30 ` Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-27 04:30 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-26 11:37:48 -0400, Robert Haas wrote:
> I mean, you're just repeating the same argument here, and it's just
> not valid. Regardless of the file size, the chances of a false
> checksum match are literally less than one in a billion. There is
> every reason to believe that users will be happy with a low-overhead
> method that has a 99.9999999+% chance of detecting corrupt files. I do
> agree that a 64-bit CRC would probably be not much more expensive and
> improve the probability of detecting errors even further
I *seriously* doubt that it's true that 64bit CRCs wouldn't be
slower. The only reason CRC32C is semi-fast is that we're accelerating
it using hardware instructions (on x86-64 and ARM at least). Before that
it was very regularly the bottleneck for processing WAL - and it still
sometimes is. Most CRCs aren't actually very fast to compute, because
they don't lend themselves to benefit from ILP or SIMD. We spent a fair
bit of time optimizing our crc implementation before the hardware
support was widespread.
> but I wanted to restrict this patch to using infrastructure we already
> have. The choices there are the various SHA functions (so I supported
> those), MD5 (which I deliberately omitted, for reasons I hope you'll
> be the first to agree with), CRC-32C (which is fast), a couple of
> other CRC-32 variants (which I omitted because they seemed redundant
> and one of them only ever existed in PostgreSQL because of a coding
> mistake), and the hacked-up version of FNV that we use for page-level
> checksums (which is only 16 bits and seems to have no advantages for
> this purpose).
FWIW, FNV is only 16bit because we reduce its size to 16 bit. See the
tail of pg_checksum_page.
I'm not sure the error detection guarantees of various CRC algorithms
are that relevant here, btw. IMO, for something like checksums in a
backup, just having a single one-bit error isn't as common as having
larger errors (e.g. entire blocks beeing zeroed). And to detect that
32bit checksums aren't that good.
> > As for folks who are that close to the edge on their backup timing that
> > they can't have it slow down- chances are pretty darn good that they're
> > not far from ending up needing to find a better solution than
> > pg_basebackup anyway. Or they don't need to generate a manifest (or, I
> > suppose, they could have one but not have checksums..).
>
> 40-50% is a lot more than "if you were on the edge."
sha256 does about approx 400MB/s per core on modern intel CPUs. That's
way below commonly accessible storage / network capabilities (and even
if you're only doing 200MB/s, you're still going to spend roughly half
of the CPU time just doing hashing. It's unlikely that you're going to
see much speedups for sha256 just by upgrading a CPU. While there are
hardware instructions available, they don't result in all that large
improvements. Of course, we could also start using the GPU (err, really
no).
Defaulting to that makes very little sense to me. You're not just going
to spend that time while backing up, but also when validating backups
(i.e. network limits suddenly aren't a relevant bottleneck anymore).
> > I fail to see the usefulness of a tool that doesn't actually verify that
> > the backup is able to be restored from.
> >
> > Even pg_basebackup (in both fetch and stream modes...) checks that we at
> > least got all the WAL that's needed for the backup from the server
> > before considering the backup to be valid and telling the user that
> > there was a successful backup. With what you're proposing here, we
> > could have someone do a pg_basebackup, get back an ERROR saying the
> > backup wasn't valid, and then run pg_validatebackup and be told that the
> > backup is valid. I don't get how that's sensible.
>
> I'm sorry that you can't see how that's sensible, but it doesn't mean
> that it isn't sensible. It is totally unrealistic to expect that any
> backup verification tool can verify that you won't get an error when
> trying to use the backup. That would require that everything that the
> validation tool try to do everything that PostgreSQL will try to do
> when the backup is used, including running recovery and updating the
> data files. Anything less than that creates a real possibility that
> the backup will verify good but fail when used. This tool has a much
> narrower purpose, which is to try to verify that we (still) have the
> files the server sent as part of the backup and that, to the best of
> our ability to detect such things, they have not been modified. As you
> know, or should know, the WAL files are not sent as part of the
> backup, and so are not verified. Other things that would also be
> useful to check are also not verified. It would be fantastic to have
> more verification tools in the future, but it is difficult to see why
> anyone would bother trying if an attempt to get the first one
> committed gets blocked because it does not yet do everything. Very few
> patches try to do everything, and those that do usually get blocked
> because, by trying to do too much, they get some of it badly wrong.
It sounds to me that if there are to be manifests for the WAL, it should
be a separate (set of) manifests. Trying to somehow tie together the
manifest for the base backup, and the one for the WAL, makes little
sense to me. They're commonly not computed in one place, often not even
stored in the same place. For PITR relevant WAL doesn't even exist yet
at the time the manifest is created (and thus obviously cannot be
included in the base backup manifest). And fairly obviously one would
want to be able to verify the correctness of WAL between two
basebackups.
I don't see much point in complicating the design to somehow capture WAL
in the manifest, when it's only going to solve a small set of cases.
Seems better to (later?) add support for generating manifests for WAL
files, and then have a tool that can verify all the manifests required
to restore a base backup.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 15:26 ` Stephen Frost <[email protected]>
2020-03-27 19:29 ` Re: backup manifests Robert Haas <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 15:26 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Andres Freund ([email protected]) wrote:
> On 2020-03-26 11:37:48 -0400, Robert Haas wrote:
> > I'm sorry that you can't see how that's sensible, but it doesn't mean
> > that it isn't sensible. It is totally unrealistic to expect that any
> > backup verification tool can verify that you won't get an error when
> > trying to use the backup. That would require that everything that the
> > validation tool try to do everything that PostgreSQL will try to do
> > when the backup is used, including running recovery and updating the
> > data files. Anything less than that creates a real possibility that
> > the backup will verify good but fail when used. This tool has a much
> > narrower purpose, which is to try to verify that we (still) have the
> > files the server sent as part of the backup and that, to the best of
> > our ability to detect such things, they have not been modified. As you
> > know, or should know, the WAL files are not sent as part of the
> > backup, and so are not verified. Other things that would also be
> > useful to check are also not verified. It would be fantastic to have
> > more verification tools in the future, but it is difficult to see why
> > anyone would bother trying if an attempt to get the first one
> > committed gets blocked because it does not yet do everything. Very few
> > patches try to do everything, and those that do usually get blocked
> > because, by trying to do too much, they get some of it badly wrong.
>
> It sounds to me that if there are to be manifests for the WAL, it should
> be a separate (set of) manifests. Trying to somehow tie together the
> manifest for the base backup, and the one for the WAL, makes little
> sense to me. They're commonly not computed in one place, often not even
> stored in the same place. For PITR relevant WAL doesn't even exist yet
> at the time the manifest is created (and thus obviously cannot be
> included in the base backup manifest). And fairly obviously one would
> want to be able to verify the correctness of WAL between two
> basebackups.
We aren't talking about generic PITR or about tools other than
pg_basebackup, which has specific options for grabbing the WAL, and
making sure that it is all there for the backup that was taken.
> I don't see much point in complicating the design to somehow capture WAL
> in the manifest, when it's only going to solve a small set of cases.
As it relates to this, I tend to think that it solves the exact case
that pg_basebackup is built for and used for. I said up-thread that if
someone does decide to use -X none then we could just throw a warning
(and perhaps have a way to override that if there's desire for it).
> Seems better to (later?) add support for generating manifests for WAL
> files, and then have a tool that can verify all the manifests required
> to restore a base backup.
I'm not trying to expand on the feature set here or move the goalposts
way down the road, which is what seems to be what's being suggested
here. To be clear, I don't have any objection to adding a generic tool
for validating WAL as you're talking about here, but I also don't think
that's required for pg_validatebackup. What I do think we need is a
check of the WAL that's fetched when people use pg_basebackup -Xstream
or -Xfetch. pg_basebackup itself has that check because it's critical
to the backup being successful and valid. Not having that basic
validation of a backup really just isn't ok- there's a reason
pg_basebackup has that check.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 19:29 ` Robert Haas <[email protected]>
2020-03-27 20:08 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 20:16 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 20:57 ` Re: backup manifests Stephen Frost <[email protected]>
0 siblings, 3 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-27 19:29 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Andres Freund <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 11:26 AM Stephen Frost <[email protected]> wrote:
> > Seems better to (later?) add support for generating manifests for WAL
> > files, and then have a tool that can verify all the manifests required
> > to restore a base backup.
>
> I'm not trying to expand on the feature set here or move the goalposts
> way down the road, which is what seems to be what's being suggested
> here. To be clear, I don't have any objection to adding a generic tool
> for validating WAL as you're talking about here, but I also don't think
> that's required for pg_validatebackup. What I do think we need is a
> check of the WAL that's fetched when people use pg_basebackup -Xstream
> or -Xfetch. pg_basebackup itself has that check because it's critical
> to the backup being successful and valid. Not having that basic
> validation of a backup really just isn't ok- there's a reason
> pg_basebackup has that check.
I don't understand how this could be done without significantly
complicating the architecture. As I said before, -Xstream sends WAL
over a separate connection that is unrelated to the one running
BASE_BACKUP, so the base-backup connection doesn't know what to
include in the manifest. Now you could do something like: once all of
the WAL files have been fetched, the client checksums all of those and
sends their names and checksums to the server, which turns around and
puts them into the manifest, which it then sends back to the client.
But that is actually quite a bit of additional complexity, and it's
pretty strange, too, because now you have the client checksumming some
files and the server checksumming others. I know you mentioned a few
different ideas before, but I think they all kinda have some problem
along these lines.
I also kinda disagree with the idea that the WAL should be considered
an integral part of the backup. I don't know how pgbackrest does
things, but BART stores each backup in a separate directly without any
associated WAL, and then keeps all the WAL together in a different
directory. I imagine that people who are using continuous archiving
also tend to use -Xnone, or if they do backups by copying the files
rather than using pg_backrest, they exclude pg_wal. In fact, for
people with big, important databases, I'd assume that would be the
normal pattern. You presumably wouldn't want to keep one copy of the
WAL files taken during the backup with the backup itself, and a
separate copy in the archive.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 19:29 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 20:08 ` Andres Freund <[email protected]>
2 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-27 20:08 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Stephen Frost <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-27 15:29:02 -0400, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 11:26 AM Stephen Frost <[email protected]> wrote:
> > > Seems better to (later?) add support for generating manifests for WAL
> > > files, and then have a tool that can verify all the manifests required
> > > to restore a base backup.
> >
> > I'm not trying to expand on the feature set here or move the goalposts
> > way down the road, which is what seems to be what's being suggested
> > here. To be clear, I don't have any objection to adding a generic tool
> > for validating WAL as you're talking about here, but I also don't think
> > that's required for pg_validatebackup. What I do think we need is a
> > check of the WAL that's fetched when people use pg_basebackup -Xstream
> > or -Xfetch. pg_basebackup itself has that check because it's critical
> > to the backup being successful and valid. Not having that basic
> > validation of a backup really just isn't ok- there's a reason
> > pg_basebackup has that check.
>
> I don't understand how this could be done without significantly
> complicating the architecture. As I said before, -Xstream sends WAL
> over a separate connection that is unrelated to the one running
> BASE_BACKUP, so the base-backup connection doesn't know what to
> include in the manifest. Now you could do something like: once all of
> the WAL files have been fetched, the client checksums all of those and
> sends their names and checksums to the server, which turns around and
> puts them into the manifest, which it then sends back to the client.
> But that is actually quite a bit of additional complexity, and it's
> pretty strange, too, because now you have the client checksumming some
> files and the server checksumming others. I know you mentioned a few
> different ideas before, but I think they all kinda have some problem
> along these lines.
How about having separate manifests for segments? And have them stay
separate? And then have an option to verify the manifests for all the
WAL files that are required for a specific restore? The easiest way
would be to just add a separate manifest file for each segment, and name
them accordingly. But inventing a naming pattern that specifies both
start-end segments wouldn't be hard either, and result in fewer
manifests.
Base backups (in the backup sense, not for bringing up replicas etc)
without the ability to apply newer WAL are fairly pointless imo. And if
newer WAL is applied, there's not much point in just verifying the WAL
that's necessary to restore the base backup. Instead you'd want to be
able to verify all the WAL since the base backup to the "current" point
(or the next base backup).
For me having something inside pg_basebackup (or the server, for
-Xfetch) that somehow includes the WAL files in the manifest doesn't
really gain us much - it's obviously not something that'll help us to
verify all the WAL that needs to be applied (to either get the base
backup into a consistent state, or to roll forward to the desired
point).
> I also kinda disagree with the idea that the WAL should be considered
> an integral part of the backup. I don't know how pgbackrest does
> things, but BART stores each backup in a separate directly without any
> associated WAL, and then keeps all the WAL together in a different
> directory. I imagine that people who are using continuous archiving
> also tend to use -Xnone, or if they do backups by copying the files
> rather than using pg_backrest, they exclude pg_wal. In fact, for
> people with big, important databases, I'd assume that would be the
> normal pattern. You presumably wouldn't want to keep one copy of the
> WAL files taken during the backup with the backup itself, and a
> separate copy in the archive.
+1
I also don't see them as being as important, due to the already existing
checksums (which are of a much much much higher quality than what we
have for database pages, both by being wider, and by being much more
frequent in most cases). There's obviously a need to validate the WAL in
a nicer way than scripting pg_waldump - but that seems separate anyway.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 19:29 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 20:16 ` David Steele <[email protected]>
2 siblings, 0 replies; 372+ messages in thread
From: David Steele @ 2020-03-27 20:16 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Stephen Frost <[email protected]>; +Cc: Andres Freund <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/27/20 3:29 PM, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 11:26 AM Stephen Frost <[email protected]> wrote:
>>> Seems better to (later?) add support for generating manifests for WAL
>>> files, and then have a tool that can verify all the manifests required
>>> to restore a base backup.
>>
>> I'm not trying to expand on the feature set here or move the goalposts
>> way down the road, which is what seems to be what's being suggested
>> here. To be clear, I don't have any objection to adding a generic tool
>> for validating WAL as you're talking about here, but I also don't think
>> that's required for pg_validatebackup. What I do think we need is a
>> check of the WAL that's fetched when people use pg_basebackup -Xstream
>> or -Xfetch. pg_basebackup itself has that check because it's critical
>> to the backup being successful and valid. Not having that basic
>> validation of a backup really just isn't ok- there's a reason
>> pg_basebackup has that check.
>
> I don't understand how this could be done without significantly
> complicating the architecture. As I said before, -Xstream sends WAL
> over a separate connection that is unrelated to the one running
> BASE_BACKUP, so the base-backup connection doesn't know what to
> include in the manifest. Now you could do something like: once all of
> the WAL files have been fetched, the client checksums all of those and
> sends their names and checksums to the server, which turns around and
> puts them into the manifest, which it then sends back to the client.
> But that is actually quite a bit of additional complexity, and it's
> pretty strange, too, because now you have the client checksumming some
> files and the server checksumming others. I know you mentioned a few
> different ideas before, but I think they all kinda have some problem
> along these lines.
>
> I also kinda disagree with the idea that the WAL should be considered
> an integral part of the backup. I don't know how pgbackrest does
> things,
We checksum each WAL file while it is read and transmitted to the repo
by the archive_command. Then at the end of the backup we ensure that
all the WAL required to make the backup consistent has made it to the repo.
> but BART stores each backup in a separate directly without any
> associated WAL, and then keeps all the WAL together in a different
> directory. I imagine that people who are using continuous archiving
> also tend to use -Xnone, or if they do backups by copying the files
> rather than using pg_backrest, they exclude pg_wal. In fact, for
> people with big, important databases, I'd assume that would be the
> normal pattern. You presumably wouldn't want to keep one copy of the
> WAL files taken during the backup with the backup itself, and a
> separate copy in the archive.
pgBackRest does provide the option to copy WAL into the backup directory
for the super-paranoid, though it is not the default. It is pretty handy
for moving individual backups some other medium like tape, though.
If -Xnone is specified then it seems like pg_validatebackup is
completely off the hook. But in the case of -Xstream or -Xfetch
couldn't we at least verify that the expected WAL segments are present
and the correct size?
Storing the start/stop lsn in the manifest would be a nice thing to have
anyway and that would make this feature pretty trivial. Yeah, that's in
the backup_label file as well but the manifest is so much easier to read.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 19:29 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 20:57 ` Stephen Frost <[email protected]>
2020-03-27 22:07 ` Re: backup manifests Andres Freund <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 20:57 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Fri, Mar 27, 2020 at 11:26 AM Stephen Frost <[email protected]> wrote:
> > > Seems better to (later?) add support for generating manifests for WAL
> > > files, and then have a tool that can verify all the manifests required
> > > to restore a base backup.
> >
> > I'm not trying to expand on the feature set here or move the goalposts
> > way down the road, which is what seems to be what's being suggested
> > here. To be clear, I don't have any objection to adding a generic tool
> > for validating WAL as you're talking about here, but I also don't think
> > that's required for pg_validatebackup. What I do think we need is a
> > check of the WAL that's fetched when people use pg_basebackup -Xstream
> > or -Xfetch. pg_basebackup itself has that check because it's critical
> > to the backup being successful and valid. Not having that basic
> > validation of a backup really just isn't ok- there's a reason
> > pg_basebackup has that check.
>
> I don't understand how this could be done without significantly
> complicating the architecture. As I said before, -Xstream sends WAL
> over a separate connection that is unrelated to the one running
> BASE_BACKUP, so the base-backup connection doesn't know what to
> include in the manifest. Now you could do something like: once all of
> the WAL files have been fetched, the client checksums all of those and
> sends their names and checksums to the server, which turns around and
> puts them into the manifest, which it then sends back to the client.
> But that is actually quite a bit of additional complexity, and it's
> pretty strange, too, because now you have the client checksumming some
> files and the server checksumming others. I know you mentioned a few
> different ideas before, but I think they all kinda have some problem
> along these lines.
I've made some suggestions before, also chatted about an idea with David
that I'll outline here.
First off- I'm a bit mystified why you are saying that the base backup
connection doesn't know what to include in the manifest regarding WAL.
The base-backup process determines the starting position (and then even
puts it into the backup_label that's sent to the client), and then it
directly returns the ending position at the end of the BASE_BACKUP
command. Given that we do know that information, then we just need to
get the checksums/hashes for each of the WAL files, if it's been asked
for. How do we know checksums or hashes have been asked for in the
WAL streaming connection? We can have the pg_basebackup process ask for
that when it connects to stream the WAL that's needed.
Now the only part that's a little grotty is dealing with passing the
checksums/hashes that the WAL stream connection calculates over to the
base backup connection to include in the manifest. Offhand though, it
seems like we could drop a file in archive_status for that, perhaps
"wal_checksums.PID" or such (the PID would be that of the PG backend
that's doing the base backup, which we'd pass to START_REPLICATION). Of
course, the backup process would have to check and make sure that it got
all the needed WAL file checksums, but since it knows the end, that
shouldn't be too bad.
> I also kinda disagree with the idea that the WAL should be considered
> an integral part of the backup. I don't know how pgbackrest does
> things, but BART stores each backup in a separate directly without any
> associated WAL, and then keeps all the WAL together in a different
> directory. I imagine that people who are using continuous archiving
> also tend to use -Xnone, or if they do backups by copying the files
> rather than using pg_backrest, they exclude pg_wal. In fact, for
> people with big, important databases, I'd assume that would be the
> normal pattern. You presumably wouldn't want to keep one copy of the
> WAL files taken during the backup with the backup itself, and a
> separate copy in the archive.
I really don't know what to say to this. WAL is absolutely critical to
a backup being valid. pgBackRest doesn't have a way to *just* validate
a backup today, unfortunately, but we're planning to support it in the
future and we will absolutely include in that validation checking all of
the WAL that's part of the backup.
I'm fine with forgoing all of this in the -X none case, as I've said
elsewhere. I think it'd be great for pg_receivewal to have a way to
validate WAL and such, but that's a clearly new feature and it's
independent from validating a backup.
As it relates to how pgBackRest stores WAL, we actually do support both
of the options you mention, because people with big important databases
like to be extra paranoid. WAL can either be stored in just the
archive, or it can be stored in both the archive and in the backup (with
'--archive-copy'). Note that this isn't done by just grabbing whatever
is in pg_wal at the time of the backup, as that wouldn't actually work,
but rather by copying the necessary WAL from the archive at the end of
the backup.
We do also check all WAL that's pulled from the archive by the restore
command, though exactly what WAL is needed isn't something we know ahead
of time (yet, anyway.. we are working on WAL parsing code that'll
change that by actually scanning the WAL and storing all restore points,
starting/ending times and transaction IDs, and anything else that can be
used as a restore target, so we can figure out exactly all WAL that's
needed to get to a particular restore target).
We actually have someone who implemented an independent tool called
check_pgbackrest which specifically has a "archives" check, for checking
that the WAL is in the archive. We plan to also provide a way to ask
pgbackrest to confirm that there's no missing WAL, and that all of the
WAL is valid.
WAL is critical to a backup that's been taken in an online manner, no
matter where it's stored. A backup isn't valid without the WAL that's
needed to reach consistency.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 19:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:57 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 22:07 ` Andres Freund <[email protected]>
2020-03-27 22:24 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 22:33 ` Re: backup manifests David Steele <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-27 22:07 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-27 16:57:46 -0400, Stephen Frost wrote:
> I really don't know what to say to this. WAL is absolutely critical to
> a backup being valid. pgBackRest doesn't have a way to *just* validate
> a backup today, unfortunately, but we're planning to support it in the
> future and we will absolutely include in that validation checking all of
> the WAL that's part of the backup.
Could you please address the fact that just about everybody uses base
backups + later WAL to have a short data loss window? Integrating the
WAL files necessary to make the base backup consistent doesn't achieve
much if we can't verify the WAL files afterwards. And fairly obviously
pg_basebackup can't do much about WAL created after its invocation.
Given that we need something separate to address that "verification
hole", I don't see why it's useful to have a special case solution (or
rather multiple ones, for stream and fetch) inside pg_basebackup.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 19:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:57 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 22:07 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 22:24 ` Stephen Frost <[email protected]>
1 sibling, 0 replies; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 22:24 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Andres Freund ([email protected]) wrote:
> On 2020-03-27 16:57:46 -0400, Stephen Frost wrote:
> > I really don't know what to say to this. WAL is absolutely critical to
> > a backup being valid. pgBackRest doesn't have a way to *just* validate
> > a backup today, unfortunately, but we're planning to support it in the
> > future and we will absolutely include in that validation checking all of
> > the WAL that's part of the backup.
>
> Could you please address the fact that just about everybody uses base
> backups + later WAL to have a short data loss window? Integrating the
> WAL files necessary to make the base backup consistent doesn't achieve
> much if we can't verify the WAL files afterwards. And fairly obviously
> pg_basebackup can't do much about WAL created after its invocation.
I feel like we have very different ideas about what "just about
everybody" does here. In my view, folks use pg_basebackup because it's
easy and they can create self-contained backups that include all the WAL
needed to get the backup up and running again and they don't typically
care about PITR all that much. Folks who care about PITR use something
that manages WAL for them, which pg_basebackup and pg_receivewal really
don't do and it's not easy to add scripting around them to figure out
what WAL is needed for what backup, etc.
If we didn't think that the ability to create a self-contained backup
was useful, it sure seems odd that we've done a lot to make that work
(having both fetch and stream modes for it) and that it's the default.
> Given that we need something separate to address that "verification
> hole", I don't see why it's useful to have a special case solution (or
> rather multiple ones, for stream and fetch) inside pg_basebackup.
Well, the proposal up-thread would end up with almost zero changes to
pg_basebackup itself, but, yes, there'd be changes to BASE_BACKUP and
different ones for STREAMING_REPLICATION to support getting the WAL
checksums into the manifest.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-23 22:42 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-24 18:04 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 13:31 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-25 16:50 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-25 20:54 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-26 15:37 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 04:30 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 15:26 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 19:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:57 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 22:07 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 22:33 ` David Steele <[email protected]>
1 sibling, 0 replies; 372+ messages in thread
From: David Steele @ 2020-03-27 22:33 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Stephen Frost <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/27/20 6:07 PM, Andres Freund wrote:
> Hi,
>
> On 2020-03-27 16:57:46 -0400, Stephen Frost wrote:
>> I really don't know what to say to this. WAL is absolutely critical to
>> a backup being valid. pgBackRest doesn't have a way to *just* validate
>> a backup today, unfortunately, but we're planning to support it in the
>> future and we will absolutely include in that validation checking all of
>> the WAL that's part of the backup.
>
> Could you please address the fact that just about everybody uses base
> backups + later WAL to have a short data loss window? Integrating the
> WAL files necessary to make the base backup consistent doesn't achieve
> much if we can't verify the WAL files afterwards. And fairly obviously
> pg_basebackup can't do much about WAL created after its invocation.
>
> Given that we need something separate to address that "verification
> hole", I don't see why it's useful to have a special case solution (or
> rather multiple ones, for stream and fetch) inside pg_basebackup.
There's a pretty big difference between not being able to play forward
to the end of WAL and not being able to get the backup to restore to
consistency at all.
The WAL that is generated during during the backup has special
importance. Without it you have no backup at all. It's the difference
between *some* data loss and *total* data loss.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-24 03:43 ` Amit Kapila <[email protected]>
2020-03-24 17:00 ` Re: backup manifests Robert Haas <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Amit Kapila @ 2020-03-24 03:43 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 23, 2020 at 9:46 PM Robert Haas <[email protected]> wrote:
>
> On Mon, Mar 23, 2020 at 7:04 AM Amit Kapila <[email protected]> wrote:
> > /src/bin/pg_validatebackup/tmp_check/t_002_algorithm_master_data/backup/none
> > --manifest-checksum none --no-sync
> > \tmp_install\bin\pg_basebackup.EXE: illegal option -- manifest-checksum
> >
> > It seems the option to be used should be --manifest-checksums. The
> > attached patch fixes this problem for me.
>
> OK, incorporated that.
>
> > > t/002_algorithm.pl ..... 4/19 # Failed test 'validate backup with
> > > algorithm "none"'
> > > # at t/002_algorithm.pl line 53.
> > >
> >
> > The error message for the above failure is:
> > pg_validatebackup: fatal: could not parse backup manifest: both
> > pathname and encoded pathname
> >
> > I don't know at this stage what could cause this? Any pointers?
>
> I think I forgot an initializer. Try this version.
>
All others except one are passing now. See the summary of the failed
test below and attached are failed run logs.
Test Summary Report
-------------------
t/003_corruption.pl (Wstat: 65280 Tests: 14 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 44 tests but ran 14.
Files=6, Tests=123, 164 wallclock secs ( 0.06 usr + 0.02 sys = 0.08 CPU)
Result: FAIL
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] 003_corruption_master.log (5.6K, ../../CAA4eK1K+QUVkAEEm8Lhgea-y=QPqO7eKFQeywNqqdTcurpKF-A@mail.gmail.com/2-003_corruption_master.log)
download
[application/octet-stream] regress_log_003_corruption (6.3K, ../../CAA4eK1K+QUVkAEEm8Lhgea-y=QPqO7eKFQeywNqqdTcurpKF-A@mail.gmail.com/3-regress_log_003_corruption)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-24 03:43 ` Re: backup manifests Amit Kapila <[email protected]>
@ 2020-03-24 17:00 ` Robert Haas <[email protected]>
2020-03-25 06:53 ` Re: backup manifests Amit Kapila <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-24 17:00 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Mon, Mar 23, 2020 at 11:43 PM Amit Kapila <[email protected]> wrote:
> All others except one are passing now. See the summary of the failed
> test below and attached are failed run logs.
>
> Test Summary Report
> -------------------
> t/003_corruption.pl (Wstat: 65280 Tests: 14 Failed: 0)
> Non-zero exit status: 255
> Parse errors: Bad plan. You planned 44 tests but ran 14.
> Files=6, Tests=123, 164 wallclock secs ( 0.06 usr + 0.02 sys = 0.08 CPU)
> Result: FAIL
Hmm. It looks like it's trying to remove the symlink that points to
the tablespace directory, and failing with no error message. I could
set that permutation to be skipped on Windows, or maybe there's an
alternate method you can suggest that would work?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-24 03:43 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-24 17:00 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-25 06:53 ` Amit Kapila <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Amit Kapila @ 2020-03-25 06:53 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Tue, Mar 24, 2020 at 10:30 PM Robert Haas <[email protected]> wrote:
>
> On Mon, Mar 23, 2020 at 11:43 PM Amit Kapila <[email protected]> wrote:
> > All others except one are passing now. See the summary of the failed
> > test below and attached are failed run logs.
> >
> > Test Summary Report
> > -------------------
> > t/003_corruption.pl (Wstat: 65280 Tests: 14 Failed: 0)
> > Non-zero exit status: 255
> > Parse errors: Bad plan. You planned 44 tests but ran 14.
> > Files=6, Tests=123, 164 wallclock secs ( 0.06 usr + 0.02 sys = 0.08 CPU)
> > Result: FAIL
>
> Hmm. It looks like it's trying to remove the symlink that points to
> the tablespace directory, and failing with no error message. I could
> set that permutation to be skipped on Windows, or maybe there's an
> alternate method you can suggest that would work?
>
We can use rmdir() for Windows. The attached patch fixes the failure
for me. I have tried the test on CentOS as well after the fix and it
passes there as well.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] fix_003_corruption.patch (661B, ../../CAA4eK1JsOvZaYAeXsU8tDvrycodPBn4-MUbo1hTZNwZaBRAO=w@mail.gmail.com/2-fix_003_corruption.patch)
download | inline diff:
diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
index 45e2220c38..6ad29a031f 100644
--- a/src/bin/pg_validatebackup/t/003_corruption.pl
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -172,7 +172,14 @@ sub mutilate_missing_tablespace
my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
slurp_dir("$backup_path/pg_tblspc");
my $pathname = "$backup_path/pg_tblspc/$tsoid";
- unlink($pathname) || die "$pathname: $!";
+ if ($windows_os)
+ {
+ rmdir($pathname) || die "$pathname: $!";
+ }
+ else
+ {
+ unlink($pathname) || die "$pathname: $!";
+ }
}
# Append an additional bytes to a file.
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 06:29 ` Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2 siblings, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-27 06:29 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-23 12:15:54 -0400, Robert Haas wrote:
> + <varlistentry>
> + <term><literal>MANIFEST</literal></term>
> + <listitem>
> + <para>
> + When this option is specified with a value of <literal>ye'</literal>
s/ye'/yes/
> + or <literal>force-escape</literal>, a backup manifest is created
> + and sent along with the backup. The latter value forces all filenames
> + to be hex-encoded; otherwise, this type of encoding is performed only
> + for files whose names are non-UTF8 octet sequences.
> + <literal>force-escape</literal> is intended primarily for testing
> + purposes, to be sure that clients which read the backup manifest
> + can handle this case. For compatibility with previous releases,
> + the default is <literal>MANIFEST 'no'</literal>.
> + </para>
> + </listitem>
> + </varlistentry>
Are you planning to include a specification of the manifest file format
anywhere? I looked through the patches and didn't find anything.
I think it'd also be good to include more information about what the
point of manifest files actually is.
> + <para>
> + <application>pg_validatebackup</application> reads the manifest file of a
> + backup, verifies the manifest against its own internal checksum, and then
> + verifies that the same files are present in the target directory as in the
> + manifest itself. It then verifies that each file has the expected checksum,
> + unless the backup was taken the checksum algorithm set to
> + <literal>none</literal>, in which case checksum verification is not
> + performed. The presence or absence of directories is not checked, except
> + indirectly: if a directory is missing, any files it should have contained
> + will necessarily also be missing. Certain files and directories are
> + excluded from verification:
> + </para>
Depending on what you want to use the manifest for, we'd also need to
check that there are no additional files. That seems to actually be
implemented, which imo should be mentioned here.
> +/*
> + * Finalize the backup manifest, and send it to the client.
> + */
> +static void
> +SendBackupManifest(manifest_info *manifest)
> +{
> + StringInfoData protobuf;
> + uint8 checksumbuf[PG_SHA256_DIGEST_LENGTH];
> + char checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH];
> + size_t manifest_bytes_done = 0;
> +
> + /*
> + * If there is no buffile, then the user doesn't want a manifest, so
> + * don't waste any time generating one.
> + */
> + if (manifest->buffile == NULL)
> + return;
> +
> + /* Terminate the list of files. */
> + AppendStringToManifest(manifest, "],\n");
> +
> + /*
> + * Append manifest checksum, so that the problems with the manifest itself
> + * can be detected.
> + *
> + * We always use SHA-256 for this, regardless of what algorithm is chosen
> + * for checksumming the files. If we ever want to make the checksum
> + * algorithm used for the manifest file variable, the client will need a
> + * way to figure out which algorithm to use as close to the beginning of
> + * the manifest file as possible, to avoid having to read the whole thing
> + * twice.
> + */
> + manifest->still_checksumming = false;
> + pg_sha256_final(&manifest->manifest_ctx, checksumbuf);
> + AppendStringToManifest(manifest, "\"Manifest-Checksum\": \"");
> + hex_encode((char *) checksumbuf, sizeof checksumbuf, checksumstringbuf);
> + checksumstringbuf[PG_SHA256_DIGEST_STRING_LENGTH - 1] = '\0';
> + AppendStringToManifest(manifest, checksumstringbuf);
> + AppendStringToManifest(manifest, "\"}\n");
Hm. Is it a great choice to include the checksum for the manifest inside
the manifest itself? With a cryptographic checksum it seems like it
could make a ton of sense to store the checksum somewhere "safe", but
keep the manifest itself alongside the base backup itself. While not
huge, they won't be tiny either.
> diff --git a/src/bin/pg_validatebackup/parse_manifest.c b/src/bin/pg_validatebackup/parse_manifest.c
> new file mode 100644
> index 0000000000..e6b42adfda
> --- /dev/null
> +++ b/src/bin/pg_validatebackup/parse_manifest.c
> @@ -0,0 +1,576 @@
> +/*-------------------------------------------------------------------------
> + *
> + * parse_manifest.c
> + * Parse a backup manifest in JSON format.
> + *
> + * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
> + * Portions Copyright (c) 1994, Regents of the University of California
> + *
> + * src/bin/pg_validatebackup/parse_manifest.c
> + *
> + *-------------------------------------------------------------------------
> + */
Doesn't have to be in the first version, but could it be useful to move
this to common/ or such?
> +/*
> + * Validate one directory.
> + *
> + * 'relpath' is NULL if we are to validate the top-level backup directory,
> + * and otherwise the relative path to the directory that is to be validated.
> + *
> + * 'fullpath' is the backup directory with 'relpath' appended; i.e. the actual
> + * filesystem path at which it can be found.
> + */
> +static void
> +validate_backup_directory(validator_context *context, char *relpath,
> + char *fullpath)
> +{
Hm. Should this warn if the directory's permissions are set too openly
(world writable?)?
> +/*
> + * Validate the checksum of a single file.
> + */
> +static void
> +validate_file_checksum(validator_context *context, manifestfile *tabent,
> + char *fullpath)
> +{
> + pg_checksum_context checksum_ctx;
> + char *relpath = tabent->pathname;
> + int fd;
> + int rc;
> + uint8 buffer[READ_CHUNK_SIZE];
> + uint8 checksumbuf[PG_CHECKSUM_MAX_LENGTH];
> + int checksumlen;
> +
> + /* Open the target file. */
> + if ((fd = open(fullpath, O_RDONLY | PG_BINARY, 0)) < 0)
> + {
> + report_backup_error(context, "could not open file \"%s\": %m",
> + relpath);
> + return;
> + }
> +
> + /* Initialize checksum context. */
> + pg_checksum_init(&checksum_ctx, tabent->checksum_type);
> +
> + /* Read the file chunk by chunk, updating the checksum as we go. */
> + while ((rc = read(fd, buffer, READ_CHUNK_SIZE)) > 0)
> + pg_checksum_update(&checksum_ctx, buffer, rc);
> + if (rc < 0)
> + report_backup_error(context, "could not read file \"%s\": %m",
> + relpath);
> +
Hm. I think it'd be good to verify that the checksummed size is the same
as the size of the file in the manifest.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 19:20 ` Robert Haas <[email protected]>
2020-03-27 20:02 ` Re: backup manifests David Steele <[email protected]>
2020-03-27 20:32 ` Re: backup manifests Andres Freund <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-27 19:20 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 2:29 AM Andres Freund <[email protected]> wrote:
> s/ye'/yes/
Ugh, sorry. Fixed in the version posted earlier.
> Are you planning to include a specification of the manifest file format
> anywhere? I looked through the patches and didn't find anything.
I thought about that. I think it would be good to have. I was sort of
hoping to leave it for a follow-on patch, but maybe that's cheating
too much.
> I think it'd also be good to include more information about what the
> point of manifest files actually is.
What kind of information do you want to see included there? Basically,
the way the documentation is written right now, it essentially says,
well, we have this manifest thing so that you can later run
pg_validatebackup, and pg_validatebackup says that it's there to check
the integrity of backups using the manifest. This is all a bit
circular, though, and maybe needs elaboration.
What I've experienced is that:
- Sometimes people take a backup and then wonder later whether the
disk has flipped some bits.
- Sometimes people restore a backup and forget some of the parts, like
the user-defined tablespaces.
- Sometimes anti-virus software, or poorly-run cron job run amok,
wander around inflicting unpredictable damage.
It would be nice to have a system that would notice these kinds of
things on a running system, but here I've got the more modest goal of
checking for in the context of a backup. If the data gets corrupted in
transit, or if the disk mutilates it, or if the user mutilates it, you
need something to check the backup against to find out that bad things
have happend; the manifest is that thing. But I don't know exactly how
much of all that should go in the docs, or in what way.
> > + <para>
> > + <application>pg_validatebackup</application> reads the manifest file of a
> > + backup, verifies the manifest against its own internal checksum, and then
> > + verifies that the same files are present in the target directory as in the
> > + manifest itself. It then verifies that each file has the expected checksum,
>
> Depending on what you want to use the manifest for, we'd also need to
> check that there are no additional files. That seems to actually be
> implemented, which imo should be mentioned here.
I intended the text to say that, because it says that it checks that
the two things are "the same," which is symmetric. In the new version
I posted a bit ago, I tried to make it more explicit, because
apparently it was not sufficiently clear.
> Hm. Is it a great choice to include the checksum for the manifest inside
> the manifest itself? With a cryptographic checksum it seems like it
> could make a ton of sense to store the checksum somewhere "safe", but
> keep the manifest itself alongside the base backup itself. While not
> huge, they won't be tiny either.
Seems like the user could just copy the manifest checksum and store it
somewhere, if they wish. Then they can check it against the manifest
itself later, if they wish. Or they can take a SHA-512 of the whole
file and store that securely. The problem is that we have no idea how
to write that checksum to a more security storage. We could write
backup_manifest and backup_manifest.checksum into separate files, but
that seems like it's adding complexity without any real benefit.
To me, the security-related uses of this patch seem to be fairly
niche. I think it's nice that they exist, but I don't think that's the
main selling point. For me, the main selling point is that you can
check that your disk didn't eat your data and that nobody nuked any
files that were supposed to be there.
> Doesn't have to be in the first version, but could it be useful to move
> this to common/ or such?
Yeah. At one point, this code was written in a way that was totally
specific to pg_validatebackup, but I then realized that it would be
better to make it more general, so I refactored it into in the form
you see now, where pg_validatebackup.c depends on parse_manifest.c but
not the reverse. I suspect that if someone wants to use this for
something else they might need to change a few more things - not sure
exactly what - but I don't think it would be too hard. I thought it
would be best to leave that task until someone has a concrete use case
in mind, but I did want it to to be relatively easy to do that down
the road, and I hope that the way I've organized the code achieves
that.
> > +static void
> > +validate_backup_directory(validator_context *context, char *relpath,
> > + char *fullpath)
> > +{
>
> Hm. Should this warn if the directory's permissions are set too openly
> (world writable?)?
I don't think so, but it's pretty clear that different people have
different ideas about what the scope of this tool ought to be, even in
this first version.
> Hm. I think it'd be good to verify that the checksummed size is the same
> as the size of the file in the manifest.
That's checked in an earlier phase. Are you worried about the file
being modified after the first pass checks the size and before we come
through to do the checksumming?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 20:02 ` David Steele <[email protected]>
2020-03-30 00:47 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: David Steele @ 2020-03-27 20:02 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Andres Freund <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/27/20 3:20 PM, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 2:29 AM Andres Freund <[email protected]> wrote:
>
>> Hm. Is it a great choice to include the checksum for the manifest inside
>> the manifest itself? With a cryptographic checksum it seems like it
>> could make a ton of sense to store the checksum somewhere "safe", but
>> keep the manifest itself alongside the base backup itself. While not
>> huge, they won't be tiny either.
>
> Seems like the user could just copy the manifest checksum and store it
> somewhere, if they wish. Then they can check it against the manifest
> itself later, if they wish. Or they can take a SHA-512 of the whole
> file and store that securely. The problem is that we have no idea how
> to write that checksum to a more security storage. We could write
> backup_manifest and backup_manifest.checksum into separate files, but
> that seems like it's adding complexity without any real benefit.
I agree that this seems like a separate problem. What Robert has done
here is detect random mutilation of the manifest.
To prevent malicious modifications you either need to store the checksum
in another place, or digitally sign the file and store that alongside it
(or inside it even). Either way seems pretty far out of scope to me.
>> Hm. I think it'd be good to verify that the checksummed size is the same
>> as the size of the file in the manifest.
>
> That's checked in an earlier phase. Are you worried about the file
> being modified after the first pass checks the size and before we come
> through to do the checksumming?
I prefer to validate the size and checksum in the same pass, but I'm not
sure it's that big a deal. If the backup is being corrupted under the
validate process that would also apply to files that had already been
validated.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:02 ` Re: backup manifests David Steele <[email protected]>
@ 2020-03-30 00:47 ` Robert Haas <[email protected]>
2020-03-30 00:59 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 01:05 ` Re: backup manifests David Steele <[email protected]>
0 siblings, 2 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-30 00:47 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Andres Freund <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 4:02 PM David Steele <[email protected]> wrote:
> I prefer to validate the size and checksum in the same pass, but I'm not
> sure it's that big a deal. If the backup is being corrupted under the
> validate process that would also apply to files that had already been
> validated.
I did it like this because I thought that in typical scenarios it
would be likely to produce useful results more quickly. For instance,
suppose that you forget to restore the tablespace directories, and
just get the main $PGDATA directory. Well, if you do it all in one
pass, you might spend a long time checksumming things before you
realize that some files are completely missing. I thought it would be
useful to complain about files that are extra or missing or the wrong
size FIRST, because that only requires us to stat() each file, and
only after that do the comparatively extensive checksumming step that
requires us to read the entire contents of each file. Granted, unless
you use --exit-on-error, you're going to get all the complaints
eventually anyway, but you might use that option, or you might hit ^C
when you start to see a slough of complaints poppoing out.
Maybe that was the wrong idea, but I thought people would like the
idea of running cheaper checks first. I wasn't worried about
concurrent modification of the backup because then you're super-hosed
no matter what.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:02 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 00:47 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 00:59 ` Andres Freund <[email protected]>
1 sibling, 0 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-30 00:59 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: David Steele <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-29 20:47:40 -0400, Robert Haas wrote:
> Maybe that was the wrong idea, but I thought people would like the
> idea of running cheaper checks first. I wasn't worried about
> concurrent modification of the backup because then you're super-hosed
> no matter what.
I do like that approach.
To be clear: I'm suggesting the additional crosscheck not because I'm
not concerned with concurrent modifications, but because I've seen
filesystem per-inode metadata and the actual data / extent-tree
differ. Leading to EOF reported while reading at a different place than
what the size via stat() would indicate.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:02 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 00:47 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 01:05 ` David Steele <[email protected]>
2020-03-30 20:43 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: David Steele @ 2020-03-30 01:05 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/29/20 8:47 PM, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 4:02 PM David Steele <[email protected]> wrote:
>> I prefer to validate the size and checksum in the same pass, but I'm not
>> sure it's that big a deal. If the backup is being corrupted under the
>> validate process that would also apply to files that had already been
>> validated.
>
> I did it like this because I thought that in typical scenarios it
> would be likely to produce useful results more quickly. For instance,
> suppose that you forget to restore the tablespace directories, and
> just get the main $PGDATA directory. Well, if you do it all in one
> pass, you might spend a long time checksumming things before you
> realize that some files are completely missing. I thought it would be
> useful to complain about files that are extra or missing or the wrong
> size FIRST, because that only requires us to stat() each file, and
> only after that do the comparatively extensive checksumming step that
> requires us to read the entire contents of each file. Granted, unless
> you use --exit-on-error, you're going to get all the complaints
> eventually anyway, but you might use that option, or you might hit ^C
> when you start to see a slough of complaints poppoing out.
Yeah, that seems reasonable.
In our case backups are nearly always compressed and/or encrypted so
even checking the original size is a bit of work. Getting the checksum
at the same time seems like an obvious win.
Currently we don't have a separate validate command outside of restore
but when we do we'll consider doing a pass to check for file presence
(and size when possible) first. Thanks!
> I wasn't worried about
> concurrent modification of the backup because then you're super-hosed
> no matter what.
Really, really, super-hosed.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:02 ` Re: backup manifests David Steele <[email protected]>
2020-03-30 00:47 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-30 01:05 ` Re: backup manifests David Steele <[email protected]>
@ 2020-03-30 20:43 ` Robert Haas <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Robert Haas @ 2020-03-30 20:43 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Andres Freund <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Sun, Mar 29, 2020 at 9:05 PM David Steele <[email protected]> wrote:
> Yeah, that seems reasonable.
>
> In our case backups are nearly always compressed and/or encrypted so
> even checking the original size is a bit of work. Getting the checksum
> at the same time seems like an obvious win.
Makes sense. If this even got extended so it could read from tar-files
instead of the filesystem directly, we'd surely want to take the
opposite approach and just make a single pass. I'm not sure whether
it's worth doing that at some point in the future, but it might be. If
we're going to add the capability to compress or encrypt backups to
pg_basebackup, we might want to do that first, and then make this tool
handle all of those formats in one go.
(As always, I don't have the ability to control how arbitrary
developers spend their development time... so this is just a thought.)
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-27 20:32 ` Andres Freund <[email protected]>
2020-03-27 21:44 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-30 00:33 ` Re: backup manifests Robert Haas <[email protected]>
1 sibling, 2 replies; 372+ messages in thread
From: Andres Freund @ 2020-03-27 20:32 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-27 15:20:27 -0400, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 2:29 AM Andres Freund <[email protected]> wrote:
> > Are you planning to include a specification of the manifest file format
> > anywhere? I looked through the patches and didn't find anything.
>
> I thought about that. I think it would be good to have. I was sort of
> hoping to leave it for a follow-on patch, but maybe that's cheating
> too much.
I don't like having a file format that's intended to be used by external
tools too that's undocumented except for code that assembles it in a
piecemeal fashion. Do you mean in a follow-on patch this release, or
later? I don't have a problem with the former.
> > I think it'd also be good to include more information about what the
> > point of manifest files actually is.
>
> What kind of information do you want to see included there? Basically,
> the way the documentation is written right now, it essentially says,
> well, we have this manifest thing so that you can later run
> pg_validatebackup, and pg_validatebackup says that it's there to check
> the integrity of backups using the manifest. This is all a bit
> circular, though, and maybe needs elaboration.
I do found it to be circular. I think we mostly need a paragraph or two
somewhere that explains on a higher level what the point of verifying
base backups is and what is verified.
> > Hm. Is it a great choice to include the checksum for the manifest inside
> > the manifest itself? With a cryptographic checksum it seems like it
> > could make a ton of sense to store the checksum somewhere "safe", but
> > keep the manifest itself alongside the base backup itself. While not
> > huge, they won't be tiny either.
>
> Seems like the user could just copy the manifest checksum and store it
> somewhere, if they wish. Then they can check it against the manifest
> itself later, if they wish. Or they can take a SHA-512 of the whole
> file and store that securely. The problem is that we have no idea how
> to write that checksum to a more security storage. We could write
> backup_manifest and backup_manifest.checksum into separate files, but
> that seems like it's adding complexity without any real benefit.
>
> To me, the security-related uses of this patch seem to be fairly
> niche. I think it's nice that they exist, but I don't think that's the
> main selling point. For me, the main selling point is that you can
> check that your disk didn't eat your data and that nobody nuked any
> files that were supposed to be there.
Oh, I agree. I wasn't really mentioning the crypto checksum because of
it being "security" stuff, but because of the quality of the guarantee
it gives. I don't know how large the manifest file will be for a setup
of with a lot of partitioned tables, but I'd expect it to not be
tiny. So not having to store it in the 'archiving sytem' is nice.
FWIW, I was thinking of backup_manifest.checksum potentially being
desirable for another reason: The need to embed the checksum inside the
document imo adds a fair bit of rigidity to the file format. See
> +static void
> +verify_manifest_checksum(JsonManifestParseState *parse, char *buffer,
> + size_t size)
> +{
...
> +
> + /* Find the last two newlines in the file. */
> + for (i = 0; i < size; ++i)
> + {
> + if (buffer[i] == '\n')
> + {
> + ++number_of_newlines;
> + penultimate_newline = ultimate_newline;
> + ultimate_newline = i;
> + }
> + }
> +
> + /*
> + * Make sure that the last newline is right at the end, and that there are
> + * at least two lines total. We need this to be true in order for the
> + * following code, which computes the manifest checksum, to work properly.
> + */
> + if (number_of_newlines < 2)
> + json_manifest_parse_failure(parse->context,
> + "expected at least 2 lines");
> + if (ultimate_newline != size - 1)
> + json_manifest_parse_failure(parse->context,
> + "last line not newline-terminated");
> +
> + /* Checksum the rest. */
> + pg_sha256_init(&manifest_ctx);
> + pg_sha256_update(&manifest_ctx, (uint8 *) buffer, penultimate_newline + 1);
> + pg_sha256_final(&manifest_ctx, manifest_checksum_actual);
which certainly isn't "free form json".
> > Doesn't have to be in the first version, but could it be useful to move
> > this to common/ or such?
>
> Yeah. At one point, this code was written in a way that was totally
> specific to pg_validatebackup, but I then realized that it would be
> better to make it more general, so I refactored it into in the form
> you see now, where pg_validatebackup.c depends on parse_manifest.c but
> not the reverse. I suspect that if someone wants to use this for
> something else they might need to change a few more things - not sure
> exactly what - but I don't think it would be too hard. I thought it
> would be best to leave that task until someone has a concrete use case
> in mind, but I did want it to to be relatively easy to do that down
> the road, and I hope that the way I've organized the code achieves
> that.
Cool.
> > > +static void
> > > +validate_backup_directory(validator_context *context, char *relpath,
> > > + char *fullpath)
> > > +{
> >
> > Hm. Should this warn if the directory's permissions are set too openly
> > (world writable?)?
>
> I don't think so, but it's pretty clear that different people have
> different ideas about what the scope of this tool ought to be, even in
> this first version.
Yea. I don't have a strong opinion on this specific issue. I was mostly
wondering because I've repeatedly seen people restore backups with world
readable properties, and with that it's obviously possible for somebody
else to change the contents after the checksum was computed.
> > Hm. I think it'd be good to verify that the checksummed size is the same
> > as the size of the file in the manifest.
>
> That's checked in an earlier phase. Are you worried about the file
> being modified after the first pass checks the size and before we come
> through to do the checksumming?
Not really, I wondered about it for a bit, and then decided that it's
too remote an issue.
What I've seen a couple of times is that actually reading a file can
result in the file ending to be reported at a different position than
what stat() said. So by crosschecking the size while reading with the
one from stat (which was compared with the source system one) we'd make
the errors much better. It's certainly easier to know where to start
looking when validate says "error: read %llu bytes from file, expected
%llu" or something along those lines, than when it just were to report a
checksum error.
There's also some crypto hash algorithm weaknesses that are easier to
exploit when it's possible to append data to a known prefix, but that
doesn't seem an obvious threat here.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:32 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 21:44 ` Stephen Frost <[email protected]>
2020-03-27 21:56 ` Re: backup manifests Andres Freund <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 21:44 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Andres Freund ([email protected]) wrote:
> On 2020-03-27 15:20:27 -0400, Robert Haas wrote:
> > On Fri, Mar 27, 2020 at 2:29 AM Andres Freund <[email protected]> wrote:
> > > Hm. Should this warn if the directory's permissions are set too openly
> > > (world writable?)?
> >
> > I don't think so, but it's pretty clear that different people have
> > different ideas about what the scope of this tool ought to be, even in
> > this first version.
>
> Yea. I don't have a strong opinion on this specific issue. I was mostly
> wondering because I've repeatedly seen people restore backups with world
> readable properties, and with that it's obviously possible for somebody
> else to change the contents after the checksum was computed.
For my 2c, at least, I don't think we need to check the directory
permissions, but I wouldn't object to including a warning if they're set
such that PG won't start. I suppose +0 for "warn if they are such that
PG won't start".
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:32 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 21:44 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 21:56 ` Andres Freund <[email protected]>
2020-03-27 22:09 ` Re: backup manifests Stephen Frost <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Andres Freund @ 2020-03-27 21:56 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Hi,
On 2020-03-27 17:44:07 -0400, Stephen Frost wrote:
> * Andres Freund ([email protected]) wrote:
> > On 2020-03-27 15:20:27 -0400, Robert Haas wrote:
> > > On Fri, Mar 27, 2020 at 2:29 AM Andres Freund <[email protected]> wrote:
> > > > Hm. Should this warn if the directory's permissions are set too openly
> > > > (world writable?)?
> > >
> > > I don't think so, but it's pretty clear that different people have
> > > different ideas about what the scope of this tool ought to be, even in
> > > this first version.
> >
> > Yea. I don't have a strong opinion on this specific issue. I was mostly
> > wondering because I've repeatedly seen people restore backups with world
> > readable properties, and with that it's obviously possible for somebody
> > else to change the contents after the checksum was computed.
>
> For my 2c, at least, I don't think we need to check the directory
> permissions, but I wouldn't object to including a warning if they're set
> such that PG won't start. I suppose +0 for "warn if they are such that
> PG won't start".
I was thinking of that check not being just at the top-level, but in
subdirectories too. It's easy to screw up the top and subdirectory
permissions in different ways, e.g. when manually creating the database
dir and then restoring a data directory directly into that. IIRC
postmaster doesn't check that at start.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:32 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 21:44 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 21:56 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-27 22:09 ` Stephen Frost <[email protected]>
2020-03-27 22:17 ` Re: backup manifests Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 372+ messages in thread
From: Stephen Frost @ 2020-03-27 22:09 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
Greetings,
* Andres Freund ([email protected]) wrote:
> On 2020-03-27 17:44:07 -0400, Stephen Frost wrote:
> > * Andres Freund ([email protected]) wrote:
> > > On 2020-03-27 15:20:27 -0400, Robert Haas wrote:
> > > > On Fri, Mar 27, 2020 at 2:29 AM Andres Freund <[email protected]> wrote:
> > > > > Hm. Should this warn if the directory's permissions are set too openly
> > > > > (world writable?)?
> > > >
> > > > I don't think so, but it's pretty clear that different people have
> > > > different ideas about what the scope of this tool ought to be, even in
> > > > this first version.
> > >
> > > Yea. I don't have a strong opinion on this specific issue. I was mostly
> > > wondering because I've repeatedly seen people restore backups with world
> > > readable properties, and with that it's obviously possible for somebody
> > > else to change the contents after the checksum was computed.
> >
> > For my 2c, at least, I don't think we need to check the directory
> > permissions, but I wouldn't object to including a warning if they're set
> > such that PG won't start. I suppose +0 for "warn if they are such that
> > PG won't start".
>
> I was thinking of that check not being just at the top-level, but in
> subdirectories too. It's easy to screw up the top and subdirectory
> permissions in different ways, e.g. when manually creating the database
> dir and then restoring a data directory directly into that. IIRC
> postmaster doesn't check that at start.
Yeah, I'm pretty sure we don't check that at postmaster start.. which
also means that we'll start up just fine even if the perms on
subdirectories are odd or wrong, unless maybe we end up in a really odd
state where a directory is 000'd or something.
Of course.. this is all a mess when it comes to pg_basebackup, really,
as previously discussed elsewhere, because what permissions and such you
end up with actually depends on what *format* you use with
pg_basebackup- it's different between 'tar' format and 'plain' format.
That is, if you use 'tar' format, and then actually use 'tar' to
extract, you get one set of privs, but if you use 'plain', you get
something different.
I mean.. pgBackRest sets all perms to whatever is in the manifest on
restore (or delta), but this patch doesn't include the permissions on
files, or ownership (something pgBackRest also tries to set, if
possible, on restore), does it...? Doesn't look like it on a quick
look. So if we want to compare to pgBackRest then, yes, we should
include the permissions in the manifest and we should check that
everything in the manifest matches what's on the filesystem.
I don't think we should just compare all permissions or ownership with
some arbitrary idea of what we think they should be, even though if you
use pg_basebackup in 'plain' format, you actually end up with
differences, today, from what the source system has. In my view, that
should actually be fixed, to the extent possible.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:32 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 21:44 ` Re: backup manifests Stephen Frost <[email protected]>
2020-03-27 21:56 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 22:09 ` Re: backup manifests Stephen Frost <[email protected]>
@ 2020-03-27 22:17 ` Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Alvaro Herrera @ 2020-03-27 22:17 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Andres Freund <[email protected]>; Robert Haas <[email protected]>; Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 2020-Mar-27, Stephen Frost wrote:
> I don't think we should just compare all permissions or ownership with
> some arbitrary idea of what we think they should be, even though if you
> use pg_basebackup in 'plain' format, you actually end up with
> differences, today, from what the source system has. In my view, that
> should actually be fixed, to the extent possible.
I posted some thoughts about this at
https://www.postgresql.org/message-id/20190904201117.GA12986%40alvherre.pgsql
I didn't get time to work on that myself.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:32 ` Re: backup manifests Andres Freund <[email protected]>
@ 2020-03-30 00:33 ` Robert Haas <[email protected]>
2020-03-30 00:48 ` Re: backup manifests David Steele <[email protected]>
1 sibling, 1 reply; 372+ messages in thread
From: Robert Haas @ 2020-03-30 00:33 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Fri, Mar 27, 2020 at 4:32 PM Andres Freund <[email protected]> wrote:
> I don't like having a file format that's intended to be used by external
> tools too that's undocumented except for code that assembles it in a
> piecemeal fashion. Do you mean in a follow-on patch this release, or
> later? I don't have a problem with the former.
This release. I'm happy to work on that as soon as this gets
committed, assuming it gets committed.
> I do found it to be circular. I think we mostly need a paragraph or two
> somewhere that explains on a higher level what the point of verifying
> base backups is and what is verified.
Fair enough.
> FWIW, I was thinking of backup_manifest.checksum potentially being
> desirable for another reason: The need to embed the checksum inside the
> document imo adds a fair bit of rigidity to the file format. See
Well, David Steele suggested this approach. I didn't particularly like
it, but nobody showed up to agree with me or propose anything
different, so here we are. I don't think it's the end of the world.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-16 05:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-21 12:26 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 11:04 ` Re: backup manifests Amit Kapila <[email protected]>
2020-03-23 16:15 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 06:29 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-27 19:20 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-27 20:32 ` Re: backup manifests Andres Freund <[email protected]>
2020-03-30 00:33 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-30 00:48 ` David Steele <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: David Steele @ 2020-03-30 00:48 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Andres Freund <[email protected]>; +Cc: Amit Kapila <[email protected]>; Suraj Kharage <[email protected]>; tushar <[email protected]>; Rajkumar Raghuwanshi <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/29/20 8:33 PM, Robert Haas wrote:
> On Fri, Mar 27, 2020 at 4:32 PM Andres Freund <[email protected]> wrote:
>
>> FWIW, I was thinking of backup_manifest.checksum potentially being
>> desirable for another reason: The need to embed the checksum inside the
>> document imo adds a fair bit of rigidity to the file format. See
>
> Well, David Steele suggested this approach. I didn't particularly like
> it, but nobody showed up to agree with me or propose anything
> different, so here we are. I don't think it's the end of the world.
I prefer the embedded checksum even though it is a pain. It's a lot less
likely to go missing.
--
-David
[email protected]
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
2020-03-04 13:51 ` Re: backup manifests tushar <[email protected]>
2020-03-05 04:07 ` Re: backup manifests Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Re: backup manifests Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` Re: backup manifests tushar <[email protected]>
2020-03-05 12:05 ` Re: backup manifests tushar <[email protected]>
2020-03-05 16:55 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-09 16:22 ` Re: backup manifests tushar <[email protected]>
2020-03-09 17:16 ` Re: backup manifests Robert Haas <[email protected]>
2020-03-12 14:46 ` Re: backup manifests tushar <[email protected]>
2020-03-13 20:34 ` Re: backup manifests Robert Haas <[email protected]>
@ 2020-03-16 10:22 ` tushar <[email protected]>
1 sibling, 0 replies; 372+ messages in thread
From: tushar @ 2020-03-16 10:22 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Rajkumar Raghuwanshi <[email protected]>; Suraj Kharage <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On 3/14/20 2:04 AM, Robert Haas wrote:
> OK. Done in the attached version
Thanks. Verified.
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 372+ messages in thread
* Re: backup manifests
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` Re: backup manifests tushar <[email protected]>
2020-03-03 14:49 ` Re: backup manifests tushar <[email protected]>
2020-03-04 09:56 ` Re: backup manifests tushar <[email protected]>
2020-03-04 10:21 ` Re: backup manifests tushar <[email protected]>
@ 2020-03-05 03:50 ` Suraj Kharage <[email protected]>
1 sibling, 0 replies; 372+ messages in thread
From: Suraj Kharage @ 2020-03-05 03:50 UTC (permalink / raw)
To: tushar <[email protected]>; +Cc: Robert Haas <[email protected]>; Rushabh Lathia <[email protected]>; Tels <[email protected]>; David Steele <[email protected]>; Andrew Dunstan <[email protected]>; pgsql-hackers; Jeevan Chalke <[email protected]>; vignesh C <[email protected]>
On Wed, Mar 4, 2020 at 3:51 PM tushar <[email protected]> wrote:
> Another scenario, in which if we modify Manifest-Checksum" value from
> backup_manifest file , we are not getting an error
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data/
> pg_validatebackup: * manifest_checksum =
> 28d082921650d0ae881de8ceb122c8d2af5f449f51ecfb446827f7f49f91f65d
> pg_validatebackup: backup successfully verified
>
> open backup_manifest file and replace
>
> "Manifest-Checksum":
> "8d082921650d0ae881de8ceb122c8d2af5f449f51ecfb446827f7f49f91f65d"}
> with
> "Manifest-Checksum": "Hello World"}
>
> rerun the pg_validatebackup
>
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data/
> pg_validatebackup: * manifest_checksum = Hello World
> pg_validatebackup: backup successfully verified
>
> regards,
>
Yeah, This handling is missing in the provided WIP patch. I believe Robert
will consider this fixing in upcoming version of validator patch.
--
--
Thanks & Regards,
Suraj kharage,
EnterpriseDB Corporation,
The Postgres Database Company.
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
* [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT()
@ 2023-01-21 00:09 Andres Freund <[email protected]>
0 siblings, 0 replies; 372+ messages in thread
From: Andres Freund @ 2023-01-21 00:09 UTC (permalink / raw)
This just updates the places that "Zero initialize instr_time uses causing
compiler warnings" changed, to see whether this is a nicer approach.
---
src/include/portability/instr_time.h | 24 ++++++++++++++------
src/backend/access/transam/xlog.c | 11 +++-------
src/backend/storage/buffer/bufmgr.c | 16 +++++---------
src/backend/storage/file/buffile.c | 16 +++++---------
src/backend/storage/ipc/latch.c | 8 +++----
src/bin/psql/common.c | 33 +++++++++++++---------------
6 files changed, 50 insertions(+), 58 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index af2ab6ec887..2d1ff4f7f82 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -17,6 +17,10 @@
*
* INSTR_TIME_IS_LT(x, y) x < y
*
+ * INSTR_TIME_ZERO() an instr_time set to 0
+ *
+ * INSTR_TIME_CURRENT() an instr_time set to current time
+ *
* INSTR_TIME_SET_ZERO(t) set t to zero (memset is acceptable too)
*
* INSTR_TIME_SET_CURRENT(t) set t to current time
@@ -110,7 +114,7 @@ typedef struct instr_time
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_clock_gettime_ns(void)
{
@@ -123,8 +127,8 @@ pg_clock_gettime_ns(void)
return now;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_clock_gettime_ns())
+#define INSTR_TIME_CURRENT(t) \
+ pg_clock_gettime_ns()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = NS_PER_S * (s))
@@ -138,7 +142,7 @@ pg_clock_gettime_ns(void)
/* Use QueryPerformanceCounter() */
-/* helper for INSTR_TIME_SET_CURRENT */
+/* helper for INSTR_TIME_CURRENT */
static inline instr_time
pg_query_performance_counter(void)
{
@@ -160,8 +164,8 @@ GetTimerFrequency(void)
return (double) f.QuadPart;
}
-#define INSTR_TIME_SET_CURRENT(t) \
- ((t) = pg_query_performance_counter())
+#define INSTR_TIME_CURRENT(t) \
+ pg_query_performance_counter()
#define INSTR_TIME_SET_SECONDS(t, s) \
((t).ticks = s * GetTimerFrequency())
@@ -181,7 +185,13 @@ GetTimerFrequency(void)
#define INSTR_TIME_IS_LT(x, y) ((x).ticks < (y).ticks)
-#define INSTR_TIME_SET_ZERO(t) ((t).ticks = 0)
+#define INSTR_TIME_ZERO(t) (instr_time){0}
+
+#define INSTR_TIME_SET_CURRENT(t) \
+ (t) = INSTR_TIME_CURRENT()
+
+#define INSTR_TIME_SET_ZERO(t) \
+ ((t) = INSTR_TIME_ZERO())
#define INSTR_TIME_SET_CURRENT_LAZY(t) \
(INSTR_TIME_IS_ZERO(t) ? INSTR_TIME_SET_CURRENT(t), true : false)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bdea..f563800c8ab 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2178,7 +2178,7 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
Size nbytes;
Size nleft;
int written;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
@@ -2191,8 +2191,6 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
/* Measure I/O timing to write WAL data */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
written = pg_pwrite(openLogFile, from, nleft, startoffset);
@@ -2204,9 +2202,8 @@ XLogWrite(XLogwrtRqst WriteRqst, TimeLineID tli, bool flexible)
*/
if (track_wal_io_timing)
{
- instr_time duration;
+ instr_time duration = INSTR_TIME_CURRENT();
- INSTR_TIME_SET_CURRENT(duration);
INSTR_TIME_SUBTRACT(duration, start);
PendingWalStats.wal_write_time += INSTR_TIME_GET_MICROSEC(duration);
}
@@ -8137,7 +8134,7 @@ void
issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
{
char *msg = NULL;
- instr_time start;
+ instr_time start = INSTR_TIME_ZERO();
Assert(tli != 0);
@@ -8153,8 +8150,6 @@ issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli)
/* Measure I/O timing to sync the WAL file */
if (track_wal_io_timing)
INSTR_TIME_SET_CURRENT(start);
- else
- INSTR_TIME_SET_ZERO(start);
pgstat_report_wait_start(WAIT_EVENT_WAL_SYNC);
switch (sync_method)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 800a4248c95..d8baf80e650 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1012,19 +1012,17 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
else
{
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
smgrread(smgr, forkNum, blockNum, (char *) bufBlock);
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_read_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_read_time, io_time);
@@ -2826,8 +2824,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
- instr_time io_start,
- io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
@@ -2904,8 +2901,6 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* bufToWrite is either the shared buffer or a copy, as appropriate.
@@ -2918,7 +2913,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
pgstat_count_buffer_write_time(INSTR_TIME_GET_MICROSEC(io_time));
INSTR_TIME_ADD(pgBufferUsage.blk_write_time, io_time);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 0a51624df3b..6f813279690 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -429,8 +429,7 @@ static void
BufFileLoadBuffer(BufFile *file)
{
File thisfile;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -446,8 +445,6 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
/*
* Read whatever we can get, up to a full bufferload.
@@ -468,7 +465,8 @@ BufFileLoadBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_read_time, io_time);
}
@@ -500,8 +498,7 @@ BufFileDumpBuffer(BufFile *file)
while (wpos < file->nbytes)
{
off_t availbytes;
- instr_time io_start;
- instr_time io_time;
+ instr_time io_start = INSTR_TIME_ZERO();
/*
* Advance to next component file if necessary and possible.
@@ -527,8 +524,6 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
- else
- INSTR_TIME_SET_ZERO(io_start);
bytestowrite = FileWrite(thisfile,
file->buffer.data + wpos,
@@ -543,7 +538,8 @@ BufFileDumpBuffer(BufFile *file)
if (track_io_timing)
{
- INSTR_TIME_SET_CURRENT(io_time);
+ instr_time io_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(io_time, io_start);
INSTR_TIME_ADD(pgBufferUsage.temp_blk_write_time, io_time);
}
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index f4123e7de7e..8092ff4a984 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -1385,8 +1385,7 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
uint32 wait_event_info)
{
int returned_events = 0;
- instr_time start_time;
- instr_time cur_time;
+ instr_time start_time = INSTR_TIME_ZERO();
long cur_timeout = -1;
Assert(nevents > 0);
@@ -1401,8 +1400,6 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
Assert(timeout >= 0 && timeout <= INT_MAX);
cur_timeout = timeout;
}
- else
- INSTR_TIME_SET_ZERO(start_time);
pgstat_report_wait_start(wait_event_info);
@@ -1489,7 +1486,8 @@ WaitEventSetWait(WaitEventSet *set, long timeout,
/* If we're not done, update cur_timeout for next iteration */
if (returned_events == 0 && timeout >= 0)
{
- INSTR_TIME_SET_CURRENT(cur_time);
+ instr_time cur_time = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(cur_time, start_time);
cur_timeout = timeout - (long) INSTR_TIME_GET_MILLISEC(cur_time);
if (cur_timeout <= 0)
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8d..5badb029e83 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1269,15 +1269,12 @@ DescribeQuery(const char *query, double *elapsed_msec)
bool timing = pset.timing;
PGresult *result;
bool OK;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
*elapsed_msec = 0;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/*
* To parse the query but not execute it, we prepare it, using the unnamed
@@ -1350,7 +1347,8 @@ DescribeQuery(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1400,16 +1398,13 @@ ExecQueryAndProcessResults(const char *query,
{
bool timing = pset.timing;
bool success;
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
PGresult *result;
FILE *gfile_fout = NULL;
bool gfile_is_pipe = false;
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
if (pset.bind_flag)
success = PQsendQueryParams(pset.db, query, pset.bind_nparams, NULL, (const char * const *) pset.bind_params, NULL, NULL, 0);
@@ -1490,7 +1485,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1595,7 +1591,8 @@ ExecQueryAndProcessResults(const char *query,
*/
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec = INSTR_TIME_GET_MILLISEC(after);
}
@@ -1693,8 +1690,7 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
int ntuples;
int fetch_count;
char fetch_cmd[64];
- instr_time before,
- after;
+ instr_time before = INSTR_TIME_ZERO();
int flush_error;
*elapsed_msec = 0;
@@ -1706,8 +1702,6 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
INSTR_TIME_SET_CURRENT(before);
- else
- INSTR_TIME_SET_ZERO(before);
/* if we're not in a transaction, start one */
if (PQtransactionStatus(pset.db) == PQTRANS_IDLE)
@@ -1738,7 +1732,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1786,7 +1781,8 @@ ExecQueryUsingCursor(const char *query, double *elapsed_msec)
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
@@ -1926,7 +1922,8 @@ cleanup:
if (timing)
{
- INSTR_TIME_SET_CURRENT(after);
+ instr_time after = INSTR_TIME_CURRENT();
+
INSTR_TIME_SUBTRACT(after, before);
*elapsed_msec += INSTR_TIME_GET_MILLISEC(after);
}
--
2.38.0
--vt42kbp2j7rjcapp--
^ permalink raw reply [nested|flat] 372+ messages in thread
end of thread, other threads:[~2023-01-21 00:09 UTC | newest]
Thread overview: 372+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-02-27 15:52 Re: backup manifests Robert Haas <[email protected]>
2020-03-03 10:34 ` tushar <[email protected]>
2020-03-03 14:49 ` tushar <[email protected]>
2020-03-04 09:56 ` tushar <[email protected]>
2020-03-04 10:21 ` tushar <[email protected]>
2020-03-04 13:51 ` tushar <[email protected]>
2020-03-05 04:07 ` Suraj Kharage <[email protected]>
2020-03-05 07:39 ` Rajkumar Raghuwanshi <[email protected]>
2020-03-05 10:10 ` tushar <[email protected]>
2020-03-05 12:05 ` tushar <[email protected]>
2020-03-05 16:55 ` Robert Haas <[email protected]>
2020-03-06 08:58 ` Suraj Kharage <[email protected]>
2020-03-11 20:08 ` Robert Haas <[email protected]>
2020-03-09 16:22 ` tushar <[email protected]>
2020-03-09 17:16 ` Robert Haas <[email protected]>
2020-03-12 14:46 ` tushar <[email protected]>
2020-03-13 13:53 ` tushar <[email protected]>
2020-03-13 16:54 ` Robert Haas <[email protected]>
2020-03-13 20:34 ` Robert Haas <[email protected]>
2020-03-16 05:07 ` Suraj Kharage <[email protected]>
2020-03-16 06:03 ` Suraj Kharage <[email protected]>
2020-03-20 22:29 ` Robert Haas <[email protected]>
2020-03-21 12:26 ` Amit Kapila <[email protected]>
2020-03-23 11:04 ` Amit Kapila <[email protected]>
2020-03-23 16:15 ` Robert Haas <[email protected]>
2020-03-23 22:42 ` Stephen Frost <[email protected]>
2020-03-24 18:04 ` Robert Haas <[email protected]>
2020-03-25 13:31 ` Stephen Frost <[email protected]>
2020-03-25 16:50 ` Robert Haas <[email protected]>
2020-03-25 20:54 ` Stephen Frost <[email protected]>
2020-03-26 15:37 ` Robert Haas <[email protected]>
2020-03-26 16:34 ` Stephen Frost <[email protected]>
2020-03-26 17:40 ` Mark Dilger <[email protected]>
2020-03-26 19:37 ` Stephen Frost <[email protected]>
2020-03-26 20:38 ` Mark Dilger <[email protected]>
2020-03-26 21:00 ` Stephen Frost <[email protected]>
2020-03-27 05:31 ` Andres Freund <[email protected]>
2020-03-26 18:02 ` Robert Haas <[email protected]>
2020-03-26 20:44 ` Stephen Frost <[email protected]>
2020-03-27 18:53 ` Robert Haas <[email protected]>
2020-03-27 19:55 ` Stephen Frost <[email protected]>
2020-03-27 20:39 ` David Steele <[email protected]>
2020-03-27 05:06 ` Andres Freund <[email protected]>
2020-03-27 18:13 ` Robert Haas <[email protected]>
2020-03-27 19:56 ` Andres Freund <[email protected]>
2020-03-27 22:36 ` Bruce Momjian <[email protected]>
2020-03-27 22:38 ` Stephen Frost <[email protected]>
2020-03-27 22:39 ` Bruce Momjian <[email protected]>
2020-03-26 20:37 ` David Steele <[email protected]>
2020-03-27 17:53 ` Robert Haas <[email protected]>
2020-03-27 19:50 ` David Steele <[email protected]>
2020-03-30 20:16 ` Robert Haas <[email protected]>
2020-03-30 23:24 ` David Steele <[email protected]>
2020-03-31 11:57 ` Robert Haas <[email protected]>
2020-03-29 03:40 ` Noah Misch <[email protected]>
2020-03-30 00:42 ` Robert Haas <[email protected]>
2020-03-30 00:54 ` David Steele <[email protected]>
2020-03-30 01:07 ` Andres Freund <[email protected]>
2020-03-30 01:23 ` David Steele <[email protected]>
2020-03-30 02:08 ` Andres Freund <[email protected]>
2020-03-30 18:35 ` Robert Haas <[email protected]>
2020-03-30 18:59 ` Andres Freund <[email protected]>
2020-03-30 19:23 ` Robert Haas <[email protected]>
2020-03-30 21:08 ` Andres Freund <[email protected]>
2020-03-30 22:56 ` David Steele <[email protected]>
2020-03-31 18:10 ` Robert Haas <[email protected]>
2020-03-31 22:50 ` Andres Freund <[email protected]>
2020-03-30 05:58 ` Noah Misch <[email protected]>
2020-03-30 06:24 ` Amit Kapila <[email protected]>
2020-03-30 19:04 ` Robert Haas <[email protected]>
2020-03-30 19:16 ` Andres Freund <[email protected]>
2020-03-31 05:40 ` Noah Misch <[email protected]>
2020-03-31 09:26 ` Amit Kapila <[email protected]>
2020-03-31 11:58 ` Stephen Frost <[email protected]>
2020-03-27 18:34 ` Robert Haas <[email protected]>
2020-03-27 19:48 ` Stephen Frost <[email protected]>
2020-03-27 20:12 ` Andres Freund <[email protected]>
2020-03-27 21:07 ` Stephen Frost <[email protected]>
2020-03-27 22:00 ` Andres Freund <[email protected]>
2020-03-27 04:30 ` Andres Freund <[email protected]>
2020-03-27 15:26 ` Stephen Frost <[email protected]>
2020-03-27 19:29 ` Robert Haas <[email protected]>
2020-03-27 20:08 ` Andres Freund <[email protected]>
2020-03-27 20:16 ` David Steele <[email protected]>
2020-03-27 20:57 ` Stephen Frost <[email protected]>
2020-03-27 22:07 ` Andres Freund <[email protected]>
2020-03-27 22:24 ` Stephen Frost <[email protected]>
2020-03-27 22:33 ` David Steele <[email protected]>
2020-03-24 03:43 ` Amit Kapila <[email protected]>
2020-03-24 17:00 ` Robert Haas <[email protected]>
2020-03-25 06:53 ` Amit Kapila <[email protected]>
2020-03-27 06:29 ` Andres Freund <[email protected]>
2020-03-27 19:20 ` Robert Haas <[email protected]>
2020-03-27 20:02 ` David Steele <[email protected]>
2020-03-30 00:47 ` Robert Haas <[email protected]>
2020-03-30 00:59 ` Andres Freund <[email protected]>
2020-03-30 01:05 ` David Steele <[email protected]>
2020-03-30 20:43 ` Robert Haas <[email protected]>
2020-03-27 20:32 ` Andres Freund <[email protected]>
2020-03-27 21:44 ` Stephen Frost <[email protected]>
2020-03-27 21:56 ` Andres Freund <[email protected]>
2020-03-27 22:09 ` Stephen Frost <[email protected]>
2020-03-27 22:17 ` Alvaro Herrera <[email protected]>
2020-03-30 00:33 ` Robert Haas <[email protected]>
2020-03-30 00:48 ` David Steele <[email protected]>
2020-03-16 10:22 ` tushar <[email protected]>
2020-03-05 03:50 ` Suraj Kharage <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
2023-01-21 00:09 [PATCH v8 5/5] wip: instr_time: Add and use INSTR_TIME_ZERO(), INSTR_TIME_CURRENT() Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox