public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v3 07/10] Use "template" initdb in tests
9+ messages / 3 participants
[nested] [flat]
* [PATCH v3 07/10] Use "template" initdb in tests
@ 2023-02-03 05:51 Andres Freund <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Andres Freund @ 2023-02-03 05:51 UTC (permalink / raw)
Discussion: https://postgr.es/m/[email protected]
---
meson.build | 30 ++++++++++
src/test/perl/PostgreSQL/Test/Cluster.pm | 46 ++++++++++++++-
src/test/regress/pg_regress.c | 74 ++++++++++++++++++------
.cirrus.tasks.yml | 3 +-
src/Makefile.global.in | 52 +++++++++--------
5 files changed, 161 insertions(+), 44 deletions(-)
diff --git a/meson.build b/meson.build
index f5ec442f9a9..8b2b521a013 100644
--- a/meson.build
+++ b/meson.build
@@ -3070,8 +3070,10 @@ testport = 40000
test_env = environment()
temp_install_bindir = test_install_location / get_option('bindir')
+test_initdb_template = meson.build_root() / 'tmp_install' / 'initdb-template'
test_env.set('PG_REGRESS', pg_regress.full_path())
test_env.set('REGRESS_SHLIB', regress_module.full_path())
+test_env.set('INITDB_TEMPLATE', test_initdb_template)
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable PG_TEST_EXTRA.
@@ -3086,6 +3088,34 @@ if library_path_var != ''
endif
+# Create (and remove old) initdb template directory. Tests use that, where
+# possible, to make it cheaper to run tests.
+#
+# Use python to remove the old cached initdb, as we cannot rely on a working
+# 'rm' binary on windows.
+test('initdb_cache',
+ python,
+ args: [
+ '-c', '''
+import shutil
+import sys
+import subprocess
+
+shutil.rmtree(sys.argv[1], ignore_errors=True)
+sp = subprocess.run(sys.argv[2:] + [sys.argv[1]])
+sys.exit(sp.returncode)
+''',
+ test_initdb_template,
+ temp_install_bindir / 'initdb',
+ '-A', 'trust', '-N', '--no-instructions'
+ ],
+ priority: setup_tests_priority - 1,
+ timeout: 300,
+ is_parallel: false,
+ env: test_env,
+ suite: ['setup'])
+
+
###############################################################
# Test Generation
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 5e161dbee60..4d449c35de9 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -522,8 +522,50 @@ sub init
mkdir $self->backup_dir;
mkdir $self->archive_dir;
- PostgreSQL::Test::Utils::system_or_bail('initdb', '-D', $pgdata, '-A',
- 'trust', '-N', @{ $params{extra} });
+ # If available and if there aren't any parameters, use a previously
+ # initdb'd cluster as a template by copying it. For a lot of tests, that's
+ # substantially cheaper. Do so only if there aren't parameters, it doesn't
+ # seem worth figuring out whether they affect compatibility.
+ #
+ # There's very similar code in pg_regress.c, but we can't easily
+ # deduplicate it until we require perl at build time.
+ if (defined $params{extra} or !defined $ENV{INITDB_TEMPLATE})
+ {
+ note("initializing database system by running initdb");
+ PostgreSQL::Test::Utils::system_or_bail('initdb', '-D', $pgdata, '-A',
+ 'trust', '-N', @{ $params{extra} });
+ }
+ else
+ {
+ my @copycmd;
+ my $expected_exitcode;
+
+ note("initializing database system by copying initdb template");
+
+ if ($PostgreSQL::Test::Utils::windows_os)
+ {
+ @copycmd = qw(robocopy /E /NJS /NJH /NFL /NDL /NP);
+ $expected_exitcode = 1; # 1 denotes files were copied
+ }
+ else
+ {
+ @copycmd = qw(cp -a);
+ $expected_exitcode = 0;
+ }
+
+ @copycmd = (@copycmd, $ENV{INITDB_TEMPLATE}, $pgdata);
+
+ my $ret = PostgreSQL::Test::Utils::system_log(@copycmd);
+
+ # See http://perldoc.perl.org/perlvar.html#%24CHILD_ERROR
+ if ($ret & 127 or $ret >> 8 != $expected_exitcode)
+ {
+ BAIL_OUT(
+ sprintf("failed to execute command \"%s\": $ret",
+ join(" ", @copycmd)));
+ }
+ }
+
PostgreSQL::Test::Utils::system_or_bail($ENV{PG_REGRESS},
'--config-auth', $pgdata, @{ $params{auth_extra} });
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index b68632320a7..407e3915cec 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -2295,6 +2295,7 @@ regression_main(int argc, char *argv[],
FILE *pg_conf;
const char *env_wait;
int wait_seconds;
+ const char *initdb_template_dir;
/*
* Prepare the temp instance
@@ -2316,25 +2317,64 @@ regression_main(int argc, char *argv[],
if (!directory_exists(buf))
make_directory(buf);
- /* initdb */
initStringInfo(&cmd);
- appendStringInfo(&cmd,
- "\"%s%sinitdb\" -D \"%s/data\" --no-clean --no-sync",
- bindir ? bindir : "",
- bindir ? "/" : "",
- temp_instance);
- if (debug)
- appendStringInfo(&cmd, " --debug");
- if (nolocale)
- appendStringInfo(&cmd, " --no-locale");
- appendStringInfo(&cmd, " > \"%s/log/initdb.log\" 2>&1", outputdir);
- fflush(NULL);
- if (system(cmd.data))
+
+ /*
+ * Create data directory.
+ *
+ * If available, use a previously initdb'd cluster as a template by
+ * copying it. For a lot of tests, that's substantially cheaper.
+ *
+ * There's very similar code in Cluster.pm, but we can't easily de
+ * duplicate it until we require perl at build time.
+ */
+ initdb_template_dir = getenv("INITDB_TEMPLATE");
+ if (initdb_template_dir == NULL || nolocale || debug)
{
- bail("initdb failed\n"
- "# Examine \"%s/log/initdb.log\" for the reason.\n"
- "# Command was: %s",
- outputdir, cmd.data);
+ note("initializing database system by running initdb");
+
+ appendStringInfo(&cmd,
+ "\"%s%sinitdb\" -D \"%s/data\" --no-clean --no-sync",
+ bindir ? bindir : "",
+ bindir ? "/" : "",
+ temp_instance);
+ if (debug)
+ appendStringInfo(&cmd, " --debug");
+ if (nolocale)
+ appendStringInfo(&cmd, " --no-locale");
+ appendStringInfo(&cmd, " > \"%s/log/initdb.log\" 2>&1", outputdir);
+ fflush(NULL);
+ if (system(cmd.data))
+ {
+ bail("initdb failed\n"
+ "# Examine \"%s/log/initdb.log\" for the reason.\n"
+ "# Command was: %s",
+ outputdir, cmd.data);
+ }
+ }
+ else
+ {
+#ifndef WIN32
+ const char *copycmd = "cp -a \"%s\" \"%s/data\"";
+ int expected_exitcode = 0;
+#else
+ const char *copycmd = "robocopy /E /NJS /NJH /NFL /NDL /NP \"%s\" \"%s/data\"";
+ int expected_exitcode = 1; /* 1 denotes files were copied */
+#endif
+
+ note("initializing database system by copying initdb template");
+
+ appendStringInfo(&cmd,
+ copycmd,
+ initdb_template_dir,
+ temp_instance);
+ if (system(cmd.data) != expected_exitcode)
+ {
+ bail("copying of initdb template failed\n"
+ "# Examine \"%s/log/initdb.log\" for the reason.\n"
+ "# Command was: %s",
+ outputdir, cmd.data);
+ }
}
pfree(cmd.data);
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 0cf7ba77996..e137769850d 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -109,8 +109,9 @@ task:
test_minimal_script: |
su postgres <<-EOF
ulimit -c unlimited
+ meson test $MTEST_ARGS --suite setup
meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \
- tmp_install cube/regress pg_ctl/001_start_stop
+ cube/regress pg_ctl/001_start_stop
EOF
on_failure:
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index df9f721a41a..0b4ca0eb6ae 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -397,30 +397,6 @@ check: temp-install
.PHONY: temp-install
-temp-install: | submake-generated-headers
-ifndef NO_TEMP_INSTALL
-ifneq ($(abs_top_builddir),)
-ifeq ($(MAKELEVEL),0)
- rm -rf '$(abs_top_builddir)'/tmp_install
- $(MKDIR_P) '$(abs_top_builddir)'/tmp_install/log
- $(MAKE) -C '$(top_builddir)' DESTDIR='$(abs_top_builddir)'/tmp_install install >'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1
- $(MAKE) -j1 $(if $(CHECKPREP_TOP),-C $(CHECKPREP_TOP),) checkprep >>'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1
-endif
-endif
-endif
-
-# Tasks to run serially at the end of temp-install. Some EXTRA_INSTALL
-# entries appear more than once in the tree, and parallel installs of the same
-# file can fail with EEXIST.
-checkprep:
- $(if $(EXTRA_INSTALL),for extra in $(EXTRA_INSTALL); do $(MAKE) -C '$(top_builddir)'/$$extra DESTDIR='$(abs_top_builddir)'/tmp_install install || exit; done)
-
-PROVE = @PROVE@
-# There are common routines in src/test/perl, and some test suites have
-# extra perl modules in their own directory.
-PG_PROVE_FLAGS = -I $(top_srcdir)/src/test/perl/ -I $(srcdir)
-# User-supplied prove flags such as --verbose can be provided in PROVE_FLAGS.
-PROVE_FLAGS =
# prepend to path if already set, else just set it
define add_to_path
@@ -437,8 +413,36 @@ ld_library_path_var = LD_LIBRARY_PATH
with_temp_install = \
PATH="$(abs_top_builddir)/tmp_install$(bindir):$(CURDIR):$$PATH" \
$(call add_to_path,$(strip $(ld_library_path_var)),$(abs_top_builddir)/tmp_install$(libdir)) \
+ INITDB_TEMPLATE='$(abs_top_builddir)'/tmp_install/initdb-template \
$(with_temp_install_extra)
+temp-install: | submake-generated-headers
+ifndef NO_TEMP_INSTALL
+ifneq ($(abs_top_builddir),)
+ifeq ($(MAKELEVEL),0)
+ rm -rf '$(abs_top_builddir)'/tmp_install
+ $(MKDIR_P) '$(abs_top_builddir)'/tmp_install/log
+ $(MAKE) -C '$(top_builddir)' DESTDIR='$(abs_top_builddir)'/tmp_install install >'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1
+ $(MAKE) -j1 $(if $(CHECKPREP_TOP),-C $(CHECKPREP_TOP),) checkprep >>'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1
+
+ $(with_temp_install) initdb -A trust -N --no-instructions '$(abs_top_builddir)'/tmp_install/initdb-template >>'$(abs_top_builddir)'/tmp_install/log/initdb-template.log 2>&1
+endif
+endif
+endif
+
+# Tasks to run serially at the end of temp-install. Some EXTRA_INSTALL
+# entries appear more than once in the tree, and parallel installs of the same
+# file can fail with EEXIST.
+checkprep:
+ $(if $(EXTRA_INSTALL),for extra in $(EXTRA_INSTALL); do $(MAKE) -C '$(top_builddir)'/$$extra DESTDIR='$(abs_top_builddir)'/tmp_install install || exit; done)
+
+PROVE = @PROVE@
+# There are common routines in src/test/perl, and some test suites have
+# extra perl modules in their own directory.
+PG_PROVE_FLAGS = -I $(top_srcdir)/src/test/perl/ -I $(srcdir)
+# User-supplied prove flags such as --verbose can be provided in PROVE_FLAGS.
+PROVE_FLAGS =
+
ifeq ($(enable_tap_tests),yes)
ifndef PGXS
--
2.38.0
--uh2yukyzfvojbe2k
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v3-0008-ci-switch-tasks-to-debugoptimized-build.patch"
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
@ 2023-03-02 07:39 Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Drouvot, Bertrand @ 2023-03-02 07:39 UTC (permalink / raw)
To: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; +Cc: Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 3/1/23 8:54 PM, Gregory Stark (as CFM) wrote:
> Looks like you have a path forward on this and it's not ready to
> commit yet? In which case I'll mark it Waiting on Author?
>
Yeah, there is some dependencies around this one.
[1]: depends on it
Current one depends of [2], [3] and [4]
Waiting on Author is then the right state, thanks for having moved it to that state.
[1]: https://www.postgresql.org/message-id/flat/[email protected]
[2]: https://www.postgresql.org/message-id/flat/[email protected]
[3]: https://www.postgresql.org/message-id/flat/[email protected]
[4]: https://www.postgresql.org/message-id/flat/[email protected]
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
@ 2023-03-24 00:04 ` Michael Paquier <[email protected]>
2023-03-24 05:58 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Michael Paquier @ 2023-03-24 00:04 UTC (permalink / raw)
To: Drouvot, Bertrand <[email protected]>; +Cc: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 02, 2023 at 08:39:14AM +0100, Drouvot, Bertrand wrote:
> Yeah, there is some dependencies around this one.
>
> [1]: depends on it
> Current one depends of [2], [3] and [4]
>
> Waiting on Author is then the right state, thanks for having moved it to that state.
>
> [1]: https://www.postgresql.org/message-id/flat/[email protected]
> [2]: https://www.postgresql.org/message-id/flat/[email protected]
> [3]: https://www.postgresql.org/message-id/flat/[email protected]
> [4]: https://www.postgresql.org/message-id/flat/[email protected]
[3] and [4] have been applied. [2] is more sensitive than it looks,
and [1] for the split of index and table stats can feed on the one of
this thread.
Roma wasn't built in one day, and from what I can see you can still do
some progress with the refactoring of pgstatfuncs.c with what's
already on HEAD. So how about handling doing that first as much as we
can based on the state of HEAD? That looks like 50~60% (?) of the
original goal to switch pgstatfuncs.c to use more macros to generate
the definition of all these SQL functions.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
@ 2023-03-24 05:58 ` Drouvot, Bertrand <[email protected]>
2023-03-25 02:50 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Drouvot, Bertrand @ 2023-03-24 05:58 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 3/24/23 1:04 AM, Michael Paquier wrote:
> On Thu, Mar 02, 2023 at 08:39:14AM +0100, Drouvot, Bertrand wrote:
>> Yeah, there is some dependencies around this one.
>>
>> [1]: depends on it
>> Current one depends of [2], [3] and [4]
>>
>> Waiting on Author is then the right state, thanks for having moved it to that state.
>>
>> [1]: https://www.postgresql.org/message-id/flat/[email protected]
>> [2]: https://www.postgresql.org/message-id/flat/[email protected]
>> [3]: https://www.postgresql.org/message-id/flat/[email protected]
>> [4]: https://www.postgresql.org/message-id/flat/[email protected]
>
> [3] and [4] have been applied.
Thanks for your help on this!
> [2] is more sensitive than it looks,
> and [1] for the split of index and table stats can feed on the one of
> this thread.
>
> Roma wasn't built in one day, and from what I can see you can still do
> some progress with the refactoring of pgstatfuncs.c with what's
> already on HEAD. So how about handling doing that first as much as we
> can based on the state of HEAD? That looks like 50~60% (?) of the
> original goal to switch pgstatfuncs.c to use more macros to generate
> the definition of all these SQL functions.
I think that's a good idea, so please find enclosed V2 which as compare to V1:
- Does not include the refactoring for pg_stat_get_xact_tuples_inserted(), pg_stat_get_xact_tuples_updated()
and pg_stat_get_xact_tuples_deleted() (as they depend of [2] mentioned above)
- Does not include the refactoring for pg_stat_get_xact_function_total_time(), pg_stat_get_xact_function_self_time(),
pg_stat_get_function_total_time() and pg_stat_get_function_self_time(). I think they can be done in a dedicated commit once
we agree on the renaming for PG_STAT_GET_DBENTRY_FLOAT8 (see Andres's comment up-thread) so that the new macros can match the future agreement.
- Does include the refactoring of the new pg_stat_get_xact_tuples_newpage_updated() function (added in ae4fdde135)
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c7ba86b3dc..e1dd1e0ad3 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1498,50 +1498,42 @@ pg_stat_get_slru(PG_FUNCTION_ARGS)
return (Datum) 0;
}
-Datum
-pg_stat_get_xact_numscans(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.numscans);
-
- PG_RETURN_INT64(result);
+#define PG_STAT_GET_XACT_RELENTRY_INT64(stat) \
+Datum \
+CppConcat(pg_stat_get_xact_,stat)(PG_FUNCTION_ARGS) \
+{ \
+ Oid relid = PG_GETARG_OID(0); \
+ int64 result; \
+ PgStat_TableStatus *tabentry; \
+ \
+ if ((tabentry = find_tabstat_entry(relid)) == NULL) \
+ result = 0; \
+ else \
+ result = (int64) (tabentry->counts.stat); \
+ \
+ PG_RETURN_INT64(result); \
}
-Datum
-pg_stat_get_xact_tuples_returned(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
+/* pg_stat_get_xact_numscans */
+PG_STAT_GET_XACT_RELENTRY_INT64(numscans)
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_returned);
+/* pg_stat_get_xact_tuples_returned */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_returned)
- PG_RETURN_INT64(result);
-}
+/* pg_stat_get_xact_tuples_fetched */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_fetched)
-Datum
-pg_stat_get_xact_tuples_fetched(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
+/* pg_stat_get_xact_tuples_hot_updated */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_hot_updated)
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_fetched);
+/* pg_stat_get_xact_tuples_newpage_updated */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_newpage_updated)
- PG_RETURN_INT64(result);
-}
+/* pg_stat_get_xact_blocks_fetched */
+PG_STAT_GET_XACT_RELENTRY_INT64(blocks_fetched)
+
+/* pg_stat_get_xact_blocks_hit */
+PG_STAT_GET_XACT_RELENTRY_INT64(blocks_hit)
Datum
pg_stat_get_xact_tuples_inserted(PG_FUNCTION_ARGS)
@@ -1606,66 +1598,6 @@ pg_stat_get_xact_tuples_deleted(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
-Datum
-pg_stat_get_xact_tuples_hot_updated(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_hot_updated);
-
- PG_RETURN_INT64(result);
-}
-
-Datum
-pg_stat_get_xact_tuples_newpage_updated(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_newpage_updated);
-
- PG_RETURN_INT64(result);
-}
-
-Datum
-pg_stat_get_xact_blocks_fetched(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.blocks_fetched);
-
- PG_RETURN_INT64(result);
-}
-
-Datum
-pg_stat_get_xact_blocks_hit(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.blocks_hit);
-
- PG_RETURN_INT64(result);
-}
-
Datum
pg_stat_get_xact_function_calls(PG_FUNCTION_ARGS)
{
Attachments:
[text/plain] v2-0001-generate-pg_stat_get_xact-functions-with-macros.patch (3.7K, ../../[email protected]/2-v2-0001-generate-pg_stat_get_xact-functions-with-macros.patch)
download | inline diff:
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c7ba86b3dc..e1dd1e0ad3 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1498,50 +1498,42 @@ pg_stat_get_slru(PG_FUNCTION_ARGS)
return (Datum) 0;
}
-Datum
-pg_stat_get_xact_numscans(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.numscans);
-
- PG_RETURN_INT64(result);
+#define PG_STAT_GET_XACT_RELENTRY_INT64(stat) \
+Datum \
+CppConcat(pg_stat_get_xact_,stat)(PG_FUNCTION_ARGS) \
+{ \
+ Oid relid = PG_GETARG_OID(0); \
+ int64 result; \
+ PgStat_TableStatus *tabentry; \
+ \
+ if ((tabentry = find_tabstat_entry(relid)) == NULL) \
+ result = 0; \
+ else \
+ result = (int64) (tabentry->counts.stat); \
+ \
+ PG_RETURN_INT64(result); \
}
-Datum
-pg_stat_get_xact_tuples_returned(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
+/* pg_stat_get_xact_numscans */
+PG_STAT_GET_XACT_RELENTRY_INT64(numscans)
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_returned);
+/* pg_stat_get_xact_tuples_returned */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_returned)
- PG_RETURN_INT64(result);
-}
+/* pg_stat_get_xact_tuples_fetched */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_fetched)
-Datum
-pg_stat_get_xact_tuples_fetched(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
+/* pg_stat_get_xact_tuples_hot_updated */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_hot_updated)
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_fetched);
+/* pg_stat_get_xact_tuples_newpage_updated */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_newpage_updated)
- PG_RETURN_INT64(result);
-}
+/* pg_stat_get_xact_blocks_fetched */
+PG_STAT_GET_XACT_RELENTRY_INT64(blocks_fetched)
+
+/* pg_stat_get_xact_blocks_hit */
+PG_STAT_GET_XACT_RELENTRY_INT64(blocks_hit)
Datum
pg_stat_get_xact_tuples_inserted(PG_FUNCTION_ARGS)
@@ -1606,66 +1598,6 @@ pg_stat_get_xact_tuples_deleted(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
-Datum
-pg_stat_get_xact_tuples_hot_updated(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_hot_updated);
-
- PG_RETURN_INT64(result);
-}
-
-Datum
-pg_stat_get_xact_tuples_newpage_updated(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.tuples_newpage_updated);
-
- PG_RETURN_INT64(result);
-}
-
-Datum
-pg_stat_get_xact_blocks_fetched(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.blocks_fetched);
-
- PG_RETURN_INT64(result);
-}
-
-Datum
-pg_stat_get_xact_blocks_hit(PG_FUNCTION_ARGS)
-{
- Oid relid = PG_GETARG_OID(0);
- int64 result;
- PgStat_TableStatus *tabentry;
-
- if ((tabentry = find_tabstat_entry(relid)) == NULL)
- result = 0;
- else
- result = (int64) (tabentry->counts.blocks_hit);
-
- PG_RETURN_INT64(result);
-}
-
Datum
pg_stat_get_xact_function_calls(PG_FUNCTION_ARGS)
{
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-24 05:58 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
@ 2023-03-25 02:50 ` Michael Paquier <[email protected]>
2023-03-27 01:20 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Michael Paquier @ 2023-03-25 02:50 UTC (permalink / raw)
To: Drouvot, Bertrand <[email protected]>; +Cc: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 24, 2023 at 06:58:30AM +0100, Drouvot, Bertrand wrote:
> - Does not include the refactoring for pg_stat_get_xact_tuples_inserted(),
> pg_stat_get_xact_tuples_updated() and pg_stat_get_xact_tuples_deleted() (as
> they depend of [2] mentioned above)
>
> - Does not include the refactoring for
> pg_stat_get_xact_function_total_time(),
> pg_stat_get_xact_function_self_time(),
> pg_stat_get_function_total_time() and
> pg_stat_get_function_self_time(). I think they can be done in a
> dedicated commit once we agree on the renaming for
> PG_STAT_GET_DBENTRY_FLOAT8 (see Andres's comment up-thread) so that
> the new macros can match the future agreement.
>
> - Does include the refactoring of the new
> - pg_stat_get_xact_tuples_newpage_updated() function (added in
> - ae4fdde135)
Fine by me. One step is better than no steps, and this helps:
1 file changed, 29 insertions(+), 97 deletions(-)
I'll go apply that if there are no objections.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-24 05:58 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-25 02:50 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
@ 2023-03-27 01:20 ` Michael Paquier <[email protected]>
2023-03-27 05:45 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Michael Paquier @ 2023-03-27 01:20 UTC (permalink / raw)
To: Drouvot, Bertrand <[email protected]>; +Cc: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
On Sat, Mar 25, 2023 at 11:50:50AM +0900, Michael Paquier wrote:
> On Fri, Mar 24, 2023 at 06:58:30AM +0100, Drouvot, Bertrand wrote:
>> - Does not include the refactoring for
>> pg_stat_get_xact_function_total_time(),
>> pg_stat_get_xact_function_self_time(),
>> pg_stat_get_function_total_time() and
>> pg_stat_get_function_self_time(). I think they can be done in a
>> dedicated commit once we agree on the renaming for
>> PG_STAT_GET_DBENTRY_FLOAT8 (see Andres's comment up-thread) so that
>> the new macros can match the future agreement.
Thanks for the reminder. I have completely missed that this is
mentioned in [1], and that it is all about 8018ffb. The suggestion to
prefix the macro names with a "_MS" to outline the conversion sounds
like a good one seen from here. So please find attached a patch to do
this adjustment, completed with a similar switch for the two counters
of the function entries.
>> - Does include the refactoring of the new
>> - pg_stat_get_xact_tuples_newpage_updated() function (added in
>> - ae4fdde135)
>
> Fine by me. One step is better than no steps, and this helps:
> 1 file changed, 29 insertions(+), 97 deletions(-)
>
> I'll go apply that if there are no objections.
Just did this part to shave a bit more code.
[1]: https://www.postgresql.org/message-id/[email protected]
--
Michael
Attachments:
[text/x-diff] pgstatfuncs-more-macros.patch (2.9K, ../../[email protected]/2-pgstatfuncs-more-macros.patch)
download | inline diff:
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e1dd1e0ad3..ff155e003f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -152,29 +152,26 @@ pg_stat_get_function_calls(PG_FUNCTION_ARGS)
PG_RETURN_INT64(funcentry->numcalls);
}
-Datum
-pg_stat_get_function_total_time(PG_FUNCTION_ARGS)
-{
- Oid funcid = PG_GETARG_OID(0);
- PgStat_StatFuncEntry *funcentry;
-
- if ((funcentry = pgstat_fetch_stat_funcentry(funcid)) == NULL)
- PG_RETURN_NULL();
- /* convert counter from microsec to millisec for display */
- PG_RETURN_FLOAT8(((double) funcentry->total_time) / 1000.0);
+/* convert counter from microsec to millisec for display */
+#define PG_STAT_GET_FUNCENTRY_FLOAT8_MS(stat) \
+Datum \
+CppConcat(pg_stat_get_function_,stat)(PG_FUNCTION_ARGS) \
+{ \
+ Oid funcid = PG_GETARG_OID(0); \
+ double result; \
+ PgStat_StatFuncEntry *funcentry; \
+ \
+ if ((funcentry = pgstat_fetch_stat_funcentry(funcid)) == NULL) \
+ PG_RETURN_NULL(); \
+ result = ((double) funcentry->stat) / 1000.0; \
+ PG_RETURN_FLOAT8(result); \
}
-Datum
-pg_stat_get_function_self_time(PG_FUNCTION_ARGS)
-{
- Oid funcid = PG_GETARG_OID(0);
- PgStat_StatFuncEntry *funcentry;
+/* pg_stat_get_function_total_time */
+PG_STAT_GET_FUNCENTRY_FLOAT8_MS(total_time)
- if ((funcentry = pgstat_fetch_stat_funcentry(funcid)) == NULL)
- PG_RETURN_NULL();
- /* convert counter from microsec to millisec for display */
- PG_RETURN_FLOAT8(((double) funcentry->self_time) / 1000.0);
-}
+/* pg_stat_get_function_self_time */
+PG_STAT_GET_FUNCENTRY_FLOAT8_MS(self_time)
Datum
pg_stat_get_backend_idset(PG_FUNCTION_ARGS)
@@ -1147,7 +1144,8 @@ pg_stat_get_db_checksum_last_failure(PG_FUNCTION_ARGS)
PG_RETURN_TIMESTAMPTZ(result);
}
-#define PG_STAT_GET_DBENTRY_FLOAT8(stat) \
+/* convert counter from microsec to millisec for display */
+#define PG_STAT_GET_DBENTRY_FLOAT8_MS(stat) \
Datum \
CppConcat(pg_stat_get_db_,stat)(PG_FUNCTION_ARGS) \
{ \
@@ -1164,19 +1162,19 @@ CppConcat(pg_stat_get_db_,stat)(PG_FUNCTION_ARGS) \
}
/* pg_stat_get_db_active_time */
-PG_STAT_GET_DBENTRY_FLOAT8(active_time)
+PG_STAT_GET_DBENTRY_FLOAT8_MS(active_time)
/* pg_stat_get_db_blk_read_time */
-PG_STAT_GET_DBENTRY_FLOAT8(blk_read_time)
+PG_STAT_GET_DBENTRY_FLOAT8_MS(blk_read_time)
/* pg_stat_get_db_blk_write_time */
-PG_STAT_GET_DBENTRY_FLOAT8(blk_write_time)
+PG_STAT_GET_DBENTRY_FLOAT8_MS(blk_write_time)
/* pg_stat_get_db_idle_in_transaction_time */
-PG_STAT_GET_DBENTRY_FLOAT8(idle_in_transaction_time)
+PG_STAT_GET_DBENTRY_FLOAT8_MS(idle_in_transaction_time)
/* pg_stat_get_db_session_time */
-PG_STAT_GET_DBENTRY_FLOAT8(session_time)
+PG_STAT_GET_DBENTRY_FLOAT8_MS(session_time)
Datum
pg_stat_get_bgwriter_timed_checkpoints(PG_FUNCTION_ARGS)
[application/pgp-signature] signature.asc (833B, ../../[email protected]/3-signature.asc)
download
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-24 05:58 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-25 02:50 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 01:20 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
@ 2023-03-27 05:45 ` Drouvot, Bertrand <[email protected]>
2023-03-27 06:40 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Drouvot, Bertrand @ 2023-03-27 05:45 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 3/27/23 3:20 AM, Michael Paquier wrote:
> On Sat, Mar 25, 2023 at 11:50:50AM +0900, Michael Paquier wrote:
>> On Fri, Mar 24, 2023 at 06:58:30AM +0100, Drouvot, Bertrand wrote:
>>> - Does not include the refactoring for
>>> pg_stat_get_xact_function_total_time(),
>>> pg_stat_get_xact_function_self_time(),
>>> pg_stat_get_function_total_time() and
>>> pg_stat_get_function_self_time(). I think they can be done in a
>>> dedicated commit once we agree on the renaming for
>>> PG_STAT_GET_DBENTRY_FLOAT8 (see Andres's comment up-thread) so that
>>> the new macros can match the future agreement.
>
> Thanks for the reminder. I have completely missed that this is
> mentioned in [1], and that it is all about 8018ffb. The suggestion to
> prefix the macro names with a "_MS" to outline the conversion sounds
> like a good one seen from here. So please find attached a patch to do
> this adjustment, completed with a similar switch for the two counters
> of the function entries.
>
Thanks! LGTM, but what about also taking care of pg_stat_get_xact_function_total_time()
and pg_stat_get_xact_function_self_time() while at it?
>>> - Does include the refactoring of the new
>>> - pg_stat_get_xact_tuples_newpage_updated() function (added in
>>> - ae4fdde135)
>>
>> Fine by me. One step is better than no steps, and this helps:
>> 1 file changed, 29 insertions(+), 97 deletions(-)
>>
>> I'll go apply that if there are no objections.
>
> Just did this part to shave a bit more code.
Thanks!
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-24 05:58 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-25 02:50 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 01:20 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 05:45 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
@ 2023-03-27 06:40 ` Michael Paquier <[email protected]>
2023-03-27 06:54 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Michael Paquier @ 2023-03-27 06:40 UTC (permalink / raw)
To: Drouvot, Bertrand <[email protected]>; +Cc: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Mar 27, 2023 at 07:45:26AM +0200, Drouvot, Bertrand wrote:
> Thanks! LGTM, but what about also taking care of pg_stat_get_xact_function_total_time()
> and pg_stat_get_xact_function_self_time() while at it?
With a macro that uses INSTR_TIME_GET_MILLISEC() to cope with
instr_time? Why not, that's one duplication less.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Generate pg_stat_get_xact*() functions with Macros
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-24 05:58 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-25 02:50 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 01:20 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 05:45 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-27 06:40 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
@ 2023-03-27 06:54 ` Drouvot, Bertrand <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Drouvot, Bertrand @ 2023-03-27 06:54 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Gregory Stark (as CFM) <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; PostgreSQL Hackers <[email protected]>
On 3/27/23 8:40 AM, Michael Paquier wrote:
> On Mon, Mar 27, 2023 at 07:45:26AM +0200, Drouvot, Bertrand wrote:
>> Thanks! LGTM, but what about also taking care of pg_stat_get_xact_function_total_time()
>> and pg_stat_get_xact_function_self_time() while at it?
>
> With a macro that uses INSTR_TIME_GET_MILLISEC() to cope with
> instr_time? Why not, that's one duplication less.
Yes, something like V1 up-thread was doing. I think it can be added with your current proposal.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2023-03-27 06:54 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-02-03 05:51 [PATCH v3 07/10] Use "template" initdb in tests Andres Freund <[email protected]>
2023-03-02 07:39 Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-24 00:04 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-24 05:58 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-25 02:50 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 01:20 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 05:45 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
2023-03-27 06:40 ` Re: Generate pg_stat_get_xact*() functions with Macros Michael Paquier <[email protected]>
2023-03-27 06:54 ` Re: Generate pg_stat_get_xact*() functions with Macros Drouvot, Bertrand <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox