Date: Wed, 29 Mar 2023 23:03:59 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Andres Freund <andres@anarazel.de>
Cc: Michael Paquier <michael@paquier.xyz>,
	Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
	nathandbossart@gmail.com, pgsql-hackers@postgresql.org
Subject: Re: fix stats_fetch_consistency value in postgresql.conf.sample
Message-ID: <ZCUKL22GutwGrrZk@telsasoft.com>
References: <20220616132306.GD29853@telsasoft.com>
 <20220617.094358.2201653243066915172.horikyota.ntt@gmail.com>
 <20220622230710.r2m4ex5tlf6mveyh@alap3.anarazel.de>
 <20220630.173801.662386227537019720.horikyota.ntt@gmail.com>
 <20220713173000.GD18011@telsasoft.com>
 <Ys9ZOk49mruHPn4q@paquier.xyz>
 <20220713234900.z4rniuaerkq34s4v@awork3.anarazel.de>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="gwf9g/xA8/k5MLj4"
Content-Disposition: inline
In-Reply-To: <20220713234900.z4rniuaerkq34s4v@awork3.anarazel.de>
Archived-At: <https://www.postgresql.org/message-id/ZCUKL22GutwGrrZk%40telsasoft.com>
Precedence: bulk


--gwf9g/xA8/k5MLj4
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Wed, Jul 13, 2022 at 04:49:00PM -0700, Andres Freund wrote:
> On 2022-07-14 08:46:02 +0900, Michael Paquier wrote:
> > On Wed, Jul 13, 2022 at 12:30:00PM -0500, Justin Pryzby wrote:
> > > How did you make this list ?  Was it by excluding things that failed for you ?
> > > 
> > > cfbot is currently failing due to io_concurrency on windows.
> > > I think there are more GUC which should be included here.
> > > 
> > > http://cfbot.cputube.org/kyotaro-horiguchi.html
> > 
> > FWIW, I am not really a fan of making this test depend on a hardcoded
> > list of GUCs.
> 
> I wonder if we should add flags indicating platform dependency etc to guc.c?
> That should allow to remove most of them?

Michael commented on this, but on another thread, so I'm copying and
pasting it here.

On Thu, Mar 23, 2023 at 08:59:57PM -0500, Justin Pryzby wrote:
> On Fri, Mar 24, 2023 at 10:24:43AM +0900, Michael Paquier wrote:
> > >> * Check consistency of GUC defaults between .sample.conf and pg_settings.boot_val
> > >   - It looks like this was pretty active until last October and might
> > > have been ready to apply at least partially? But no further work or
> > > review has happened since.
> > 
> > FWIW, I don't find much appealing the addition of two GUC flags for
> > only the sole purpose of that,
> 
> The flags seem independently interesting - adding them here follows
> a suggestion Andres made in response to your complaint.
> 20220713234900.z4rniuaerkq34s4v@awork3.anarazel.de
> 
> > particularly as we get a stronger
> > dependency between GUCs that can be switched dynamically at
> > initialization and at compile-time.
> 
> What do you mean by "stronger dependency between GUCs" ?

I'm still not clear what that means ?

I updated the patch to handle the GUC added at 1671f990d.

-- 
Justin

--gwf9g/xA8/k5MLj4
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
	filename="0001-pg_settings_get_flags-add-DEFAULT_COMPILE-and-DEFAUL.patch"

From b38dce442fa20439c6b75b9f4bac20d52d3dd94b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Tue, 19 Jul 2022 08:31:56 -0500
Subject: [PATCH 1/2] pg_settings_get_flags(): add DEFAULT_COMPILE and
 DEFAULT_INITDB ..

for settings which vary by ./configure or platform or initdb

Note that this is independent from PGC_S_DYNAMIC_DEFAULT.
---
 doc/src/sgml/func.sgml              | 15 ++++++
 src/backend/utils/misc/guc_funcs.c  |  6 ++-
 src/backend/utils/misc/guc_tables.c | 76 ++++++++++++++++++-----------
 src/include/utils/guc.h             |  2 +
 4 files changed, 69 insertions(+), 30 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d599b243b10..66451a6c759 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24825,6 +24825,21 @@ SELECT collation for ('foo' COLLATE "de_DE");
      <row><entry>Flag</entry><entry>Description</entry></row>
     </thead>
     <tbody>
+
+     <row>
+      <entry><literal>DEFAULT_COMPILE</literal></entry>
+      <entry>Parameters with this flag have default values which vary by
+      platform, or compile-time options.
+      </entry>
+     </row>
+
+     <row>
+      <entry><literal>DEFAULT_INITDB</literal></entry>
+      <entry>Parameters with this flag have default values which are
+      determined dynamically during initdb.
+      </entry>
+     </row>
+
      <row>
       <entry><literal>EXPLAIN</literal></entry>
       <entry>Parameters with this flag are included in
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 2ce1e53ec54..70fdce3868d 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -577,7 +577,7 @@ ShowAllGUCConfig(DestReceiver *dest)
 Datum
 pg_settings_get_flags(PG_FUNCTION_ARGS)
 {
-#define MAX_GUC_FLAGS	6
+#define MAX_GUC_FLAGS	8
 	char	   *varname = TextDatumGetCString(PG_GETARG_DATUM(0));
 	struct config_generic *record;
 	int			cnt = 0;
@@ -590,6 +590,10 @@ pg_settings_get_flags(PG_FUNCTION_ARGS)
 	if (record == NULL)
 		PG_RETURN_NULL();
 
+	if (record->flags & GUC_DEFAULT_COMPILE)
+		flags[cnt++] = CStringGetTextDatum("DEFAULT_COMPILE");
+	if (record->flags & GUC_DEFAULT_INITDB)
+		flags[cnt++] = CStringGetTextDatum("DEFAULT_INITDB");
 	if (record->flags & GUC_EXPLAIN)
 		flags[cnt++] = CStringGetTextDatum("EXPLAIN");
 	if (record->flags & GUC_NO_RESET)
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 2c3061b7359..0d6c782a1df 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1233,7 +1233,7 @@ struct config_bool ConfigureNamesBool[] =
 		{"debug_assertions", PGC_INTERNAL, PRESET_OPTIONS,
 			gettext_noop("Shows whether the running server has assertion checks enabled."),
 			NULL,
-			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_DEFAULT_COMPILE
 		},
 		&assert_enabled,
 		DEFAULT_ASSERT_ENABLED,
@@ -1425,7 +1425,8 @@ struct config_bool ConfigureNamesBool[] =
 	{
 		{"update_process_title", PGC_SUSET, PROCESS_TITLE,
 			gettext_noop("Updates the process title to show the active SQL command."),
-			gettext_noop("Enables updating of the process title every time a new SQL command is received by the server.")
+			gettext_noop("Enables updating of the process title every time a new SQL command is received by the server."),
+			GUC_DEFAULT_COMPILE
 		},
 		&update_process_title,
 		DEFAULT_UPDATE_PROCESS_TITLE,
@@ -2170,7 +2171,8 @@ struct config_int ConfigureNamesInt[] =
 	{
 		{"max_connections", PGC_POSTMASTER, CONN_AUTH_SETTINGS,
 			gettext_noop("Sets the maximum number of concurrent connections."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
 		},
 		&MaxConnections,
 		100, 1, MAX_BACKENDS,
@@ -2218,7 +2220,7 @@ struct config_int ConfigureNamesInt[] =
 		{"shared_buffers", PGC_POSTMASTER, RESOURCES_MEM,
 			gettext_noop("Sets the number of shared memory buffers used by the server."),
 			NULL,
-			GUC_UNIT_BLOCKS
+			GUC_UNIT_BLOCKS | GUC_DEFAULT_INITDB
 		},
 		&NBuffers,
 		16384, 16, INT_MAX / 2,
@@ -2261,7 +2263,8 @@ struct config_int ConfigureNamesInt[] =
 	{
 		{"port", PGC_POSTMASTER, CONN_AUTH_SETTINGS,
 			gettext_noop("Sets the TCP port the server listens on."),
-			NULL
+			NULL,
+			GUC_DEFAULT_COMPILE
 		},
 		&PostPortNumber,
 		DEF_PGPORT, 1, 65535,
@@ -2290,7 +2293,8 @@ struct config_int ConfigureNamesInt[] =
 						 "to be a numeric mode specification in the form "
 						 "accepted by the chmod and umask system calls. "
 						 "(To use the customary octal format the number must "
-						 "start with a 0 (zero).)")
+						 "start with a 0 (zero).)"),
+			GUC_DEFAULT_INITDB
 		},
 		&Log_file_mode,
 		0600, 0000, 0777,
@@ -2723,7 +2727,7 @@ struct config_int ConfigureNamesInt[] =
 		{"checkpoint_flush_after", PGC_SIGHUP, WAL_CHECKPOINTS,
 			gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
 			NULL,
-			GUC_UNIT_BLOCKS
+			GUC_UNIT_BLOCKS | GUC_DEFAULT_COMPILE
 		},
 		&checkpoint_flush_after,
 		DEFAULT_CHECKPOINT_FLUSH_AFTER, 0, WRITEBACK_MAX_PENDING_FLUSHES,
@@ -2941,7 +2945,7 @@ struct config_int ConfigureNamesInt[] =
 		{"bgwriter_flush_after", PGC_SIGHUP, RESOURCES_BGWRITER,
 			gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
 			NULL,
-			GUC_UNIT_BLOCKS
+			GUC_UNIT_BLOCKS | GUC_DEFAULT_COMPILE
 		},
 		&bgwriter_flush_after,
 		DEFAULT_BGWRITER_FLUSH_AFTER, 0, WRITEBACK_MAX_PENDING_FLUSHES,
@@ -2954,7 +2958,7 @@ struct config_int ConfigureNamesInt[] =
 			RESOURCES_ASYNCHRONOUS,
 			gettext_noop("Number of simultaneous requests that can be handled efficiently by the disk subsystem."),
 			NULL,
-			GUC_EXPLAIN
+			GUC_EXPLAIN | GUC_DEFAULT_COMPILE
 		},
 		&effective_io_concurrency,
 		DEFAULT_EFFECTIVE_IO_CONCURRENCY,
@@ -2968,7 +2972,7 @@ struct config_int ConfigureNamesInt[] =
 			RESOURCES_ASYNCHRONOUS,
 			gettext_noop("A variant of effective_io_concurrency that is used for maintenance work."),
 			NULL,
-			GUC_EXPLAIN
+			GUC_EXPLAIN | GUC_DEFAULT_COMPILE
 		},
 		&maintenance_io_concurrency,
 		DEFAULT_MAINTENANCE_IO_CONCURRENCY,
@@ -2981,7 +2985,7 @@ struct config_int ConfigureNamesInt[] =
 		{"backend_flush_after", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
 			gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
 			NULL,
-			GUC_UNIT_BLOCKS
+			GUC_UNIT_BLOCKS | GUC_DEFAULT_COMPILE
 		},
 		&backend_flush_after,
 		DEFAULT_BACKEND_FLUSH_AFTER, 0, WRITEBACK_MAX_PENDING_FLUSHES,
@@ -3436,7 +3440,7 @@ struct config_int ConfigureNamesInt[] =
 		{"debug_discard_caches", PGC_SUSET, DEVELOPER_OPTIONS,
 			gettext_noop("Aggressively flush system caches for debugging purposes."),
 			NULL,
-			GUC_NOT_IN_SAMPLE
+			GUC_NOT_IN_SAMPLE | GUC_DEFAULT_COMPILE
 		},
 		&debug_discard_caches,
 #ifdef DISCARD_CACHES_ENABLED
@@ -3930,7 +3934,8 @@ struct config_string ConfigureNamesString[] =
 	{
 		{"log_timezone", PGC_SIGHUP, LOGGING_WHAT,
 			gettext_noop("Sets the time zone to use in log messages."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
 		},
 		&log_timezone_string,
 		"GMT",
@@ -3942,7 +3947,7 @@ struct config_string ConfigureNamesString[] =
 			gettext_noop("Sets the display format for date and time values."),
 			gettext_noop("Also controls interpretation of ambiguous "
 						 "date inputs."),
-			GUC_LIST_INPUT | GUC_REPORT
+			GUC_LIST_INPUT | GUC_REPORT | GUC_DEFAULT_INITDB
 		},
 		&datestyle_string,
 		"ISO, MDY",
@@ -4056,7 +4061,8 @@ struct config_string ConfigureNamesString[] =
 	{
 		{"lc_messages", PGC_SUSET, CLIENT_CONN_LOCALE,
 			gettext_noop("Sets the language in which messages are displayed."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
 		},
 		&locale_messages,
 		"",
@@ -4066,7 +4072,8 @@ struct config_string ConfigureNamesString[] =
 	{
 		{"lc_monetary", PGC_USERSET, CLIENT_CONN_LOCALE,
 			gettext_noop("Sets the locale for formatting monetary amounts."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
 		},
 		&locale_monetary,
 		"C",
@@ -4076,7 +4083,8 @@ struct config_string ConfigureNamesString[] =
 	{
 		{"lc_numeric", PGC_USERSET, CLIENT_CONN_LOCALE,
 			gettext_noop("Sets the locale for formatting numbers."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
 		},
 		&locale_numeric,
 		"C",
@@ -4086,7 +4094,8 @@ struct config_string ConfigureNamesString[] =
 	{
 		{"lc_time", PGC_USERSET, CLIENT_CONN_LOCALE,
 			gettext_noop("Sets the locale for formatting date and time values."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
 		},
 		&locale_time,
 		"C",
@@ -4245,7 +4254,8 @@ struct config_string ConfigureNamesString[] =
 		{"TimeZone", PGC_USERSET, CLIENT_CONN_LOCALE,
 			gettext_noop("Sets the time zone for displaying and interpreting time stamps."),
 			NULL,
-			GUC_REPORT
+			GUC_REPORT | GUC_DEFAULT_INITDB
+
 		},
 		&timezone_string,
 		"GMT",
@@ -4276,7 +4286,7 @@ struct config_string ConfigureNamesString[] =
 		{"unix_socket_directories", PGC_POSTMASTER, CONN_AUTH_SETTINGS,
 			gettext_noop("Sets the directories where Unix-domain sockets will be created."),
 			NULL,
-			GUC_LIST_INPUT | GUC_LIST_QUOTE | GUC_SUPERUSER_ONLY
+			GUC_LIST_INPUT | GUC_LIST_QUOTE | GUC_SUPERUSER_ONLY | GUC_DEFAULT_COMPILE
 		},
 		&Unix_socket_directories,
 		DEFAULT_PGSOCKET_DIR,
@@ -4357,7 +4367,7 @@ struct config_string ConfigureNamesString[] =
 		{"ssl_library", PGC_INTERNAL, PRESET_OPTIONS,
 			gettext_noop("Shows the name of the SSL library."),
 			NULL,
-			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_DEFAULT_COMPILE
 		},
 		&ssl_library,
 #ifdef USE_SSL
@@ -4432,7 +4442,9 @@ struct config_string ConfigureNamesString[] =
 	{
 		{"default_text_search_config", PGC_USERSET, CLIENT_CONN_LOCALE,
 			gettext_noop("Sets default text search configuration."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
+
 		},
 		&TSCurrentConfig,
 		"pg_catalog.simple",
@@ -4443,7 +4455,7 @@ struct config_string ConfigureNamesString[] =
 		{"ssl_ciphers", PGC_SIGHUP, CONN_AUTH_SSL,
 			gettext_noop("Sets the list of allowed SSL ciphers."),
 			NULL,
-			GUC_SUPERUSER_ONLY
+			GUC_SUPERUSER_ONLY | GUC_DEFAULT_COMPILE
 		},
 		&SSLCipherSuites,
 #ifdef USE_OPENSSL
@@ -4458,7 +4470,7 @@ struct config_string ConfigureNamesString[] =
 		{"ssl_ecdh_curve", PGC_SIGHUP, CONN_AUTH_SSL,
 			gettext_noop("Sets the curve to use for ECDH."),
 			NULL,
-			GUC_SUPERUSER_ONLY
+			GUC_SUPERUSER_ONLY | GUC_DEFAULT_COMPILE
 		},
 		&SSLECDHCurve,
 #ifdef USE_SSL
@@ -4717,7 +4729,8 @@ struct config_enum ConfigureNamesEnum[] =
 	{
 		{"syslog_facility", PGC_SIGHUP, LOGGING_WHERE,
 			gettext_noop("Sets the syslog \"facility\" to be used when syslog enabled."),
-			NULL
+			NULL,
+			GUC_DEFAULT_COMPILE
 		},
 		&syslog_facility,
 		DEFAULT_SYSLOG_FACILITY,
@@ -4826,7 +4839,9 @@ struct config_enum ConfigureNamesEnum[] =
 	{
 		{"dynamic_shared_memory_type", PGC_POSTMASTER, RESOURCES_MEM,
 			gettext_noop("Selects the dynamic shared memory implementation used."),
-			NULL
+			NULL,
+			/* platform-specific default, which is also overriden during initdb */
+			GUC_DEFAULT_COMPILE | GUC_DEFAULT_INITDB
 		},
 		&dynamic_shared_memory_type,
 		DEFAULT_DYNAMIC_SHARED_MEMORY_TYPE, dynamic_shared_memory_options,
@@ -4836,7 +4851,8 @@ struct config_enum ConfigureNamesEnum[] =
 	{
 		{"shared_memory_type", PGC_POSTMASTER, RESOURCES_MEM,
 			gettext_noop("Selects the shared memory implementation used for the main shared memory region."),
-			NULL
+			NULL,
+			GUC_DEFAULT_COMPILE
 		},
 		&shared_memory_type,
 		DEFAULT_SHARED_MEMORY_TYPE, shared_memory_options,
@@ -4846,7 +4862,8 @@ struct config_enum ConfigureNamesEnum[] =
 	{
 		{"wal_sync_method", PGC_SIGHUP, WAL_SETTINGS,
 			gettext_noop("Selects the method used for forcing WAL updates to disk."),
-			NULL
+			NULL,
+			GUC_DEFAULT_COMPILE
 		},
 		&sync_method,
 		DEFAULT_SYNC_METHOD, sync_method_options,
@@ -4910,7 +4927,8 @@ struct config_enum ConfigureNamesEnum[] =
 	{
 		{"password_encryption", PGC_USERSET, CONN_AUTH_AUTH,
 			gettext_noop("Chooses the algorithm for encrypting passwords."),
-			NULL
+			NULL,
+			GUC_DEFAULT_INITDB
 		},
 		&Password_encryption,
 		PASSWORD_TYPE_SCRAM_SHA_256, password_encryption_options,
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index 15c61992f06..2b17cd08fd9 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -222,6 +222,8 @@ typedef enum
 #define GUC_DISALLOW_IN_AUTO_FILE \
 							   0x002000 /* can't set in PG_AUTOCONF_FILENAME */
 #define GUC_RUNTIME_COMPUTED   0x004000 /* delay processing in 'postgres -C' */
+#define GUC_DEFAULT_COMPILE	   0x008000	/* default is determined at compile time */
+#define GUC_DEFAULT_INITDB	   0x010000	/* default is determined at during initdb */
 
 #define GUC_UNIT_KB			 0x01000000 /* value is in kilobytes */
 #define GUC_UNIT_BLOCKS		 0x02000000 /* value is in blocks */
-- 
2.34.1


--gwf9g/xA8/k5MLj4
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
	filename="0002-test-GUC-default-values-in-postgresql.conf.sample.patch"

From 0cc261a37af071554a0e2954b3a49ed6b3be2b60 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 25 May 2022 05:14:43 -0500
Subject: [PATCH 2/2] test GUC default values in postgresql.conf.sample

---
 src/test/modules/test_misc/t/003_check_guc.pl | 33 +++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index e9f33f3c775..ab6f16dad7b 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -106,4 +106,37 @@ foreach my $param (@sample_intersect)
 	);
 }
 
+# Test that GUCs in postgresql.conf.sample show the correct default values
+my $check_defaults = $node->safe_psql(
+	'postgres',
+	"
+	CREATE TABLE sample_conf AS
+	SELECT m[1] AS name, COALESCE(m[3], m[5]) AS sample_value
+	FROM (SELECT regexp_split_to_table(pg_read_file('$sample_file'), '\n') AS ln) conf,
+	regexp_match(ln, '^#?([_[:alpha:]]+) (= ''([^'']*)''|(= ([^[:space:]]*))).*') AS m
+	WHERE ln ~ '^#?[[:alpha:]]'
+	");
+
+$check_defaults = $node->safe_psql(
+	'postgres',
+	"
+	SELECT name, current_setting(name), sc.sample_value, boot_val
+	FROM pg_show_all_settings() psas
+	JOIN sample_conf sc USING(name)
+	WHERE lower(sc.sample_value) != lower(boot_val) -- same value (specified by Cluster.pm)
+	AND sc.sample_value != current_setting(name) -- same value, with human units
+	AND sc.sample_value != current_setting(name)||'.0' -- same value with .0 suffix
+	AND 'DEFAULT_COMPILE' != ALL(pg_settings_get_flags(psas.name)) -- dynamically-set defaults
+	AND 'DEFAULT_INITDB' != ALL(pg_settings_get_flags(psas.name)) -- dynamically-set defaults
+	AND NOT (sc.sample_value ~ '^0' AND current_setting(name) ~ '^0') -- zeros may be written differently
+	AND NOT (sc.sample_value='60s' AND current_setting(name) = '1min') -- two ways to write 1min
+	AND name NOT IN ('krb_server_keyfile', 'wal_retrieve_retry_interval'); -- exceptions
+	");
+
+my @check_defaults_array = split("\n", lc($check_defaults));
+print(STDERR "$check_defaults\n");
+
+is (@check_defaults_array, 0,
+	'check for consistency of defaults in postgresql.conf.sample');
+
 done_testing();
-- 
2.34.1


--gwf9g/xA8/k5MLj4--