public inbox for [email protected]
help / color / mirror / Atom feedRe: [RFC] building postgres with meson
27+ messages / 4 participants
[nested] [flat]
* Re: [RFC] building postgres with meson
@ 2022-08-10 17:19 Andres Freund <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: Andres Freund @ 2022-08-10 17:19 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers
Hi,
On 2022-06-02 10:26:09 -0700, Andres Freund wrote:
> > Could we have the meson build check that, say, if gram.c exists it
> > is newer than gram.y? Or get it to ignore an in-tree gram.c?
>
> I suspect the problem with ignoring is gram.h, that's probably a bit harder to
> ignore.
I tried to ignore various generated files in the source tree, but I don't
think it's doable for all of them. Consider
e.g. src/backend/utils/misc/guc-file.c which is gets built via #include
"guc-file.c" from gram.c
Because it's a "" include, the search path starts in the current directory and
only then -I is searched. To my knowledge there's no way of changing
that. Quoting the gcc manpage:
-I dir
-iquote dir
-isystem dir
-idirafter dir
Add the directory dir to the list of directories to be searched for header files during preprocessing. If dir begins with = or $SYSROOT, then
the = or $SYSROOT is replaced by the sysroot prefix; see --sysroot and -isysroot.
Directories specified with -iquote apply only to the quote form of the directive, "#include "file"". Directories specified with -I, -isystem, or
-idirafter apply to lookup for both the "#include "file"" and "#include <file>" directives.
You can specify any number or combination of these options on the command line to search for header files in several directories. The lookup
order is as follows:
1. For the quote form of the include directive, the directory of the current file is searched first.
2. For the quote form of the include directive, the directories specified by -iquote options are searched in left-to-right order, as they appear
on the command line.
3. Directories specified with -I options are scanned in left-to-right order.
[...]
Except for copying guc.c from source to build tree before building, I don't
see a way of ignoring the in-build-tree guc-file.c.
Not sure what a good way of dealing with this is. For now I'll make it just
error out if there's any known such file in the source tree, but that's not a
good solution forever. If it were just "normal" build leftovers I'd propose
to (optionally) just remove them, but that's not good for tarballs.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: [RFC] building postgres with meson
@ 2022-08-11 03:33 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-08-11 03:33 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers
On Thu, Aug 11, 2022 at 12:19 AM Andres Freund <[email protected]> wrote:
> I tried to ignore various generated files in the source tree, but I don't
> think it's doable for all of them. Consider
> e.g. src/backend/utils/misc/guc-file.c which is gets built via #include
> "guc-file.c" from gram.c
With a bit of work, we could probably get rid of those includes. See
27199058d98ef7f for one example.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: [RFC] building postgres with meson
@ 2022-08-11 03:37 Tom Lane <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 2 replies; 27+ messages in thread
From: Tom Lane @ 2022-08-11 03:37 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Andres Freund <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers
John Naylor <[email protected]> writes:
> With a bit of work, we could probably get rid of those includes. See
> 27199058d98ef7f for one example.
Yeah --- it would mean creating gram.h files for all the bison grammars
not just a few of them, but it's certainly do-able if there's motivation
to make the changes. Most of the files that are done that way date
from before we knew about flex's %top.
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: [RFC] building postgres with meson
@ 2022-08-11 03:45 Tom Lane <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 0 replies; 27+ messages in thread
From: Tom Lane @ 2022-08-11 03:45 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Andres Freund <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers
I wrote:
> John Naylor <[email protected]> writes:
>> With a bit of work, we could probably get rid of those includes. See
>> 27199058d98ef7f for one example.
> Yeah --- it would mean creating gram.h files for all the bison grammars
> not just a few of them, but it's certainly do-able if there's motivation
> to make the changes. Most of the files that are done that way date
> from before we knew about flex's %top.
BTW, 72b1e3a21 is another useful precedent in this area.
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: [RFC] building postgres with meson
@ 2022-08-11 03:57 John Naylor <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 2 replies; 27+ messages in thread
From: John Naylor @ 2022-08-11 03:57 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers
On Thu, Aug 11, 2022 at 10:37 AM Tom Lane <[email protected]> wrote:
>
> John Naylor <[email protected]> writes:
> > With a bit of work, we could probably get rid of those includes. See
> > 27199058d98ef7f for one example.
>
> Yeah --- it would mean creating gram.h files for all the bison grammars
> not just a few of them, but it's certainly do-able if there's motivation
> to make the changes. Most of the files that are done that way date
> from before we knew about flex's %top.
I'll volunteer to work on this unless an easier solution happens to
come along in the next couple days. (aside: guc-file.l doesn't have a
grammar, so not yet sure if that makes the issue easier or harder...)
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: [RFC] building postgres with meson
@ 2022-08-11 04:07 Andres Freund <[email protected]>
parent: John Naylor <[email protected]>
1 sibling, 1 reply; 27+ messages in thread
From: Andres Freund @ 2022-08-11 04:07 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Tom Lane <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers
Hi,
On 2022-08-11 10:57:33 +0700, John Naylor wrote:
> I'll volunteer to work on this unless an easier solution happens to
> come along in the next couple days.
Cool!
> (aside: guc-file.l doesn't have a grammar, so not yet sure if that makes the
> issue easier or harder...)
I think we should consider compiling it separately from guc.c. guc.c already
compiles quite slowly (iirc beat only by ecpg and main grammar), and it's a
relatively commonly changed source file.
It might even be a good idea to split guc.c so it only contains the settings
arrays + direct dependencies...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: [RFC] building postgres with meson
@ 2022-08-11 04:32 Tom Lane <[email protected]>
parent: John Naylor <[email protected]>
1 sibling, 0 replies; 27+ messages in thread
From: Tom Lane @ 2022-08-11 04:32 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Andres Freund <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers
John Naylor <[email protected]> writes:
> I'll volunteer to work on this unless an easier solution happens to
> come along in the next couple days. (aside: guc-file.l doesn't have a
> grammar, so not yet sure if that makes the issue easier or harder...)
That one's probably mostly about the issue mentioned in the other
commit you identified. Without %top, it's impossible to make a
standalone flex module honor the rule about thou-shalt-have-no-
other-includes-before-postgres.h. So embedding it in some other
file was originally a necessity for that. Now that we know how
to fix that, it's just a matter of making sure that any other stuff
the scanner needs is available from a .h file.
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* build remaining Flex files standalone
@ 2022-08-12 06:01 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-08-12 06:01 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
Starting a new thread to control clutter. [was: Re: [RFC] building
postgres with meson]
motivation: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
On Thu, Aug 11, 2022 at 11:07 AM Andres Freund <[email protected]> wrote:
> I think we should consider compiling it separately from guc.c. guc.c already
> compiles quite slowly (iirc beat only by ecpg and main grammar), and it's a
> relatively commonly changed source file.
Done in the attached, and will do the rest in time. It seemed most
straightforward to put ProcessConfigFileInternal() in guc.c since
that's where most of its callers are, and it relies on some vars and
types declared there. There are a couple new extern declarations in
guc.h that are only for guc.c and guc-file.c:
+/* functions shared between guc.c and guc-file.l */
+extern int guc_name_compare(const char *namea, const char *nameb);
+extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
+ bool applySettings, int elevel);
+extern void record_config_file_error(const char *errmsg,
+ const char *config_file,
+ int lineno,
+ ConfigVariable **head_p,
+ ConfigVariable **tail_p);
These might be better placed in a new guc_internal.h. Thoughts?
> It might even be a good idea to split guc.c so it only contains the settings
> arrays + direct dependencies...
Perhaps this can be a TODO item, one which falls under "[E] marks
items that are easier to implement". I've been slacking on removing
the old/intractable cruft from the TODO list, but we should also be
sticking small nice-but-not-necessary things in there. That said, if
this idea has any bearing on the guc_internal.h idea, it might be
better dealt with now.
--
John Naylor
EDB: http://www.enterprisedb.com
Attachments:
[text/x-patch] v1-0001-Build-guc-file.c-standalone.patch (27.4K, ../../CAFBsxsF8Gc2StS3haXofshHCzqNMRXiSxvQEYGwnFsTmsdwNeg@mail.gmail.com/2-v1-0001-Build-guc-file.c-standalone.patch)
download | inline diff:
From d723ba14acf56fd432e9e263db937fcc13fc0355 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Thu, 11 Aug 2022 19:38:37 +0700
Subject: [PATCH v1] Build guc-file.c standalone
The proposed Meson build system will need a way to ignore certain
generated files in order to coexist with the autoconf build system,
and #include'd C files generated by Flex make this more difficult.
Build guc-file.c separately from guc.c, as was done in 72b1e3a21.
TODO: other Flex-generated files
Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
---
src/backend/utils/misc/Makefile | 5 +-
src/backend/utils/misc/guc-file.l | 367 +-----------------------------
src/backend/utils/misc/guc.c | 362 ++++++++++++++++++++++++++++-
src/include/utils/guc.h | 9 +
4 files changed, 370 insertions(+), 373 deletions(-)
diff --git a/src/backend/utils/misc/Makefile b/src/backend/utils/misc/Makefile
index 1d5327cf64..cf7ce9bc83 100644
--- a/src/backend/utils/misc/Makefile
+++ b/src/backend/utils/misc/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
guc.o \
+ guc-file.o \
help_config.o \
pg_config.o \
pg_controldata.o \
@@ -37,10 +38,6 @@ endif
include $(top_srcdir)/src/backend/common.mk
-# guc-file is compiled as part of guc
-guc.o: guc-file.c
-
# Note: guc-file.c is not deleted by 'make clean',
# since we want to ship it in distribution tarballs.
clean:
- @rm -f lex.yy.c
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index ce5633844c..08adb454de 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -7,7 +7,7 @@
* src/backend/utils/misc/guc-file.l
*/
-%{
+%top{
#include "postgres.h"
@@ -17,9 +17,12 @@
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "storage/fd.h"
+#include <sys/stat.h>
#include "utils/guc.h"
+#include "utils/memutils.h"
+}
-
+%{
/*
* flex emits a yy_fatal_error() function that it calls in response to
* critical errors like malloc failure, file I/O errors, and detection of
@@ -48,12 +51,6 @@ static sigjmp_buf *GUC_flex_fatal_jmp;
static void FreeConfigVariable(ConfigVariable *item);
-static void record_config_file_error(const char *errmsg,
- const char *config_file,
- int lineno,
- ConfigVariable **head_p,
- ConfigVariable **tail_p);
-
static int GUC_flex_fatal(const char *msg);
/* LCOV_EXCL_START */
@@ -159,358 +156,6 @@ ProcessConfigFile(GucContext context)
MemoryContextDelete(config_cxt);
}
-/*
- * This function handles both actual config file (re)loads and execution of
- * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
- * case we don't apply any of the settings, but we make all the usual validity
- * checks, and we return the ConfigVariable list so that it can be printed out
- * by show_all_file_settings().
- */
-static ConfigVariable *
-ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
-{
- bool error = false;
- bool applying = false;
- const char *ConfFileWithError;
- ConfigVariable *item,
- *head,
- *tail;
- int i;
-
- /* Parse the main config file into a list of option names and values */
- ConfFileWithError = ConfigFileName;
- head = tail = NULL;
-
- if (!ParseConfigFile(ConfigFileName, true,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- goto bail_out;
- }
-
- /*
- * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
- * replace any parameters set by ALTER SYSTEM command. Because this file
- * is in the data directory, we can't read it until the DataDir has been
- * set.
- */
- if (DataDir)
- {
- if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- ConfFileWithError = PG_AUTOCONF_FILENAME;
- goto bail_out;
- }
- }
- else
- {
- /*
- * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
- * read. In this case, we don't want to accept any settings but
- * data_directory from postgresql.conf, because they might be
- * overwritten with settings in the PG_AUTOCONF_FILENAME file which
- * will be read later. OTOH, since data_directory isn't allowed in the
- * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
- */
- ConfigVariable *newlist = NULL;
-
- /*
- * Prune all items except the last "data_directory" from the list.
- */
- for (item = head; item; item = item->next)
- {
- if (!item->ignore &&
- strcmp(item->name, "data_directory") == 0)
- newlist = item;
- }
-
- if (newlist)
- newlist->next = NULL;
- head = tail = newlist;
-
- /*
- * Quick exit if data_directory is not present in file.
- *
- * We need not do any further processing, in particular we don't set
- * PgReloadTime; that will be set soon by subsequent full loading of
- * the config file.
- */
- if (head == NULL)
- goto bail_out;
- }
-
- /*
- * Mark all extant GUC variables as not present in the config file. We
- * need this so that we can tell below which ones have been removed from
- * the file since we last processed it.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
-
- gconf->status &= ~GUC_IS_IN_FILE;
- }
-
- /*
- * Check if all the supplied option names are valid, as an additional
- * quasi-syntactic check on the validity of the config file. It is
- * important that the postmaster and all backends agree on the results of
- * this phase, else we will have strange inconsistencies about which
- * processes accept a config file update and which don't. Hence, unknown
- * custom variable names have to be accepted without complaint. For the
- * same reason, we don't attempt to validate the options' values here.
- *
- * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
- * variable mentioned in the file; and we detect duplicate entries in the
- * file and mark the earlier occurrences as ignorable.
- */
- for (item = head; item; item = item->next)
- {
- struct config_generic *record;
-
- /* Ignore anything already marked as ignorable */
- if (item->ignore)
- continue;
-
- /*
- * Try to find the variable; but do not create a custom placeholder if
- * it's not there already.
- */
- record = find_option(item->name, false, true, elevel);
-
- if (record)
- {
- /* If it's already marked, then this is a duplicate entry */
- if (record->status & GUC_IS_IN_FILE)
- {
- /*
- * Mark the earlier occurrence(s) as dead/ignorable. We could
- * avoid the O(N^2) behavior here with some additional state,
- * but it seems unlikely to be worth the trouble.
- */
- ConfigVariable *pitem;
-
- for (pitem = head; pitem != item; pitem = pitem->next)
- {
- if (!pitem->ignore &&
- strcmp(pitem->name, item->name) == 0)
- pitem->ignore = true;
- }
- }
- /* Now mark it as present in file */
- record->status |= GUC_IS_IN_FILE;
- }
- else if (!valid_custom_variable_name(item->name))
- {
- /* Invalid non-custom variable, so complain */
- ereport(elevel,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
- item->name,
- item->filename, item->sourceline)));
- item->errmsg = pstrdup("unrecognized configuration parameter");
- error = true;
- ConfFileWithError = item->filename;
- }
- }
-
- /*
- * If we've detected any errors so far, we don't want to risk applying any
- * changes.
- */
- if (error)
- goto bail_out;
-
- /* Otherwise, set flag that we're beginning to apply changes */
- applying = true;
-
- /*
- * Check for variables having been removed from the config file, and
- * revert their reset values (and perhaps also effective values) to the
- * boot-time defaults. If such a variable can't be changed after startup,
- * report that and continue.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
- GucStack *stack;
-
- if (gconf->reset_source != PGC_S_FILE ||
- (gconf->status & GUC_IS_IN_FILE))
- continue;
- if (gconf->context < PGC_SIGHUP)
- {
- /* The removal can't be effective without a restart */
- gconf->status |= GUC_PENDING_RESTART;
- ereport(elevel,
- (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
- errmsg("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name)));
- record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name),
- NULL, 0,
- &head, &tail);
- error = true;
- continue;
- }
-
- /* No more to do if we're just doing show_all_file_settings() */
- if (!applySettings)
- continue;
-
- /*
- * Reset any "file" sources to "default", else set_config_option will
- * not override those settings.
- */
- if (gconf->reset_source == PGC_S_FILE)
- gconf->reset_source = PGC_S_DEFAULT;
- if (gconf->source == PGC_S_FILE)
- gconf->source = PGC_S_DEFAULT;
- for (stack = gconf->stack; stack; stack = stack->prev)
- {
- if (stack->source == PGC_S_FILE)
- stack->source = PGC_S_DEFAULT;
- }
-
- /* Now we can re-apply the wired-in default (i.e., the boot_val) */
- if (set_config_option(gconf->name, NULL,
- context, PGC_S_DEFAULT,
- GUC_ACTION_SET, true, 0, false) > 0)
- {
- /* Log the change if appropriate */
- if (context == PGC_SIGHUP)
- ereport(elevel,
- (errmsg("parameter \"%s\" removed from configuration file, reset to default",
- gconf->name)));
- }
- }
-
- /*
- * Restore any variables determined by environment variables or
- * dynamically-computed defaults. This is a no-op except in the case
- * where one of these had been in the config file and is now removed.
- *
- * In particular, we *must not* do this during the postmaster's initial
- * loading of the file, since the timezone functions in particular should
- * be run only after initialization is complete.
- *
- * XXX this is an unmaintainable crock, because we have to know how to set
- * (or at least what to call to set) every non-PGC_INTERNAL variable that
- * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
- */
- if (context == PGC_SIGHUP && applySettings)
- {
- InitializeGUCOptionsFromEnvironment();
- pg_timezone_abbrev_initialize();
- /* this selects SQL_ASCII in processes not connected to a database */
- SetConfigOption("client_encoding", GetDatabaseEncodingName(),
- PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
- }
-
- /*
- * Now apply the values from the config file.
- */
- for (item = head; item; item = item->next)
- {
- char *pre_value = NULL;
- int scres;
-
- /* Ignore anything marked as ignorable */
- if (item->ignore)
- continue;
-
- /* In SIGHUP cases in the postmaster, we want to report changes */
- if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
- {
- const char *preval = GetConfigOption(item->name, true, false);
-
- /* If option doesn't exist yet or is NULL, treat as empty string */
- if (!preval)
- preval = "";
- /* must dup, else might have dangling pointer below */
- pre_value = pstrdup(preval);
- }
-
- scres = set_config_option(item->name, item->value,
- context, PGC_S_FILE,
- GUC_ACTION_SET, applySettings, 0, false);
- if (scres > 0)
- {
- /* variable was updated, so log the change if appropriate */
- if (pre_value)
- {
- const char *post_value = GetConfigOption(item->name, true, false);
-
- if (!post_value)
- post_value = "";
- if (strcmp(pre_value, post_value) != 0)
- ereport(elevel,
- (errmsg("parameter \"%s\" changed to \"%s\"",
- item->name, item->value)));
- }
- item->applied = true;
- }
- else if (scres == 0)
- {
- error = true;
- item->errmsg = pstrdup("setting could not be applied");
- ConfFileWithError = item->filename;
- }
- else
- {
- /* no error, but variable's active value was not changed */
- item->applied = true;
- }
-
- /*
- * We should update source location unless there was an error, since
- * even if the active value didn't change, the reset value might have.
- * (In the postmaster, there won't be a difference, but it does matter
- * in backends.)
- */
- if (scres != 0 && applySettings)
- set_config_sourcefile(item->name, item->filename,
- item->sourceline);
-
- if (pre_value)
- pfree(pre_value);
- }
-
- /* Remember when we last successfully loaded the config file. */
- if (applySettings)
- PgReloadTime = GetCurrentTimestamp();
-
-bail_out:
- if (error && applySettings)
- {
- /* During postmaster startup, any error is fatal */
- if (context == PGC_POSTMASTER)
- ereport(ERROR,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors",
- ConfFileWithError)));
- else if (applying)
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
- ConfFileWithError)));
- else
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; no changes were applied",
- ConfFileWithError)));
- }
-
- /* Successful or otherwise, return the collected data list */
- return head;
-}
-
/*
* Given a configuration file or directory location that may be a relative
* path, return an absolute one. We consider the location to be relative to
@@ -659,7 +304,7 @@ cleanup:
* Capture an error message in the ConfigVariable list returned by
* config file parsing.
*/
-static void
+void
record_config_file_error(const char *errmsg,
const char *config_file,
int lineno,
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 5db5df6285..e4a00f1205 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -243,10 +243,6 @@ static void assign_recovery_target_lsn(const char *newval, void *extra);
static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
-/* Private functions in guc-file.l that need to be called from guc.c */
-static ConfigVariable *ProcessConfigFileInternal(GucContext context,
- bool applySettings, int elevel);
-
/*
* Track whether there were any deferred checks for custom resource managers
* specified in wal_consistency_checking.
@@ -5164,8 +5160,8 @@ static bool report_needed; /* true if any GUC_REPORT reports are needed */
static int GUCNestLevel = 0; /* 1 when in main transaction */
+static struct config_generic *find_option(const char *name, bool create_placeholders, bool skip_errors, int elevel);
static int guc_var_compare(const void *a, const void *b);
-static int guc_name_compare(const char *namea, const char *nameb);
static void InitializeGUCOptionsFromEnvironment(void);
static void InitializeOneGUCOption(struct config_generic *gconf);
static void push_old_value(struct config_generic *gconf, GucAction action);
@@ -5184,7 +5180,359 @@ static bool validate_option_array_item(const char *name, const char *value,
static void write_auto_conf_file(int fd, const char *filename, ConfigVariable *head_p);
static void replace_auto_config_value(ConfigVariable **head_p, ConfigVariable **tail_p,
const char *name, const char *value);
+static bool valid_custom_variable_name(const char *name);
+
+/*
+ * This function handles both actual config file (re)loads and execution of
+ * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
+ * case we don't apply any of the settings, but we make all the usual validity
+ * checks, and we return the ConfigVariable list so that it can be printed out
+ * by show_all_file_settings().
+ */
+ConfigVariable *
+ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
+{
+ bool error = false;
+ bool applying = false;
+ const char *ConfFileWithError;
+ ConfigVariable *item,
+ *head,
+ *tail;
+ int i;
+
+ /* Parse the main config file into a list of option names and values */
+ ConfFileWithError = ConfigFileName;
+ head = tail = NULL;
+
+ if (!ParseConfigFile(ConfigFileName, true,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ goto bail_out;
+ }
+
+ /*
+ * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
+ * replace any parameters set by ALTER SYSTEM command. Because this file
+ * is in the data directory, we can't read it until the DataDir has been
+ * set.
+ */
+ if (DataDir)
+ {
+ if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ ConfFileWithError = PG_AUTOCONF_FILENAME;
+ goto bail_out;
+ }
+ }
+ else
+ {
+ /*
+ * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
+ * read. In this case, we don't want to accept any settings but
+ * data_directory from postgresql.conf, because they might be
+ * overwritten with settings in the PG_AUTOCONF_FILENAME file which
+ * will be read later. OTOH, since data_directory isn't allowed in the
+ * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
+ */
+ ConfigVariable *newlist = NULL;
+
+ /*
+ * Prune all items except the last "data_directory" from the list.
+ */
+ for (item = head; item; item = item->next)
+ {
+ if (!item->ignore &&
+ strcmp(item->name, "data_directory") == 0)
+ newlist = item;
+ }
+
+ if (newlist)
+ newlist->next = NULL;
+ head = tail = newlist;
+
+ /*
+ * Quick exit if data_directory is not present in file.
+ *
+ * We need not do any further processing, in particular we don't set
+ * PgReloadTime; that will be set soon by subsequent full loading of
+ * the config file.
+ */
+ if (head == NULL)
+ goto bail_out;
+ }
+
+ /*
+ * Mark all extant GUC variables as not present in the config file. We
+ * need this so that we can tell below which ones have been removed from
+ * the file since we last processed it.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+
+ gconf->status &= ~GUC_IS_IN_FILE;
+ }
+
+ /*
+ * Check if all the supplied option names are valid, as an additional
+ * quasi-syntactic check on the validity of the config file. It is
+ * important that the postmaster and all backends agree on the results of
+ * this phase, else we will have strange inconsistencies about which
+ * processes accept a config file update and which don't. Hence, unknown
+ * custom variable names have to be accepted without complaint. For the
+ * same reason, we don't attempt to validate the options' values here.
+ *
+ * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
+ * variable mentioned in the file; and we detect duplicate entries in the
+ * file and mark the earlier occurrences as ignorable.
+ */
+ for (item = head; item; item = item->next)
+ {
+ struct config_generic *record;
+
+ /* Ignore anything already marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /*
+ * Try to find the variable; but do not create a custom placeholder if
+ * it's not there already.
+ */
+ record = find_option(item->name, false, true, elevel);
+
+ if (record)
+ {
+ /* If it's already marked, then this is a duplicate entry */
+ if (record->status & GUC_IS_IN_FILE)
+ {
+ /*
+ * Mark the earlier occurrence(s) as dead/ignorable. We could
+ * avoid the O(N^2) behavior here with some additional state,
+ * but it seems unlikely to be worth the trouble.
+ */
+ ConfigVariable *pitem;
+
+ for (pitem = head; pitem != item; pitem = pitem->next)
+ {
+ if (!pitem->ignore &&
+ strcmp(pitem->name, item->name) == 0)
+ pitem->ignore = true;
+ }
+ }
+ /* Now mark it as present in file */
+ record->status |= GUC_IS_IN_FILE;
+ }
+ else if (!valid_custom_variable_name(item->name))
+ {
+ /* Invalid non-custom variable, so complain */
+ ereport(elevel,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
+ item->name,
+ item->filename, item->sourceline)));
+ item->errmsg = pstrdup("unrecognized configuration parameter");
+ error = true;
+ ConfFileWithError = item->filename;
+ }
+ }
+
+ /*
+ * If we've detected any errors so far, we don't want to risk applying any
+ * changes.
+ */
+ if (error)
+ goto bail_out;
+
+ /* Otherwise, set flag that we're beginning to apply changes */
+ applying = true;
+
+ /*
+ * Check for variables having been removed from the config file, and
+ * revert their reset values (and perhaps also effective values) to the
+ * boot-time defaults. If such a variable can't be changed after startup,
+ * report that and continue.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+ GucStack *stack;
+
+ if (gconf->reset_source != PGC_S_FILE ||
+ (gconf->status & GUC_IS_IN_FILE))
+ continue;
+ if (gconf->context < PGC_SIGHUP)
+ {
+ /* The removal can't be effective without a restart */
+ gconf->status |= GUC_PENDING_RESTART;
+ ereport(elevel,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ errmsg("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name)));
+ record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name),
+ NULL, 0,
+ &head, &tail);
+ error = true;
+ continue;
+ }
+
+ /* No more to do if we're just doing show_all_file_settings() */
+ if (!applySettings)
+ continue;
+
+ /*
+ * Reset any "file" sources to "default", else set_config_option will
+ * not override those settings.
+ */
+ if (gconf->reset_source == PGC_S_FILE)
+ gconf->reset_source = PGC_S_DEFAULT;
+ if (gconf->source == PGC_S_FILE)
+ gconf->source = PGC_S_DEFAULT;
+ for (stack = gconf->stack; stack; stack = stack->prev)
+ {
+ if (stack->source == PGC_S_FILE)
+ stack->source = PGC_S_DEFAULT;
+ }
+
+ /* Now we can re-apply the wired-in default (i.e., the boot_val) */
+ if (set_config_option(gconf->name, NULL,
+ context, PGC_S_DEFAULT,
+ GUC_ACTION_SET, true, 0, false) > 0)
+ {
+ /* Log the change if appropriate */
+ if (context == PGC_SIGHUP)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" removed from configuration file, reset to default",
+ gconf->name)));
+ }
+ }
+
+ /*
+ * Restore any variables determined by environment variables or
+ * dynamically-computed defaults. This is a no-op except in the case
+ * where one of these had been in the config file and is now removed.
+ *
+ * In particular, we *must not* do this during the postmaster's initial
+ * loading of the file, since the timezone functions in particular should
+ * be run only after initialization is complete.
+ *
+ * XXX this is an unmaintainable crock, because we have to know how to set
+ * (or at least what to call to set) every non-PGC_INTERNAL variable that
+ * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
+ */
+ if (context == PGC_SIGHUP && applySettings)
+ {
+ InitializeGUCOptionsFromEnvironment();
+ pg_timezone_abbrev_initialize();
+ /* this selects SQL_ASCII in processes not connected to a database */
+ SetConfigOption("client_encoding", GetDatabaseEncodingName(),
+ PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
+ }
+ /*
+ * Now apply the values from the config file.
+ */
+ for (item = head; item; item = item->next)
+ {
+ char *pre_value = NULL;
+ int scres;
+
+ /* Ignore anything marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /* In SIGHUP cases in the postmaster, we want to report changes */
+ if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
+ {
+ const char *preval = GetConfigOption(item->name, true, false);
+
+ /* If option doesn't exist yet or is NULL, treat as empty string */
+ if (!preval)
+ preval = "";
+ /* must dup, else might have dangling pointer below */
+ pre_value = pstrdup(preval);
+ }
+
+ scres = set_config_option(item->name, item->value,
+ context, PGC_S_FILE,
+ GUC_ACTION_SET, applySettings, 0, false);
+ if (scres > 0)
+ {
+ /* variable was updated, so log the change if appropriate */
+ if (pre_value)
+ {
+ const char *post_value = GetConfigOption(item->name, true, false);
+
+ if (!post_value)
+ post_value = "";
+ if (strcmp(pre_value, post_value) != 0)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" changed to \"%s\"",
+ item->name, item->value)));
+ }
+ item->applied = true;
+ }
+ else if (scres == 0)
+ {
+ error = true;
+ item->errmsg = pstrdup("setting could not be applied");
+ ConfFileWithError = item->filename;
+ }
+ else
+ {
+ /* no error, but variable's active value was not changed */
+ item->applied = true;
+ }
+
+ /*
+ * We should update source location unless there was an error, since
+ * even if the active value didn't change, the reset value might have.
+ * (In the postmaster, there won't be a difference, but it does matter
+ * in backends.)
+ */
+ if (scres != 0 && applySettings)
+ set_config_sourcefile(item->name, item->filename,
+ item->sourceline);
+
+ if (pre_value)
+ pfree(pre_value);
+ }
+
+ /* Remember when we last successfully loaded the config file. */
+ if (applySettings)
+ PgReloadTime = GetCurrentTimestamp();
+
+bail_out:
+ if (error && applySettings)
+ {
+ /* During postmaster startup, any error is fatal */
+ if (context == PGC_POSTMASTER)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors",
+ ConfFileWithError)));
+ else if (applying)
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
+ ConfFileWithError)));
+ else
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; no changes were applied",
+ ConfFileWithError)));
+ }
+
+ /* Successful or otherwise, return the collected data list */
+ return head;
+}
/*
* Some infrastructure for checking malloc/strdup/realloc calls
@@ -5741,7 +6089,7 @@ guc_var_compare(const void *a, const void *b)
/*
* the bare comparison function for GUC names
*/
-static int
+int
guc_name_compare(const char *namea, const char *nameb)
{
/*
@@ -12988,5 +13336,3 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
-
-#include "guc-file.c"
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index e734493a48..aae071cd82 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -442,6 +442,15 @@ extern void GUC_check_errcode(int sqlerrcode);
pre_format_elog_string(errno, TEXTDOMAIN), \
GUC_check_errhint_string = format_elog_string
+/* functions shared between guc.c and guc-file.l */
+extern int guc_name_compare(const char *namea, const char *nameb);
+extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
+ bool applySettings, int elevel);
+extern void record_config_file_error(const char *errmsg,
+ const char *config_file,
+ int lineno,
+ ConfigVariable **head_p,
+ ConfigVariable **tail_p);
/*
* The following functions are not in guc.c, but are declared here to avoid
--
2.36.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-13 08:39 John Naylor <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-08-13 08:39 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
Here are the rest. Most of it was pretty straightforward, with the
main exception of jsonpath_scan.c, which is not quite finished. That
one passes tests but still has one compiler warning. I'm unsure how
much of what is there already is really necessary or was cargo-culted
from elsewhere without explanation. For starters, I'm not sure why the
grammar has a forward declaration of "union YYSTYPE". It's noteworthy
that it used to compile standalone, but with a bit more stuff, and
that was reverted in 550b9d26f80fa30. I can hack on it some more later
but I ran out of steam today.
Other questions thus far:
- "BISONFLAGS += -d" is now in every make file with a .y file -- can
we just force that everywhere?
- Include order seems to matter for the grammar's .h file. I didn't
test if that was the case every time, and after a few miscompiles just
always made it the last inclusion, but I'm wondering if we should keep
those inclusions outside %top{} and put it at the start of the next
%{} ?
- contrib/cubeparse.y now has a global variable -- not terrific, but I
wanted to get something working first.
- I'm actually okay with guc-file.c now, but I'll still welcome
comments on that.
--
John Naylor
EDB: http://www.enterprisedb.com
Attachments:
[text/x-patch] v201-0003-Build-repl_scanner.c-standalone.patch (4.1K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/2-v201-0003-Build-repl_scanner.c-standalone.patch)
download | inline diff:
From da2b610b8608e6759f5ed9cc32b487ea8e750ce4 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 12 Aug 2022 17:09:45 +0700
Subject: [PATCH v201 3/9] Build repl_scanner.c standalone
---
src/backend/replication/Makefile | 11 +++++++++--
src/backend/replication/repl_gram.y | 2 --
src/backend/replication/repl_scanner.l | 27 +++++++++++++++-----------
3 files changed, 25 insertions(+), 15 deletions(-)
diff --git a/src/backend/replication/Makefile b/src/backend/replication/Makefile
index 2bffac58c0..bc8170418f 100644
--- a/src/backend/replication/Makefile
+++ b/src/backend/replication/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
repl_gram.o \
+ repl_scanner.o \
slot.o \
slotfuncs.o \
syncrep.o \
@@ -28,8 +29,14 @@ SUBDIRS = logical
include $(top_srcdir)/src/backend/common.mk
-# repl_scanner is compiled as part of repl_gram
-repl_gram.o: repl_scanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+repl_gram.h: repl_gram.c
+ touch $@
+
+repl_gram.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+repl_gram.o repl_scanner.o: repl_gram.h
# syncrep_scanner is compiled as part of syncrep_gram
syncrep_gram.o: syncrep_scanner.c
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 4cf087e602..b343f108d3 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -416,5 +416,3 @@ ident_or_keyword:
;
%%
-
-#include "repl_scanner.c"
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 586f0d3a5c..95a933c9a8 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* repl_scanner.l
@@ -17,7 +17,12 @@
#include "utils/builtins.h"
#include "parser/scansup.h"
+#include "replication/walsender_private.h"
+
+#include "repl_gram.h"
+}
+%{
/* Avoid exit() on fatal scanner errors (a bit ugly -- see yy_fatal_error) */
#undef fprintf
#define fprintf(file, fmt, msg) fprintf_to_ereport(fmt, msg)
@@ -130,7 +135,7 @@ WAIT { return K_WAIT; }
{space}+ { /* do nothing */ }
{digit}+ {
- yylval.uintval = strtoul(yytext, NULL, 10);
+ replication_yylval.uintval = strtoul(yytext, NULL, 10);
return UCONST;
}
@@ -138,8 +143,8 @@ WAIT { return K_WAIT; }
uint32 hi,
lo;
if (sscanf(yytext, "%X/%X", &hi, &lo) != 2)
- yyerror("invalid streaming start location");
- yylval.recptr = ((uint64) hi) << 32 | lo;
+ replication_yyerror("invalid streaming start location");
+ replication_yylval.recptr = ((uint64) hi) << 32 | lo;
return RECPTR;
}
@@ -151,7 +156,7 @@ WAIT { return K_WAIT; }
<xq>{quotestop} {
yyless(1);
BEGIN(INITIAL);
- yylval.str = litbufdup();
+ replication_yylval.str = litbufdup();
return SCONST;
}
@@ -173,9 +178,9 @@ WAIT { return K_WAIT; }
yyless(1);
BEGIN(INITIAL);
- yylval.str = litbufdup();
- len = strlen(yylval.str);
- truncate_identifier(yylval.str, len, true);
+ replication_yylval.str = litbufdup();
+ len = strlen(replication_yylval.str);
+ truncate_identifier(replication_yylval.str, len, true);
return IDENT;
}
@@ -186,7 +191,7 @@ WAIT { return K_WAIT; }
{identifier} {
int len = strlen(yytext);
- yylval.str = downcase_truncate_identifier(yytext, len, true);
+ replication_yylval.str = downcase_truncate_identifier(yytext, len, true);
return IDENT;
}
@@ -195,7 +200,7 @@ WAIT { return K_WAIT; }
return yytext[0];
}
-<xq,xd><<EOF>> { yyerror("unterminated quoted string"); }
+<xq,xd><<EOF>> { replication_yyerror("unterminated quoted string"); }
<<EOF>> {
@@ -231,7 +236,7 @@ addlitchar(unsigned char ychar)
}
void
-yyerror(const char *message)
+replication_yyerror(const char *message)
{
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
--
2.36.1
[text/x-patch] v201-0001-Build-guc-file.c-standalone.patch (27.5K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/3-v201-0001-Build-guc-file.c-standalone.patch)
download | inline diff:
From d723ba14acf56fd432e9e263db937fcc13fc0355 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Thu, 11 Aug 2022 19:38:37 +0700
Subject: [PATCH v201 1/9] Build guc-file.c standalone
The proposed Meson build system will need a way to ignore certain
generated files in order to coexist with the autoconf build system,
and #include'd C files generated by Flex make this more difficult.
Build guc-file.c separately from guc.c, as was done in 72b1e3a21.
TODO: other Flex-generated files
Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
---
src/backend/utils/misc/Makefile | 5 +-
src/backend/utils/misc/guc-file.l | 367 +-----------------------------
src/backend/utils/misc/guc.c | 362 ++++++++++++++++++++++++++++-
src/include/utils/guc.h | 9 +
4 files changed, 370 insertions(+), 373 deletions(-)
diff --git a/src/backend/utils/misc/Makefile b/src/backend/utils/misc/Makefile
index 1d5327cf64..cf7ce9bc83 100644
--- a/src/backend/utils/misc/Makefile
+++ b/src/backend/utils/misc/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
guc.o \
+ guc-file.o \
help_config.o \
pg_config.o \
pg_controldata.o \
@@ -37,10 +38,6 @@ endif
include $(top_srcdir)/src/backend/common.mk
-# guc-file is compiled as part of guc
-guc.o: guc-file.c
-
# Note: guc-file.c is not deleted by 'make clean',
# since we want to ship it in distribution tarballs.
clean:
- @rm -f lex.yy.c
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index ce5633844c..08adb454de 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -7,7 +7,7 @@
* src/backend/utils/misc/guc-file.l
*/
-%{
+%top{
#include "postgres.h"
@@ -17,9 +17,12 @@
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "storage/fd.h"
+#include <sys/stat.h>
#include "utils/guc.h"
+#include "utils/memutils.h"
+}
-
+%{
/*
* flex emits a yy_fatal_error() function that it calls in response to
* critical errors like malloc failure, file I/O errors, and detection of
@@ -48,12 +51,6 @@ static sigjmp_buf *GUC_flex_fatal_jmp;
static void FreeConfigVariable(ConfigVariable *item);
-static void record_config_file_error(const char *errmsg,
- const char *config_file,
- int lineno,
- ConfigVariable **head_p,
- ConfigVariable **tail_p);
-
static int GUC_flex_fatal(const char *msg);
/* LCOV_EXCL_START */
@@ -159,358 +156,6 @@ ProcessConfigFile(GucContext context)
MemoryContextDelete(config_cxt);
}
-/*
- * This function handles both actual config file (re)loads and execution of
- * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
- * case we don't apply any of the settings, but we make all the usual validity
- * checks, and we return the ConfigVariable list so that it can be printed out
- * by show_all_file_settings().
- */
-static ConfigVariable *
-ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
-{
- bool error = false;
- bool applying = false;
- const char *ConfFileWithError;
- ConfigVariable *item,
- *head,
- *tail;
- int i;
-
- /* Parse the main config file into a list of option names and values */
- ConfFileWithError = ConfigFileName;
- head = tail = NULL;
-
- if (!ParseConfigFile(ConfigFileName, true,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- goto bail_out;
- }
-
- /*
- * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
- * replace any parameters set by ALTER SYSTEM command. Because this file
- * is in the data directory, we can't read it until the DataDir has been
- * set.
- */
- if (DataDir)
- {
- if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- ConfFileWithError = PG_AUTOCONF_FILENAME;
- goto bail_out;
- }
- }
- else
- {
- /*
- * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
- * read. In this case, we don't want to accept any settings but
- * data_directory from postgresql.conf, because they might be
- * overwritten with settings in the PG_AUTOCONF_FILENAME file which
- * will be read later. OTOH, since data_directory isn't allowed in the
- * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
- */
- ConfigVariable *newlist = NULL;
-
- /*
- * Prune all items except the last "data_directory" from the list.
- */
- for (item = head; item; item = item->next)
- {
- if (!item->ignore &&
- strcmp(item->name, "data_directory") == 0)
- newlist = item;
- }
-
- if (newlist)
- newlist->next = NULL;
- head = tail = newlist;
-
- /*
- * Quick exit if data_directory is not present in file.
- *
- * We need not do any further processing, in particular we don't set
- * PgReloadTime; that will be set soon by subsequent full loading of
- * the config file.
- */
- if (head == NULL)
- goto bail_out;
- }
-
- /*
- * Mark all extant GUC variables as not present in the config file. We
- * need this so that we can tell below which ones have been removed from
- * the file since we last processed it.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
-
- gconf->status &= ~GUC_IS_IN_FILE;
- }
-
- /*
- * Check if all the supplied option names are valid, as an additional
- * quasi-syntactic check on the validity of the config file. It is
- * important that the postmaster and all backends agree on the results of
- * this phase, else we will have strange inconsistencies about which
- * processes accept a config file update and which don't. Hence, unknown
- * custom variable names have to be accepted without complaint. For the
- * same reason, we don't attempt to validate the options' values here.
- *
- * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
- * variable mentioned in the file; and we detect duplicate entries in the
- * file and mark the earlier occurrences as ignorable.
- */
- for (item = head; item; item = item->next)
- {
- struct config_generic *record;
-
- /* Ignore anything already marked as ignorable */
- if (item->ignore)
- continue;
-
- /*
- * Try to find the variable; but do not create a custom placeholder if
- * it's not there already.
- */
- record = find_option(item->name, false, true, elevel);
-
- if (record)
- {
- /* If it's already marked, then this is a duplicate entry */
- if (record->status & GUC_IS_IN_FILE)
- {
- /*
- * Mark the earlier occurrence(s) as dead/ignorable. We could
- * avoid the O(N^2) behavior here with some additional state,
- * but it seems unlikely to be worth the trouble.
- */
- ConfigVariable *pitem;
-
- for (pitem = head; pitem != item; pitem = pitem->next)
- {
- if (!pitem->ignore &&
- strcmp(pitem->name, item->name) == 0)
- pitem->ignore = true;
- }
- }
- /* Now mark it as present in file */
- record->status |= GUC_IS_IN_FILE;
- }
- else if (!valid_custom_variable_name(item->name))
- {
- /* Invalid non-custom variable, so complain */
- ereport(elevel,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
- item->name,
- item->filename, item->sourceline)));
- item->errmsg = pstrdup("unrecognized configuration parameter");
- error = true;
- ConfFileWithError = item->filename;
- }
- }
-
- /*
- * If we've detected any errors so far, we don't want to risk applying any
- * changes.
- */
- if (error)
- goto bail_out;
-
- /* Otherwise, set flag that we're beginning to apply changes */
- applying = true;
-
- /*
- * Check for variables having been removed from the config file, and
- * revert their reset values (and perhaps also effective values) to the
- * boot-time defaults. If such a variable can't be changed after startup,
- * report that and continue.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
- GucStack *stack;
-
- if (gconf->reset_source != PGC_S_FILE ||
- (gconf->status & GUC_IS_IN_FILE))
- continue;
- if (gconf->context < PGC_SIGHUP)
- {
- /* The removal can't be effective without a restart */
- gconf->status |= GUC_PENDING_RESTART;
- ereport(elevel,
- (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
- errmsg("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name)));
- record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name),
- NULL, 0,
- &head, &tail);
- error = true;
- continue;
- }
-
- /* No more to do if we're just doing show_all_file_settings() */
- if (!applySettings)
- continue;
-
- /*
- * Reset any "file" sources to "default", else set_config_option will
- * not override those settings.
- */
- if (gconf->reset_source == PGC_S_FILE)
- gconf->reset_source = PGC_S_DEFAULT;
- if (gconf->source == PGC_S_FILE)
- gconf->source = PGC_S_DEFAULT;
- for (stack = gconf->stack; stack; stack = stack->prev)
- {
- if (stack->source == PGC_S_FILE)
- stack->source = PGC_S_DEFAULT;
- }
-
- /* Now we can re-apply the wired-in default (i.e., the boot_val) */
- if (set_config_option(gconf->name, NULL,
- context, PGC_S_DEFAULT,
- GUC_ACTION_SET, true, 0, false) > 0)
- {
- /* Log the change if appropriate */
- if (context == PGC_SIGHUP)
- ereport(elevel,
- (errmsg("parameter \"%s\" removed from configuration file, reset to default",
- gconf->name)));
- }
- }
-
- /*
- * Restore any variables determined by environment variables or
- * dynamically-computed defaults. This is a no-op except in the case
- * where one of these had been in the config file and is now removed.
- *
- * In particular, we *must not* do this during the postmaster's initial
- * loading of the file, since the timezone functions in particular should
- * be run only after initialization is complete.
- *
- * XXX this is an unmaintainable crock, because we have to know how to set
- * (or at least what to call to set) every non-PGC_INTERNAL variable that
- * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
- */
- if (context == PGC_SIGHUP && applySettings)
- {
- InitializeGUCOptionsFromEnvironment();
- pg_timezone_abbrev_initialize();
- /* this selects SQL_ASCII in processes not connected to a database */
- SetConfigOption("client_encoding", GetDatabaseEncodingName(),
- PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
- }
-
- /*
- * Now apply the values from the config file.
- */
- for (item = head; item; item = item->next)
- {
- char *pre_value = NULL;
- int scres;
-
- /* Ignore anything marked as ignorable */
- if (item->ignore)
- continue;
-
- /* In SIGHUP cases in the postmaster, we want to report changes */
- if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
- {
- const char *preval = GetConfigOption(item->name, true, false);
-
- /* If option doesn't exist yet or is NULL, treat as empty string */
- if (!preval)
- preval = "";
- /* must dup, else might have dangling pointer below */
- pre_value = pstrdup(preval);
- }
-
- scres = set_config_option(item->name, item->value,
- context, PGC_S_FILE,
- GUC_ACTION_SET, applySettings, 0, false);
- if (scres > 0)
- {
- /* variable was updated, so log the change if appropriate */
- if (pre_value)
- {
- const char *post_value = GetConfigOption(item->name, true, false);
-
- if (!post_value)
- post_value = "";
- if (strcmp(pre_value, post_value) != 0)
- ereport(elevel,
- (errmsg("parameter \"%s\" changed to \"%s\"",
- item->name, item->value)));
- }
- item->applied = true;
- }
- else if (scres == 0)
- {
- error = true;
- item->errmsg = pstrdup("setting could not be applied");
- ConfFileWithError = item->filename;
- }
- else
- {
- /* no error, but variable's active value was not changed */
- item->applied = true;
- }
-
- /*
- * We should update source location unless there was an error, since
- * even if the active value didn't change, the reset value might have.
- * (In the postmaster, there won't be a difference, but it does matter
- * in backends.)
- */
- if (scres != 0 && applySettings)
- set_config_sourcefile(item->name, item->filename,
- item->sourceline);
-
- if (pre_value)
- pfree(pre_value);
- }
-
- /* Remember when we last successfully loaded the config file. */
- if (applySettings)
- PgReloadTime = GetCurrentTimestamp();
-
-bail_out:
- if (error && applySettings)
- {
- /* During postmaster startup, any error is fatal */
- if (context == PGC_POSTMASTER)
- ereport(ERROR,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors",
- ConfFileWithError)));
- else if (applying)
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
- ConfFileWithError)));
- else
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; no changes were applied",
- ConfFileWithError)));
- }
-
- /* Successful or otherwise, return the collected data list */
- return head;
-}
-
/*
* Given a configuration file or directory location that may be a relative
* path, return an absolute one. We consider the location to be relative to
@@ -659,7 +304,7 @@ cleanup:
* Capture an error message in the ConfigVariable list returned by
* config file parsing.
*/
-static void
+void
record_config_file_error(const char *errmsg,
const char *config_file,
int lineno,
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 5db5df6285..e4a00f1205 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -243,10 +243,6 @@ static void assign_recovery_target_lsn(const char *newval, void *extra);
static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
-/* Private functions in guc-file.l that need to be called from guc.c */
-static ConfigVariable *ProcessConfigFileInternal(GucContext context,
- bool applySettings, int elevel);
-
/*
* Track whether there were any deferred checks for custom resource managers
* specified in wal_consistency_checking.
@@ -5164,8 +5160,8 @@ static bool report_needed; /* true if any GUC_REPORT reports are needed */
static int GUCNestLevel = 0; /* 1 when in main transaction */
+static struct config_generic *find_option(const char *name, bool create_placeholders, bool skip_errors, int elevel);
static int guc_var_compare(const void *a, const void *b);
-static int guc_name_compare(const char *namea, const char *nameb);
static void InitializeGUCOptionsFromEnvironment(void);
static void InitializeOneGUCOption(struct config_generic *gconf);
static void push_old_value(struct config_generic *gconf, GucAction action);
@@ -5184,7 +5180,359 @@ static bool validate_option_array_item(const char *name, const char *value,
static void write_auto_conf_file(int fd, const char *filename, ConfigVariable *head_p);
static void replace_auto_config_value(ConfigVariable **head_p, ConfigVariable **tail_p,
const char *name, const char *value);
+static bool valid_custom_variable_name(const char *name);
+
+/*
+ * This function handles both actual config file (re)loads and execution of
+ * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
+ * case we don't apply any of the settings, but we make all the usual validity
+ * checks, and we return the ConfigVariable list so that it can be printed out
+ * by show_all_file_settings().
+ */
+ConfigVariable *
+ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
+{
+ bool error = false;
+ bool applying = false;
+ const char *ConfFileWithError;
+ ConfigVariable *item,
+ *head,
+ *tail;
+ int i;
+
+ /* Parse the main config file into a list of option names and values */
+ ConfFileWithError = ConfigFileName;
+ head = tail = NULL;
+
+ if (!ParseConfigFile(ConfigFileName, true,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ goto bail_out;
+ }
+
+ /*
+ * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
+ * replace any parameters set by ALTER SYSTEM command. Because this file
+ * is in the data directory, we can't read it until the DataDir has been
+ * set.
+ */
+ if (DataDir)
+ {
+ if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ ConfFileWithError = PG_AUTOCONF_FILENAME;
+ goto bail_out;
+ }
+ }
+ else
+ {
+ /*
+ * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
+ * read. In this case, we don't want to accept any settings but
+ * data_directory from postgresql.conf, because they might be
+ * overwritten with settings in the PG_AUTOCONF_FILENAME file which
+ * will be read later. OTOH, since data_directory isn't allowed in the
+ * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
+ */
+ ConfigVariable *newlist = NULL;
+
+ /*
+ * Prune all items except the last "data_directory" from the list.
+ */
+ for (item = head; item; item = item->next)
+ {
+ if (!item->ignore &&
+ strcmp(item->name, "data_directory") == 0)
+ newlist = item;
+ }
+
+ if (newlist)
+ newlist->next = NULL;
+ head = tail = newlist;
+
+ /*
+ * Quick exit if data_directory is not present in file.
+ *
+ * We need not do any further processing, in particular we don't set
+ * PgReloadTime; that will be set soon by subsequent full loading of
+ * the config file.
+ */
+ if (head == NULL)
+ goto bail_out;
+ }
+
+ /*
+ * Mark all extant GUC variables as not present in the config file. We
+ * need this so that we can tell below which ones have been removed from
+ * the file since we last processed it.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+
+ gconf->status &= ~GUC_IS_IN_FILE;
+ }
+
+ /*
+ * Check if all the supplied option names are valid, as an additional
+ * quasi-syntactic check on the validity of the config file. It is
+ * important that the postmaster and all backends agree on the results of
+ * this phase, else we will have strange inconsistencies about which
+ * processes accept a config file update and which don't. Hence, unknown
+ * custom variable names have to be accepted without complaint. For the
+ * same reason, we don't attempt to validate the options' values here.
+ *
+ * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
+ * variable mentioned in the file; and we detect duplicate entries in the
+ * file and mark the earlier occurrences as ignorable.
+ */
+ for (item = head; item; item = item->next)
+ {
+ struct config_generic *record;
+
+ /* Ignore anything already marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /*
+ * Try to find the variable; but do not create a custom placeholder if
+ * it's not there already.
+ */
+ record = find_option(item->name, false, true, elevel);
+
+ if (record)
+ {
+ /* If it's already marked, then this is a duplicate entry */
+ if (record->status & GUC_IS_IN_FILE)
+ {
+ /*
+ * Mark the earlier occurrence(s) as dead/ignorable. We could
+ * avoid the O(N^2) behavior here with some additional state,
+ * but it seems unlikely to be worth the trouble.
+ */
+ ConfigVariable *pitem;
+
+ for (pitem = head; pitem != item; pitem = pitem->next)
+ {
+ if (!pitem->ignore &&
+ strcmp(pitem->name, item->name) == 0)
+ pitem->ignore = true;
+ }
+ }
+ /* Now mark it as present in file */
+ record->status |= GUC_IS_IN_FILE;
+ }
+ else if (!valid_custom_variable_name(item->name))
+ {
+ /* Invalid non-custom variable, so complain */
+ ereport(elevel,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
+ item->name,
+ item->filename, item->sourceline)));
+ item->errmsg = pstrdup("unrecognized configuration parameter");
+ error = true;
+ ConfFileWithError = item->filename;
+ }
+ }
+
+ /*
+ * If we've detected any errors so far, we don't want to risk applying any
+ * changes.
+ */
+ if (error)
+ goto bail_out;
+
+ /* Otherwise, set flag that we're beginning to apply changes */
+ applying = true;
+
+ /*
+ * Check for variables having been removed from the config file, and
+ * revert their reset values (and perhaps also effective values) to the
+ * boot-time defaults. If such a variable can't be changed after startup,
+ * report that and continue.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+ GucStack *stack;
+
+ if (gconf->reset_source != PGC_S_FILE ||
+ (gconf->status & GUC_IS_IN_FILE))
+ continue;
+ if (gconf->context < PGC_SIGHUP)
+ {
+ /* The removal can't be effective without a restart */
+ gconf->status |= GUC_PENDING_RESTART;
+ ereport(elevel,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ errmsg("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name)));
+ record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name),
+ NULL, 0,
+ &head, &tail);
+ error = true;
+ continue;
+ }
+
+ /* No more to do if we're just doing show_all_file_settings() */
+ if (!applySettings)
+ continue;
+
+ /*
+ * Reset any "file" sources to "default", else set_config_option will
+ * not override those settings.
+ */
+ if (gconf->reset_source == PGC_S_FILE)
+ gconf->reset_source = PGC_S_DEFAULT;
+ if (gconf->source == PGC_S_FILE)
+ gconf->source = PGC_S_DEFAULT;
+ for (stack = gconf->stack; stack; stack = stack->prev)
+ {
+ if (stack->source == PGC_S_FILE)
+ stack->source = PGC_S_DEFAULT;
+ }
+
+ /* Now we can re-apply the wired-in default (i.e., the boot_val) */
+ if (set_config_option(gconf->name, NULL,
+ context, PGC_S_DEFAULT,
+ GUC_ACTION_SET, true, 0, false) > 0)
+ {
+ /* Log the change if appropriate */
+ if (context == PGC_SIGHUP)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" removed from configuration file, reset to default",
+ gconf->name)));
+ }
+ }
+
+ /*
+ * Restore any variables determined by environment variables or
+ * dynamically-computed defaults. This is a no-op except in the case
+ * where one of these had been in the config file and is now removed.
+ *
+ * In particular, we *must not* do this during the postmaster's initial
+ * loading of the file, since the timezone functions in particular should
+ * be run only after initialization is complete.
+ *
+ * XXX this is an unmaintainable crock, because we have to know how to set
+ * (or at least what to call to set) every non-PGC_INTERNAL variable that
+ * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
+ */
+ if (context == PGC_SIGHUP && applySettings)
+ {
+ InitializeGUCOptionsFromEnvironment();
+ pg_timezone_abbrev_initialize();
+ /* this selects SQL_ASCII in processes not connected to a database */
+ SetConfigOption("client_encoding", GetDatabaseEncodingName(),
+ PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
+ }
+ /*
+ * Now apply the values from the config file.
+ */
+ for (item = head; item; item = item->next)
+ {
+ char *pre_value = NULL;
+ int scres;
+
+ /* Ignore anything marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /* In SIGHUP cases in the postmaster, we want to report changes */
+ if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
+ {
+ const char *preval = GetConfigOption(item->name, true, false);
+
+ /* If option doesn't exist yet or is NULL, treat as empty string */
+ if (!preval)
+ preval = "";
+ /* must dup, else might have dangling pointer below */
+ pre_value = pstrdup(preval);
+ }
+
+ scres = set_config_option(item->name, item->value,
+ context, PGC_S_FILE,
+ GUC_ACTION_SET, applySettings, 0, false);
+ if (scres > 0)
+ {
+ /* variable was updated, so log the change if appropriate */
+ if (pre_value)
+ {
+ const char *post_value = GetConfigOption(item->name, true, false);
+
+ if (!post_value)
+ post_value = "";
+ if (strcmp(pre_value, post_value) != 0)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" changed to \"%s\"",
+ item->name, item->value)));
+ }
+ item->applied = true;
+ }
+ else if (scres == 0)
+ {
+ error = true;
+ item->errmsg = pstrdup("setting could not be applied");
+ ConfFileWithError = item->filename;
+ }
+ else
+ {
+ /* no error, but variable's active value was not changed */
+ item->applied = true;
+ }
+
+ /*
+ * We should update source location unless there was an error, since
+ * even if the active value didn't change, the reset value might have.
+ * (In the postmaster, there won't be a difference, but it does matter
+ * in backends.)
+ */
+ if (scres != 0 && applySettings)
+ set_config_sourcefile(item->name, item->filename,
+ item->sourceline);
+
+ if (pre_value)
+ pfree(pre_value);
+ }
+
+ /* Remember when we last successfully loaded the config file. */
+ if (applySettings)
+ PgReloadTime = GetCurrentTimestamp();
+
+bail_out:
+ if (error && applySettings)
+ {
+ /* During postmaster startup, any error is fatal */
+ if (context == PGC_POSTMASTER)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors",
+ ConfFileWithError)));
+ else if (applying)
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
+ ConfFileWithError)));
+ else
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; no changes were applied",
+ ConfFileWithError)));
+ }
+
+ /* Successful or otherwise, return the collected data list */
+ return head;
+}
/*
* Some infrastructure for checking malloc/strdup/realloc calls
@@ -5741,7 +6089,7 @@ guc_var_compare(const void *a, const void *b)
/*
* the bare comparison function for GUC names
*/
-static int
+int
guc_name_compare(const char *namea, const char *nameb)
{
/*
@@ -12988,5 +13336,3 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
-
-#include "guc-file.c"
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index e734493a48..aae071cd82 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -442,6 +442,15 @@ extern void GUC_check_errcode(int sqlerrcode);
pre_format_elog_string(errno, TEXTDOMAIN), \
GUC_check_errhint_string = format_elog_string
+/* functions shared between guc.c and guc-file.l */
+extern int guc_name_compare(const char *namea, const char *nameb);
+extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
+ bool applySettings, int elevel);
+extern void record_config_file_error(const char *errmsg,
+ const char *config_file,
+ int lineno,
+ ConfigVariable **head_p,
+ ConfigVariable **tail_p);
/*
* The following functions are not in guc.c, but are declared here to avoid
--
2.36.1
[text/x-patch] v201-0005-Build-specscanner.c-standalone.patch (4.5K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/4-v201-0005-Build-specscanner.c-standalone.patch)
download | inline diff:
From 4c105b7b415f0937e7fa42e8313c9452a8db1ad4 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 09:34:17 +0700
Subject: [PATCH v201 5/9] Build specscanner.c standalone
---
src/test/isolation/.gitignore | 1 +
src/test/isolation/Makefile | 15 +++++++++++----
src/test/isolation/specparse.y | 2 --
src/test/isolation/specscanner.l | 25 ++++++++++++++++---------
4 files changed, 28 insertions(+), 15 deletions(-)
diff --git a/src/test/isolation/.gitignore b/src/test/isolation/.gitignore
index 870dac4d28..2c13b4bf98 100644
--- a/src/test/isolation/.gitignore
+++ b/src/test/isolation/.gitignore
@@ -3,6 +3,7 @@
/pg_isolation_regress
# Local generated source files
+/specparse.h
/specparse.c
/specscanner.c
diff --git a/src/test/isolation/Makefile b/src/test/isolation/Makefile
index 0d452c89d4..b8738b7c1b 100644
--- a/src/test/isolation/Makefile
+++ b/src/test/isolation/Makefile
@@ -15,7 +15,8 @@ override CPPFLAGS := -I. -I$(srcdir) -I$(libpq_srcdir) \
OBJS = \
$(WIN32RES) \
isolationtester.o \
- specparse.o
+ specparse.o \
+ specscanner.o
all: isolationtester$(X) pg_isolation_regress$(X)
@@ -44,8 +45,14 @@ isolationtester$(X): $(OBJS) | submake-libpq submake-libpgport
distprep: specparse.c specscanner.c
-# specscanner is compiled as part of specparse
-specparse.o: specscanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+specparse.h: specparse.c
+ touch $@
+
+specparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+specparse.o specscanner.o: specparse.h
# specparse.c and specscanner.c are in the distribution tarball,
# so do not clean them here
@@ -55,7 +62,7 @@ clean distclean:
rm -rf $(pg_regress_clean_files)
maintainer-clean: distclean
- rm -f specparse.c specscanner.c
+ rm -f specparse.h specparse.c specscanner.c
installcheck: all
$(pg_isolation_regress_installcheck) --schedule=$(srcdir)/isolation_schedule
diff --git a/src/test/isolation/specparse.y b/src/test/isolation/specparse.y
index eb368184b8..657285cc23 100644
--- a/src/test/isolation/specparse.y
+++ b/src/test/isolation/specparse.y
@@ -276,5 +276,3 @@ blocker:
;
%%
-
-#include "specscanner.c"
diff --git a/src/test/isolation/specscanner.l b/src/test/isolation/specscanner.l
index aa6e89268e..2dc292c21d 100644
--- a/src/test/isolation/specscanner.l
+++ b/src/test/isolation/specscanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* specscanner.l
@@ -9,7 +9,14 @@
*
*-------------------------------------------------------------------------
*/
+#include "postgres_fe.h"
+
+#include "isolationtester.h"
+#include "specparse.h"
+}
+
+%{
static int yyline = 1; /* line number for error reporting */
#define LITBUF_INIT 1024 /* initial size of litbuf */
@@ -75,7 +82,7 @@ teardown { return TEARDOWN; }
/* Plain identifiers */
{identifier} {
- yylval.str = pg_strdup(yytext);
+ spec_yylval.str = pg_strdup(yytext);
return(identifier);
}
@@ -87,13 +94,13 @@ teardown { return TEARDOWN; }
<qident>\"\" { addlitchar(yytext[0]); }
<qident>\" {
litbuf[litbufpos] = '\0';
- yylval.str = pg_strdup(litbuf);
+ spec_yylval.str = pg_strdup(litbuf);
BEGIN(INITIAL);
return(identifier);
}
<qident>. { addlitchar(yytext[0]); }
-<qident>\n { yyerror("unexpected newline in quoted identifier"); }
-<qident><<EOF>> { yyerror("unterminated quoted identifier"); }
+<qident>\n { spec_yyerror("unexpected newline in quoted identifier"); }
+<qident><<EOF>> { spec_yyerror("unterminated quoted identifier"); }
/* SQL blocks: { UPDATE ... } */
/* We trim leading/trailing whitespace, otherwise they're unprocessed */
@@ -104,7 +111,7 @@ teardown { return TEARDOWN; }
}
<sql>{space}*"}" {
litbuf[litbufpos] = '\0';
- yylval.str = pg_strdup(litbuf);
+ spec_yylval.str = pg_strdup(litbuf);
BEGIN(INITIAL);
return(sqlblock);
}
@@ -116,12 +123,12 @@ teardown { return TEARDOWN; }
addlitchar(yytext[0]);
}
<sql><<EOF>> {
- yyerror("unterminated sql block");
+ spec_yyerror("unterminated sql block");
}
/* Numbers and punctuation */
{digit}+ {
- yylval.integer = atoi(yytext);
+ spec_yylval.integer = atoi(yytext);
return INTEGER;
}
@@ -150,7 +157,7 @@ addlitchar(char c)
}
void
-yyerror(const char *message)
+spec_yyerror(const char *message)
{
fprintf(stderr, "%s at line %d\n", message, yyline);
exit(1);
--
2.36.1
[text/x-patch] v201-0004-Build-syncrep_scanner.c-standalone.patch (3.5K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/5-v201-0004-Build-syncrep_scanner.c-standalone.patch)
download | inline diff:
From 92d48ac8354bce8bb57a43e01448835e1cd75871 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 12 Aug 2022 18:27:39 +0700
Subject: [PATCH v201 4/9] Build syncrep_scanner.c standalone
---
src/backend/replication/.gitignore | 1 +
src/backend/replication/Makefile | 11 +++++++++--
src/backend/replication/syncrep_gram.y | 2 --
src/backend/replication/syncrep_scanner.l | 17 +++++++++++------
4 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/src/backend/replication/.gitignore b/src/backend/replication/.gitignore
index d1df6147bd..ad138c0c98 100644
--- a/src/backend/replication/.gitignore
+++ b/src/backend/replication/.gitignore
@@ -1,4 +1,5 @@
/repl_gram.c
/repl_scanner.c
+/syncrep_gram.h
/syncrep_gram.c
/syncrep_scanner.c
diff --git a/src/backend/replication/Makefile b/src/backend/replication/Makefile
index bc8170418f..23f29ba545 100644
--- a/src/backend/replication/Makefile
+++ b/src/backend/replication/Makefile
@@ -21,6 +21,7 @@ OBJS = \
slotfuncs.o \
syncrep.o \
syncrep_gram.o \
+ syncrep_scanner.o \
walreceiver.o \
walreceiverfuncs.o \
walsender.o
@@ -38,8 +39,14 @@ repl_gram.c: BISONFLAGS += -d
# Force these dependencies to be known even without dependency info built:
repl_gram.o repl_scanner.o: repl_gram.h
-# syncrep_scanner is compiled as part of syncrep_gram
-syncrep_gram.o: syncrep_scanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+syncrep_gram.h: syncrep_gram.c
+ touch $@
+
+syncrep_gram.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+syncrep_gram.o syncrep_scanner.o: syncrep_gram.h
# repl_gram.c, repl_scanner.c, syncrep_gram.c and syncrep_scanner.c
# are in the distribution tarball, so they are not cleaned here.
diff --git a/src/backend/replication/syncrep_gram.y b/src/backend/replication/syncrep_gram.y
index d932f2cda3..4fc3647da1 100644
--- a/src/backend/replication/syncrep_gram.y
+++ b/src/backend/replication/syncrep_gram.y
@@ -112,5 +112,3 @@ create_syncrep_config(const char *num_sync, List *members, uint8 syncrep_method)
return config;
}
-
-#include "syncrep_scanner.c"
diff --git a/src/backend/replication/syncrep_scanner.l b/src/backend/replication/syncrep_scanner.l
index 1952c8c6e0..8f38cb4613 100644
--- a/src/backend/replication/syncrep_scanner.l
+++ b/src/backend/replication/syncrep_scanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* syncrep_scanner.l
@@ -16,7 +16,12 @@
#include "postgres.h"
#include "lib/stringinfo.h"
+#include "replication/syncrep.h"
+
+#include "syncrep_gram.h"
+}
+%{
/* Avoid exit() on fatal scanner errors (a bit ugly -- see yy_fatal_error) */
#undef fprintf
#define fprintf(file, fmt, msg) fprintf_to_ereport(fmt, msg)
@@ -82,28 +87,28 @@ xdinside [^"]+
appendStringInfoString(&xdbuf, yytext);
}
<xd>{xdstop} {
- yylval.str = xdbuf.data;
+ syncrep_yylval.str = xdbuf.data;
xdbuf.data = NULL;
BEGIN(INITIAL);
return NAME;
}
<xd><<EOF>> {
- yyerror("unterminated quoted identifier");
+ syncrep_yyerror("unterminated quoted identifier");
return JUNK;
}
{identifier} {
- yylval.str = pstrdup(yytext);
+ syncrep_yylval.str = pstrdup(yytext);
return NAME;
}
{digit}+ {
- yylval.str = pstrdup(yytext);
+ syncrep_yylval.str = pstrdup(yytext);
return NUM;
}
"*" {
- yylval.str = "*";
+ syncrep_yylval.str = "*";
return NAME;
}
--
2.36.1
[text/x-patch] v201-0002-Build-booscanner.c-standalone.patch (5.8K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/6-v201-0002-Build-booscanner.c-standalone.patch)
download | inline diff:
From 7d4ecfcb3e91f3b45e94b9e64c7c40f1bbd22aa8 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 12 Aug 2022 15:45:24 +0700
Subject: [PATCH v201 2/9] Build booscanner.c standalone
---
src/backend/bootstrap/.gitignore | 1 +
src/backend/bootstrap/Makefile | 11 +++++-
src/backend/bootstrap/bootparse.y | 2 -
src/backend/bootstrap/bootscanner.l | 57 +++++++++++++++--------------
4 files changed, 40 insertions(+), 31 deletions(-)
diff --git a/src/backend/bootstrap/.gitignore b/src/backend/bootstrap/.gitignore
index 1ffe8ca39e..6351b920fd 100644
--- a/src/backend/bootstrap/.gitignore
+++ b/src/backend/bootstrap/.gitignore
@@ -1,2 +1,3 @@
+/bootparse.h
/bootparse.c
/bootscanner.c
diff --git a/src/backend/bootstrap/Makefile b/src/backend/bootstrap/Makefile
index 6421efb227..c39eb7089c 100644
--- a/src/backend/bootstrap/Makefile
+++ b/src/backend/bootstrap/Makefile
@@ -14,12 +14,19 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
bootparse.o \
+ bootscanner.o \
bootstrap.o
include $(top_srcdir)/src/backend/common.mk
-# bootscanner is compiled as part of bootparse
-bootparse.o: bootscanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+bootparse.h: bootparse.c
+ touch $@
+
+bootparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+bootparse.o bootscan.o: bootparse.h
# bootparse.c and bootscanner.c are in the distribution tarball, so
# they are not cleaned here.
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 7d7655d295..c45ddde67f 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -488,5 +488,3 @@ boot_ident:
| XNULL { $$ = pstrdup($1); }
;
%%
-
-#include "bootscanner.c"
diff --git a/src/backend/bootstrap/bootscanner.l b/src/backend/bootstrap/bootscanner.l
index 3094ccb93f..3f916fbb93 100644
--- a/src/backend/bootstrap/bootscanner.l
+++ b/src/backend/bootstrap/bootscanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* bootscanner.l
@@ -18,8 +18,11 @@
#include "bootstrap/bootstrap.h"
#include "utils/guc.h"
-/* Not needed now that this file is compiled as part of bootparse. */
-/* #include "bootparse.h" */
+/* XXX must be included after bootstrap.h */
+#include "bootparse.h"
+}
+
+%{
/* LCOV_EXCL_START */
@@ -52,7 +55,7 @@ id [-A-Za-z0-9_]+
sid \'([^']|\'\')*\'
/*
- * Keyword tokens return the keyword text (as a constant string) in yylval.kw,
+ * Keyword tokens return the keyword text (as a constant string) in boot_yylval.kw,
* just in case that's needed because we want to treat the keyword as an
* unreserved identifier. Note that _null_ is not treated as a keyword
* for this purpose; it's the one "reserved word" in the bootstrap syntax.
@@ -60,23 +63,23 @@ sid \'([^']|\'\')*\'
* Notice that all the keywords are case-sensitive, and for historical
* reasons some must be upper case.
*
- * String tokens return a palloc'd string in yylval.str.
+ * String tokens return a palloc'd string in boot_yylval.str.
*/
%%
-open { yylval.kw = "open"; return OPEN; }
+open { boot_yylval.kw = "open"; return OPEN; }
-close { yylval.kw = "close"; return XCLOSE; }
+close { boot_yylval.kw = "close"; return XCLOSE; }
-create { yylval.kw = "create"; return XCREATE; }
+create { boot_yylval.kw = "create"; return XCREATE; }
-OID { yylval.kw = "OID"; return OBJ_ID; }
-bootstrap { yylval.kw = "bootstrap"; return XBOOTSTRAP; }
-shared_relation { yylval.kw = "shared_relation"; return XSHARED_RELATION; }
-rowtype_oid { yylval.kw = "rowtype_oid"; return XROWTYPE_OID; }
+OID { boot_yylval.kw = "OID"; return OBJ_ID; }
+bootstrap { boot_yylval.kw = "bootstrap"; return XBOOTSTRAP; }
+shared_relation { boot_yylval.kw = "shared_relation"; return XSHARED_RELATION; }
+rowtype_oid { boot_yylval.kw = "rowtype_oid"; return XROWTYPE_OID; }
-insert { yylval.kw = "insert"; return INSERT_TUPLE; }
+insert { boot_yylval.kw = "insert"; return INSERT_TUPLE; }
_null_ { return NULLVAL; }
@@ -90,25 +93,25 @@ _null_ { return NULLVAL; }
^\#[^\n]* ; /* drop everything after "#" for comments */
-declare { yylval.kw = "declare"; return XDECLARE; }
-build { yylval.kw = "build"; return XBUILD; }
-indices { yylval.kw = "indices"; return INDICES; }
-unique { yylval.kw = "unique"; return UNIQUE; }
-index { yylval.kw = "index"; return INDEX; }
-on { yylval.kw = "on"; return ON; }
-using { yylval.kw = "using"; return USING; }
-toast { yylval.kw = "toast"; return XTOAST; }
-FORCE { yylval.kw = "FORCE"; return XFORCE; }
-NOT { yylval.kw = "NOT"; return XNOT; }
-NULL { yylval.kw = "NULL"; return XNULL; }
+declare { boot_yylval.kw = "declare"; return XDECLARE; }
+build { boot_yylval.kw = "build"; return XBUILD; }
+indices { boot_yylval.kw = "indices"; return INDICES; }
+unique { boot_yylval.kw = "unique"; return UNIQUE; }
+index { boot_yylval.kw = "index"; return INDEX; }
+on { boot_yylval.kw = "on"; return ON; }
+using { boot_yylval.kw = "using"; return USING; }
+toast { boot_yylval.kw = "toast"; return XTOAST; }
+FORCE { boot_yylval.kw = "FORCE"; return XFORCE; }
+NOT { boot_yylval.kw = "NOT"; return XNOT; }
+NULL { boot_yylval.kw = "NULL"; return XNULL; }
{id} {
- yylval.str = pstrdup(yytext);
+ boot_yylval.str = pstrdup(yytext);
return ID;
}
{sid} {
/* strip quotes and escapes */
- yylval.str = DeescapeQuotedString(yytext);
+ boot_yylval.str = DeescapeQuotedString(yytext);
return ID;
}
@@ -121,7 +124,7 @@ NULL { yylval.kw = "NULL"; return XNULL; }
/* LCOV_EXCL_STOP */
void
-yyerror(const char *message)
+boot_yyerror(const char *message)
{
elog(ERROR, "%s at line %d", message, yyline);
}
--
2.36.1
[text/x-patch] v201-0006-Build-cubescan.c-standalone.patch (5.1K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/7-v201-0006-Build-cubescan.c-standalone.patch)
download | inline diff:
From 75168640fbd4e2a23f2765b2918cd6a0676d0f74 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 11:18:06 +0700
Subject: [PATCH v201 6/9] Build cubescan.c standalone
---
contrib/cube/.gitignore | 1 +
contrib/cube/Makefile | 16 ++++++++++------
contrib/cube/cubedata.h | 4 ++++
contrib/cube/cubeparse.y | 9 ++-------
contrib/cube/cubescan.l | 33 +++++++++++++++++++--------------
5 files changed, 36 insertions(+), 27 deletions(-)
diff --git a/contrib/cube/.gitignore b/contrib/cube/.gitignore
index cb4c989fff..f788440c79 100644
--- a/contrib/cube/.gitignore
+++ b/contrib/cube/.gitignore
@@ -1,3 +1,4 @@
+/cubeparse.h
/cubeparse.c
/cubescan.c
# Generated subdirectories
diff --git a/contrib/cube/Makefile b/contrib/cube/Makefile
index cf195506c7..4fd19aac35 100644
--- a/contrib/cube/Makefile
+++ b/contrib/cube/Makefile
@@ -4,7 +4,8 @@ MODULE_big = cube
OBJS = \
$(WIN32RES) \
cube.o \
- cubeparse.o
+ cubeparse.o \
+ cubescan.o
EXTENSION = cube
DATA = cube--1.2.sql cube--1.2--1.3.sql cube--1.3--1.4.sql cube--1.4--1.5.sql \
@@ -15,8 +16,6 @@ HEADERS = cubedata.h
REGRESS = cube cube_sci
-EXTRA_CLEAN = y.tab.c y.tab.h
-
SHLIB_LINK += $(filter -lm, $(LIBS))
ifdef USE_PGXS
@@ -30,11 +29,16 @@ include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
+# See notes in src/backend/parser/Makefile about the following two rules
+cubeparse.h: cubeparse.c
+ touch $@
+
+cubeparse.c: BISONFLAGS += -d
-# cubescan is compiled as part of cubeparse
-cubeparse.o: cubescan.c
+# Force these dependencies to be known even without dependency info built:
+cubeparse.o cubescan.o: cubeparse.h
distprep: cubeparse.c cubescan.c
maintainer-clean:
- rm -f cubeparse.c cubescan.c
+ rm -f cubeparse.h cubeparse.c cubescan.c
diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
index dbe7d4f742..0b373048b5 100644
--- a/contrib/cube/cubedata.h
+++ b/contrib/cube/cubedata.h
@@ -67,3 +67,7 @@ extern void cube_scanner_finish(void);
/* in cubeparse.y */
extern int cube_yyparse(NDBOX **result);
+
+/* All grammar constructs return strings */
+#define YYSTYPE char *
+extern int scanbuflen;
diff --git a/contrib/cube/cubeparse.y b/contrib/cube/cubeparse.y
index 7577c4515c..d3fd1fb475 100644
--- a/contrib/cube/cubeparse.y
+++ b/contrib/cube/cubeparse.y
@@ -9,9 +9,6 @@
#include "cubedata.h"
#include "utils/float.h"
-/* All grammar constructs return strings */
-#define YYSTYPE char *
-
/*
* Bison doesn't allocate anything that needs to live across parser calls,
* so we can easily have it use palloc instead of malloc. This prevents
@@ -23,8 +20,8 @@
#define YYMALLOC palloc
#define YYFREE pfree
-static char *scanbuf;
-static int scanbuflen;
+// TODO: get rid of global var
+int scanbuflen;
static int item_count(const char *s, char delim);
static NDBOX *write_box(int dim, char *str1, char *str2);
@@ -265,5 +262,3 @@ write_point_as_box(int dim, char *str)
return bp;
}
-
-#include "cubescan.c"
diff --git a/contrib/cube/cubescan.l b/contrib/cube/cubescan.l
index bd400e3684..b7d35c6f78 100644
--- a/contrib/cube/cubescan.l
+++ b/contrib/cube/cubescan.l
@@ -1,9 +1,16 @@
-%{
+%top{
/*
* A scanner for EMP-style numeric ranges
* contrib/cube/cubescan.l
*/
+#include "postgres.h"
+#include "cubedata.h"
+
+#include "cubeparse.h"
+}
+
+%{
/* LCOV_EXCL_START */
/* No reason to constrain amount of data slurped */
@@ -21,9 +28,7 @@ fprintf_to_ereport(const char *fmt, const char *msg)
/* Handles to the buffer that the lexer uses internally */
static YY_BUFFER_STATE scanbufhandle;
-/* this is now declared in cubeparse.y: */
-/* static char *scanbuf; */
-/* static int scanbuflen; */
+static char *scanbuf;
%}
%option 8bit
@@ -45,14 +50,14 @@ NaN [nN][aA][nN]
%%
-{float} yylval = yytext; return CUBEFLOAT;
-{infinity} yylval = yytext; return CUBEFLOAT;
-{NaN} yylval = yytext; return CUBEFLOAT;
-\[ yylval = "("; return O_BRACKET;
-\] yylval = ")"; return C_BRACKET;
-\( yylval = "("; return O_PAREN;
-\) yylval = ")"; return C_PAREN;
-\, yylval = ","; return COMMA;
+{float} cube_yylval = yytext; return CUBEFLOAT;
+{infinity} cube_yylval = yytext; return CUBEFLOAT;
+{NaN} cube_yylval = yytext; return CUBEFLOAT;
+\[ cube_yylval = "("; return O_BRACKET;
+\] cube_yylval = ")"; return C_BRACKET;
+\( cube_yylval = "("; return O_PAREN;
+\) cube_yylval = ")"; return C_PAREN;
+\, cube_yylval = ","; return COMMA;
[ \t\n\r\f]+ /* discard spaces */
. return yytext[0]; /* alert parser of the garbage */
@@ -62,7 +67,7 @@ NaN [nN][aA][nN]
/* result is not used, but Bison expects this signature */
void
-yyerror(NDBOX **result, const char *message)
+cube_yyerror(NDBOX **result, const char *message)
{
if (*yytext == YY_END_OF_BUFFER_CHAR)
{
@@ -89,7 +94,7 @@ yyerror(NDBOX **result, const char *message)
void
cube_scanner_init(const char *str)
{
- Size slen = strlen(str);
+ Size slen = strlen(str);
/*
* Might be left over after ereport()
--
2.36.1
[text/x-patch] v201-0007-Build-segscan.c-standalone.patch (3.7K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/8-v201-0007-Build-segscan.c-standalone.patch)
download | inline diff:
From 5170d8fac8bbb9e101e10f18ed280e446b305e26 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 12:00:33 +0700
Subject: [PATCH v201 7/9] Build segscan.c standalone
---
contrib/seg/.gitignore | 1 +
contrib/seg/Makefile | 15 +++++++++++----
contrib/seg/segparse.y | 3 ---
contrib/seg/segscan.l | 25 +++++++++++++++----------
4 files changed, 27 insertions(+), 17 deletions(-)
diff --git a/contrib/seg/.gitignore b/contrib/seg/.gitignore
index 69e73d2096..fa247a4e67 100644
--- a/contrib/seg/.gitignore
+++ b/contrib/seg/.gitignore
@@ -1,3 +1,4 @@
+/segparse.h
/segparse.c
/segscan.c
# Generated subdirectories
diff --git a/contrib/seg/Makefile b/contrib/seg/Makefile
index bb63e83506..c6c134b8f1 100644
--- a/contrib/seg/Makefile
+++ b/contrib/seg/Makefile
@@ -4,7 +4,8 @@ MODULE_big = seg
OBJS = \
$(WIN32RES) \
seg.o \
- segparse.o
+ segparse.o \
+ segscan.o
EXTENSION = seg
DATA = seg--1.1.sql seg--1.1--1.2.sql seg--1.2--1.3.sql seg--1.3--1.4.sql \
@@ -29,10 +30,16 @@ include $(top_srcdir)/contrib/contrib-global.mk
endif
-# segscan is compiled as part of segparse
-segparse.o: segscan.c
+# See notes in src/backend/parser/Makefile about the following two rules
+segparse.h: segparse.c
+ touch $@
+
+segparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+segparse.o segscan.o: segparse.h
distprep: segparse.c segscan.c
maintainer-clean:
- rm -f segparse.c segscan.c
+ rm -f segparse.h segparse.c segscan.c
diff --git a/contrib/seg/segparse.y b/contrib/seg/segparse.y
index 33e3a9f35f..637eacd1a6 100644
--- a/contrib/seg/segparse.y
+++ b/contrib/seg/segparse.y
@@ -160,6 +160,3 @@ seg_atof(const char *value)
datum = DirectFunctionCall1(float4in, CStringGetDatum(value));
return DatumGetFloat4(datum);
}
-
-
-#include "segscan.c"
diff --git a/contrib/seg/segscan.l b/contrib/seg/segscan.l
index 5f6595e9eb..db0db1aa70 100644
--- a/contrib/seg/segscan.l
+++ b/contrib/seg/segscan.l
@@ -1,8 +1,15 @@
-%{
+%top{
/*
* A scanner for EMP-style numeric ranges
*/
+#include "postgres.h"
+
+#include "segdata.h"
+#include "segparse.h"
+}
+
+%{
/* LCOV_EXCL_START */
/* No reason to constrain amount of data slurped */
@@ -21,7 +28,6 @@ fprintf_to_ereport(const char *fmt, const char *msg)
/* Handles to the buffer that the lexer uses internally */
static YY_BUFFER_STATE scanbufhandle;
static char *scanbuf;
-static int scanbuflen;
%}
%option 8bit
@@ -42,12 +48,12 @@ float ({integer}|{real})([eE]{integer})?
%%
-{range} yylval.text = yytext; return RANGE;
-{plumin} yylval.text = yytext; return PLUMIN;
-{float} yylval.text = yytext; return SEGFLOAT;
-\< yylval.text = "<"; return EXTENSION;
-\> yylval.text = ">"; return EXTENSION;
-\~ yylval.text = "~"; return EXTENSION;
+{range} seg_yylval.text = yytext; return RANGE;
+{plumin} seg_yylval.text = yytext; return PLUMIN;
+{float} seg_yylval.text = yytext; return SEGFLOAT;
+\< seg_yylval.text = "<"; return EXTENSION;
+\> seg_yylval.text = ">"; return EXTENSION;
+\~ seg_yylval.text = "~"; return EXTENSION;
[ \t\n\r\f]+ /* discard spaces */
. return yytext[0]; /* alert parser of the garbage */
@@ -56,7 +62,7 @@ float ({integer}|{real})([eE]{integer})?
/* LCOV_EXCL_STOP */
void
-yyerror(SEG *result, const char *message)
+seg_yyerror(SEG *result, const char *message)
{
if (*yytext == YY_END_OF_BUFFER_CHAR)
{
@@ -94,7 +100,6 @@ seg_scanner_init(const char *str)
/*
* Make a scan buffer with special termination needed by flex.
*/
- scanbuflen = slen;
scanbuf = palloc(slen + 2);
memcpy(scanbuf, str, slen);
scanbuf[slen] = scanbuf[slen + 1] = YY_END_OF_BUFFER_CHAR;
--
2.36.1
[text/x-patch] v201-0008-Build-jsonpath_scan.c-standalone.patch (6.6K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/9-v201-0008-Build-jsonpath_scan.c-standalone.patch)
download | inline diff:
From 4d16b395978e8bc830e91d363cf9cadda0c00365 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 12:35:55 +0700
Subject: [PATCH v201 8/9] Build jsonpath_scan.c standalone
XXX: warnings about missing jsonpath_yylex
---
src/backend/utils/adt/.gitignore | 1 +
src/backend/utils/adt/Makefile | 11 +++++++++--
src/backend/utils/adt/jsonpath_gram.y | 23 -----------------------
src/backend/utils/adt/jsonpath_scan.l | 25 +++++++++++++++----------
src/include/utils/jsonpath.h | 13 +++++++++++++
5 files changed, 38 insertions(+), 35 deletions(-)
diff --git a/src/backend/utils/adt/.gitignore b/src/backend/utils/adt/.gitignore
index 48cf941a52..7fab054407 100644
--- a/src/backend/utils/adt/.gitignore
+++ b/src/backend/utils/adt/.gitignore
@@ -1,2 +1,3 @@
+/jsonpath_gram.h
/jsonpath_gram.c
/jsonpath_scan.c
diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile
index 7c722ea2ce..d03f897478 100644
--- a/src/backend/utils/adt/Makefile
+++ b/src/backend/utils/adt/Makefile
@@ -57,6 +57,7 @@ OBJS = \
jsonpath.o \
jsonpath_exec.o \
jsonpath_gram.o \
+ jsonpath_scan.o \
like.o \
like_support.o \
lockfuncs.o \
@@ -119,11 +120,17 @@ OBJS = \
xid8funcs.o \
xml.o
+# See notes in src/backend/parser/Makefile about the following two rules
+jsonpath_gram.h: jsonpath_gram.c
+ touch $@
+
+jsonpath_gram.c: BISONFLAGS += -d
+
jsonpath_scan.c: FLEXFLAGS = -CF -p -p
jsonpath_scan.c: FLEX_NO_BACKUP=yes
-# jsonpath_scan is compiled as part of jsonpath_gram
-jsonpath_gram.o: jsonpath_scan.c
+# Force these dependencies to be known even without dependency info built:
+jsonpath_gram.o jsonpath_gram.o jsonpath_parser.o: jsonpath_gram.h
# jsonpath_gram.c and jsonpath_scan.c are in the distribution tarball,
# so they are not cleaned here.
diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y
index f903dba3e3..a7557d325e 100644
--- a/src/backend/utils/adt/jsonpath_gram.y
+++ b/src/backend/utils/adt/jsonpath_gram.y
@@ -24,21 +24,8 @@
#include "utils/builtins.h"
#include "utils/jsonpath.h"
-/* struct JsonPathString is shared between scan and gram */
-typedef struct JsonPathString
-{
- char *val;
- int len;
- int total;
-} JsonPathString;
-
union YYSTYPE;
-/* flex 2.5.4 doesn't bother with a decl for this */
-int jsonpath_yylex(union YYSTYPE *yylval_param);
-int jsonpath_yyparse(JsonPathParseResult **result);
-void jsonpath_yyerror(JsonPathParseResult **result, const char *message);
-
static JsonPathParseItem *makeItemType(JsonPathItemType type);
static JsonPathParseItem *makeItemString(JsonPathString *s);
static JsonPathParseItem *makeItemVariable(JsonPathString *s);
@@ -593,13 +580,3 @@ jspConvertRegexFlags(uint32 xflags)
return cflags;
}
-
-/*
- * jsonpath_scan.l is compiled as part of jsonpath_gram.y. Currently, this is
- * unavoidable because jsonpath_gram does not create a .h file to export its
- * token symbols. If these files ever grow large enough to be worth compiling
- * separately, that could be fixed; but for now it seems like useless
- * complication.
- */
-
-#include "jsonpath_scan.c"
diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l
index 4351f6ec98..edd9d1c706 100644
--- a/src/backend/utils/adt/jsonpath_scan.l
+++ b/src/backend/utils/adt/jsonpath_scan.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* jsonpath_scan.l
@@ -17,9 +17,14 @@
#include "postgres.h"
+#include "utils/jsonpath.h"
#include "mb/pg_wchar.h"
#include "nodes/pg_list.h"
+#include "jsonpath_gram.h"
+}
+
+%{
static JsonPathString scanstring;
/* Handles to the buffer that the lexer uses internally */
@@ -142,9 +147,9 @@ hex_fail \\x{hex_dig}{0,1}
<xnq,xq,xvq>{hex_char} { parseHexChar(yytext); }
-<xnq,xq,xvq>{unicode}*{unicodefail} { yyerror(NULL, "invalid unicode sequence"); }
+<xnq,xq,xvq>{unicode}*{unicodefail} { jsonpath_yyerror(NULL, "invalid unicode sequence"); }
-<xnq,xq,xvq>{hex_fail} { yyerror(NULL, "invalid hex character sequence"); }
+<xnq,xq,xvq>{hex_fail} { jsonpath_yyerror(NULL, "invalid hex character sequence"); }
<xnq,xq,xvq>{unicode}+\\ {
/* throw back the \\, and treat as unicode */
@@ -154,9 +159,9 @@ hex_fail \\x{hex_dig}{0,1}
<xnq,xq,xvq>\\. { addchar(false, yytext[1]); }
-<xnq,xq,xvq>\\ { yyerror(NULL, "unexpected end after backslash"); }
+<xnq,xq,xvq>\\ { jsonpath_yyerror(NULL, "unexpected end after backslash"); }
-<xq,xvq><<EOF>> { yyerror(NULL, "unexpected end of quoted string"); }
+<xq,xvq><<EOF>> { jsonpath_yyerror(NULL, "unexpected end of quoted string"); }
<xq>\" {
yylval->str = scanstring;
@@ -178,7 +183,7 @@ hex_fail \\x{hex_dig}{0,1}
<xc>\* { }
-<xc><<EOF>> { yyerror(NULL, "unexpected end of comment"); }
+<xc><<EOF>> { jsonpath_yyerror(NULL, "unexpected end of comment"); }
\&\& { return AND_P; }
@@ -244,10 +249,10 @@ hex_fail \\x{hex_dig}{0,1}
return INT_P;
}
-{realfail} { yyerror(NULL, "invalid numeric literal"); }
-{integer_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
-{decimal_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
-{real_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
+{realfail} { jsonpath_yyerror(NULL, "invalid numeric literal"); }
+{integer_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
+{decimal_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
+{real_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
\" {
addchar(true, '\0');
diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h
index 8e79b8dc9f..b9b56209b2 100644
--- a/src/include/utils/jsonpath.h
+++ b/src/include/utils/jsonpath.h
@@ -21,6 +21,14 @@
#include "utils/jsonb.h"
#include "utils/jsonfuncs.h"
+/* struct JsonPathString is shared between scan and gram */
+typedef struct JsonPathString
+{
+ char *val;
+ int len;
+ int total;
+} JsonPathString;
+
typedef struct
{
int32 vl_len_; /* varlena header (do not touch directly!) */
@@ -250,6 +258,11 @@ typedef struct JsonPathParseResult
extern JsonPathParseResult *parsejsonpath(const char *str, int len);
+/* flex 2.5.4 doesn't bother with a decl for this */
+//int jsonpath_yylex(union YYSTYPE *yylval_param);
+//int jsonpath_yyparse(JsonPathParseResult **result);
+void jsonpath_yyerror(JsonPathParseResult **result, const char *message);
+
extern int jspConvertRegexFlags(uint32 xflags);
/*
--
2.36.1
[text/x-patch] v201-0009-Build-exprscan.c-standalone.patch (3.2K, ../../CAFBsxsGq=jLMofR3BN1S1xJcaJsYcQnkWFRtRDObYVtDST0+Vg@mail.gmail.com/10-v201-0009-Build-exprscan.c-standalone.patch)
download | inline diff:
From cae4dccc5aaee93e0ed258448f60d4178163ec1c Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 13:35:14 +0700
Subject: [PATCH v201 9/9] Build exprscan.c standalone
---
src/bin/pgbench/.gitignore | 1 +
src/bin/pgbench/Makefile | 13 ++++++++++---
src/bin/pgbench/exprparse.y | 15 ---------------
src/bin/pgbench/exprscan.l | 9 ++++++++-
4 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/src/bin/pgbench/.gitignore b/src/bin/pgbench/.gitignore
index 983a3cd7a6..07492a993c 100644
--- a/src/bin/pgbench/.gitignore
+++ b/src/bin/pgbench/.gitignore
@@ -1,3 +1,4 @@
+/exprparse.h
/exprparse.c
/exprscan.c
/pgbench
diff --git a/src/bin/pgbench/Makefile b/src/bin/pgbench/Makefile
index f402fe7b91..6647c9fe97 100644
--- a/src/bin/pgbench/Makefile
+++ b/src/bin/pgbench/Makefile
@@ -10,6 +10,7 @@ include $(top_builddir)/src/Makefile.global
OBJS = \
$(WIN32RES) \
exprparse.o \
+ exprscan.o \
pgbench.o
override CPPFLAGS := -I. -I$(srcdir) -I$(libpq_srcdir) $(CPPFLAGS)
@@ -26,8 +27,14 @@ all: pgbench
pgbench: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-# exprscan is compiled as part of exprparse
-exprparse.o: exprscan.c
+# See notes in src/backend/parser/Makefile about the following two rules
+exprparse.h: exprparse.c
+ touch $@
+
+exprparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+exprparse.o exprscan.o: exprparse.h
distprep: exprparse.c exprscan.c
@@ -45,7 +52,7 @@ clean distclean:
rm -rf tmp_check
maintainer-clean: distclean
- rm -f exprparse.c exprscan.c
+ rm -f exprparse.h exprparse.c exprscan.c
check:
$(prove_check)
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b5592d4b97..ade2ecdaab 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -526,18 +526,3 @@ make_case(yyscan_t yyscanner, PgBenchExprList *when_then_list, PgBenchExpr *else
find_func(yyscanner, "!case_end"),
make_elist(else_part, when_then_list));
}
-
-/*
- * exprscan.l is compiled as part of exprparse.y. Currently, this is
- * unavoidable because exprparse does not create a .h file to export
- * its token symbols. If these files ever grow large enough to be
- * worth compiling separately, that could be fixed; but for now it
- * seems like useless complication.
- */
-
-/* First, get rid of "#define yyscan_t" from pgbench.h */
-#undef yyscan_t
-/* ... and the yylval macro, which flex will have its own definition for */
-#undef yylval
-
-#include "exprscan.c"
diff --git a/src/bin/pgbench/exprscan.l b/src/bin/pgbench/exprscan.l
index 4f63818606..6e9d949dcf 100644
--- a/src/bin/pgbench/exprscan.l
+++ b/src/bin/pgbench/exprscan.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* exprscan.l
@@ -23,8 +23,15 @@
*-------------------------------------------------------------------------
*/
+#include "postgres_fe.h"
+
#include "fe_utils/psqlscan_int.h"
+#include "pgbench.h"
+#include "exprparse.h"
+}
+
+%{
/* context information for reporting errors in expressions */
static const char *expr_source = NULL;
static int expr_lineno = 0;
--
2.36.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-15 18:11 Andres Freund <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: Andres Freund @ 2022-08-15 18:11 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
Hi,
Thanks for your work on this!
On 2022-08-13 15:39:06 +0700, John Naylor wrote:
> Here are the rest. Most of it was pretty straightforward, with the
> main exception of jsonpath_scan.c, which is not quite finished. That
> one passes tests but still has one compiler warning. I'm unsure how
> much of what is there already is really necessary or was cargo-culted
> from elsewhere without explanation. For starters, I'm not sure why the
> grammar has a forward declaration of "union YYSTYPE". It's noteworthy
> that it used to compile standalone, but with a bit more stuff, and
> that was reverted in 550b9d26f80fa30. I can hack on it some more later
> but I ran out of steam today.
I'm not sure either...
> Other questions thus far:
>
> - "BISONFLAGS += -d" is now in every make file with a .y file -- can
> we just force that everywhere?
Hm. Not sure it's worth it, extensions might use our BISON stuff...
> - Include order seems to matter for the grammar's .h file. I didn't
> test if that was the case every time, and after a few miscompiles just
> always made it the last inclusion, but I'm wondering if we should keep
> those inclusions outside %top{} and put it at the start of the next
> %{} ?
I think we have a few of those dependencies already, see e.g.
/*
* NB: include gram.h only AFTER including scanner.h, because scanner.h
* is what #defines YYLTYPE.
*/
> From d723ba14acf56fd432e9e263db937fcc13fc0355 Mon Sep 17 00:00:00 2001
> From: John Naylor <[email protected]>
> Date: Thu, 11 Aug 2022 19:38:37 +0700
> Subject: [PATCH v201 1/9] Build guc-file.c standalone
Might be worth doing some of the moving around here separately from the
parser/scanner specific bits.
> +/* functions shared between guc.c and guc-file.l */
> +extern int guc_name_compare(const char *namea, const char *nameb);
> +extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
> + bool applySettings, int elevel);
> +extern void record_config_file_error(const char *errmsg,
> + const char *config_file,
> + int lineno,
> + ConfigVariable **head_p,
> + ConfigVariable **tail_p);
>
> /*
> * The following functions are not in guc.c, but are declared here to avoid
> --
> 2.36.1
>
I think I prefer your suggestion of a guc_internal.h upthread.
> From 7d4ecfcb3e91f3b45e94b9e64c7c40f1bbd22aa8 Mon Sep 17 00:00:00 2001
> From: John Naylor <[email protected]>
> Date: Fri, 12 Aug 2022 15:45:24 +0700
> Subject: [PATCH v201 2/9] Build booscanner.c standalone
> -# bootscanner is compiled as part of bootparse
> -bootparse.o: bootscanner.c
> +# See notes in src/backend/parser/Makefile about the following two rules
> +bootparse.h: bootparse.c
> + touch $@
> +
> +bootparse.c: BISONFLAGS += -d
> +
> +# Force these dependencies to be known even without dependency info built:
> +bootparse.o bootscan.o: bootparse.h
Wonder if we could / should wrap this is something common. It's somewhat
annoying to repeat this stuff everywhere.
> diff --git a/src/test/isolation/specscanner.l b/src/test/isolation/specscanner.l
> index aa6e89268e..2dc292c21d 100644
> --- a/src/test/isolation/specscanner.l
> +++ b/src/test/isolation/specscanner.l
> @@ -1,4 +1,4 @@
> -%{
> +%top{
> /*-------------------------------------------------------------------------
> *
> * specscanner.l
> @@ -9,7 +9,14 @@
> *
> *-------------------------------------------------------------------------
> */
> +#include "postgres_fe.h"
Miniscule nitpick: I think we typically leave an empty line between header and
first include.
> diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
> index dbe7d4f742..0b373048b5 100644
> --- a/contrib/cube/cubedata.h
> +++ b/contrib/cube/cubedata.h
> @@ -67,3 +67,7 @@ extern void cube_scanner_finish(void);
>
> /* in cubeparse.y */
> extern int cube_yyparse(NDBOX **result);
> +
> +/* All grammar constructs return strings */
> +#define YYSTYPE char *
Why does this need to be defined in a semi-public header? If we do this in
multiple files we'll end up with the danger of macro redefinition warnings.
> +extern int scanbuflen;
The code around scanbuflen seems pretty darn grotty. Allocating enough memory
for the entire list by allocating the entire string size... I don't know
anything about contrib/cube, but isn't that in effect O(inputlen^2) memory?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-16 10:41 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-08-16 10:41 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
For v3, I addressed some comments and added .h files to the
headerscheck exceptions.
On Tue, Aug 16, 2022 at 1:11 AM Andres Freund <[email protected]> wrote:
> On 2022-08-13 15:39:06 +0700, John Naylor wrote:
> > Here are the rest. Most of it was pretty straightforward, with the
> > main exception of jsonpath_scan.c, which is not quite finished. That
> > one passes tests but still has one compiler warning. I'm unsure how
> > much of what is there already is really necessary or was cargo-culted
> > from elsewhere without explanation. For starters, I'm not sure why the
> > grammar has a forward declaration of "union YYSTYPE". It's noteworthy
> > that it used to compile standalone, but with a bit more stuff, and
> > that was reverted in 550b9d26f80fa30. I can hack on it some more later
> > but I ran out of steam today.
I've got it in half-way decent shape now, with an *internal.h header
and some cleanups.
> > - Include order seems to matter for the grammar's .h file. I didn't
> > test if that was the case every time, and after a few miscompiles just
> > always made it the last inclusion, but I'm wondering if we should keep
> > those inclusions outside %top{} and put it at the start of the next
> > %{} ?
>
> I think we have a few of those dependencies already, see e.g.
> /*
> * NB: include gram.h only AFTER including scanner.h, because scanner.h
> * is what #defines YYLTYPE.
> */
Went with something like this in all cases:
/*
* NB: include bootparse.h only AFTER including bootstrap.h, because bootstrap.h
* includes node definitions needed for YYSTYPE.
*/
Future cleanup: I see this in headerscheck:
# We can't make these Bison output files compilable standalone
# without using "%code require", which old Bison versions lack.
# parser/gram.h will be included by parser/gramparse.h anyway.
That directive has been supported in Bison since 2.4.2.
> > From d723ba14acf56fd432e9e263db937fcc13fc0355 Mon Sep 17 00:00:00 2001
> > From: John Naylor <[email protected]>
> > Date: Thu, 11 Aug 2022 19:38:37 +0700
> > Subject: [PATCH v201 1/9] Build guc-file.c standalone
>
> Might be worth doing some of the moving around here separately from the
> parser/scanner specific bits.
Done in 0001/0003.
> > +/* functions shared between guc.c and guc-file.l */
> > [...]
> I think I prefer your suggestion of a guc_internal.h upthread.
Started in 0002, but left open the headerscheck failure.
Also, if such a thing is meant to be #include'd only by two generated
files, maybe it should just live in the directory where they live, and
not in the src/include dir?
> > From 7d4ecfcb3e91f3b45e94b9e64c7c40f1bbd22aa8 Mon Sep 17 00:00:00 2001
> > From: John Naylor <[email protected]>
> > Date: Fri, 12 Aug 2022 15:45:24 +0700
> > Subject: [PATCH v201 2/9] Build booscanner.c standalone
>
> > -# bootscanner is compiled as part of bootparse
> > -bootparse.o: bootscanner.c
> > +# See notes in src/backend/parser/Makefile about the following two rules
> > +bootparse.h: bootparse.c
> > + touch $@
> > +
> > +bootparse.c: BISONFLAGS += -d
> > +
> > +# Force these dependencies to be known even without dependency info built:
> > +bootparse.o bootscan.o: bootparse.h
>
> Wonder if we could / should wrap this is something common. It's somewhat
> annoying to repeat this stuff everywhere.
I haven't looked at the Meson effort recently, but if the build rule
is less annoying there, I'm inclined to leave this as a wart until
autotools are retired.
> > diff --git a/src/test/isolation/specscanner.l b/src/test/isolation/specscanner.l
> > index aa6e89268e..2dc292c21d 100644
> > --- a/src/test/isolation/specscanner.l
> > +++ b/src/test/isolation/specscanner.l
> > @@ -1,4 +1,4 @@
> > -%{
> > +%top{
> > /*-------------------------------------------------------------------------
> > *
> > * specscanner.l
> > @@ -9,7 +9,14 @@
> > *
> > *-------------------------------------------------------------------------
> > */
> > +#include "postgres_fe.h"
>
> Miniscule nitpick: I think we typically leave an empty line between header and
> first include.
In a small unscientific sample it seems like the opposite is true
actually, but I'll at least try to be consistent within the patch set.
> > diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
> > index dbe7d4f742..0b373048b5 100644
> > --- a/contrib/cube/cubedata.h
> > +++ b/contrib/cube/cubedata.h
> > @@ -67,3 +67,7 @@ extern void cube_scanner_finish(void);
> >
> > /* in cubeparse.y */
> > extern int cube_yyparse(NDBOX **result);
> > +
> > +/* All grammar constructs return strings */
> > +#define YYSTYPE char *
>
> Why does this need to be defined in a semi-public header? If we do this in
> multiple files we'll end up with the danger of macro redefinition warnings.
I tried to put all the Flex/Bison stuff in another *_internal header,
but that breaks the build. Putting just this one symbol in a header is
silly, but done that way for now. Maybe two copies of the symbol?
Another future cleanup: "%define api.prefix {cube_yy}" etc would cause
it to be spelled CUBE_YYSTYPE (other macros too), sidestepping this
problem (requires Bison 2.6). IIUC, doing it our way has been
deprecated for 9 years.
> > +extern int scanbuflen;
>
> The code around scanbuflen seems pretty darn grotty. Allocating enough memory
> for the entire list by allocating the entire string size... I don't know
> anything about contrib/cube, but isn't that in effect O(inputlen^2) memory?
Neither do I.
--
John Naylor
EDB: http://www.enterprisedb.com
Attachments:
[text/x-patch] v3-0001-Preparatory-refactoring-for-compiling-guc-file.c-.patch (26.0K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/2-v3-0001-Preparatory-refactoring-for-compiling-guc-file.c-.patch)
download | inline diff:
From 6e780de69ba59e3d921c3115de920de1f98994cd Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Tue, 16 Aug 2022 10:42:19 +0700
Subject: [PATCH v3 01/11] Preparatory refactoring for compiling guc-file.c
standalone
Mostly this involves moving ProcessConfigFileInternal() to guc.c
and fixing the shared API to match.
---
src/backend/utils/misc/guc-file.l | 360 +-----------------------------
src/backend/utils/misc/guc.c | 360 +++++++++++++++++++++++++++++-
src/include/utils/guc.h | 9 +
3 files changed, 364 insertions(+), 365 deletions(-)
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index ce5633844c..b4fa09749b 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -48,12 +48,6 @@ static sigjmp_buf *GUC_flex_fatal_jmp;
static void FreeConfigVariable(ConfigVariable *item);
-static void record_config_file_error(const char *errmsg,
- const char *config_file,
- int lineno,
- ConfigVariable **head_p,
- ConfigVariable **tail_p);
-
static int GUC_flex_fatal(const char *msg);
/* LCOV_EXCL_START */
@@ -159,358 +153,6 @@ ProcessConfigFile(GucContext context)
MemoryContextDelete(config_cxt);
}
-/*
- * This function handles both actual config file (re)loads and execution of
- * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
- * case we don't apply any of the settings, but we make all the usual validity
- * checks, and we return the ConfigVariable list so that it can be printed out
- * by show_all_file_settings().
- */
-static ConfigVariable *
-ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
-{
- bool error = false;
- bool applying = false;
- const char *ConfFileWithError;
- ConfigVariable *item,
- *head,
- *tail;
- int i;
-
- /* Parse the main config file into a list of option names and values */
- ConfFileWithError = ConfigFileName;
- head = tail = NULL;
-
- if (!ParseConfigFile(ConfigFileName, true,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- goto bail_out;
- }
-
- /*
- * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
- * replace any parameters set by ALTER SYSTEM command. Because this file
- * is in the data directory, we can't read it until the DataDir has been
- * set.
- */
- if (DataDir)
- {
- if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- ConfFileWithError = PG_AUTOCONF_FILENAME;
- goto bail_out;
- }
- }
- else
- {
- /*
- * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
- * read. In this case, we don't want to accept any settings but
- * data_directory from postgresql.conf, because they might be
- * overwritten with settings in the PG_AUTOCONF_FILENAME file which
- * will be read later. OTOH, since data_directory isn't allowed in the
- * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
- */
- ConfigVariable *newlist = NULL;
-
- /*
- * Prune all items except the last "data_directory" from the list.
- */
- for (item = head; item; item = item->next)
- {
- if (!item->ignore &&
- strcmp(item->name, "data_directory") == 0)
- newlist = item;
- }
-
- if (newlist)
- newlist->next = NULL;
- head = tail = newlist;
-
- /*
- * Quick exit if data_directory is not present in file.
- *
- * We need not do any further processing, in particular we don't set
- * PgReloadTime; that will be set soon by subsequent full loading of
- * the config file.
- */
- if (head == NULL)
- goto bail_out;
- }
-
- /*
- * Mark all extant GUC variables as not present in the config file. We
- * need this so that we can tell below which ones have been removed from
- * the file since we last processed it.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
-
- gconf->status &= ~GUC_IS_IN_FILE;
- }
-
- /*
- * Check if all the supplied option names are valid, as an additional
- * quasi-syntactic check on the validity of the config file. It is
- * important that the postmaster and all backends agree on the results of
- * this phase, else we will have strange inconsistencies about which
- * processes accept a config file update and which don't. Hence, unknown
- * custom variable names have to be accepted without complaint. For the
- * same reason, we don't attempt to validate the options' values here.
- *
- * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
- * variable mentioned in the file; and we detect duplicate entries in the
- * file and mark the earlier occurrences as ignorable.
- */
- for (item = head; item; item = item->next)
- {
- struct config_generic *record;
-
- /* Ignore anything already marked as ignorable */
- if (item->ignore)
- continue;
-
- /*
- * Try to find the variable; but do not create a custom placeholder if
- * it's not there already.
- */
- record = find_option(item->name, false, true, elevel);
-
- if (record)
- {
- /* If it's already marked, then this is a duplicate entry */
- if (record->status & GUC_IS_IN_FILE)
- {
- /*
- * Mark the earlier occurrence(s) as dead/ignorable. We could
- * avoid the O(N^2) behavior here with some additional state,
- * but it seems unlikely to be worth the trouble.
- */
- ConfigVariable *pitem;
-
- for (pitem = head; pitem != item; pitem = pitem->next)
- {
- if (!pitem->ignore &&
- strcmp(pitem->name, item->name) == 0)
- pitem->ignore = true;
- }
- }
- /* Now mark it as present in file */
- record->status |= GUC_IS_IN_FILE;
- }
- else if (!valid_custom_variable_name(item->name))
- {
- /* Invalid non-custom variable, so complain */
- ereport(elevel,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
- item->name,
- item->filename, item->sourceline)));
- item->errmsg = pstrdup("unrecognized configuration parameter");
- error = true;
- ConfFileWithError = item->filename;
- }
- }
-
- /*
- * If we've detected any errors so far, we don't want to risk applying any
- * changes.
- */
- if (error)
- goto bail_out;
-
- /* Otherwise, set flag that we're beginning to apply changes */
- applying = true;
-
- /*
- * Check for variables having been removed from the config file, and
- * revert their reset values (and perhaps also effective values) to the
- * boot-time defaults. If such a variable can't be changed after startup,
- * report that and continue.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
- GucStack *stack;
-
- if (gconf->reset_source != PGC_S_FILE ||
- (gconf->status & GUC_IS_IN_FILE))
- continue;
- if (gconf->context < PGC_SIGHUP)
- {
- /* The removal can't be effective without a restart */
- gconf->status |= GUC_PENDING_RESTART;
- ereport(elevel,
- (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
- errmsg("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name)));
- record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name),
- NULL, 0,
- &head, &tail);
- error = true;
- continue;
- }
-
- /* No more to do if we're just doing show_all_file_settings() */
- if (!applySettings)
- continue;
-
- /*
- * Reset any "file" sources to "default", else set_config_option will
- * not override those settings.
- */
- if (gconf->reset_source == PGC_S_FILE)
- gconf->reset_source = PGC_S_DEFAULT;
- if (gconf->source == PGC_S_FILE)
- gconf->source = PGC_S_DEFAULT;
- for (stack = gconf->stack; stack; stack = stack->prev)
- {
- if (stack->source == PGC_S_FILE)
- stack->source = PGC_S_DEFAULT;
- }
-
- /* Now we can re-apply the wired-in default (i.e., the boot_val) */
- if (set_config_option(gconf->name, NULL,
- context, PGC_S_DEFAULT,
- GUC_ACTION_SET, true, 0, false) > 0)
- {
- /* Log the change if appropriate */
- if (context == PGC_SIGHUP)
- ereport(elevel,
- (errmsg("parameter \"%s\" removed from configuration file, reset to default",
- gconf->name)));
- }
- }
-
- /*
- * Restore any variables determined by environment variables or
- * dynamically-computed defaults. This is a no-op except in the case
- * where one of these had been in the config file and is now removed.
- *
- * In particular, we *must not* do this during the postmaster's initial
- * loading of the file, since the timezone functions in particular should
- * be run only after initialization is complete.
- *
- * XXX this is an unmaintainable crock, because we have to know how to set
- * (or at least what to call to set) every non-PGC_INTERNAL variable that
- * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
- */
- if (context == PGC_SIGHUP && applySettings)
- {
- InitializeGUCOptionsFromEnvironment();
- pg_timezone_abbrev_initialize();
- /* this selects SQL_ASCII in processes not connected to a database */
- SetConfigOption("client_encoding", GetDatabaseEncodingName(),
- PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
- }
-
- /*
- * Now apply the values from the config file.
- */
- for (item = head; item; item = item->next)
- {
- char *pre_value = NULL;
- int scres;
-
- /* Ignore anything marked as ignorable */
- if (item->ignore)
- continue;
-
- /* In SIGHUP cases in the postmaster, we want to report changes */
- if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
- {
- const char *preval = GetConfigOption(item->name, true, false);
-
- /* If option doesn't exist yet or is NULL, treat as empty string */
- if (!preval)
- preval = "";
- /* must dup, else might have dangling pointer below */
- pre_value = pstrdup(preval);
- }
-
- scres = set_config_option(item->name, item->value,
- context, PGC_S_FILE,
- GUC_ACTION_SET, applySettings, 0, false);
- if (scres > 0)
- {
- /* variable was updated, so log the change if appropriate */
- if (pre_value)
- {
- const char *post_value = GetConfigOption(item->name, true, false);
-
- if (!post_value)
- post_value = "";
- if (strcmp(pre_value, post_value) != 0)
- ereport(elevel,
- (errmsg("parameter \"%s\" changed to \"%s\"",
- item->name, item->value)));
- }
- item->applied = true;
- }
- else if (scres == 0)
- {
- error = true;
- item->errmsg = pstrdup("setting could not be applied");
- ConfFileWithError = item->filename;
- }
- else
- {
- /* no error, but variable's active value was not changed */
- item->applied = true;
- }
-
- /*
- * We should update source location unless there was an error, since
- * even if the active value didn't change, the reset value might have.
- * (In the postmaster, there won't be a difference, but it does matter
- * in backends.)
- */
- if (scres != 0 && applySettings)
- set_config_sourcefile(item->name, item->filename,
- item->sourceline);
-
- if (pre_value)
- pfree(pre_value);
- }
-
- /* Remember when we last successfully loaded the config file. */
- if (applySettings)
- PgReloadTime = GetCurrentTimestamp();
-
-bail_out:
- if (error && applySettings)
- {
- /* During postmaster startup, any error is fatal */
- if (context == PGC_POSTMASTER)
- ereport(ERROR,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors",
- ConfFileWithError)));
- else if (applying)
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
- ConfFileWithError)));
- else
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; no changes were applied",
- ConfFileWithError)));
- }
-
- /* Successful or otherwise, return the collected data list */
- return head;
-}
-
/*
* Given a configuration file or directory location that may be a relative
* path, return an absolute one. We consider the location to be relative to
@@ -659,7 +301,7 @@ cleanup:
* Capture an error message in the ConfigVariable list returned by
* config file parsing.
*/
-static void
+void
record_config_file_error(const char *errmsg,
const char *config_file,
int lineno,
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 9fbbfb1be5..66ab3912a0 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -243,10 +243,6 @@ static void assign_recovery_target_lsn(const char *newval, void *extra);
static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
-/* Private functions in guc-file.l that need to be called from guc.c */
-static ConfigVariable *ProcessConfigFileInternal(GucContext context,
- bool applySettings, int elevel);
-
/*
* Track whether there were any deferred checks for custom resource managers
* specified in wal_consistency_checking.
@@ -5160,8 +5156,8 @@ static bool report_needed; /* true if any GUC_REPORT reports are needed */
static int GUCNestLevel = 0; /* 1 when in main transaction */
+static struct config_generic *find_option(const char *name, bool create_placeholders, bool skip_errors, int elevel);
static int guc_var_compare(const void *a, const void *b);
-static int guc_name_compare(const char *namea, const char *nameb);
static void InitializeGUCOptionsFromEnvironment(void);
static void InitializeOneGUCOption(struct config_generic *gconf);
static void push_old_value(struct config_generic *gconf, GucAction action);
@@ -5180,7 +5176,359 @@ static bool validate_option_array_item(const char *name, const char *value,
static void write_auto_conf_file(int fd, const char *filename, ConfigVariable *head_p);
static void replace_auto_config_value(ConfigVariable **head_p, ConfigVariable **tail_p,
const char *name, const char *value);
+static bool valid_custom_variable_name(const char *name);
+
+/*
+ * This function handles both actual config file (re)loads and execution of
+ * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
+ * case we don't apply any of the settings, but we make all the usual validity
+ * checks, and we return the ConfigVariable list so that it can be printed out
+ * by show_all_file_settings().
+ */
+ConfigVariable *
+ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
+{
+ bool error = false;
+ bool applying = false;
+ const char *ConfFileWithError;
+ ConfigVariable *item,
+ *head,
+ *tail;
+ int i;
+
+ /* Parse the main config file into a list of option names and values */
+ ConfFileWithError = ConfigFileName;
+ head = tail = NULL;
+
+ if (!ParseConfigFile(ConfigFileName, true,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ goto bail_out;
+ }
+
+ /*
+ * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
+ * replace any parameters set by ALTER SYSTEM command. Because this file
+ * is in the data directory, we can't read it until the DataDir has been
+ * set.
+ */
+ if (DataDir)
+ {
+ if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ ConfFileWithError = PG_AUTOCONF_FILENAME;
+ goto bail_out;
+ }
+ }
+ else
+ {
+ /*
+ * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
+ * read. In this case, we don't want to accept any settings but
+ * data_directory from postgresql.conf, because they might be
+ * overwritten with settings in the PG_AUTOCONF_FILENAME file which
+ * will be read later. OTOH, since data_directory isn't allowed in the
+ * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
+ */
+ ConfigVariable *newlist = NULL;
+
+ /*
+ * Prune all items except the last "data_directory" from the list.
+ */
+ for (item = head; item; item = item->next)
+ {
+ if (!item->ignore &&
+ strcmp(item->name, "data_directory") == 0)
+ newlist = item;
+ }
+ if (newlist)
+ newlist->next = NULL;
+ head = tail = newlist;
+
+ /*
+ * Quick exit if data_directory is not present in file.
+ *
+ * We need not do any further processing, in particular we don't set
+ * PgReloadTime; that will be set soon by subsequent full loading of
+ * the config file.
+ */
+ if (head == NULL)
+ goto bail_out;
+ }
+
+ /*
+ * Mark all extant GUC variables as not present in the config file. We
+ * need this so that we can tell below which ones have been removed from
+ * the file since we last processed it.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+
+ gconf->status &= ~GUC_IS_IN_FILE;
+ }
+
+ /*
+ * Check if all the supplied option names are valid, as an additional
+ * quasi-syntactic check on the validity of the config file. It is
+ * important that the postmaster and all backends agree on the results of
+ * this phase, else we will have strange inconsistencies about which
+ * processes accept a config file update and which don't. Hence, unknown
+ * custom variable names have to be accepted without complaint. For the
+ * same reason, we don't attempt to validate the options' values here.
+ *
+ * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
+ * variable mentioned in the file; and we detect duplicate entries in the
+ * file and mark the earlier occurrences as ignorable.
+ */
+ for (item = head; item; item = item->next)
+ {
+ struct config_generic *record;
+
+ /* Ignore anything already marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /*
+ * Try to find the variable; but do not create a custom placeholder if
+ * it's not there already.
+ */
+ record = find_option(item->name, false, true, elevel);
+
+ if (record)
+ {
+ /* If it's already marked, then this is a duplicate entry */
+ if (record->status & GUC_IS_IN_FILE)
+ {
+ /*
+ * Mark the earlier occurrence(s) as dead/ignorable. We could
+ * avoid the O(N^2) behavior here with some additional state,
+ * but it seems unlikely to be worth the trouble.
+ */
+ ConfigVariable *pitem;
+
+ for (pitem = head; pitem != item; pitem = pitem->next)
+ {
+ if (!pitem->ignore &&
+ strcmp(pitem->name, item->name) == 0)
+ pitem->ignore = true;
+ }
+ }
+ /* Now mark it as present in file */
+ record->status |= GUC_IS_IN_FILE;
+ }
+ else if (!valid_custom_variable_name(item->name))
+ {
+ /* Invalid non-custom variable, so complain */
+ ereport(elevel,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
+ item->name,
+ item->filename, item->sourceline)));
+ item->errmsg = pstrdup("unrecognized configuration parameter");
+ error = true;
+ ConfFileWithError = item->filename;
+ }
+ }
+
+ /*
+ * If we've detected any errors so far, we don't want to risk applying any
+ * changes.
+ */
+ if (error)
+ goto bail_out;
+
+ /* Otherwise, set flag that we're beginning to apply changes */
+ applying = true;
+
+ /*
+ * Check for variables having been removed from the config file, and
+ * revert their reset values (and perhaps also effective values) to the
+ * boot-time defaults. If such a variable can't be changed after startup,
+ * report that and continue.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+ GucStack *stack;
+
+ if (gconf->reset_source != PGC_S_FILE ||
+ (gconf->status & GUC_IS_IN_FILE))
+ continue;
+ if (gconf->context < PGC_SIGHUP)
+ {
+ /* The removal can't be effective without a restart */
+ gconf->status |= GUC_PENDING_RESTART;
+ ereport(elevel,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ errmsg("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name)));
+ record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name),
+ NULL, 0,
+ &head, &tail);
+ error = true;
+ continue;
+ }
+
+ /* No more to do if we're just doing show_all_file_settings() */
+ if (!applySettings)
+ continue;
+
+ /*
+ * Reset any "file" sources to "default", else set_config_option will
+ * not override those settings.
+ */
+ if (gconf->reset_source == PGC_S_FILE)
+ gconf->reset_source = PGC_S_DEFAULT;
+ if (gconf->source == PGC_S_FILE)
+ gconf->source = PGC_S_DEFAULT;
+ for (stack = gconf->stack; stack; stack = stack->prev)
+ {
+ if (stack->source == PGC_S_FILE)
+ stack->source = PGC_S_DEFAULT;
+ }
+
+ /* Now we can re-apply the wired-in default (i.e., the boot_val) */
+ if (set_config_option(gconf->name, NULL,
+ context, PGC_S_DEFAULT,
+ GUC_ACTION_SET, true, 0, false) > 0)
+ {
+ /* Log the change if appropriate */
+ if (context == PGC_SIGHUP)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" removed from configuration file, reset to default",
+ gconf->name)));
+ }
+ }
+
+ /*
+ * Restore any variables determined by environment variables or
+ * dynamically-computed defaults. This is a no-op except in the case
+ * where one of these had been in the config file and is now removed.
+ *
+ * In particular, we *must not* do this during the postmaster's initial
+ * loading of the file, since the timezone functions in particular should
+ * be run only after initialization is complete.
+ *
+ * XXX this is an unmaintainable crock, because we have to know how to set
+ * (or at least what to call to set) every non-PGC_INTERNAL variable that
+ * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
+ */
+ if (context == PGC_SIGHUP && applySettings)
+ {
+ InitializeGUCOptionsFromEnvironment();
+ pg_timezone_abbrev_initialize();
+ /* this selects SQL_ASCII in processes not connected to a database */
+ SetConfigOption("client_encoding", GetDatabaseEncodingName(),
+ PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
+ }
+
+ /*
+ * Now apply the values from the config file.
+ */
+ for (item = head; item; item = item->next)
+ {
+ char *pre_value = NULL;
+ int scres;
+
+ /* Ignore anything marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /* In SIGHUP cases in the postmaster, we want to report changes */
+ if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
+ {
+ const char *preval = GetConfigOption(item->name, true, false);
+
+ /* If option doesn't exist yet or is NULL, treat as empty string */
+ if (!preval)
+ preval = "";
+ /* must dup, else might have dangling pointer below */
+ pre_value = pstrdup(preval);
+ }
+
+ scres = set_config_option(item->name, item->value,
+ context, PGC_S_FILE,
+ GUC_ACTION_SET, applySettings, 0, false);
+ if (scres > 0)
+ {
+ /* variable was updated, so log the change if appropriate */
+ if (pre_value)
+ {
+ const char *post_value = GetConfigOption(item->name, true, false);
+
+ if (!post_value)
+ post_value = "";
+ if (strcmp(pre_value, post_value) != 0)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" changed to \"%s\"",
+ item->name, item->value)));
+ }
+ item->applied = true;
+ }
+ else if (scres == 0)
+ {
+ error = true;
+ item->errmsg = pstrdup("setting could not be applied");
+ ConfFileWithError = item->filename;
+ }
+ else
+ {
+ /* no error, but variable's active value was not changed */
+ item->applied = true;
+ }
+
+ /*
+ * We should update source location unless there was an error, since
+ * even if the active value didn't change, the reset value might have.
+ * (In the postmaster, there won't be a difference, but it does matter
+ * in backends.)
+ */
+ if (scres != 0 && applySettings)
+ set_config_sourcefile(item->name, item->filename,
+ item->sourceline);
+
+ if (pre_value)
+ pfree(pre_value);
+ }
+
+ /* Remember when we last successfully loaded the config file. */
+ if (applySettings)
+ PgReloadTime = GetCurrentTimestamp();
+
+bail_out:
+ if (error && applySettings)
+ {
+ /* During postmaster startup, any error is fatal */
+ if (context == PGC_POSTMASTER)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors",
+ ConfFileWithError)));
+ else if (applying)
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
+ ConfFileWithError)));
+ else
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; no changes were applied",
+ ConfFileWithError)));
+ }
+
+ /* Successful or otherwise, return the collected data list */
+ return head;
+}
/*
* Some infrastructure for checking malloc/strdup/realloc calls
@@ -5737,7 +6085,7 @@ guc_var_compare(const void *a, const void *b)
/*
* the bare comparison function for GUC names
*/
-static int
+int
guc_name_compare(const char *namea, const char *nameb)
{
/*
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index e734493a48..aae071cd82 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -442,6 +442,15 @@ extern void GUC_check_errcode(int sqlerrcode);
pre_format_elog_string(errno, TEXTDOMAIN), \
GUC_check_errhint_string = format_elog_string
+/* functions shared between guc.c and guc-file.l */
+extern int guc_name_compare(const char *namea, const char *nameb);
+extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
+ bool applySettings, int elevel);
+extern void record_config_file_error(const char *errmsg,
+ const char *config_file,
+ int lineno,
+ ConfigVariable **head_p,
+ ConfigVariable **tail_p);
/*
* The following functions are not in guc.c, but are declared here to avoid
--
2.36.1
[text/x-patch] v3-0005-Build-repl_scanner.c-standalone.patch (6.1K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/3-v3-0005-Build-repl_scanner.c-standalone.patch)
download | inline diff:
From 3d6fc71eaf615bee5a8789f3fdec9cb361b711ff Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 12 Aug 2022 17:09:45 +0700
Subject: [PATCH v3 05/11] Build repl_scanner.c standalone
---
src/backend/Makefile | 3 ++-
src/backend/replication/.gitignore | 1 +
src/backend/replication/Makefile | 11 +++++++--
src/backend/replication/repl_gram.y | 2 --
src/backend/replication/repl_scanner.l | 31 +++++++++++++++++---------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 33 insertions(+), 16 deletions(-)
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 5a12666918..f527659a7b 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -183,7 +183,7 @@ distprep:
$(MAKE) -C bootstrap bootparse.c bootparse.h bootscanner.c
$(MAKE) -C catalog distprep
$(MAKE) -C nodes distprep
- $(MAKE) -C replication repl_gram.c repl_scanner.c syncrep_gram.c syncrep_scanner.c
+ $(MAKE) -C replication repl_gram.c repl_gram.h repl_scanner.c syncrep_gram.c syncrep_scanner.c
$(MAKE) -C storage/lmgr lwlocknames.h lwlocknames.c
$(MAKE) -C utils distprep
$(MAKE) -C utils/adt jsonpath_gram.c jsonpath_scan.c
@@ -304,6 +304,7 @@ maintainer-clean: distclean
parser/gram.h \
parser/scan.c \
replication/repl_gram.c \
+ replication/repl_gram.h \
replication/repl_scanner.c \
replication/syncrep_gram.c \
replication/syncrep_scanner.c \
diff --git a/src/backend/replication/.gitignore b/src/backend/replication/.gitignore
index d1df6147bd..a5f600232f 100644
--- a/src/backend/replication/.gitignore
+++ b/src/backend/replication/.gitignore
@@ -1,3 +1,4 @@
+/repl_gram.h
/repl_gram.c
/repl_scanner.c
/syncrep_gram.c
diff --git a/src/backend/replication/Makefile b/src/backend/replication/Makefile
index 2bffac58c0..bc8170418f 100644
--- a/src/backend/replication/Makefile
+++ b/src/backend/replication/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
repl_gram.o \
+ repl_scanner.o \
slot.o \
slotfuncs.o \
syncrep.o \
@@ -28,8 +29,14 @@ SUBDIRS = logical
include $(top_srcdir)/src/backend/common.mk
-# repl_scanner is compiled as part of repl_gram
-repl_gram.o: repl_scanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+repl_gram.h: repl_gram.c
+ touch $@
+
+repl_gram.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+repl_gram.o repl_scanner.o: repl_gram.h
# syncrep_scanner is compiled as part of syncrep_gram
syncrep_gram.o: syncrep_scanner.c
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 4cf087e602..b343f108d3 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -416,5 +416,3 @@ ident_or_keyword:
;
%%
-
-#include "repl_scanner.c"
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 586f0d3a5c..23fcb2a11d 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* repl_scanner.l
@@ -18,6 +18,15 @@
#include "utils/builtins.h"
#include "parser/scansup.h"
+/*
+ * NB: include repl_gram.h only AFTER including walsender_private.h, because
+ * walsender_private includes headers that define XLogRecPtr.
+ */
+#include "replication/walsender_private.h"
+#include "repl_gram.h"
+}
+
+%{
/* Avoid exit() on fatal scanner errors (a bit ugly -- see yy_fatal_error) */
#undef fprintf
#define fprintf(file, fmt, msg) fprintf_to_ereport(fmt, msg)
@@ -130,7 +139,7 @@ WAIT { return K_WAIT; }
{space}+ { /* do nothing */ }
{digit}+ {
- yylval.uintval = strtoul(yytext, NULL, 10);
+ replication_yylval.uintval = strtoul(yytext, NULL, 10);
return UCONST;
}
@@ -138,8 +147,8 @@ WAIT { return K_WAIT; }
uint32 hi,
lo;
if (sscanf(yytext, "%X/%X", &hi, &lo) != 2)
- yyerror("invalid streaming start location");
- yylval.recptr = ((uint64) hi) << 32 | lo;
+ replication_yyerror("invalid streaming start location");
+ replication_yylval.recptr = ((uint64) hi) << 32 | lo;
return RECPTR;
}
@@ -151,7 +160,7 @@ WAIT { return K_WAIT; }
<xq>{quotestop} {
yyless(1);
BEGIN(INITIAL);
- yylval.str = litbufdup();
+ replication_yylval.str = litbufdup();
return SCONST;
}
@@ -173,9 +182,9 @@ WAIT { return K_WAIT; }
yyless(1);
BEGIN(INITIAL);
- yylval.str = litbufdup();
- len = strlen(yylval.str);
- truncate_identifier(yylval.str, len, true);
+ replication_yylval.str = litbufdup();
+ len = strlen(replication_yylval.str);
+ truncate_identifier(replication_yylval.str, len, true);
return IDENT;
}
@@ -186,7 +195,7 @@ WAIT { return K_WAIT; }
{identifier} {
int len = strlen(yytext);
- yylval.str = downcase_truncate_identifier(yytext, len, true);
+ replication_yylval.str = downcase_truncate_identifier(yytext, len, true);
return IDENT;
}
@@ -195,7 +204,7 @@ WAIT { return K_WAIT; }
return yytext[0];
}
-<xq,xd><<EOF>> { yyerror("unterminated quoted string"); }
+<xq,xd><<EOF>> { replication_yyerror("unterminated quoted string"); }
<<EOF>> {
@@ -231,7 +240,7 @@ addlitchar(unsigned char ychar)
}
void
-yyerror(const char *message)
+replication_yyerror(const char *message)
{
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 96a95cad9b..c102adf434 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -118,6 +118,7 @@ do
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
+ test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v3-0002-Move-private-declarations-shared-between-guc.c-an.patch (3.2K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/4-v3-0002-Move-private-declarations-shared-between-guc.c-an.patch)
download | inline diff:
From 0d33e92fcd15b57977d8eb9d021996912e69ab81 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Tue, 16 Aug 2022 12:01:41 +0700
Subject: [PATCH v3 02/11] Move private declarations shared between guc.c and
guc-file.l to new header
FIXME: fails headerscheck
---
src/backend/utils/misc/guc-file.l | 1 +
src/backend/utils/misc/guc.c | 1 +
src/include/utils/guc.h | 10 ----------
src/include/utils/guc_internal.h | 24 ++++++++++++++++++++++++
4 files changed, 26 insertions(+), 10 deletions(-)
create mode 100644 src/include/utils/guc_internal.h
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index b4fa09749b..843838b1df 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -18,6 +18,7 @@
#include "miscadmin.h"
#include "storage/fd.h"
#include "utils/guc.h"
+#include "utils/guc_internal.h"
/*
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 66ab3912a0..293834fc13 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -100,6 +100,7 @@
#include "utils/builtins.h"
#include "utils/bytea.h"
#include "utils/float.h"
+#include "utils/guc_internal.h"
#include "utils/guc_tables.h"
#include "utils/memutils.h"
#include "utils/pg_locale.h"
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index aae071cd82..45ae1b537f 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -442,16 +442,6 @@ extern void GUC_check_errcode(int sqlerrcode);
pre_format_elog_string(errno, TEXTDOMAIN), \
GUC_check_errhint_string = format_elog_string
-/* functions shared between guc.c and guc-file.l */
-extern int guc_name_compare(const char *namea, const char *nameb);
-extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
- bool applySettings, int elevel);
-extern void record_config_file_error(const char *errmsg,
- const char *config_file,
- int lineno,
- ConfigVariable **head_p,
- ConfigVariable **tail_p);
-
/*
* The following functions are not in guc.c, but are declared here to avoid
* having to include guc.h in some widely used headers that it really doesn't
diff --git a/src/include/utils/guc_internal.h b/src/include/utils/guc_internal.h
new file mode 100644
index 0000000000..5d5db6bdce
--- /dev/null
+++ b/src/include/utils/guc_internal.h
@@ -0,0 +1,24 @@
+/*--------------------------------------------------------------------
+ * guc_internals.h
+ *
+ * Declarations shared between backend/utils/misc/guc.c and
+ * backend/utils/misc/guc-file.l
+ *
+ * Copyright (c) 2000-2022, PostgreSQL Global Development Group
+ *
+ * src/include/utils/guc_internals.h
+ *--------------------------------------------------------------------
+ */
+#ifndef GUC_INTERNALS_H
+#define GUC_INTERNALS_H
+
+extern int guc_name_compare(const char *namea, const char *nameb);
+extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
+ bool applySettings, int elevel);
+extern void record_config_file_error(const char *errmsg,
+ const char *config_file,
+ int lineno,
+ ConfigVariable **head_p,
+ ConfigVariable **tail_p);
+
+#endif /* GUC_INTERNALS_H */
--
2.36.1
[text/x-patch] v3-0004-Build-bootscanner.c-standalone.patch (7.4K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/5-v3-0004-Build-bootscanner.c-standalone.patch)
download | inline diff:
From ba67b6b8e5c26631d89d4a188a6b06c24bcc5f39 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 12 Aug 2022 15:45:24 +0700
Subject: [PATCH v3 04/11] Build bootscanner.c standalone
---
src/backend/Makefile | 3 +-
src/backend/bootstrap/.gitignore | 1 +
src/backend/bootstrap/Makefile | 11 +++++-
src/backend/bootstrap/bootparse.y | 2 -
src/backend/bootstrap/bootscanner.l | 60 ++++++++++++++++-------------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 46 insertions(+), 32 deletions(-)
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 3f01c65592..5a12666918 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -180,7 +180,7 @@ utils/probes.o: utils/probes.d $(SUBDIROBJS)
# Be sure that these files get removed by the maintainer-clean target
distprep:
$(MAKE) -C parser gram.c gram.h scan.c
- $(MAKE) -C bootstrap bootparse.c bootscanner.c
+ $(MAKE) -C bootstrap bootparse.c bootparse.h bootscanner.c
$(MAKE) -C catalog distprep
$(MAKE) -C nodes distprep
$(MAKE) -C replication repl_gram.c repl_scanner.c syncrep_gram.c syncrep_scanner.c
@@ -298,6 +298,7 @@ maintainer-clean: distclean
$(MAKE) -C nodes $@
$(MAKE) -C utils $@
rm -f bootstrap/bootparse.c \
+ bootstrap/bootparse.h \
bootstrap/bootscanner.c \
parser/gram.c \
parser/gram.h \
diff --git a/src/backend/bootstrap/.gitignore b/src/backend/bootstrap/.gitignore
index 1ffe8ca39e..6351b920fd 100644
--- a/src/backend/bootstrap/.gitignore
+++ b/src/backend/bootstrap/.gitignore
@@ -1,2 +1,3 @@
+/bootparse.h
/bootparse.c
/bootscanner.c
diff --git a/src/backend/bootstrap/Makefile b/src/backend/bootstrap/Makefile
index 6421efb227..606c8021e7 100644
--- a/src/backend/bootstrap/Makefile
+++ b/src/backend/bootstrap/Makefile
@@ -14,12 +14,19 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
bootparse.o \
+ bootscanner.o \
bootstrap.o
include $(top_srcdir)/src/backend/common.mk
-# bootscanner is compiled as part of bootparse
-bootparse.o: bootscanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+bootparse.h: bootparse.c
+ touch $@
+
+bootparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+bootparse.o bootscanner.o: bootparse.h
# bootparse.c and bootscanner.c are in the distribution tarball, so
# they are not cleaned here.
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 7d7655d295..c45ddde67f 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -488,5 +488,3 @@ boot_ident:
| XNULL { $$ = pstrdup($1); }
;
%%
-
-#include "bootscanner.c"
diff --git a/src/backend/bootstrap/bootscanner.l b/src/backend/bootstrap/bootscanner.l
index 3094ccb93f..d6eae84816 100644
--- a/src/backend/bootstrap/bootscanner.l
+++ b/src/backend/bootstrap/bootscanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* bootscanner.l
@@ -15,11 +15,17 @@
*/
#include "postgres.h"
+/*
+ * NB: include bootparse.h only AFTER including bootstrap.h, because bootstrap.h
+ * includes node definitions needed for YYSTYPE.
+ */
#include "bootstrap/bootstrap.h"
+#include "bootparse.h"
#include "utils/guc.h"
-/* Not needed now that this file is compiled as part of bootparse. */
-/* #include "bootparse.h" */
+}
+
+%{
/* LCOV_EXCL_START */
@@ -52,7 +58,7 @@ id [-A-Za-z0-9_]+
sid \'([^']|\'\')*\'
/*
- * Keyword tokens return the keyword text (as a constant string) in yylval.kw,
+ * Keyword tokens return the keyword text (as a constant string) in boot_yylval.kw,
* just in case that's needed because we want to treat the keyword as an
* unreserved identifier. Note that _null_ is not treated as a keyword
* for this purpose; it's the one "reserved word" in the bootstrap syntax.
@@ -60,23 +66,23 @@ sid \'([^']|\'\')*\'
* Notice that all the keywords are case-sensitive, and for historical
* reasons some must be upper case.
*
- * String tokens return a palloc'd string in yylval.str.
+ * String tokens return a palloc'd string in boot_yylval.str.
*/
%%
-open { yylval.kw = "open"; return OPEN; }
+open { boot_yylval.kw = "open"; return OPEN; }
-close { yylval.kw = "close"; return XCLOSE; }
+close { boot_yylval.kw = "close"; return XCLOSE; }
-create { yylval.kw = "create"; return XCREATE; }
+create { boot_yylval.kw = "create"; return XCREATE; }
-OID { yylval.kw = "OID"; return OBJ_ID; }
-bootstrap { yylval.kw = "bootstrap"; return XBOOTSTRAP; }
-shared_relation { yylval.kw = "shared_relation"; return XSHARED_RELATION; }
-rowtype_oid { yylval.kw = "rowtype_oid"; return XROWTYPE_OID; }
+OID { boot_yylval.kw = "OID"; return OBJ_ID; }
+bootstrap { boot_yylval.kw = "bootstrap"; return XBOOTSTRAP; }
+shared_relation { boot_yylval.kw = "shared_relation"; return XSHARED_RELATION; }
+rowtype_oid { boot_yylval.kw = "rowtype_oid"; return XROWTYPE_OID; }
-insert { yylval.kw = "insert"; return INSERT_TUPLE; }
+insert { boot_yylval.kw = "insert"; return INSERT_TUPLE; }
_null_ { return NULLVAL; }
@@ -90,25 +96,25 @@ _null_ { return NULLVAL; }
^\#[^\n]* ; /* drop everything after "#" for comments */
-declare { yylval.kw = "declare"; return XDECLARE; }
-build { yylval.kw = "build"; return XBUILD; }
-indices { yylval.kw = "indices"; return INDICES; }
-unique { yylval.kw = "unique"; return UNIQUE; }
-index { yylval.kw = "index"; return INDEX; }
-on { yylval.kw = "on"; return ON; }
-using { yylval.kw = "using"; return USING; }
-toast { yylval.kw = "toast"; return XTOAST; }
-FORCE { yylval.kw = "FORCE"; return XFORCE; }
-NOT { yylval.kw = "NOT"; return XNOT; }
-NULL { yylval.kw = "NULL"; return XNULL; }
+declare { boot_yylval.kw = "declare"; return XDECLARE; }
+build { boot_yylval.kw = "build"; return XBUILD; }
+indices { boot_yylval.kw = "indices"; return INDICES; }
+unique { boot_yylval.kw = "unique"; return UNIQUE; }
+index { boot_yylval.kw = "index"; return INDEX; }
+on { boot_yylval.kw = "on"; return ON; }
+using { boot_yylval.kw = "using"; return USING; }
+toast { boot_yylval.kw = "toast"; return XTOAST; }
+FORCE { boot_yylval.kw = "FORCE"; return XFORCE; }
+NOT { boot_yylval.kw = "NOT"; return XNOT; }
+NULL { boot_yylval.kw = "NULL"; return XNULL; }
{id} {
- yylval.str = pstrdup(yytext);
+ boot_yylval.str = pstrdup(yytext);
return ID;
}
{sid} {
/* strip quotes and escapes */
- yylval.str = DeescapeQuotedString(yytext);
+ boot_yylval.str = DeescapeQuotedString(yytext);
return ID;
}
@@ -121,7 +127,7 @@ NULL { yylval.kw = "NULL"; return XNULL; }
/* LCOV_EXCL_STOP */
void
-yyerror(const char *message)
+boot_yyerror(const char *message)
{
elog(ERROR, "%s at line %d", message, yyline);
}
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 3f8640a03d..96a95cad9b 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -117,6 +117,7 @@ do
# parser/gram.h will be included by parser/gramparse.h anyway.
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
+ test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v3-0003-Build-guc-file.c-standalone.patch (2.5K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/6-v3-0003-Build-guc-file.c-standalone.patch)
download | inline diff:
From 68becb3064c8c573212033339735b3e335eb3af6 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Thu, 11 Aug 2022 19:38:37 +0700
Subject: [PATCH v3 03/11] Build guc-file.c standalone
The proposed Meson build system will need a way to ignore certain
generated files in order to coexist with the autoconf build system,
and #include'd C files generated by Flex make this more difficult.
Build guc-file.c separately from guc.c, as was done in 72b1e3a21.
Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
---
src/backend/utils/misc/Makefile | 5 +----
src/backend/utils/misc/guc-file.l | 9 +++++----
src/backend/utils/misc/guc.c | 2 --
3 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/src/backend/utils/misc/Makefile b/src/backend/utils/misc/Makefile
index 1d5327cf64..cf7ce9bc83 100644
--- a/src/backend/utils/misc/Makefile
+++ b/src/backend/utils/misc/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
guc.o \
+ guc-file.o \
help_config.o \
pg_config.o \
pg_controldata.o \
@@ -37,10 +38,6 @@ endif
include $(top_srcdir)/src/backend/common.mk
-# guc-file is compiled as part of guc
-guc.o: guc-file.c
-
# Note: guc-file.c is not deleted by 'make clean',
# since we want to ship it in distribution tarballs.
clean:
- @rm -f lex.yy.c
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index 843838b1df..9aa3abe1ba 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -1,4 +1,4 @@
-/* -*-pgsql-c-*- */
+%top{
/*
* Scanner for the configuration file
*
@@ -7,8 +7,6 @@
* src/backend/utils/misc/guc-file.l
*/
-%{
-
#include "postgres.h"
#include <ctype.h>
@@ -17,10 +15,13 @@
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "storage/fd.h"
+#include <sys/stat.h>
#include "utils/guc.h"
#include "utils/guc_internal.h"
+#include "utils/memutils.h"
+}
-
+%{
/*
* flex emits a yy_fatal_error() function that it calls in response to
* critical errors like malloc failure, file I/O errors, and detection of
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 293834fc13..92b5b18c8f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -13333,5 +13333,3 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
-
-#include "guc-file.c"
--
2.36.1
[text/x-patch] v3-0007-Build-specscanner.c-standalone.patch (5.3K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/7-v3-0007-Build-specscanner.c-standalone.patch)
download | inline diff:
From 829925b21f94ed70c33882490e6fe3351f16697e Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 09:34:17 +0700
Subject: [PATCH v3 07/11] Build specscanner.c standalone
---
src/test/isolation/.gitignore | 1 +
src/test/isolation/Makefile | 15 +++++++++++----
src/test/isolation/specparse.y | 2 --
src/test/isolation/specscanner.l | 28 +++++++++++++++++++---------
src/tools/pginclude/headerscheck | 1 +
5 files changed, 32 insertions(+), 15 deletions(-)
diff --git a/src/test/isolation/.gitignore b/src/test/isolation/.gitignore
index 870dac4d28..2c13b4bf98 100644
--- a/src/test/isolation/.gitignore
+++ b/src/test/isolation/.gitignore
@@ -3,6 +3,7 @@
/pg_isolation_regress
# Local generated source files
+/specparse.h
/specparse.c
/specscanner.c
diff --git a/src/test/isolation/Makefile b/src/test/isolation/Makefile
index 0d452c89d4..b8738b7c1b 100644
--- a/src/test/isolation/Makefile
+++ b/src/test/isolation/Makefile
@@ -15,7 +15,8 @@ override CPPFLAGS := -I. -I$(srcdir) -I$(libpq_srcdir) \
OBJS = \
$(WIN32RES) \
isolationtester.o \
- specparse.o
+ specparse.o \
+ specscanner.o
all: isolationtester$(X) pg_isolation_regress$(X)
@@ -44,8 +45,14 @@ isolationtester$(X): $(OBJS) | submake-libpq submake-libpgport
distprep: specparse.c specscanner.c
-# specscanner is compiled as part of specparse
-specparse.o: specscanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+specparse.h: specparse.c
+ touch $@
+
+specparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+specparse.o specscanner.o: specparse.h
# specparse.c and specscanner.c are in the distribution tarball,
# so do not clean them here
@@ -55,7 +62,7 @@ clean distclean:
rm -rf $(pg_regress_clean_files)
maintainer-clean: distclean
- rm -f specparse.c specscanner.c
+ rm -f specparse.h specparse.c specscanner.c
installcheck: all
$(pg_isolation_regress_installcheck) --schedule=$(srcdir)/isolation_schedule
diff --git a/src/test/isolation/specparse.y b/src/test/isolation/specparse.y
index eb368184b8..657285cc23 100644
--- a/src/test/isolation/specparse.y
+++ b/src/test/isolation/specparse.y
@@ -276,5 +276,3 @@ blocker:
;
%%
-
-#include "specscanner.c"
diff --git a/src/test/isolation/specscanner.l b/src/test/isolation/specscanner.l
index aa6e89268e..b04696f52d 100644
--- a/src/test/isolation/specscanner.l
+++ b/src/test/isolation/specscanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* specscanner.l
@@ -9,7 +9,17 @@
*
*-------------------------------------------------------------------------
*/
+#include "postgres_fe.h"
+/*
+ * NB: include specparse.h only AFTER including isolationtester.h, because
+ * isolationtester.h includes node definitions needed for YYSTYPE.
+ */
+#include "isolationtester.h"
+#include "specparse.h"
+}
+
+%{
static int yyline = 1; /* line number for error reporting */
#define LITBUF_INIT 1024 /* initial size of litbuf */
@@ -75,7 +85,7 @@ teardown { return TEARDOWN; }
/* Plain identifiers */
{identifier} {
- yylval.str = pg_strdup(yytext);
+ spec_yylval.str = pg_strdup(yytext);
return(identifier);
}
@@ -87,13 +97,13 @@ teardown { return TEARDOWN; }
<qident>\"\" { addlitchar(yytext[0]); }
<qident>\" {
litbuf[litbufpos] = '\0';
- yylval.str = pg_strdup(litbuf);
+ spec_yylval.str = pg_strdup(litbuf);
BEGIN(INITIAL);
return(identifier);
}
<qident>. { addlitchar(yytext[0]); }
-<qident>\n { yyerror("unexpected newline in quoted identifier"); }
-<qident><<EOF>> { yyerror("unterminated quoted identifier"); }
+<qident>\n { spec_yyerror("unexpected newline in quoted identifier"); }
+<qident><<EOF>> { spec_yyerror("unterminated quoted identifier"); }
/* SQL blocks: { UPDATE ... } */
/* We trim leading/trailing whitespace, otherwise they're unprocessed */
@@ -104,7 +114,7 @@ teardown { return TEARDOWN; }
}
<sql>{space}*"}" {
litbuf[litbufpos] = '\0';
- yylval.str = pg_strdup(litbuf);
+ spec_yylval.str = pg_strdup(litbuf);
BEGIN(INITIAL);
return(sqlblock);
}
@@ -116,12 +126,12 @@ teardown { return TEARDOWN; }
addlitchar(yytext[0]);
}
<sql><<EOF>> {
- yyerror("unterminated sql block");
+ spec_yyerror("unterminated sql block");
}
/* Numbers and punctuation */
{digit}+ {
- yylval.integer = atoi(yytext);
+ spec_yylval.integer = atoi(yytext);
return INTEGER;
}
@@ -150,7 +160,7 @@ addlitchar(char c)
}
void
-yyerror(const char *message)
+spec_yyerror(const char *message)
{
fprintf(stderr, "%s at line %d\n", message, yyline);
exit(1);
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 77b697e293..cf75e93943 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -120,6 +120,7 @@ do
test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
+ test "$f" = src/test/isolation/specparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v3-0008-Build-exprscan.c-standalone.patch (4.1K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/8-v3-0008-Build-exprscan.c-standalone.patch)
download | inline diff:
From 4b9237ae205a4eb817124d0347c985de7eb194e0 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 13:35:14 +0700
Subject: [PATCH v3 08/11] Build exprscan.c standalone
---
src/bin/pgbench/.gitignore | 1 +
src/bin/pgbench/Makefile | 13 ++++++++++---
src/bin/pgbench/exprparse.y | 15 ---------------
src/bin/pgbench/exprscan.l | 12 +++++++++++-
src/tools/pginclude/headerscheck | 1 +
5 files changed, 23 insertions(+), 19 deletions(-)
diff --git a/src/bin/pgbench/.gitignore b/src/bin/pgbench/.gitignore
index 983a3cd7a6..07492a993c 100644
--- a/src/bin/pgbench/.gitignore
+++ b/src/bin/pgbench/.gitignore
@@ -1,3 +1,4 @@
+/exprparse.h
/exprparse.c
/exprscan.c
/pgbench
diff --git a/src/bin/pgbench/Makefile b/src/bin/pgbench/Makefile
index f402fe7b91..6647c9fe97 100644
--- a/src/bin/pgbench/Makefile
+++ b/src/bin/pgbench/Makefile
@@ -10,6 +10,7 @@ include $(top_builddir)/src/Makefile.global
OBJS = \
$(WIN32RES) \
exprparse.o \
+ exprscan.o \
pgbench.o
override CPPFLAGS := -I. -I$(srcdir) -I$(libpq_srcdir) $(CPPFLAGS)
@@ -26,8 +27,14 @@ all: pgbench
pgbench: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-# exprscan is compiled as part of exprparse
-exprparse.o: exprscan.c
+# See notes in src/backend/parser/Makefile about the following two rules
+exprparse.h: exprparse.c
+ touch $@
+
+exprparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+exprparse.o exprscan.o: exprparse.h
distprep: exprparse.c exprscan.c
@@ -45,7 +52,7 @@ clean distclean:
rm -rf tmp_check
maintainer-clean: distclean
- rm -f exprparse.c exprscan.c
+ rm -f exprparse.h exprparse.c exprscan.c
check:
$(prove_check)
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b5592d4b97..ade2ecdaab 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -526,18 +526,3 @@ make_case(yyscan_t yyscanner, PgBenchExprList *when_then_list, PgBenchExpr *else
find_func(yyscanner, "!case_end"),
make_elist(else_part, when_then_list));
}
-
-/*
- * exprscan.l is compiled as part of exprparse.y. Currently, this is
- * unavoidable because exprparse does not create a .h file to export
- * its token symbols. If these files ever grow large enough to be
- * worth compiling separately, that could be fixed; but for now it
- * seems like useless complication.
- */
-
-/* First, get rid of "#define yyscan_t" from pgbench.h */
-#undef yyscan_t
-/* ... and the yylval macro, which flex will have its own definition for */
-#undef yylval
-
-#include "exprscan.c"
diff --git a/src/bin/pgbench/exprscan.l b/src/bin/pgbench/exprscan.l
index 4f63818606..fe8e32838a 100644
--- a/src/bin/pgbench/exprscan.l
+++ b/src/bin/pgbench/exprscan.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* exprscan.l
@@ -22,9 +22,19 @@
*
*-------------------------------------------------------------------------
*/
+#include "postgres_fe.h"
+/*
+ * NB: include exprparse.h only AFTER including pgbench.h, because pgbench.h
+ * contains definitions needed for YYSTYPE. Likewise, pgbench.h must come after
+ * psqlscan_int.h for yyscan_t.
+ */
#include "fe_utils/psqlscan_int.h"
+#include "pgbench.h"
+#include "exprparse.h"
+}
+%{
/* context information for reporting errors in expressions */
static const char *expr_source = NULL;
static int expr_lineno = 0;
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index cf75e93943..e2c4ba9ae4 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -121,6 +121,7 @@ do
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
test "$f" = src/test/isolation/specparse.h && continue
+ test "$f" = src/bin/pgbench/exprparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v3-0006-Build-syncrep_scanner.c-standalone.patch (5.2K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/9-v3-0006-Build-syncrep_scanner.c-standalone.patch)
download | inline diff:
From 5cff476dd0588bc6c1a641990ea5050f0998130b Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 15:02:30 +0700
Subject: [PATCH v3 06/11] Build syncrep_scanner.c standalone
---
src/backend/Makefile | 3 ++-
src/backend/replication/.gitignore | 1 +
src/backend/replication/Makefile | 11 +++++++++--
src/backend/replication/syncrep_gram.y | 2 --
src/backend/replication/syncrep_scanner.l | 21 +++++++++++++++------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/src/backend/Makefile b/src/backend/Makefile
index f527659a7b..86cbe03677 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -183,7 +183,7 @@ distprep:
$(MAKE) -C bootstrap bootparse.c bootparse.h bootscanner.c
$(MAKE) -C catalog distprep
$(MAKE) -C nodes distprep
- $(MAKE) -C replication repl_gram.c repl_gram.h repl_scanner.c syncrep_gram.c syncrep_scanner.c
+ $(MAKE) -C replication repl_gram.c repl_gram.h repl_scanner.c syncrep_gram.c syncrep_gram.h syncrep_scanner.c
$(MAKE) -C storage/lmgr lwlocknames.h lwlocknames.c
$(MAKE) -C utils distprep
$(MAKE) -C utils/adt jsonpath_gram.c jsonpath_scan.c
@@ -307,6 +307,7 @@ maintainer-clean: distclean
replication/repl_gram.h \
replication/repl_scanner.c \
replication/syncrep_gram.c \
+ replication/syncrep_gram.h \
replication/syncrep_scanner.c \
storage/lmgr/lwlocknames.c \
storage/lmgr/lwlocknames.h \
diff --git a/src/backend/replication/.gitignore b/src/backend/replication/.gitignore
index a5f600232f..77d5a51068 100644
--- a/src/backend/replication/.gitignore
+++ b/src/backend/replication/.gitignore
@@ -1,5 +1,6 @@
/repl_gram.h
/repl_gram.c
/repl_scanner.c
+/syncrep_gram.h
/syncrep_gram.c
/syncrep_scanner.c
diff --git a/src/backend/replication/Makefile b/src/backend/replication/Makefile
index bc8170418f..23f29ba545 100644
--- a/src/backend/replication/Makefile
+++ b/src/backend/replication/Makefile
@@ -21,6 +21,7 @@ OBJS = \
slotfuncs.o \
syncrep.o \
syncrep_gram.o \
+ syncrep_scanner.o \
walreceiver.o \
walreceiverfuncs.o \
walsender.o
@@ -38,8 +39,14 @@ repl_gram.c: BISONFLAGS += -d
# Force these dependencies to be known even without dependency info built:
repl_gram.o repl_scanner.o: repl_gram.h
-# syncrep_scanner is compiled as part of syncrep_gram
-syncrep_gram.o: syncrep_scanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+syncrep_gram.h: syncrep_gram.c
+ touch $@
+
+syncrep_gram.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+syncrep_gram.o syncrep_scanner.o: syncrep_gram.h
# repl_gram.c, repl_scanner.c, syncrep_gram.c and syncrep_scanner.c
# are in the distribution tarball, so they are not cleaned here.
diff --git a/src/backend/replication/syncrep_gram.y b/src/backend/replication/syncrep_gram.y
index d932f2cda3..4fc3647da1 100644
--- a/src/backend/replication/syncrep_gram.y
+++ b/src/backend/replication/syncrep_gram.y
@@ -112,5 +112,3 @@ create_syncrep_config(const char *num_sync, List *members, uint8 syncrep_method)
return config;
}
-
-#include "syncrep_scanner.c"
diff --git a/src/backend/replication/syncrep_scanner.l b/src/backend/replication/syncrep_scanner.l
index 1952c8c6e0..bdb1a3391c 100644
--- a/src/backend/replication/syncrep_scanner.l
+++ b/src/backend/replication/syncrep_scanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* syncrep_scanner.l
@@ -17,6 +17,15 @@
#include "lib/stringinfo.h"
+/*
+ * NB: include syncrep_gram.h only AFTER including syncrep.h, because syncrep.h
+ * includes node definitions needed for YYSTYPE.
+ */
+#include "replication/syncrep.h"
+#include "syncrep_gram.h"
+}
+
+%{
/* Avoid exit() on fatal scanner errors (a bit ugly -- see yy_fatal_error) */
#undef fprintf
#define fprintf(file, fmt, msg) fprintf_to_ereport(fmt, msg)
@@ -82,28 +91,28 @@ xdinside [^"]+
appendStringInfoString(&xdbuf, yytext);
}
<xd>{xdstop} {
- yylval.str = xdbuf.data;
+ syncrep_yylval.str = xdbuf.data;
xdbuf.data = NULL;
BEGIN(INITIAL);
return NAME;
}
<xd><<EOF>> {
- yyerror("unterminated quoted identifier");
+ syncrep_yyerror("unterminated quoted identifier");
return JUNK;
}
{identifier} {
- yylval.str = pstrdup(yytext);
+ syncrep_yylval.str = pstrdup(yytext);
return NAME;
}
{digit}+ {
- yylval.str = pstrdup(yytext);
+ syncrep_yylval.str = pstrdup(yytext);
return NUM;
}
"*" {
- yylval.str = "*";
+ syncrep_yylval.str = "*";
return NAME;
}
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index c102adf434..77b697e293 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -119,6 +119,7 @@ do
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
+ test "$f" = src/backend/replication/syncrep_gram.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v3-0009-Build-cubescan.c-standalone.patch (5.9K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/10-v3-0009-Build-cubescan.c-standalone.patch)
download | inline diff:
From e38e77bd77c803acfc66de8c76798d9d7be67c42 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 11:18:06 +0700
Subject: [PATCH v3 09/11] Build cubescan.c standalone
---
contrib/cube/.gitignore | 1 +
contrib/cube/Makefile | 16 +++++++++-----
contrib/cube/cubedata.h | 1 +
contrib/cube/cubeparse.y | 10 +++------
contrib/cube/cubescan.l | 38 ++++++++++++++++++++------------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 40 insertions(+), 27 deletions(-)
diff --git a/contrib/cube/.gitignore b/contrib/cube/.gitignore
index cb4c989fff..f788440c79 100644
--- a/contrib/cube/.gitignore
+++ b/contrib/cube/.gitignore
@@ -1,3 +1,4 @@
+/cubeparse.h
/cubeparse.c
/cubescan.c
# Generated subdirectories
diff --git a/contrib/cube/Makefile b/contrib/cube/Makefile
index cf195506c7..4fd19aac35 100644
--- a/contrib/cube/Makefile
+++ b/contrib/cube/Makefile
@@ -4,7 +4,8 @@ MODULE_big = cube
OBJS = \
$(WIN32RES) \
cube.o \
- cubeparse.o
+ cubeparse.o \
+ cubescan.o
EXTENSION = cube
DATA = cube--1.2.sql cube--1.2--1.3.sql cube--1.3--1.4.sql cube--1.4--1.5.sql \
@@ -15,8 +16,6 @@ HEADERS = cubedata.h
REGRESS = cube cube_sci
-EXTRA_CLEAN = y.tab.c y.tab.h
-
SHLIB_LINK += $(filter -lm, $(LIBS))
ifdef USE_PGXS
@@ -30,11 +29,16 @@ include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
+# See notes in src/backend/parser/Makefile about the following two rules
+cubeparse.h: cubeparse.c
+ touch $@
+
+cubeparse.c: BISONFLAGS += -d
-# cubescan is compiled as part of cubeparse
-cubeparse.o: cubescan.c
+# Force these dependencies to be known even without dependency info built:
+cubeparse.o cubescan.o: cubeparse.h
distprep: cubeparse.c cubescan.c
maintainer-clean:
- rm -f cubeparse.c cubescan.c
+ rm -f cubeparse.h cubeparse.c cubescan.c
diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
index dbe7d4f742..a04fc504ad 100644
--- a/contrib/cube/cubedata.h
+++ b/contrib/cube/cubedata.h
@@ -67,3 +67,4 @@ extern void cube_scanner_finish(void);
/* in cubeparse.y */
extern int cube_yyparse(NDBOX **result);
+extern int scanbuflen;
diff --git a/contrib/cube/cubeparse.y b/contrib/cube/cubeparse.y
index 7577c4515c..8e210b49ec 100644
--- a/contrib/cube/cubeparse.y
+++ b/contrib/cube/cubeparse.y
@@ -7,11 +7,9 @@
#include "postgres.h"
#include "cubedata.h"
+#include "cube_internal.h"
#include "utils/float.h"
-/* All grammar constructs return strings */
-#define YYSTYPE char *
-
/*
* Bison doesn't allocate anything that needs to live across parser calls,
* so we can easily have it use palloc instead of malloc. This prevents
@@ -23,8 +21,8 @@
#define YYMALLOC palloc
#define YYFREE pfree
-static char *scanbuf;
-static int scanbuflen;
+/* TODO: get rid of global variable */
+int scanbuflen;
static int item_count(const char *s, char delim);
static NDBOX *write_box(int dim, char *str1, char *str2);
@@ -265,5 +263,3 @@ write_point_as_box(int dim, char *str)
return bp;
}
-
-#include "cubescan.c"
diff --git a/contrib/cube/cubescan.l b/contrib/cube/cubescan.l
index bd400e3684..569a4e992e 100644
--- a/contrib/cube/cubescan.l
+++ b/contrib/cube/cubescan.l
@@ -1,9 +1,21 @@
-%{
+%top{
/*
* A scanner for EMP-style numeric ranges
* contrib/cube/cubescan.l
*/
+#include "postgres.h"
+
+/*
+ * NB: include cubeparse.h only AFTER including cube_internal.h for YYSTYPE
+ * and cubedata.h for NDBOX.
+ */
+#include "cubedata.h"
+#include "cube_internal.h"
+#include "cubeparse.h"
+}
+
+%{
/* LCOV_EXCL_START */
/* No reason to constrain amount of data slurped */
@@ -21,9 +33,7 @@ fprintf_to_ereport(const char *fmt, const char *msg)
/* Handles to the buffer that the lexer uses internally */
static YY_BUFFER_STATE scanbufhandle;
-/* this is now declared in cubeparse.y: */
-/* static char *scanbuf; */
-/* static int scanbuflen; */
+static char *scanbuf;
%}
%option 8bit
@@ -45,14 +55,14 @@ NaN [nN][aA][nN]
%%
-{float} yylval = yytext; return CUBEFLOAT;
-{infinity} yylval = yytext; return CUBEFLOAT;
-{NaN} yylval = yytext; return CUBEFLOAT;
-\[ yylval = "("; return O_BRACKET;
-\] yylval = ")"; return C_BRACKET;
-\( yylval = "("; return O_PAREN;
-\) yylval = ")"; return C_PAREN;
-\, yylval = ","; return COMMA;
+{float} cube_yylval = yytext; return CUBEFLOAT;
+{infinity} cube_yylval = yytext; return CUBEFLOAT;
+{NaN} cube_yylval = yytext; return CUBEFLOAT;
+\[ cube_yylval = "("; return O_BRACKET;
+\] cube_yylval = ")"; return C_BRACKET;
+\( cube_yylval = "("; return O_PAREN;
+\) cube_yylval = ")"; return C_PAREN;
+\, cube_yylval = ","; return COMMA;
[ \t\n\r\f]+ /* discard spaces */
. return yytext[0]; /* alert parser of the garbage */
@@ -62,7 +72,7 @@ NaN [nN][aA][nN]
/* result is not used, but Bison expects this signature */
void
-yyerror(NDBOX **result, const char *message)
+cube_yyerror(NDBOX **result, const char *message)
{
if (*yytext == YY_END_OF_BUFFER_CHAR)
{
@@ -89,7 +99,7 @@ yyerror(NDBOX **result, const char *message)
void
cube_scanner_init(const char *str)
{
- Size slen = strlen(str);
+ Size slen = strlen(str);
/*
* Might be left over after ereport()
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index e2c4ba9ae4..a778067570 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -115,6 +115,7 @@ do
# We can't make these Bison output files compilable standalone
# without using "%code require", which old Bison versions lack.
# parser/gram.h will be included by parser/gramparse.h anyway.
+ test "$f" = contrib/cube/cubeparse.h && continue
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
--
2.36.1
[text/x-patch] v3-0010-Build-segscan.c-standalone.patch (4.5K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/11-v3-0010-Build-segscan.c-standalone.patch)
download | inline diff:
From d687b826a302c0aa7123b1f6bebdef3b9a75d443 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 12:00:33 +0700
Subject: [PATCH v3 10/11] Build segscan.c standalone
---
contrib/seg/.gitignore | 1 +
contrib/seg/Makefile | 15 +++++++++++----
contrib/seg/segparse.y | 3 ---
contrib/seg/segscan.l | 28 ++++++++++++++++++----------
src/tools/pginclude/headerscheck | 1 +
5 files changed, 31 insertions(+), 17 deletions(-)
diff --git a/contrib/seg/.gitignore b/contrib/seg/.gitignore
index 69e73d2096..fa247a4e67 100644
--- a/contrib/seg/.gitignore
+++ b/contrib/seg/.gitignore
@@ -1,3 +1,4 @@
+/segparse.h
/segparse.c
/segscan.c
# Generated subdirectories
diff --git a/contrib/seg/Makefile b/contrib/seg/Makefile
index bb63e83506..c6c134b8f1 100644
--- a/contrib/seg/Makefile
+++ b/contrib/seg/Makefile
@@ -4,7 +4,8 @@ MODULE_big = seg
OBJS = \
$(WIN32RES) \
seg.o \
- segparse.o
+ segparse.o \
+ segscan.o
EXTENSION = seg
DATA = seg--1.1.sql seg--1.1--1.2.sql seg--1.2--1.3.sql seg--1.3--1.4.sql \
@@ -29,10 +30,16 @@ include $(top_srcdir)/contrib/contrib-global.mk
endif
-# segscan is compiled as part of segparse
-segparse.o: segscan.c
+# See notes in src/backend/parser/Makefile about the following two rules
+segparse.h: segparse.c
+ touch $@
+
+segparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+segparse.o segscan.o: segparse.h
distprep: segparse.c segscan.c
maintainer-clean:
- rm -f segparse.c segscan.c
+ rm -f segparse.h segparse.c segscan.c
diff --git a/contrib/seg/segparse.y b/contrib/seg/segparse.y
index 33e3a9f35f..637eacd1a6 100644
--- a/contrib/seg/segparse.y
+++ b/contrib/seg/segparse.y
@@ -160,6 +160,3 @@ seg_atof(const char *value)
datum = DirectFunctionCall1(float4in, CStringGetDatum(value));
return DatumGetFloat4(datum);
}
-
-
-#include "segscan.c"
diff --git a/contrib/seg/segscan.l b/contrib/seg/segscan.l
index 5f6595e9eb..4744fd5e9e 100644
--- a/contrib/seg/segscan.l
+++ b/contrib/seg/segscan.l
@@ -1,8 +1,18 @@
-%{
+%top{
/*
* A scanner for EMP-style numeric ranges
*/
+#include "postgres.h"
+
+/*
+ * NB: include segparse.h only AFTER including segdata.h, because segdata.h
+ * contains the definition for SEG.
+ */
+#include "segdata.h"
+#include "segparse.h"
+}
+%{
/* LCOV_EXCL_START */
/* No reason to constrain amount of data slurped */
@@ -21,7 +31,6 @@ fprintf_to_ereport(const char *fmt, const char *msg)
/* Handles to the buffer that the lexer uses internally */
static YY_BUFFER_STATE scanbufhandle;
static char *scanbuf;
-static int scanbuflen;
%}
%option 8bit
@@ -42,12 +51,12 @@ float ({integer}|{real})([eE]{integer})?
%%
-{range} yylval.text = yytext; return RANGE;
-{plumin} yylval.text = yytext; return PLUMIN;
-{float} yylval.text = yytext; return SEGFLOAT;
-\< yylval.text = "<"; return EXTENSION;
-\> yylval.text = ">"; return EXTENSION;
-\~ yylval.text = "~"; return EXTENSION;
+{range} seg_yylval.text = yytext; return RANGE;
+{plumin} seg_yylval.text = yytext; return PLUMIN;
+{float} seg_yylval.text = yytext; return SEGFLOAT;
+\< seg_yylval.text = "<"; return EXTENSION;
+\> seg_yylval.text = ">"; return EXTENSION;
+\~ seg_yylval.text = "~"; return EXTENSION;
[ \t\n\r\f]+ /* discard spaces */
. return yytext[0]; /* alert parser of the garbage */
@@ -56,7 +65,7 @@ float ({integer}|{real})([eE]{integer})?
/* LCOV_EXCL_STOP */
void
-yyerror(SEG *result, const char *message)
+seg_yyerror(SEG *result, const char *message)
{
if (*yytext == YY_END_OF_BUFFER_CHAR)
{
@@ -94,7 +103,6 @@ seg_scanner_init(const char *str)
/*
* Make a scan buffer with special termination needed by flex.
*/
- scanbuflen = slen;
scanbuf = palloc(slen + 2);
memcpy(scanbuf, str, slen);
scanbuf[slen] = scanbuf[slen + 1] = YY_END_OF_BUFFER_CHAR;
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index a778067570..203703afa7 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -116,6 +116,7 @@ do
# without using "%code require", which old Bison versions lack.
# parser/gram.h will be included by parser/gramparse.h anyway.
test "$f" = contrib/cube/cubeparse.h && continue
+ test "$f" = contrib/seg/segparse.h && continue
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
--
2.36.1
[text/x-patch] v3-0011-Build-jsonpath_scan.c-standalone.patch (7.8K, ../../CAFBsxsEospoUX=QYkfC=WcJqNB+iZtBf=BaRwn-zbHa48X0NKQ@mail.gmail.com/12-v3-0011-Build-jsonpath_scan.c-standalone.patch)
download | inline diff:
From ecf3e99df1c71a579f307a9a7be06e06a88687b5 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 12:35:55 +0700
Subject: [PATCH v3 11/11] Build jsonpath_scan.c standalone
---
src/backend/utils/adt/.gitignore | 1 +
src/backend/utils/adt/Makefile | 11 ++++++--
src/backend/utils/adt/jsonpath_gram.y | 27 +------------------
src/backend/utils/adt/jsonpath_internal.h | 32 +++++++++++++++++++++++
src/backend/utils/adt/jsonpath_scan.l | 29 +++++++++++++-------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 63 insertions(+), 38 deletions(-)
create mode 100644 src/backend/utils/adt/jsonpath_internal.h
diff --git a/src/backend/utils/adt/.gitignore b/src/backend/utils/adt/.gitignore
index 48cf941a52..7fab054407 100644
--- a/src/backend/utils/adt/.gitignore
+++ b/src/backend/utils/adt/.gitignore
@@ -1,2 +1,3 @@
+/jsonpath_gram.h
/jsonpath_gram.c
/jsonpath_scan.c
diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile
index 7c722ea2ce..0de0bbb1b8 100644
--- a/src/backend/utils/adt/Makefile
+++ b/src/backend/utils/adt/Makefile
@@ -57,6 +57,7 @@ OBJS = \
jsonpath.o \
jsonpath_exec.o \
jsonpath_gram.o \
+ jsonpath_scan.o \
like.o \
like_support.o \
lockfuncs.o \
@@ -119,11 +120,17 @@ OBJS = \
xid8funcs.o \
xml.o
+# See notes in src/backend/parser/Makefile about the following two rules
+jsonpath_gram.h: jsonpath_gram.c
+ touch $@
+
+jsonpath_gram.c: BISONFLAGS += -d
+
jsonpath_scan.c: FLEXFLAGS = -CF -p -p
jsonpath_scan.c: FLEX_NO_BACKUP=yes
-# jsonpath_scan is compiled as part of jsonpath_gram
-jsonpath_gram.o: jsonpath_scan.c
+# Force these dependencies to be known even without dependency info built:
+jsonpath_gram.o jsonpath_scan.o: jsonpath_gram.h
# jsonpath_gram.c and jsonpath_scan.c are in the distribution tarball,
# so they are not cleaned here.
diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y
index f903dba3e3..e7027595ac 100644
--- a/src/backend/utils/adt/jsonpath_gram.y
+++ b/src/backend/utils/adt/jsonpath_gram.y
@@ -18,26 +18,11 @@
#include "catalog/pg_collation.h"
#include "fmgr.h"
+#include "jsonpath_internal.h"
#include "miscadmin.h"
#include "nodes/pg_list.h"
#include "regex/regex.h"
#include "utils/builtins.h"
-#include "utils/jsonpath.h"
-
-/* struct JsonPathString is shared between scan and gram */
-typedef struct JsonPathString
-{
- char *val;
- int len;
- int total;
-} JsonPathString;
-
-union YYSTYPE;
-
-/* flex 2.5.4 doesn't bother with a decl for this */
-int jsonpath_yylex(union YYSTYPE *yylval_param);
-int jsonpath_yyparse(JsonPathParseResult **result);
-void jsonpath_yyerror(JsonPathParseResult **result, const char *message);
static JsonPathParseItem *makeItemType(JsonPathItemType type);
static JsonPathParseItem *makeItemString(JsonPathString *s);
@@ -593,13 +578,3 @@ jspConvertRegexFlags(uint32 xflags)
return cflags;
}
-
-/*
- * jsonpath_scan.l is compiled as part of jsonpath_gram.y. Currently, this is
- * unavoidable because jsonpath_gram does not create a .h file to export its
- * token symbols. If these files ever grow large enough to be worth compiling
- * separately, that could be fixed; but for now it seems like useless
- * complication.
- */
-
-#include "jsonpath_scan.c"
diff --git a/src/backend/utils/adt/jsonpath_internal.h b/src/backend/utils/adt/jsonpath_internal.h
new file mode 100644
index 0000000000..edfc6191a0
--- /dev/null
+++ b/src/backend/utils/adt/jsonpath_internal.h
@@ -0,0 +1,32 @@
+/*-------------------------------------------------------------------------
+ *
+ * jsonpath_internal.h
+ * Private definitions for jsonpath scanner & parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/utils/adt/jsonpath_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef JSONPATH_INTERNAL_H
+#define JSONPATH_INTERNAL_H
+
+/* struct JsonPathString is shared between scan and gram */
+typedef struct JsonPathString
+{
+ char *val;
+ int len;
+ int total;
+} JsonPathString;
+
+#include "utils/jsonpath.h"
+#include "jsonpath_gram.h"
+
+extern int jsonpath_yylex(YYSTYPE *yylval_param);
+extern int jsonpath_yyparse(JsonPathParseResult **result);
+extern void jsonpath_yyerror(JsonPathParseResult **result, const char *message);
+
+#endif /* JSONPATH_INTERNAL_H */
diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l
index 4351f6ec98..ea824bae73 100644
--- a/src/backend/utils/adt/jsonpath_scan.l
+++ b/src/backend/utils/adt/jsonpath_scan.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* jsonpath_scan.l
@@ -17,9 +17,18 @@
#include "postgres.h"
+/*
+ * NB: include jsonpath_gram.h only AFTER including jsonpath_internal.h,
+ * because jsonpath_internal.h contains the declaration for JsonPathString.
+ */
+#include "jsonpath_internal.h"
+#include "jsonpath_gram.h"
+
#include "mb/pg_wchar.h"
#include "nodes/pg_list.h"
+}
+%{
static JsonPathString scanstring;
/* Handles to the buffer that the lexer uses internally */
@@ -142,9 +151,9 @@ hex_fail \\x{hex_dig}{0,1}
<xnq,xq,xvq>{hex_char} { parseHexChar(yytext); }
-<xnq,xq,xvq>{unicode}*{unicodefail} { yyerror(NULL, "invalid unicode sequence"); }
+<xnq,xq,xvq>{unicode}*{unicodefail} { jsonpath_yyerror(NULL, "invalid unicode sequence"); }
-<xnq,xq,xvq>{hex_fail} { yyerror(NULL, "invalid hex character sequence"); }
+<xnq,xq,xvq>{hex_fail} { jsonpath_yyerror(NULL, "invalid hex character sequence"); }
<xnq,xq,xvq>{unicode}+\\ {
/* throw back the \\, and treat as unicode */
@@ -154,9 +163,9 @@ hex_fail \\x{hex_dig}{0,1}
<xnq,xq,xvq>\\. { addchar(false, yytext[1]); }
-<xnq,xq,xvq>\\ { yyerror(NULL, "unexpected end after backslash"); }
+<xnq,xq,xvq>\\ { jsonpath_yyerror(NULL, "unexpected end after backslash"); }
-<xq,xvq><<EOF>> { yyerror(NULL, "unexpected end of quoted string"); }
+<xq,xvq><<EOF>> { jsonpath_yyerror(NULL, "unexpected end of quoted string"); }
<xq>\" {
yylval->str = scanstring;
@@ -178,7 +187,7 @@ hex_fail \\x{hex_dig}{0,1}
<xc>\* { }
-<xc><<EOF>> { yyerror(NULL, "unexpected end of comment"); }
+<xc><<EOF>> { jsonpath_yyerror(NULL, "unexpected end of comment"); }
\&\& { return AND_P; }
@@ -244,10 +253,10 @@ hex_fail \\x{hex_dig}{0,1}
return INT_P;
}
-{realfail} { yyerror(NULL, "invalid numeric literal"); }
-{integer_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
-{decimal_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
-{real_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
+{realfail} { jsonpath_yyerror(NULL, "invalid numeric literal"); }
+{integer_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
+{decimal_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
+{real_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
\" {
addchar(true, '\0');
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 203703afa7..2e90f52c33 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -122,6 +122,7 @@ do
test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
+ test "$f" = src/backend/utils/adt/jsonpath_gram.h && continue
test "$f" = src/test/isolation/specparse.h && continue
test "$f" = src/bin/pgbench/exprparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
--
2.36.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-17 01:14 Andres Freund <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 3 replies; 27+ messages in thread
From: Andres Freund @ 2022-08-17 01:14 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
Hi,
On 2022-08-16 17:41:43 +0700, John Naylor wrote:
> For v3, I addressed some comments and added .h files to the
> headerscheck exceptions.
Thanks!
> /*
> * NB: include bootparse.h only AFTER including bootstrap.h, because bootstrap.h
> * includes node definitions needed for YYSTYPE.
> */
>
> Future cleanup: I see this in headerscheck:
>
> # We can't make these Bison output files compilable standalone
> # without using "%code require", which old Bison versions lack.
> # parser/gram.h will be included by parser/gramparse.h anyway.
>
> That directive has been supported in Bison since 2.4.2.
2.4.2 is from 2010. So I think we could just start relying on it?
> > > +/* functions shared between guc.c and guc-file.l */
> > > [...]
> > I think I prefer your suggestion of a guc_internal.h upthread.
>
> Started in 0002, but left open the headerscheck failure.
>
> Also, if such a thing is meant to be #include'd only by two generated
> files, maybe it should just live in the directory where they live, and
> not in the src/include dir?
It's not something we've done for the backend afaics, but I don't see a reason
not to start at some point.
> > > From 7d4ecfcb3e91f3b45e94b9e64c7c40f1bbd22aa8 Mon Sep 17 00:00:00 2001
> > > From: John Naylor <[email protected]>
> > > Date: Fri, 12 Aug 2022 15:45:24 +0700
> > > Subject: [PATCH v201 2/9] Build booscanner.c standalone
> >
> > > -# bootscanner is compiled as part of bootparse
> > > -bootparse.o: bootscanner.c
> > > +# See notes in src/backend/parser/Makefile about the following two rules
> > > +bootparse.h: bootparse.c
> > > + touch $@
> > > +
> > > +bootparse.c: BISONFLAGS += -d
> > > +
> > > +# Force these dependencies to be known even without dependency info built:
> > > +bootparse.o bootscan.o: bootparse.h
> >
> > Wonder if we could / should wrap this is something common. It's somewhat
> > annoying to repeat this stuff everywhere.
>
> I haven't looked at the Meson effort recently, but if the build rule
> is less annoying there, I'm inclined to leave this as a wart until
> autotools are retired.
The only complicating thing in the rules there is the dependencies from one .c
file to another .c file.
> > > diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
> > > index dbe7d4f742..0b373048b5 100644
> > > --- a/contrib/cube/cubedata.h
> > > +++ b/contrib/cube/cubedata.h
> > > @@ -67,3 +67,7 @@ extern void cube_scanner_finish(void);
> > >
> > > /* in cubeparse.y */
> > > extern int cube_yyparse(NDBOX **result);
> > > +
> > > +/* All grammar constructs return strings */
> > > +#define YYSTYPE char *
> >
> > Why does this need to be defined in a semi-public header? If we do this in
> > multiple files we'll end up with the danger of macro redefinition warnings.
>
> I tried to put all the Flex/Bison stuff in another *_internal header,
> but that breaks the build. Putting just this one symbol in a header is
> silly, but done that way for now. Maybe two copies of the symbol?
The problem is that if it's in a header you can't include another header with
such a define. That's fine if it's a .h that's just intended to be included by
a limited set of files, but for something like a header for a datatype that
might need to be included to e.g. define a PL transform or a new operator or
... This would be solved by the %code requires thing, right?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-17 01:47 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 0 replies; 27+ messages in thread
From: Tom Lane @ 2022-08-17 01:47 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: John Naylor <[email protected]>; pgsql-hackers
Andres Freund <[email protected]> writes:
> On 2022-08-16 17:41:43 +0700, John Naylor wrote:
>> That directive has been supported in Bison since 2.4.2.
> 2.4.2 is from 2010. So I think we could just start relying on it?
Apple is still shipping 2.3. Is this worth enough to make Mac
users install a non-default Bison? I seriously doubt it.
I don't say that there won't be a reason that justifies that
at some point, but letting headerscheck test autogenerated
files seems of only microscopic benefit :-(
regards, tom lane
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-17 02:53 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 0 replies; 27+ messages in thread
From: John Naylor @ 2022-08-17 02:53 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
On Wed, Aug 17, 2022 at 8:14 AM Andres Freund <[email protected]> wrote:
> > > > +/* functions shared between guc.c and guc-file.l */
> > > > [...]
> > > I think I prefer your suggestion of a guc_internal.h upthread.
> >
> > Started in 0002, but left open the headerscheck failure.
> >
> > Also, if such a thing is meant to be #include'd only by two generated
> > files, maybe it should just live in the directory where they live, and
> > not in the src/include dir?
>
> It's not something we've done for the backend afaics, but I don't see a reason
> not to start at some point.
BTW, I forgot to mention I did this for the json path parser, which
makes the makefile code simpler than what was there before
550b9d26f80fa30. AFAICS, we could also do the same for gramparse.h,
which is internal to parser.c. If I'm not mistaken, the only reason we
symlink gram.h to src/include/* is so that gramparse.h can include it.
So keeping gramparse.h in the backend could allow removing some gram.h
makefile incantations.
> > > Why does this need to be defined in a semi-public header? If we do this in
> > > multiple files we'll end up with the danger of macro redefinition warnings.
> >
> > I tried to put all the Flex/Bison stuff in another *_internal header,
> > but that breaks the build. Putting just this one symbol in a header is
> > silly, but done that way for now. Maybe two copies of the symbol?
>
> The problem is that if it's in a header you can't include another header with
> such a define. That's fine if it's a .h that's just intended to be included by
> a limited set of files, but for something like a header for a datatype that
> might need to be included to e.g. define a PL transform or a new operator or
> ... This would be solved by the %code requires thing, right?
I believe it would.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-18 07:40 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-08-18 07:40 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
On Wed, Aug 17, 2022 at 8:14 AM Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2022-08-16 17:41:43 +0700, John Naylor wrote:
> > For v3, I addressed some comments and added .h files to the
> > headerscheck exceptions.
>
> Thanks!
>
>
> > /*
> > * NB: include bootparse.h only AFTER including bootstrap.h, because bootstrap.h
> > * includes node definitions needed for YYSTYPE.
> > */
> >
> > Future cleanup: I see this in headerscheck:
> >
> > # We can't make these Bison output files compilable standalone
> > # without using "%code require", which old Bison versions lack.
> > # parser/gram.h will be included by parser/gramparse.h anyway.
> >
> > That directive has been supported in Bison since 2.4.2.
>
> 2.4.2 is from 2010. So I think we could just start relying on it?
>
>
> > > > +/* functions shared between guc.c and guc-file.l */
> > > > [...]
> > > I think I prefer your suggestion of a guc_internal.h upthread.
> >
> > Started in 0002, but left open the headerscheck failure.
> >
> > Also, if such a thing is meant to be #include'd only by two generated
> > files, maybe it should just live in the directory where they live, and
> > not in the src/include dir?
>
> It's not something we've done for the backend afaics, but I don't see a reason
> not to start at some point.
>
>
> > > > From 7d4ecfcb3e91f3b45e94b9e64c7c40f1bbd22aa8 Mon Sep 17 00:00:00 2001
> > > > From: John Naylor <[email protected]>
> > > > Date: Fri, 12 Aug 2022 15:45:24 +0700
> > > > Subject: [PATCH v201 2/9] Build booscanner.c standalone
> > >
> > > > -# bootscanner is compiled as part of bootparse
> > > > -bootparse.o: bootscanner.c
> > > > +# See notes in src/backend/parser/Makefile about the following two rules
> > > > +bootparse.h: bootparse.c
> > > > + touch $@
> > > > +
> > > > +bootparse.c: BISONFLAGS += -d
> > > > +
> > > > +# Force these dependencies to be known even without dependency info built:
> > > > +bootparse.o bootscan.o: bootparse.h
> > >
> > > Wonder if we could / should wrap this is something common. It's somewhat
> > > annoying to repeat this stuff everywhere.
> >
> > I haven't looked at the Meson effort recently, but if the build rule
> > is less annoying there, I'm inclined to leave this as a wart until
> > autotools are retired.
>
> The only complicating thing in the rules there is the dependencies from one .c
> file to another .c file.
>
>
> > > > diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
> > > > index dbe7d4f742..0b373048b5 100644
> > > > --- a/contrib/cube/cubedata.h
> > > > +++ b/contrib/cube/cubedata.h
> > > > @@ -67,3 +67,7 @@ extern void cube_scanner_finish(void);
> > > >
> > > > /* in cubeparse.y */
> > > > extern int cube_yyparse(NDBOX **result);
> > > > +
> > > > +/* All grammar constructs return strings */
> > > > +#define YYSTYPE char *
> > >
> > > Why does this need to be defined in a semi-public header? If we do this in
> > > multiple files we'll end up with the danger of macro redefinition warnings.
For v4, I #defined YYSTYPE
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-18 07:43 John Naylor <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 2 replies; 27+ messages in thread
From: John Naylor @ 2022-08-18 07:43 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
> > > > > index dbe7d4f742..0b373048b5 100644
> > > > > --- a/contrib/cube/cubedata.h
> > > > > +++ b/contrib/cube/cubedata.h
> > > > > @@ -67,3 +67,7 @@ extern void cube_scanner_finish(void);
> > > > >
> > > > > /* in cubeparse.y */
> > > > > extern int cube_yyparse(NDBOX **result);
> > > > > +
> > > > > +/* All grammar constructs return strings */
> > > > > +#define YYSTYPE char *
> > > >
> > > > Why does this need to be defined in a semi-public header? If we do this in
> > > > multiple files we'll end up with the danger of macro redefinition warnings.
>
> For v4, I #defined YYSTYPE
Sorry for the misfire. Continuing on, I #defined YYSTYPE in cubescan.l
before #including cubeparse.h.
I also added scanbuflen to the %parse-param to prevent resorting to a
global variable. The rest of the patches are unchanged.
--
John Naylor
EDB: http://www.enterprisedb.com
Attachments:
[text/x-patch] v4-0002-Move-private-declarations-shared-between-guc.c-an.patch (3.2K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/2-v4-0002-Move-private-declarations-shared-between-guc.c-an.patch)
download | inline diff:
From 2b95401d925bed67b2cb1eb9e8cdb1f1dd3bcc8e Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Tue, 16 Aug 2022 12:01:41 +0700
Subject: [PATCH v4 02/11] Move private declarations shared between guc.c and
guc-file.l to new header
FIXME: fails headerscheck
---
src/backend/utils/misc/guc-file.l | 1 +
src/backend/utils/misc/guc.c | 1 +
src/include/utils/guc.h | 10 ----------
src/include/utils/guc_internal.h | 24 ++++++++++++++++++++++++
4 files changed, 26 insertions(+), 10 deletions(-)
create mode 100644 src/include/utils/guc_internal.h
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index b4fa09749b..843838b1df 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -18,6 +18,7 @@
#include "miscadmin.h"
#include "storage/fd.h"
#include "utils/guc.h"
+#include "utils/guc_internal.h"
/*
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 66ab3912a0..293834fc13 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -100,6 +100,7 @@
#include "utils/builtins.h"
#include "utils/bytea.h"
#include "utils/float.h"
+#include "utils/guc_internal.h"
#include "utils/guc_tables.h"
#include "utils/memutils.h"
#include "utils/pg_locale.h"
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index aae071cd82..45ae1b537f 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -442,16 +442,6 @@ extern void GUC_check_errcode(int sqlerrcode);
pre_format_elog_string(errno, TEXTDOMAIN), \
GUC_check_errhint_string = format_elog_string
-/* functions shared between guc.c and guc-file.l */
-extern int guc_name_compare(const char *namea, const char *nameb);
-extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
- bool applySettings, int elevel);
-extern void record_config_file_error(const char *errmsg,
- const char *config_file,
- int lineno,
- ConfigVariable **head_p,
- ConfigVariable **tail_p);
-
/*
* The following functions are not in guc.c, but are declared here to avoid
* having to include guc.h in some widely used headers that it really doesn't
diff --git a/src/include/utils/guc_internal.h b/src/include/utils/guc_internal.h
new file mode 100644
index 0000000000..5d5db6bdce
--- /dev/null
+++ b/src/include/utils/guc_internal.h
@@ -0,0 +1,24 @@
+/*--------------------------------------------------------------------
+ * guc_internals.h
+ *
+ * Declarations shared between backend/utils/misc/guc.c and
+ * backend/utils/misc/guc-file.l
+ *
+ * Copyright (c) 2000-2022, PostgreSQL Global Development Group
+ *
+ * src/include/utils/guc_internals.h
+ *--------------------------------------------------------------------
+ */
+#ifndef GUC_INTERNALS_H
+#define GUC_INTERNALS_H
+
+extern int guc_name_compare(const char *namea, const char *nameb);
+extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
+ bool applySettings, int elevel);
+extern void record_config_file_error(const char *errmsg,
+ const char *config_file,
+ int lineno,
+ ConfigVariable **head_p,
+ ConfigVariable **tail_p);
+
+#endif /* GUC_INTERNALS_H */
--
2.36.1
[text/x-patch] v4-0001-Preparatory-refactoring-for-compiling-guc-file.c-.patch (26.0K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/3-v4-0001-Preparatory-refactoring-for-compiling-guc-file.c-.patch)
download | inline diff:
From c066efea2193be8e7b21bb44c067383f34f37ec8 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Tue, 16 Aug 2022 10:42:19 +0700
Subject: [PATCH v4 01/11] Preparatory refactoring for compiling guc-file.c
standalone
Mostly this involves moving ProcessConfigFileInternal() to guc.c
and fixing the shared API to match.
---
src/backend/utils/misc/guc-file.l | 360 +-----------------------------
src/backend/utils/misc/guc.c | 360 +++++++++++++++++++++++++++++-
src/include/utils/guc.h | 9 +
3 files changed, 364 insertions(+), 365 deletions(-)
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index ce5633844c..b4fa09749b 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -48,12 +48,6 @@ static sigjmp_buf *GUC_flex_fatal_jmp;
static void FreeConfigVariable(ConfigVariable *item);
-static void record_config_file_error(const char *errmsg,
- const char *config_file,
- int lineno,
- ConfigVariable **head_p,
- ConfigVariable **tail_p);
-
static int GUC_flex_fatal(const char *msg);
/* LCOV_EXCL_START */
@@ -159,358 +153,6 @@ ProcessConfigFile(GucContext context)
MemoryContextDelete(config_cxt);
}
-/*
- * This function handles both actual config file (re)loads and execution of
- * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
- * case we don't apply any of the settings, but we make all the usual validity
- * checks, and we return the ConfigVariable list so that it can be printed out
- * by show_all_file_settings().
- */
-static ConfigVariable *
-ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
-{
- bool error = false;
- bool applying = false;
- const char *ConfFileWithError;
- ConfigVariable *item,
- *head,
- *tail;
- int i;
-
- /* Parse the main config file into a list of option names and values */
- ConfFileWithError = ConfigFileName;
- head = tail = NULL;
-
- if (!ParseConfigFile(ConfigFileName, true,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- goto bail_out;
- }
-
- /*
- * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
- * replace any parameters set by ALTER SYSTEM command. Because this file
- * is in the data directory, we can't read it until the DataDir has been
- * set.
- */
- if (DataDir)
- {
- if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
- NULL, 0, 0, elevel,
- &head, &tail))
- {
- /* Syntax error(s) detected in the file, so bail out */
- error = true;
- ConfFileWithError = PG_AUTOCONF_FILENAME;
- goto bail_out;
- }
- }
- else
- {
- /*
- * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
- * read. In this case, we don't want to accept any settings but
- * data_directory from postgresql.conf, because they might be
- * overwritten with settings in the PG_AUTOCONF_FILENAME file which
- * will be read later. OTOH, since data_directory isn't allowed in the
- * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
- */
- ConfigVariable *newlist = NULL;
-
- /*
- * Prune all items except the last "data_directory" from the list.
- */
- for (item = head; item; item = item->next)
- {
- if (!item->ignore &&
- strcmp(item->name, "data_directory") == 0)
- newlist = item;
- }
-
- if (newlist)
- newlist->next = NULL;
- head = tail = newlist;
-
- /*
- * Quick exit if data_directory is not present in file.
- *
- * We need not do any further processing, in particular we don't set
- * PgReloadTime; that will be set soon by subsequent full loading of
- * the config file.
- */
- if (head == NULL)
- goto bail_out;
- }
-
- /*
- * Mark all extant GUC variables as not present in the config file. We
- * need this so that we can tell below which ones have been removed from
- * the file since we last processed it.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
-
- gconf->status &= ~GUC_IS_IN_FILE;
- }
-
- /*
- * Check if all the supplied option names are valid, as an additional
- * quasi-syntactic check on the validity of the config file. It is
- * important that the postmaster and all backends agree on the results of
- * this phase, else we will have strange inconsistencies about which
- * processes accept a config file update and which don't. Hence, unknown
- * custom variable names have to be accepted without complaint. For the
- * same reason, we don't attempt to validate the options' values here.
- *
- * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
- * variable mentioned in the file; and we detect duplicate entries in the
- * file and mark the earlier occurrences as ignorable.
- */
- for (item = head; item; item = item->next)
- {
- struct config_generic *record;
-
- /* Ignore anything already marked as ignorable */
- if (item->ignore)
- continue;
-
- /*
- * Try to find the variable; but do not create a custom placeholder if
- * it's not there already.
- */
- record = find_option(item->name, false, true, elevel);
-
- if (record)
- {
- /* If it's already marked, then this is a duplicate entry */
- if (record->status & GUC_IS_IN_FILE)
- {
- /*
- * Mark the earlier occurrence(s) as dead/ignorable. We could
- * avoid the O(N^2) behavior here with some additional state,
- * but it seems unlikely to be worth the trouble.
- */
- ConfigVariable *pitem;
-
- for (pitem = head; pitem != item; pitem = pitem->next)
- {
- if (!pitem->ignore &&
- strcmp(pitem->name, item->name) == 0)
- pitem->ignore = true;
- }
- }
- /* Now mark it as present in file */
- record->status |= GUC_IS_IN_FILE;
- }
- else if (!valid_custom_variable_name(item->name))
- {
- /* Invalid non-custom variable, so complain */
- ereport(elevel,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
- item->name,
- item->filename, item->sourceline)));
- item->errmsg = pstrdup("unrecognized configuration parameter");
- error = true;
- ConfFileWithError = item->filename;
- }
- }
-
- /*
- * If we've detected any errors so far, we don't want to risk applying any
- * changes.
- */
- if (error)
- goto bail_out;
-
- /* Otherwise, set flag that we're beginning to apply changes */
- applying = true;
-
- /*
- * Check for variables having been removed from the config file, and
- * revert their reset values (and perhaps also effective values) to the
- * boot-time defaults. If such a variable can't be changed after startup,
- * report that and continue.
- */
- for (i = 0; i < num_guc_variables; i++)
- {
- struct config_generic *gconf = guc_variables[i];
- GucStack *stack;
-
- if (gconf->reset_source != PGC_S_FILE ||
- (gconf->status & GUC_IS_IN_FILE))
- continue;
- if (gconf->context < PGC_SIGHUP)
- {
- /* The removal can't be effective without a restart */
- gconf->status |= GUC_PENDING_RESTART;
- ereport(elevel,
- (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
- errmsg("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name)));
- record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
- gconf->name),
- NULL, 0,
- &head, &tail);
- error = true;
- continue;
- }
-
- /* No more to do if we're just doing show_all_file_settings() */
- if (!applySettings)
- continue;
-
- /*
- * Reset any "file" sources to "default", else set_config_option will
- * not override those settings.
- */
- if (gconf->reset_source == PGC_S_FILE)
- gconf->reset_source = PGC_S_DEFAULT;
- if (gconf->source == PGC_S_FILE)
- gconf->source = PGC_S_DEFAULT;
- for (stack = gconf->stack; stack; stack = stack->prev)
- {
- if (stack->source == PGC_S_FILE)
- stack->source = PGC_S_DEFAULT;
- }
-
- /* Now we can re-apply the wired-in default (i.e., the boot_val) */
- if (set_config_option(gconf->name, NULL,
- context, PGC_S_DEFAULT,
- GUC_ACTION_SET, true, 0, false) > 0)
- {
- /* Log the change if appropriate */
- if (context == PGC_SIGHUP)
- ereport(elevel,
- (errmsg("parameter \"%s\" removed from configuration file, reset to default",
- gconf->name)));
- }
- }
-
- /*
- * Restore any variables determined by environment variables or
- * dynamically-computed defaults. This is a no-op except in the case
- * where one of these had been in the config file and is now removed.
- *
- * In particular, we *must not* do this during the postmaster's initial
- * loading of the file, since the timezone functions in particular should
- * be run only after initialization is complete.
- *
- * XXX this is an unmaintainable crock, because we have to know how to set
- * (or at least what to call to set) every non-PGC_INTERNAL variable that
- * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
- */
- if (context == PGC_SIGHUP && applySettings)
- {
- InitializeGUCOptionsFromEnvironment();
- pg_timezone_abbrev_initialize();
- /* this selects SQL_ASCII in processes not connected to a database */
- SetConfigOption("client_encoding", GetDatabaseEncodingName(),
- PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
- }
-
- /*
- * Now apply the values from the config file.
- */
- for (item = head; item; item = item->next)
- {
- char *pre_value = NULL;
- int scres;
-
- /* Ignore anything marked as ignorable */
- if (item->ignore)
- continue;
-
- /* In SIGHUP cases in the postmaster, we want to report changes */
- if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
- {
- const char *preval = GetConfigOption(item->name, true, false);
-
- /* If option doesn't exist yet or is NULL, treat as empty string */
- if (!preval)
- preval = "";
- /* must dup, else might have dangling pointer below */
- pre_value = pstrdup(preval);
- }
-
- scres = set_config_option(item->name, item->value,
- context, PGC_S_FILE,
- GUC_ACTION_SET, applySettings, 0, false);
- if (scres > 0)
- {
- /* variable was updated, so log the change if appropriate */
- if (pre_value)
- {
- const char *post_value = GetConfigOption(item->name, true, false);
-
- if (!post_value)
- post_value = "";
- if (strcmp(pre_value, post_value) != 0)
- ereport(elevel,
- (errmsg("parameter \"%s\" changed to \"%s\"",
- item->name, item->value)));
- }
- item->applied = true;
- }
- else if (scres == 0)
- {
- error = true;
- item->errmsg = pstrdup("setting could not be applied");
- ConfFileWithError = item->filename;
- }
- else
- {
- /* no error, but variable's active value was not changed */
- item->applied = true;
- }
-
- /*
- * We should update source location unless there was an error, since
- * even if the active value didn't change, the reset value might have.
- * (In the postmaster, there won't be a difference, but it does matter
- * in backends.)
- */
- if (scres != 0 && applySettings)
- set_config_sourcefile(item->name, item->filename,
- item->sourceline);
-
- if (pre_value)
- pfree(pre_value);
- }
-
- /* Remember when we last successfully loaded the config file. */
- if (applySettings)
- PgReloadTime = GetCurrentTimestamp();
-
-bail_out:
- if (error && applySettings)
- {
- /* During postmaster startup, any error is fatal */
- if (context == PGC_POSTMASTER)
- ereport(ERROR,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors",
- ConfFileWithError)));
- else if (applying)
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
- ConfFileWithError)));
- else
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("configuration file \"%s\" contains errors; no changes were applied",
- ConfFileWithError)));
- }
-
- /* Successful or otherwise, return the collected data list */
- return head;
-}
-
/*
* Given a configuration file or directory location that may be a relative
* path, return an absolute one. We consider the location to be relative to
@@ -659,7 +301,7 @@ cleanup:
* Capture an error message in the ConfigVariable list returned by
* config file parsing.
*/
-static void
+void
record_config_file_error(const char *errmsg,
const char *config_file,
int lineno,
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 9fbbfb1be5..66ab3912a0 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -243,10 +243,6 @@ static void assign_recovery_target_lsn(const char *newval, void *extra);
static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
-/* Private functions in guc-file.l that need to be called from guc.c */
-static ConfigVariable *ProcessConfigFileInternal(GucContext context,
- bool applySettings, int elevel);
-
/*
* Track whether there were any deferred checks for custom resource managers
* specified in wal_consistency_checking.
@@ -5160,8 +5156,8 @@ static bool report_needed; /* true if any GUC_REPORT reports are needed */
static int GUCNestLevel = 0; /* 1 when in main transaction */
+static struct config_generic *find_option(const char *name, bool create_placeholders, bool skip_errors, int elevel);
static int guc_var_compare(const void *a, const void *b);
-static int guc_name_compare(const char *namea, const char *nameb);
static void InitializeGUCOptionsFromEnvironment(void);
static void InitializeOneGUCOption(struct config_generic *gconf);
static void push_old_value(struct config_generic *gconf, GucAction action);
@@ -5180,7 +5176,359 @@ static bool validate_option_array_item(const char *name, const char *value,
static void write_auto_conf_file(int fd, const char *filename, ConfigVariable *head_p);
static void replace_auto_config_value(ConfigVariable **head_p, ConfigVariable **tail_p,
const char *name, const char *value);
+static bool valid_custom_variable_name(const char *name);
+
+/*
+ * This function handles both actual config file (re)loads and execution of
+ * show_all_file_settings() (i.e., the pg_file_settings view). In the latter
+ * case we don't apply any of the settings, but we make all the usual validity
+ * checks, and we return the ConfigVariable list so that it can be printed out
+ * by show_all_file_settings().
+ */
+ConfigVariable *
+ProcessConfigFileInternal(GucContext context, bool applySettings, int elevel)
+{
+ bool error = false;
+ bool applying = false;
+ const char *ConfFileWithError;
+ ConfigVariable *item,
+ *head,
+ *tail;
+ int i;
+
+ /* Parse the main config file into a list of option names and values */
+ ConfFileWithError = ConfigFileName;
+ head = tail = NULL;
+
+ if (!ParseConfigFile(ConfigFileName, true,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ goto bail_out;
+ }
+
+ /*
+ * Parse the PG_AUTOCONF_FILENAME file, if present, after the main file to
+ * replace any parameters set by ALTER SYSTEM command. Because this file
+ * is in the data directory, we can't read it until the DataDir has been
+ * set.
+ */
+ if (DataDir)
+ {
+ if (!ParseConfigFile(PG_AUTOCONF_FILENAME, false,
+ NULL, 0, 0, elevel,
+ &head, &tail))
+ {
+ /* Syntax error(s) detected in the file, so bail out */
+ error = true;
+ ConfFileWithError = PG_AUTOCONF_FILENAME;
+ goto bail_out;
+ }
+ }
+ else
+ {
+ /*
+ * If DataDir is not set, the PG_AUTOCONF_FILENAME file cannot be
+ * read. In this case, we don't want to accept any settings but
+ * data_directory from postgresql.conf, because they might be
+ * overwritten with settings in the PG_AUTOCONF_FILENAME file which
+ * will be read later. OTOH, since data_directory isn't allowed in the
+ * PG_AUTOCONF_FILENAME file, it will never be overwritten later.
+ */
+ ConfigVariable *newlist = NULL;
+
+ /*
+ * Prune all items except the last "data_directory" from the list.
+ */
+ for (item = head; item; item = item->next)
+ {
+ if (!item->ignore &&
+ strcmp(item->name, "data_directory") == 0)
+ newlist = item;
+ }
+ if (newlist)
+ newlist->next = NULL;
+ head = tail = newlist;
+
+ /*
+ * Quick exit if data_directory is not present in file.
+ *
+ * We need not do any further processing, in particular we don't set
+ * PgReloadTime; that will be set soon by subsequent full loading of
+ * the config file.
+ */
+ if (head == NULL)
+ goto bail_out;
+ }
+
+ /*
+ * Mark all extant GUC variables as not present in the config file. We
+ * need this so that we can tell below which ones have been removed from
+ * the file since we last processed it.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+
+ gconf->status &= ~GUC_IS_IN_FILE;
+ }
+
+ /*
+ * Check if all the supplied option names are valid, as an additional
+ * quasi-syntactic check on the validity of the config file. It is
+ * important that the postmaster and all backends agree on the results of
+ * this phase, else we will have strange inconsistencies about which
+ * processes accept a config file update and which don't. Hence, unknown
+ * custom variable names have to be accepted without complaint. For the
+ * same reason, we don't attempt to validate the options' values here.
+ *
+ * In addition, the GUC_IS_IN_FILE flag is set on each existing GUC
+ * variable mentioned in the file; and we detect duplicate entries in the
+ * file and mark the earlier occurrences as ignorable.
+ */
+ for (item = head; item; item = item->next)
+ {
+ struct config_generic *record;
+
+ /* Ignore anything already marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /*
+ * Try to find the variable; but do not create a custom placeholder if
+ * it's not there already.
+ */
+ record = find_option(item->name, false, true, elevel);
+
+ if (record)
+ {
+ /* If it's already marked, then this is a duplicate entry */
+ if (record->status & GUC_IS_IN_FILE)
+ {
+ /*
+ * Mark the earlier occurrence(s) as dead/ignorable. We could
+ * avoid the O(N^2) behavior here with some additional state,
+ * but it seems unlikely to be worth the trouble.
+ */
+ ConfigVariable *pitem;
+
+ for (pitem = head; pitem != item; pitem = pitem->next)
+ {
+ if (!pitem->ignore &&
+ strcmp(pitem->name, item->name) == 0)
+ pitem->ignore = true;
+ }
+ }
+ /* Now mark it as present in file */
+ record->status |= GUC_IS_IN_FILE;
+ }
+ else if (!valid_custom_variable_name(item->name))
+ {
+ /* Invalid non-custom variable, so complain */
+ ereport(elevel,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("unrecognized configuration parameter \"%s\" in file \"%s\" line %d",
+ item->name,
+ item->filename, item->sourceline)));
+ item->errmsg = pstrdup("unrecognized configuration parameter");
+ error = true;
+ ConfFileWithError = item->filename;
+ }
+ }
+
+ /*
+ * If we've detected any errors so far, we don't want to risk applying any
+ * changes.
+ */
+ if (error)
+ goto bail_out;
+
+ /* Otherwise, set flag that we're beginning to apply changes */
+ applying = true;
+
+ /*
+ * Check for variables having been removed from the config file, and
+ * revert their reset values (and perhaps also effective values) to the
+ * boot-time defaults. If such a variable can't be changed after startup,
+ * report that and continue.
+ */
+ for (i = 0; i < num_guc_variables; i++)
+ {
+ struct config_generic *gconf = guc_variables[i];
+ GucStack *stack;
+
+ if (gconf->reset_source != PGC_S_FILE ||
+ (gconf->status & GUC_IS_IN_FILE))
+ continue;
+ if (gconf->context < PGC_SIGHUP)
+ {
+ /* The removal can't be effective without a restart */
+ gconf->status |= GUC_PENDING_RESTART;
+ ereport(elevel,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ errmsg("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name)));
+ record_config_file_error(psprintf("parameter \"%s\" cannot be changed without restarting the server",
+ gconf->name),
+ NULL, 0,
+ &head, &tail);
+ error = true;
+ continue;
+ }
+
+ /* No more to do if we're just doing show_all_file_settings() */
+ if (!applySettings)
+ continue;
+
+ /*
+ * Reset any "file" sources to "default", else set_config_option will
+ * not override those settings.
+ */
+ if (gconf->reset_source == PGC_S_FILE)
+ gconf->reset_source = PGC_S_DEFAULT;
+ if (gconf->source == PGC_S_FILE)
+ gconf->source = PGC_S_DEFAULT;
+ for (stack = gconf->stack; stack; stack = stack->prev)
+ {
+ if (stack->source == PGC_S_FILE)
+ stack->source = PGC_S_DEFAULT;
+ }
+
+ /* Now we can re-apply the wired-in default (i.e., the boot_val) */
+ if (set_config_option(gconf->name, NULL,
+ context, PGC_S_DEFAULT,
+ GUC_ACTION_SET, true, 0, false) > 0)
+ {
+ /* Log the change if appropriate */
+ if (context == PGC_SIGHUP)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" removed from configuration file, reset to default",
+ gconf->name)));
+ }
+ }
+
+ /*
+ * Restore any variables determined by environment variables or
+ * dynamically-computed defaults. This is a no-op except in the case
+ * where one of these had been in the config file and is now removed.
+ *
+ * In particular, we *must not* do this during the postmaster's initial
+ * loading of the file, since the timezone functions in particular should
+ * be run only after initialization is complete.
+ *
+ * XXX this is an unmaintainable crock, because we have to know how to set
+ * (or at least what to call to set) every non-PGC_INTERNAL variable that
+ * could potentially have PGC_S_DYNAMIC_DEFAULT or PGC_S_ENV_VAR source.
+ */
+ if (context == PGC_SIGHUP && applySettings)
+ {
+ InitializeGUCOptionsFromEnvironment();
+ pg_timezone_abbrev_initialize();
+ /* this selects SQL_ASCII in processes not connected to a database */
+ SetConfigOption("client_encoding", GetDatabaseEncodingName(),
+ PGC_BACKEND, PGC_S_DYNAMIC_DEFAULT);
+ }
+
+ /*
+ * Now apply the values from the config file.
+ */
+ for (item = head; item; item = item->next)
+ {
+ char *pre_value = NULL;
+ int scres;
+
+ /* Ignore anything marked as ignorable */
+ if (item->ignore)
+ continue;
+
+ /* In SIGHUP cases in the postmaster, we want to report changes */
+ if (context == PGC_SIGHUP && applySettings && !IsUnderPostmaster)
+ {
+ const char *preval = GetConfigOption(item->name, true, false);
+
+ /* If option doesn't exist yet or is NULL, treat as empty string */
+ if (!preval)
+ preval = "";
+ /* must dup, else might have dangling pointer below */
+ pre_value = pstrdup(preval);
+ }
+
+ scres = set_config_option(item->name, item->value,
+ context, PGC_S_FILE,
+ GUC_ACTION_SET, applySettings, 0, false);
+ if (scres > 0)
+ {
+ /* variable was updated, so log the change if appropriate */
+ if (pre_value)
+ {
+ const char *post_value = GetConfigOption(item->name, true, false);
+
+ if (!post_value)
+ post_value = "";
+ if (strcmp(pre_value, post_value) != 0)
+ ereport(elevel,
+ (errmsg("parameter \"%s\" changed to \"%s\"",
+ item->name, item->value)));
+ }
+ item->applied = true;
+ }
+ else if (scres == 0)
+ {
+ error = true;
+ item->errmsg = pstrdup("setting could not be applied");
+ ConfFileWithError = item->filename;
+ }
+ else
+ {
+ /* no error, but variable's active value was not changed */
+ item->applied = true;
+ }
+
+ /*
+ * We should update source location unless there was an error, since
+ * even if the active value didn't change, the reset value might have.
+ * (In the postmaster, there won't be a difference, but it does matter
+ * in backends.)
+ */
+ if (scres != 0 && applySettings)
+ set_config_sourcefile(item->name, item->filename,
+ item->sourceline);
+
+ if (pre_value)
+ pfree(pre_value);
+ }
+
+ /* Remember when we last successfully loaded the config file. */
+ if (applySettings)
+ PgReloadTime = GetCurrentTimestamp();
+
+bail_out:
+ if (error && applySettings)
+ {
+ /* During postmaster startup, any error is fatal */
+ if (context == PGC_POSTMASTER)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors",
+ ConfFileWithError)));
+ else if (applying)
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; unaffected changes were applied",
+ ConfFileWithError)));
+ else
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("configuration file \"%s\" contains errors; no changes were applied",
+ ConfFileWithError)));
+ }
+
+ /* Successful or otherwise, return the collected data list */
+ return head;
+}
/*
* Some infrastructure for checking malloc/strdup/realloc calls
@@ -5737,7 +6085,7 @@ guc_var_compare(const void *a, const void *b)
/*
* the bare comparison function for GUC names
*/
-static int
+int
guc_name_compare(const char *namea, const char *nameb)
{
/*
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index e734493a48..aae071cd82 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -442,6 +442,15 @@ extern void GUC_check_errcode(int sqlerrcode);
pre_format_elog_string(errno, TEXTDOMAIN), \
GUC_check_errhint_string = format_elog_string
+/* functions shared between guc.c and guc-file.l */
+extern int guc_name_compare(const char *namea, const char *nameb);
+extern ConfigVariable *ProcessConfigFileInternal(GucContext context,
+ bool applySettings, int elevel);
+extern void record_config_file_error(const char *errmsg,
+ const char *config_file,
+ int lineno,
+ ConfigVariable **head_p,
+ ConfigVariable **tail_p);
/*
* The following functions are not in guc.c, but are declared here to avoid
--
2.36.1
[text/x-patch] v4-0003-Build-guc-file.c-standalone.patch (2.5K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/4-v4-0003-Build-guc-file.c-standalone.patch)
download | inline diff:
From 65c0203bac3abf127a5900f1a0025b7a62a52d70 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Thu, 11 Aug 2022 19:38:37 +0700
Subject: [PATCH v4 03/11] Build guc-file.c standalone
The proposed Meson build system will need a way to ignore certain
generated files in order to coexist with the autoconf build system,
and #include'd C files generated by Flex make this more difficult.
Build guc-file.c separately from guc.c, as was done in 72b1e3a21.
Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
---
src/backend/utils/misc/Makefile | 5 +----
src/backend/utils/misc/guc-file.l | 9 +++++----
src/backend/utils/misc/guc.c | 2 --
3 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/src/backend/utils/misc/Makefile b/src/backend/utils/misc/Makefile
index 1d5327cf64..cf7ce9bc83 100644
--- a/src/backend/utils/misc/Makefile
+++ b/src/backend/utils/misc/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
guc.o \
+ guc-file.o \
help_config.o \
pg_config.o \
pg_controldata.o \
@@ -37,10 +38,6 @@ endif
include $(top_srcdir)/src/backend/common.mk
-# guc-file is compiled as part of guc
-guc.o: guc-file.c
-
# Note: guc-file.c is not deleted by 'make clean',
# since we want to ship it in distribution tarballs.
clean:
- @rm -f lex.yy.c
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
index 843838b1df..9aa3abe1ba 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -1,4 +1,4 @@
-/* -*-pgsql-c-*- */
+%top{
/*
* Scanner for the configuration file
*
@@ -7,8 +7,6 @@
* src/backend/utils/misc/guc-file.l
*/
-%{
-
#include "postgres.h"
#include <ctype.h>
@@ -17,10 +15,13 @@
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "storage/fd.h"
+#include <sys/stat.h>
#include "utils/guc.h"
#include "utils/guc_internal.h"
+#include "utils/memutils.h"
+}
-
+%{
/*
* flex emits a yy_fatal_error() function that it calls in response to
* critical errors like malloc failure, file I/O errors, and detection of
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 293834fc13..92b5b18c8f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -13333,5 +13333,3 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
-
-#include "guc-file.c"
--
2.36.1
[text/x-patch] v4-0005-Build-repl_scanner.c-standalone.patch (6.1K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/5-v4-0005-Build-repl_scanner.c-standalone.patch)
download | inline diff:
From 67d63a9e186e6e393e344627c5b70f2dd7b8609e Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 12 Aug 2022 17:09:45 +0700
Subject: [PATCH v4 05/11] Build repl_scanner.c standalone
---
src/backend/Makefile | 3 ++-
src/backend/replication/.gitignore | 1 +
src/backend/replication/Makefile | 11 +++++++--
src/backend/replication/repl_gram.y | 2 --
src/backend/replication/repl_scanner.l | 31 +++++++++++++++++---------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 33 insertions(+), 16 deletions(-)
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 5a12666918..f527659a7b 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -183,7 +183,7 @@ distprep:
$(MAKE) -C bootstrap bootparse.c bootparse.h bootscanner.c
$(MAKE) -C catalog distprep
$(MAKE) -C nodes distprep
- $(MAKE) -C replication repl_gram.c repl_scanner.c syncrep_gram.c syncrep_scanner.c
+ $(MAKE) -C replication repl_gram.c repl_gram.h repl_scanner.c syncrep_gram.c syncrep_scanner.c
$(MAKE) -C storage/lmgr lwlocknames.h lwlocknames.c
$(MAKE) -C utils distprep
$(MAKE) -C utils/adt jsonpath_gram.c jsonpath_scan.c
@@ -304,6 +304,7 @@ maintainer-clean: distclean
parser/gram.h \
parser/scan.c \
replication/repl_gram.c \
+ replication/repl_gram.h \
replication/repl_scanner.c \
replication/syncrep_gram.c \
replication/syncrep_scanner.c \
diff --git a/src/backend/replication/.gitignore b/src/backend/replication/.gitignore
index d1df6147bd..a5f600232f 100644
--- a/src/backend/replication/.gitignore
+++ b/src/backend/replication/.gitignore
@@ -1,3 +1,4 @@
+/repl_gram.h
/repl_gram.c
/repl_scanner.c
/syncrep_gram.c
diff --git a/src/backend/replication/Makefile b/src/backend/replication/Makefile
index 2bffac58c0..bc8170418f 100644
--- a/src/backend/replication/Makefile
+++ b/src/backend/replication/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
repl_gram.o \
+ repl_scanner.o \
slot.o \
slotfuncs.o \
syncrep.o \
@@ -28,8 +29,14 @@ SUBDIRS = logical
include $(top_srcdir)/src/backend/common.mk
-# repl_scanner is compiled as part of repl_gram
-repl_gram.o: repl_scanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+repl_gram.h: repl_gram.c
+ touch $@
+
+repl_gram.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+repl_gram.o repl_scanner.o: repl_gram.h
# syncrep_scanner is compiled as part of syncrep_gram
syncrep_gram.o: syncrep_scanner.c
diff --git a/src/backend/replication/repl_gram.y b/src/backend/replication/repl_gram.y
index 4cf087e602..b343f108d3 100644
--- a/src/backend/replication/repl_gram.y
+++ b/src/backend/replication/repl_gram.y
@@ -416,5 +416,3 @@ ident_or_keyword:
;
%%
-
-#include "repl_scanner.c"
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.l
index 586f0d3a5c..23fcb2a11d 100644
--- a/src/backend/replication/repl_scanner.l
+++ b/src/backend/replication/repl_scanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* repl_scanner.l
@@ -18,6 +18,15 @@
#include "utils/builtins.h"
#include "parser/scansup.h"
+/*
+ * NB: include repl_gram.h only AFTER including walsender_private.h, because
+ * walsender_private includes headers that define XLogRecPtr.
+ */
+#include "replication/walsender_private.h"
+#include "repl_gram.h"
+}
+
+%{
/* Avoid exit() on fatal scanner errors (a bit ugly -- see yy_fatal_error) */
#undef fprintf
#define fprintf(file, fmt, msg) fprintf_to_ereport(fmt, msg)
@@ -130,7 +139,7 @@ WAIT { return K_WAIT; }
{space}+ { /* do nothing */ }
{digit}+ {
- yylval.uintval = strtoul(yytext, NULL, 10);
+ replication_yylval.uintval = strtoul(yytext, NULL, 10);
return UCONST;
}
@@ -138,8 +147,8 @@ WAIT { return K_WAIT; }
uint32 hi,
lo;
if (sscanf(yytext, "%X/%X", &hi, &lo) != 2)
- yyerror("invalid streaming start location");
- yylval.recptr = ((uint64) hi) << 32 | lo;
+ replication_yyerror("invalid streaming start location");
+ replication_yylval.recptr = ((uint64) hi) << 32 | lo;
return RECPTR;
}
@@ -151,7 +160,7 @@ WAIT { return K_WAIT; }
<xq>{quotestop} {
yyless(1);
BEGIN(INITIAL);
- yylval.str = litbufdup();
+ replication_yylval.str = litbufdup();
return SCONST;
}
@@ -173,9 +182,9 @@ WAIT { return K_WAIT; }
yyless(1);
BEGIN(INITIAL);
- yylval.str = litbufdup();
- len = strlen(yylval.str);
- truncate_identifier(yylval.str, len, true);
+ replication_yylval.str = litbufdup();
+ len = strlen(replication_yylval.str);
+ truncate_identifier(replication_yylval.str, len, true);
return IDENT;
}
@@ -186,7 +195,7 @@ WAIT { return K_WAIT; }
{identifier} {
int len = strlen(yytext);
- yylval.str = downcase_truncate_identifier(yytext, len, true);
+ replication_yylval.str = downcase_truncate_identifier(yytext, len, true);
return IDENT;
}
@@ -195,7 +204,7 @@ WAIT { return K_WAIT; }
return yytext[0];
}
-<xq,xd><<EOF>> { yyerror("unterminated quoted string"); }
+<xq,xd><<EOF>> { replication_yyerror("unterminated quoted string"); }
<<EOF>> {
@@ -231,7 +240,7 @@ addlitchar(unsigned char ychar)
}
void
-yyerror(const char *message)
+replication_yyerror(const char *message)
{
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index ce33ec1f68..917ef29c15 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -120,6 +120,7 @@ do
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
+ test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v4-0004-Build-bootscanner.c-standalone.patch (7.4K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/6-v4-0004-Build-bootscanner.c-standalone.patch)
download | inline diff:
From e85b6441ca47566f8edd933e4145bfb2c8f553fa Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 12 Aug 2022 15:45:24 +0700
Subject: [PATCH v4 04/11] Build bootscanner.c standalone
---
src/backend/Makefile | 3 +-
src/backend/bootstrap/.gitignore | 1 +
src/backend/bootstrap/Makefile | 11 +++++-
src/backend/bootstrap/bootparse.y | 2 -
src/backend/bootstrap/bootscanner.l | 60 ++++++++++++++++-------------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 46 insertions(+), 32 deletions(-)
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 3f01c65592..5a12666918 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -180,7 +180,7 @@ utils/probes.o: utils/probes.d $(SUBDIROBJS)
# Be sure that these files get removed by the maintainer-clean target
distprep:
$(MAKE) -C parser gram.c gram.h scan.c
- $(MAKE) -C bootstrap bootparse.c bootscanner.c
+ $(MAKE) -C bootstrap bootparse.c bootparse.h bootscanner.c
$(MAKE) -C catalog distprep
$(MAKE) -C nodes distprep
$(MAKE) -C replication repl_gram.c repl_scanner.c syncrep_gram.c syncrep_scanner.c
@@ -298,6 +298,7 @@ maintainer-clean: distclean
$(MAKE) -C nodes $@
$(MAKE) -C utils $@
rm -f bootstrap/bootparse.c \
+ bootstrap/bootparse.h \
bootstrap/bootscanner.c \
parser/gram.c \
parser/gram.h \
diff --git a/src/backend/bootstrap/.gitignore b/src/backend/bootstrap/.gitignore
index 1ffe8ca39e..6351b920fd 100644
--- a/src/backend/bootstrap/.gitignore
+++ b/src/backend/bootstrap/.gitignore
@@ -1,2 +1,3 @@
+/bootparse.h
/bootparse.c
/bootscanner.c
diff --git a/src/backend/bootstrap/Makefile b/src/backend/bootstrap/Makefile
index 6421efb227..606c8021e7 100644
--- a/src/backend/bootstrap/Makefile
+++ b/src/backend/bootstrap/Makefile
@@ -14,12 +14,19 @@ override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS)
OBJS = \
bootparse.o \
+ bootscanner.o \
bootstrap.o
include $(top_srcdir)/src/backend/common.mk
-# bootscanner is compiled as part of bootparse
-bootparse.o: bootscanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+bootparse.h: bootparse.c
+ touch $@
+
+bootparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+bootparse.o bootscanner.o: bootparse.h
# bootparse.c and bootscanner.c are in the distribution tarball, so
# they are not cleaned here.
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 7d7655d295..c45ddde67f 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -488,5 +488,3 @@ boot_ident:
| XNULL { $$ = pstrdup($1); }
;
%%
-
-#include "bootscanner.c"
diff --git a/src/backend/bootstrap/bootscanner.l b/src/backend/bootstrap/bootscanner.l
index 3094ccb93f..d6eae84816 100644
--- a/src/backend/bootstrap/bootscanner.l
+++ b/src/backend/bootstrap/bootscanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* bootscanner.l
@@ -15,11 +15,17 @@
*/
#include "postgres.h"
+/*
+ * NB: include bootparse.h only AFTER including bootstrap.h, because bootstrap.h
+ * includes node definitions needed for YYSTYPE.
+ */
#include "bootstrap/bootstrap.h"
+#include "bootparse.h"
#include "utils/guc.h"
-/* Not needed now that this file is compiled as part of bootparse. */
-/* #include "bootparse.h" */
+}
+
+%{
/* LCOV_EXCL_START */
@@ -52,7 +58,7 @@ id [-A-Za-z0-9_]+
sid \'([^']|\'\')*\'
/*
- * Keyword tokens return the keyword text (as a constant string) in yylval.kw,
+ * Keyword tokens return the keyword text (as a constant string) in boot_yylval.kw,
* just in case that's needed because we want to treat the keyword as an
* unreserved identifier. Note that _null_ is not treated as a keyword
* for this purpose; it's the one "reserved word" in the bootstrap syntax.
@@ -60,23 +66,23 @@ sid \'([^']|\'\')*\'
* Notice that all the keywords are case-sensitive, and for historical
* reasons some must be upper case.
*
- * String tokens return a palloc'd string in yylval.str.
+ * String tokens return a palloc'd string in boot_yylval.str.
*/
%%
-open { yylval.kw = "open"; return OPEN; }
+open { boot_yylval.kw = "open"; return OPEN; }
-close { yylval.kw = "close"; return XCLOSE; }
+close { boot_yylval.kw = "close"; return XCLOSE; }
-create { yylval.kw = "create"; return XCREATE; }
+create { boot_yylval.kw = "create"; return XCREATE; }
-OID { yylval.kw = "OID"; return OBJ_ID; }
-bootstrap { yylval.kw = "bootstrap"; return XBOOTSTRAP; }
-shared_relation { yylval.kw = "shared_relation"; return XSHARED_RELATION; }
-rowtype_oid { yylval.kw = "rowtype_oid"; return XROWTYPE_OID; }
+OID { boot_yylval.kw = "OID"; return OBJ_ID; }
+bootstrap { boot_yylval.kw = "bootstrap"; return XBOOTSTRAP; }
+shared_relation { boot_yylval.kw = "shared_relation"; return XSHARED_RELATION; }
+rowtype_oid { boot_yylval.kw = "rowtype_oid"; return XROWTYPE_OID; }
-insert { yylval.kw = "insert"; return INSERT_TUPLE; }
+insert { boot_yylval.kw = "insert"; return INSERT_TUPLE; }
_null_ { return NULLVAL; }
@@ -90,25 +96,25 @@ _null_ { return NULLVAL; }
^\#[^\n]* ; /* drop everything after "#" for comments */
-declare { yylval.kw = "declare"; return XDECLARE; }
-build { yylval.kw = "build"; return XBUILD; }
-indices { yylval.kw = "indices"; return INDICES; }
-unique { yylval.kw = "unique"; return UNIQUE; }
-index { yylval.kw = "index"; return INDEX; }
-on { yylval.kw = "on"; return ON; }
-using { yylval.kw = "using"; return USING; }
-toast { yylval.kw = "toast"; return XTOAST; }
-FORCE { yylval.kw = "FORCE"; return XFORCE; }
-NOT { yylval.kw = "NOT"; return XNOT; }
-NULL { yylval.kw = "NULL"; return XNULL; }
+declare { boot_yylval.kw = "declare"; return XDECLARE; }
+build { boot_yylval.kw = "build"; return XBUILD; }
+indices { boot_yylval.kw = "indices"; return INDICES; }
+unique { boot_yylval.kw = "unique"; return UNIQUE; }
+index { boot_yylval.kw = "index"; return INDEX; }
+on { boot_yylval.kw = "on"; return ON; }
+using { boot_yylval.kw = "using"; return USING; }
+toast { boot_yylval.kw = "toast"; return XTOAST; }
+FORCE { boot_yylval.kw = "FORCE"; return XFORCE; }
+NOT { boot_yylval.kw = "NOT"; return XNOT; }
+NULL { boot_yylval.kw = "NULL"; return XNULL; }
{id} {
- yylval.str = pstrdup(yytext);
+ boot_yylval.str = pstrdup(yytext);
return ID;
}
{sid} {
/* strip quotes and escapes */
- yylval.str = DeescapeQuotedString(yytext);
+ boot_yylval.str = DeescapeQuotedString(yytext);
return ID;
}
@@ -121,7 +127,7 @@ NULL { yylval.kw = "NULL"; return XNULL; }
/* LCOV_EXCL_STOP */
void
-yyerror(const char *message)
+boot_yyerror(const char *message)
{
elog(ERROR, "%s at line %d", message, yyline);
}
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index b8419e46a4..ce33ec1f68 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -119,6 +119,7 @@ do
# parser/gram.h will be included by parser/gramparse.h anyway.
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
+ test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v4-0008-Build-exprscan.c-standalone.patch (4.1K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/7-v4-0008-Build-exprscan.c-standalone.patch)
download | inline diff:
From 8a39bfb75a69847dd6d1839091f71f36fc9579ba Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 13:35:14 +0700
Subject: [PATCH v4 08/11] Build exprscan.c standalone
---
src/bin/pgbench/.gitignore | 1 +
src/bin/pgbench/Makefile | 13 ++++++++++---
src/bin/pgbench/exprparse.y | 15 ---------------
src/bin/pgbench/exprscan.l | 12 +++++++++++-
src/tools/pginclude/headerscheck | 1 +
5 files changed, 23 insertions(+), 19 deletions(-)
diff --git a/src/bin/pgbench/.gitignore b/src/bin/pgbench/.gitignore
index 983a3cd7a6..07492a993c 100644
--- a/src/bin/pgbench/.gitignore
+++ b/src/bin/pgbench/.gitignore
@@ -1,3 +1,4 @@
+/exprparse.h
/exprparse.c
/exprscan.c
/pgbench
diff --git a/src/bin/pgbench/Makefile b/src/bin/pgbench/Makefile
index f402fe7b91..6647c9fe97 100644
--- a/src/bin/pgbench/Makefile
+++ b/src/bin/pgbench/Makefile
@@ -10,6 +10,7 @@ include $(top_builddir)/src/Makefile.global
OBJS = \
$(WIN32RES) \
exprparse.o \
+ exprscan.o \
pgbench.o
override CPPFLAGS := -I. -I$(srcdir) -I$(libpq_srcdir) $(CPPFLAGS)
@@ -26,8 +27,14 @@ all: pgbench
pgbench: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-# exprscan is compiled as part of exprparse
-exprparse.o: exprscan.c
+# See notes in src/backend/parser/Makefile about the following two rules
+exprparse.h: exprparse.c
+ touch $@
+
+exprparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+exprparse.o exprscan.o: exprparse.h
distprep: exprparse.c exprscan.c
@@ -45,7 +52,7 @@ clean distclean:
rm -rf tmp_check
maintainer-clean: distclean
- rm -f exprparse.c exprscan.c
+ rm -f exprparse.h exprparse.c exprscan.c
check:
$(prove_check)
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b5592d4b97..ade2ecdaab 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -526,18 +526,3 @@ make_case(yyscan_t yyscanner, PgBenchExprList *when_then_list, PgBenchExpr *else
find_func(yyscanner, "!case_end"),
make_elist(else_part, when_then_list));
}
-
-/*
- * exprscan.l is compiled as part of exprparse.y. Currently, this is
- * unavoidable because exprparse does not create a .h file to export
- * its token symbols. If these files ever grow large enough to be
- * worth compiling separately, that could be fixed; but for now it
- * seems like useless complication.
- */
-
-/* First, get rid of "#define yyscan_t" from pgbench.h */
-#undef yyscan_t
-/* ... and the yylval macro, which flex will have its own definition for */
-#undef yylval
-
-#include "exprscan.c"
diff --git a/src/bin/pgbench/exprscan.l b/src/bin/pgbench/exprscan.l
index 4f63818606..fe8e32838a 100644
--- a/src/bin/pgbench/exprscan.l
+++ b/src/bin/pgbench/exprscan.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* exprscan.l
@@ -22,9 +22,19 @@
*
*-------------------------------------------------------------------------
*/
+#include "postgres_fe.h"
+/*
+ * NB: include exprparse.h only AFTER including pgbench.h, because pgbench.h
+ * contains definitions needed for YYSTYPE. Likewise, pgbench.h must come after
+ * psqlscan_int.h for yyscan_t.
+ */
#include "fe_utils/psqlscan_int.h"
+#include "pgbench.h"
+#include "exprparse.h"
+}
+%{
/* context information for reporting errors in expressions */
static const char *expr_source = NULL;
static int expr_lineno = 0;
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index c1b1f3bd2a..1a56a74cda 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -123,6 +123,7 @@ do
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
test "$f" = src/test/isolation/specparse.h && continue
+ test "$f" = src/bin/pgbench/exprparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v4-0006-Build-syncrep_scanner.c-standalone.patch (5.2K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/8-v4-0006-Build-syncrep_scanner.c-standalone.patch)
download | inline diff:
From b8f2e6a055af9f2bffd2078364077fe6d2942e53 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 15:02:30 +0700
Subject: [PATCH v4 06/11] Build syncrep_scanner.c standalone
---
src/backend/Makefile | 3 ++-
src/backend/replication/.gitignore | 1 +
src/backend/replication/Makefile | 11 +++++++++--
src/backend/replication/syncrep_gram.y | 2 --
src/backend/replication/syncrep_scanner.l | 21 +++++++++++++++------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/src/backend/Makefile b/src/backend/Makefile
index f527659a7b..86cbe03677 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -183,7 +183,7 @@ distprep:
$(MAKE) -C bootstrap bootparse.c bootparse.h bootscanner.c
$(MAKE) -C catalog distprep
$(MAKE) -C nodes distprep
- $(MAKE) -C replication repl_gram.c repl_gram.h repl_scanner.c syncrep_gram.c syncrep_scanner.c
+ $(MAKE) -C replication repl_gram.c repl_gram.h repl_scanner.c syncrep_gram.c syncrep_gram.h syncrep_scanner.c
$(MAKE) -C storage/lmgr lwlocknames.h lwlocknames.c
$(MAKE) -C utils distprep
$(MAKE) -C utils/adt jsonpath_gram.c jsonpath_scan.c
@@ -307,6 +307,7 @@ maintainer-clean: distclean
replication/repl_gram.h \
replication/repl_scanner.c \
replication/syncrep_gram.c \
+ replication/syncrep_gram.h \
replication/syncrep_scanner.c \
storage/lmgr/lwlocknames.c \
storage/lmgr/lwlocknames.h \
diff --git a/src/backend/replication/.gitignore b/src/backend/replication/.gitignore
index a5f600232f..77d5a51068 100644
--- a/src/backend/replication/.gitignore
+++ b/src/backend/replication/.gitignore
@@ -1,5 +1,6 @@
/repl_gram.h
/repl_gram.c
/repl_scanner.c
+/syncrep_gram.h
/syncrep_gram.c
/syncrep_scanner.c
diff --git a/src/backend/replication/Makefile b/src/backend/replication/Makefile
index bc8170418f..23f29ba545 100644
--- a/src/backend/replication/Makefile
+++ b/src/backend/replication/Makefile
@@ -21,6 +21,7 @@ OBJS = \
slotfuncs.o \
syncrep.o \
syncrep_gram.o \
+ syncrep_scanner.o \
walreceiver.o \
walreceiverfuncs.o \
walsender.o
@@ -38,8 +39,14 @@ repl_gram.c: BISONFLAGS += -d
# Force these dependencies to be known even without dependency info built:
repl_gram.o repl_scanner.o: repl_gram.h
-# syncrep_scanner is compiled as part of syncrep_gram
-syncrep_gram.o: syncrep_scanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+syncrep_gram.h: syncrep_gram.c
+ touch $@
+
+syncrep_gram.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+syncrep_gram.o syncrep_scanner.o: syncrep_gram.h
# repl_gram.c, repl_scanner.c, syncrep_gram.c and syncrep_scanner.c
# are in the distribution tarball, so they are not cleaned here.
diff --git a/src/backend/replication/syncrep_gram.y b/src/backend/replication/syncrep_gram.y
index d932f2cda3..4fc3647da1 100644
--- a/src/backend/replication/syncrep_gram.y
+++ b/src/backend/replication/syncrep_gram.y
@@ -112,5 +112,3 @@ create_syncrep_config(const char *num_sync, List *members, uint8 syncrep_method)
return config;
}
-
-#include "syncrep_scanner.c"
diff --git a/src/backend/replication/syncrep_scanner.l b/src/backend/replication/syncrep_scanner.l
index 1952c8c6e0..bdb1a3391c 100644
--- a/src/backend/replication/syncrep_scanner.l
+++ b/src/backend/replication/syncrep_scanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* syncrep_scanner.l
@@ -17,6 +17,15 @@
#include "lib/stringinfo.h"
+/*
+ * NB: include syncrep_gram.h only AFTER including syncrep.h, because syncrep.h
+ * includes node definitions needed for YYSTYPE.
+ */
+#include "replication/syncrep.h"
+#include "syncrep_gram.h"
+}
+
+%{
/* Avoid exit() on fatal scanner errors (a bit ugly -- see yy_fatal_error) */
#undef fprintf
#define fprintf(file, fmt, msg) fprintf_to_ereport(fmt, msg)
@@ -82,28 +91,28 @@ xdinside [^"]+
appendStringInfoString(&xdbuf, yytext);
}
<xd>{xdstop} {
- yylval.str = xdbuf.data;
+ syncrep_yylval.str = xdbuf.data;
xdbuf.data = NULL;
BEGIN(INITIAL);
return NAME;
}
<xd><<EOF>> {
- yyerror("unterminated quoted identifier");
+ syncrep_yyerror("unterminated quoted identifier");
return JUNK;
}
{identifier} {
- yylval.str = pstrdup(yytext);
+ syncrep_yylval.str = pstrdup(yytext);
return NAME;
}
{digit}+ {
- yylval.str = pstrdup(yytext);
+ syncrep_yylval.str = pstrdup(yytext);
return NUM;
}
"*" {
- yylval.str = "*";
+ syncrep_yylval.str = "*";
return NAME;
}
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 917ef29c15..1e953d503f 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -121,6 +121,7 @@ do
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
+ test "$f" = src/backend/replication/syncrep_gram.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v4-0009-Build-cubescan.c-standalone.patch (7.5K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/9-v4-0009-Build-cubescan.c-standalone.patch)
download | inline diff:
From 4ec5efdae1b02b4e0a34d5deffa5dcbb119445be Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 11:18:06 +0700
Subject: [PATCH v4 09/11] Build cubescan.c standalone
Pass scanbuflen as a parameter to yyparse rather than
resorting to a global variable.
---
contrib/cube/.gitignore | 1 +
contrib/cube/Makefile | 16 +++++++-----
contrib/cube/cube.c | 6 ++---
contrib/cube/cubedata.h | 6 ++---
contrib/cube/cubeparse.y | 8 ++----
contrib/cube/cubescan.l | 44 ++++++++++++++++++++------------
src/tools/pginclude/headerscheck | 1 +
7 files changed, 47 insertions(+), 35 deletions(-)
diff --git a/contrib/cube/.gitignore b/contrib/cube/.gitignore
index cb4c989fff..f788440c79 100644
--- a/contrib/cube/.gitignore
+++ b/contrib/cube/.gitignore
@@ -1,3 +1,4 @@
+/cubeparse.h
/cubeparse.c
/cubescan.c
# Generated subdirectories
diff --git a/contrib/cube/Makefile b/contrib/cube/Makefile
index cf195506c7..4fd19aac35 100644
--- a/contrib/cube/Makefile
+++ b/contrib/cube/Makefile
@@ -4,7 +4,8 @@ MODULE_big = cube
OBJS = \
$(WIN32RES) \
cube.o \
- cubeparse.o
+ cubeparse.o \
+ cubescan.o
EXTENSION = cube
DATA = cube--1.2.sql cube--1.2--1.3.sql cube--1.3--1.4.sql cube--1.4--1.5.sql \
@@ -15,8 +16,6 @@ HEADERS = cubedata.h
REGRESS = cube cube_sci
-EXTRA_CLEAN = y.tab.c y.tab.h
-
SHLIB_LINK += $(filter -lm, $(LIBS))
ifdef USE_PGXS
@@ -30,11 +29,16 @@ include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
+# See notes in src/backend/parser/Makefile about the following two rules
+cubeparse.h: cubeparse.c
+ touch $@
+
+cubeparse.c: BISONFLAGS += -d
-# cubescan is compiled as part of cubeparse
-cubeparse.o: cubescan.c
+# Force these dependencies to be known even without dependency info built:
+cubeparse.o cubescan.o: cubeparse.h
distprep: cubeparse.c cubescan.c
maintainer-clean:
- rm -f cubeparse.c cubescan.c
+ rm -f cubeparse.h cubeparse.c cubescan.c
diff --git a/contrib/cube/cube.c b/contrib/cube/cube.c
index a5d1ba6733..6e01800a4a 100644
--- a/contrib/cube/cube.c
+++ b/contrib/cube/cube.c
@@ -119,11 +119,11 @@ cube_in(PG_FUNCTION_ARGS)
{
char *str = PG_GETARG_CSTRING(0);
NDBOX *result;
+ Size scanbuflen;
- cube_scanner_init(str);
+ cube_scanner_init(str, &scanbuflen);
- if (cube_yyparse(&result) != 0)
- cube_yyerror(&result, "cube parser failed");
+ cube_yyparse(&result, scanbuflen);
cube_scanner_finish();
diff --git a/contrib/cube/cubedata.h b/contrib/cube/cubedata.h
index dbe7d4f742..640a7ca580 100644
--- a/contrib/cube/cubedata.h
+++ b/contrib/cube/cubedata.h
@@ -61,9 +61,9 @@ typedef struct NDBOX
/* in cubescan.l */
extern int cube_yylex(void);
-extern void cube_yyerror(NDBOX **result, const char *message) pg_attribute_noreturn();
-extern void cube_scanner_init(const char *str);
+extern void cube_yyerror(NDBOX **result, Size scanbuflen, const char *message) pg_attribute_noreturn();
+extern void cube_scanner_init(const char *str, Size *scanbuflen);
extern void cube_scanner_finish(void);
/* in cubeparse.y */
-extern int cube_yyparse(NDBOX **result);
+extern int cube_yyparse(NDBOX **result, Size scanbuflen);
diff --git a/contrib/cube/cubeparse.y b/contrib/cube/cubeparse.y
index 7577c4515c..e3b750b695 100644
--- a/contrib/cube/cubeparse.y
+++ b/contrib/cube/cubeparse.y
@@ -7,6 +7,7 @@
#include "postgres.h"
#include "cubedata.h"
+#include "cube_internal.h"
#include "utils/float.h"
/* All grammar constructs return strings */
@@ -23,9 +24,6 @@
#define YYMALLOC palloc
#define YYFREE pfree
-static char *scanbuf;
-static int scanbuflen;
-
static int item_count(const char *s, char delim);
static NDBOX *write_box(int dim, char *str1, char *str2);
static NDBOX *write_point_as_box(int dim, char *str);
@@ -33,7 +31,7 @@ static NDBOX *write_point_as_box(int dim, char *str);
%}
/* BISON Declarations */
-%parse-param {NDBOX **result}
+%parse-param {NDBOX **result} {Size scanbuflen}
%expect 0
%name-prefix="cube_yy"
@@ -265,5 +263,3 @@ write_point_as_box(int dim, char *str)
return bp;
}
-
-#include "cubescan.c"
diff --git a/contrib/cube/cubescan.l b/contrib/cube/cubescan.l
index bd400e3684..6b316f2d54 100644
--- a/contrib/cube/cubescan.l
+++ b/contrib/cube/cubescan.l
@@ -1,9 +1,21 @@
-%{
+%top{
/*
* A scanner for EMP-style numeric ranges
* contrib/cube/cubescan.l
*/
+#include "postgres.h"
+
+/*
+ * NB: include cubeparse.h only AFTER defining YYSTYPE (to match cubeparse.y)
+ * and cubedata.h for NDBOX.
+ */
+#include "cubedata.h"
+#define YYSTYPE char *
+#include "cubeparse.h"
+}
+
+%{
/* LCOV_EXCL_START */
/* No reason to constrain amount of data slurped */
@@ -21,9 +33,7 @@ fprintf_to_ereport(const char *fmt, const char *msg)
/* Handles to the buffer that the lexer uses internally */
static YY_BUFFER_STATE scanbufhandle;
-/* this is now declared in cubeparse.y: */
-/* static char *scanbuf; */
-/* static int scanbuflen; */
+static char *scanbuf;
%}
%option 8bit
@@ -45,14 +55,14 @@ NaN [nN][aA][nN]
%%
-{float} yylval = yytext; return CUBEFLOAT;
-{infinity} yylval = yytext; return CUBEFLOAT;
-{NaN} yylval = yytext; return CUBEFLOAT;
-\[ yylval = "("; return O_BRACKET;
-\] yylval = ")"; return C_BRACKET;
-\( yylval = "("; return O_PAREN;
-\) yylval = ")"; return C_PAREN;
-\, yylval = ","; return COMMA;
+{float} cube_yylval = yytext; return CUBEFLOAT;
+{infinity} cube_yylval = yytext; return CUBEFLOAT;
+{NaN} cube_yylval = yytext; return CUBEFLOAT;
+\[ cube_yylval = "("; return O_BRACKET;
+\] cube_yylval = ")"; return C_BRACKET;
+\( cube_yylval = "("; return O_PAREN;
+\) cube_yylval = ")"; return C_PAREN;
+\, cube_yylval = ","; return COMMA;
[ \t\n\r\f]+ /* discard spaces */
. return yytext[0]; /* alert parser of the garbage */
@@ -60,9 +70,9 @@ NaN [nN][aA][nN]
/* LCOV_EXCL_STOP */
-/* result is not used, but Bison expects this signature */
+/* result and scanbuflen are not used, but Bison expects this signature */
void
-yyerror(NDBOX **result, const char *message)
+cube_yyerror(NDBOX **result, Size scanbuflen, const char *message)
{
if (*yytext == YY_END_OF_BUFFER_CHAR)
{
@@ -87,9 +97,9 @@ yyerror(NDBOX **result, const char *message)
* Called before any actual parsing is done
*/
void
-cube_scanner_init(const char *str)
+cube_scanner_init(const char *str, Size *scanbuflen)
{
- Size slen = strlen(str);
+ Size slen = strlen(str);
/*
* Might be left over after ereport()
@@ -100,7 +110,7 @@ cube_scanner_init(const char *str)
/*
* Make a scan buffer with special termination needed by flex.
*/
- scanbuflen = slen;
+ *scanbuflen = slen;
scanbuf = palloc(slen + 2);
memcpy(scanbuf, str, slen);
scanbuf[slen] = scanbuf[slen + 1] = YY_END_OF_BUFFER_CHAR;
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 1a56a74cda..c3f3a0acee 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -117,6 +117,7 @@ do
# We can't make these Bison output files compilable standalone
# without using "%code require", which old Bison versions lack.
# parser/gram.h will be included by parser/gramparse.h anyway.
+ test "$f" = contrib/cube/cubeparse.h && continue
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
--
2.36.1
[text/x-patch] v4-0010-Build-segscan.c-standalone.patch (4.5K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/10-v4-0010-Build-segscan.c-standalone.patch)
download | inline diff:
From dac7ffb5dd095c83570b87b47fb0c60869bf7064 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 12:00:33 +0700
Subject: [PATCH v4 10/11] Build segscan.c standalone
---
contrib/seg/.gitignore | 1 +
contrib/seg/Makefile | 15 +++++++++++----
contrib/seg/segparse.y | 3 ---
contrib/seg/segscan.l | 28 ++++++++++++++++++----------
src/tools/pginclude/headerscheck | 1 +
5 files changed, 31 insertions(+), 17 deletions(-)
diff --git a/contrib/seg/.gitignore b/contrib/seg/.gitignore
index 69e73d2096..fa247a4e67 100644
--- a/contrib/seg/.gitignore
+++ b/contrib/seg/.gitignore
@@ -1,3 +1,4 @@
+/segparse.h
/segparse.c
/segscan.c
# Generated subdirectories
diff --git a/contrib/seg/Makefile b/contrib/seg/Makefile
index bb63e83506..c6c134b8f1 100644
--- a/contrib/seg/Makefile
+++ b/contrib/seg/Makefile
@@ -4,7 +4,8 @@ MODULE_big = seg
OBJS = \
$(WIN32RES) \
seg.o \
- segparse.o
+ segparse.o \
+ segscan.o
EXTENSION = seg
DATA = seg--1.1.sql seg--1.1--1.2.sql seg--1.2--1.3.sql seg--1.3--1.4.sql \
@@ -29,10 +30,16 @@ include $(top_srcdir)/contrib/contrib-global.mk
endif
-# segscan is compiled as part of segparse
-segparse.o: segscan.c
+# See notes in src/backend/parser/Makefile about the following two rules
+segparse.h: segparse.c
+ touch $@
+
+segparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+segparse.o segscan.o: segparse.h
distprep: segparse.c segscan.c
maintainer-clean:
- rm -f segparse.c segscan.c
+ rm -f segparse.h segparse.c segscan.c
diff --git a/contrib/seg/segparse.y b/contrib/seg/segparse.y
index 33e3a9f35f..637eacd1a6 100644
--- a/contrib/seg/segparse.y
+++ b/contrib/seg/segparse.y
@@ -160,6 +160,3 @@ seg_atof(const char *value)
datum = DirectFunctionCall1(float4in, CStringGetDatum(value));
return DatumGetFloat4(datum);
}
-
-
-#include "segscan.c"
diff --git a/contrib/seg/segscan.l b/contrib/seg/segscan.l
index 5f6595e9eb..4744fd5e9e 100644
--- a/contrib/seg/segscan.l
+++ b/contrib/seg/segscan.l
@@ -1,8 +1,18 @@
-%{
+%top{
/*
* A scanner for EMP-style numeric ranges
*/
+#include "postgres.h"
+
+/*
+ * NB: include segparse.h only AFTER including segdata.h, because segdata.h
+ * contains the definition for SEG.
+ */
+#include "segdata.h"
+#include "segparse.h"
+}
+%{
/* LCOV_EXCL_START */
/* No reason to constrain amount of data slurped */
@@ -21,7 +31,6 @@ fprintf_to_ereport(const char *fmt, const char *msg)
/* Handles to the buffer that the lexer uses internally */
static YY_BUFFER_STATE scanbufhandle;
static char *scanbuf;
-static int scanbuflen;
%}
%option 8bit
@@ -42,12 +51,12 @@ float ({integer}|{real})([eE]{integer})?
%%
-{range} yylval.text = yytext; return RANGE;
-{plumin} yylval.text = yytext; return PLUMIN;
-{float} yylval.text = yytext; return SEGFLOAT;
-\< yylval.text = "<"; return EXTENSION;
-\> yylval.text = ">"; return EXTENSION;
-\~ yylval.text = "~"; return EXTENSION;
+{range} seg_yylval.text = yytext; return RANGE;
+{plumin} seg_yylval.text = yytext; return PLUMIN;
+{float} seg_yylval.text = yytext; return SEGFLOAT;
+\< seg_yylval.text = "<"; return EXTENSION;
+\> seg_yylval.text = ">"; return EXTENSION;
+\~ seg_yylval.text = "~"; return EXTENSION;
[ \t\n\r\f]+ /* discard spaces */
. return yytext[0]; /* alert parser of the garbage */
@@ -56,7 +65,7 @@ float ({integer}|{real})([eE]{integer})?
/* LCOV_EXCL_STOP */
void
-yyerror(SEG *result, const char *message)
+seg_yyerror(SEG *result, const char *message)
{
if (*yytext == YY_END_OF_BUFFER_CHAR)
{
@@ -94,7 +103,6 @@ seg_scanner_init(const char *str)
/*
* Make a scan buffer with special termination needed by flex.
*/
- scanbuflen = slen;
scanbuf = palloc(slen + 2);
memcpy(scanbuf, str, slen);
scanbuf[slen] = scanbuf[slen + 1] = YY_END_OF_BUFFER_CHAR;
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index c3f3a0acee..7022ac6c39 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -118,6 +118,7 @@ do
# without using "%code require", which old Bison versions lack.
# parser/gram.h will be included by parser/gramparse.h anyway.
test "$f" = contrib/cube/cubeparse.h && continue
+ test "$f" = contrib/seg/segparse.h && continue
test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
--
2.36.1
[text/x-patch] v4-0007-Build-specscanner.c-standalone.patch (5.3K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/11-v4-0007-Build-specscanner.c-standalone.patch)
download | inline diff:
From 9ac807e74024d6c2e989763bb1dda15f1ad5f41a Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 09:34:17 +0700
Subject: [PATCH v4 07/11] Build specscanner.c standalone
---
src/test/isolation/.gitignore | 1 +
src/test/isolation/Makefile | 15 +++++++++++----
src/test/isolation/specparse.y | 2 --
src/test/isolation/specscanner.l | 28 +++++++++++++++++++---------
src/tools/pginclude/headerscheck | 1 +
5 files changed, 32 insertions(+), 15 deletions(-)
diff --git a/src/test/isolation/.gitignore b/src/test/isolation/.gitignore
index 870dac4d28..2c13b4bf98 100644
--- a/src/test/isolation/.gitignore
+++ b/src/test/isolation/.gitignore
@@ -3,6 +3,7 @@
/pg_isolation_regress
# Local generated source files
+/specparse.h
/specparse.c
/specscanner.c
diff --git a/src/test/isolation/Makefile b/src/test/isolation/Makefile
index 0d452c89d4..b8738b7c1b 100644
--- a/src/test/isolation/Makefile
+++ b/src/test/isolation/Makefile
@@ -15,7 +15,8 @@ override CPPFLAGS := -I. -I$(srcdir) -I$(libpq_srcdir) \
OBJS = \
$(WIN32RES) \
isolationtester.o \
- specparse.o
+ specparse.o \
+ specscanner.o
all: isolationtester$(X) pg_isolation_regress$(X)
@@ -44,8 +45,14 @@ isolationtester$(X): $(OBJS) | submake-libpq submake-libpgport
distprep: specparse.c specscanner.c
-# specscanner is compiled as part of specparse
-specparse.o: specscanner.c
+# See notes in src/backend/parser/Makefile about the following two rules
+specparse.h: specparse.c
+ touch $@
+
+specparse.c: BISONFLAGS += -d
+
+# Force these dependencies to be known even without dependency info built:
+specparse.o specscanner.o: specparse.h
# specparse.c and specscanner.c are in the distribution tarball,
# so do not clean them here
@@ -55,7 +62,7 @@ clean distclean:
rm -rf $(pg_regress_clean_files)
maintainer-clean: distclean
- rm -f specparse.c specscanner.c
+ rm -f specparse.h specparse.c specscanner.c
installcheck: all
$(pg_isolation_regress_installcheck) --schedule=$(srcdir)/isolation_schedule
diff --git a/src/test/isolation/specparse.y b/src/test/isolation/specparse.y
index eb368184b8..657285cc23 100644
--- a/src/test/isolation/specparse.y
+++ b/src/test/isolation/specparse.y
@@ -276,5 +276,3 @@ blocker:
;
%%
-
-#include "specscanner.c"
diff --git a/src/test/isolation/specscanner.l b/src/test/isolation/specscanner.l
index aa6e89268e..b04696f52d 100644
--- a/src/test/isolation/specscanner.l
+++ b/src/test/isolation/specscanner.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* specscanner.l
@@ -9,7 +9,17 @@
*
*-------------------------------------------------------------------------
*/
+#include "postgres_fe.h"
+/*
+ * NB: include specparse.h only AFTER including isolationtester.h, because
+ * isolationtester.h includes node definitions needed for YYSTYPE.
+ */
+#include "isolationtester.h"
+#include "specparse.h"
+}
+
+%{
static int yyline = 1; /* line number for error reporting */
#define LITBUF_INIT 1024 /* initial size of litbuf */
@@ -75,7 +85,7 @@ teardown { return TEARDOWN; }
/* Plain identifiers */
{identifier} {
- yylval.str = pg_strdup(yytext);
+ spec_yylval.str = pg_strdup(yytext);
return(identifier);
}
@@ -87,13 +97,13 @@ teardown { return TEARDOWN; }
<qident>\"\" { addlitchar(yytext[0]); }
<qident>\" {
litbuf[litbufpos] = '\0';
- yylval.str = pg_strdup(litbuf);
+ spec_yylval.str = pg_strdup(litbuf);
BEGIN(INITIAL);
return(identifier);
}
<qident>. { addlitchar(yytext[0]); }
-<qident>\n { yyerror("unexpected newline in quoted identifier"); }
-<qident><<EOF>> { yyerror("unterminated quoted identifier"); }
+<qident>\n { spec_yyerror("unexpected newline in quoted identifier"); }
+<qident><<EOF>> { spec_yyerror("unterminated quoted identifier"); }
/* SQL blocks: { UPDATE ... } */
/* We trim leading/trailing whitespace, otherwise they're unprocessed */
@@ -104,7 +114,7 @@ teardown { return TEARDOWN; }
}
<sql>{space}*"}" {
litbuf[litbufpos] = '\0';
- yylval.str = pg_strdup(litbuf);
+ spec_yylval.str = pg_strdup(litbuf);
BEGIN(INITIAL);
return(sqlblock);
}
@@ -116,12 +126,12 @@ teardown { return TEARDOWN; }
addlitchar(yytext[0]);
}
<sql><<EOF>> {
- yyerror("unterminated sql block");
+ spec_yyerror("unterminated sql block");
}
/* Numbers and punctuation */
{digit}+ {
- yylval.integer = atoi(yytext);
+ spec_yylval.integer = atoi(yytext);
return INTEGER;
}
@@ -150,7 +160,7 @@ addlitchar(char c)
}
void
-yyerror(const char *message)
+spec_yyerror(const char *message)
{
fprintf(stderr, "%s at line %d\n", message, yyline);
exit(1);
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 1e953d503f..c1b1f3bd2a 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -122,6 +122,7 @@ do
test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
+ test "$f" = src/test/isolation/specparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
test "$f" = src/interfaces/ecpg/preproc/preproc.h && continue
--
2.36.1
[text/x-patch] v4-0011-Build-jsonpath_scan.c-standalone.patch (7.8K, ../../CAFBsxsHdAP0U3AjYuJ_SpWBxj2vmpdqXX3Yg=TLBrq0rAWhFgg@mail.gmail.com/12-v4-0011-Build-jsonpath_scan.c-standalone.patch)
download | inline diff:
From 697b7d18f59b51ebc40672e18791b8deccc3733c Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Sat, 13 Aug 2022 12:35:55 +0700
Subject: [PATCH v4 11/11] Build jsonpath_scan.c standalone
---
src/backend/utils/adt/.gitignore | 1 +
src/backend/utils/adt/Makefile | 11 ++++++--
src/backend/utils/adt/jsonpath_gram.y | 27 +------------------
src/backend/utils/adt/jsonpath_internal.h | 32 +++++++++++++++++++++++
src/backend/utils/adt/jsonpath_scan.l | 29 +++++++++++++-------
src/tools/pginclude/headerscheck | 1 +
6 files changed, 63 insertions(+), 38 deletions(-)
create mode 100644 src/backend/utils/adt/jsonpath_internal.h
diff --git a/src/backend/utils/adt/.gitignore b/src/backend/utils/adt/.gitignore
index 48cf941a52..7fab054407 100644
--- a/src/backend/utils/adt/.gitignore
+++ b/src/backend/utils/adt/.gitignore
@@ -1,2 +1,3 @@
+/jsonpath_gram.h
/jsonpath_gram.c
/jsonpath_scan.c
diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile
index 7c722ea2ce..0de0bbb1b8 100644
--- a/src/backend/utils/adt/Makefile
+++ b/src/backend/utils/adt/Makefile
@@ -57,6 +57,7 @@ OBJS = \
jsonpath.o \
jsonpath_exec.o \
jsonpath_gram.o \
+ jsonpath_scan.o \
like.o \
like_support.o \
lockfuncs.o \
@@ -119,11 +120,17 @@ OBJS = \
xid8funcs.o \
xml.o
+# See notes in src/backend/parser/Makefile about the following two rules
+jsonpath_gram.h: jsonpath_gram.c
+ touch $@
+
+jsonpath_gram.c: BISONFLAGS += -d
+
jsonpath_scan.c: FLEXFLAGS = -CF -p -p
jsonpath_scan.c: FLEX_NO_BACKUP=yes
-# jsonpath_scan is compiled as part of jsonpath_gram
-jsonpath_gram.o: jsonpath_scan.c
+# Force these dependencies to be known even without dependency info built:
+jsonpath_gram.o jsonpath_scan.o: jsonpath_gram.h
# jsonpath_gram.c and jsonpath_scan.c are in the distribution tarball,
# so they are not cleaned here.
diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y
index ce5d5af891..35a79ca965 100644
--- a/src/backend/utils/adt/jsonpath_gram.y
+++ b/src/backend/utils/adt/jsonpath_gram.y
@@ -18,26 +18,11 @@
#include "catalog/pg_collation.h"
#include "fmgr.h"
+#include "jsonpath_internal.h"
#include "miscadmin.h"
#include "nodes/pg_list.h"
#include "regex/regex.h"
#include "utils/builtins.h"
-#include "utils/jsonpath.h"
-
-/* struct JsonPathString is shared between scan and gram */
-typedef struct JsonPathString
-{
- char *val;
- int len;
- int total;
-} JsonPathString;
-
-union YYSTYPE;
-
-/* flex 2.5.4 doesn't bother with a decl for this */
-int jsonpath_yylex(union YYSTYPE *yylval_param);
-int jsonpath_yyparse(JsonPathParseResult **result);
-void jsonpath_yyerror(JsonPathParseResult **result, const char *message);
static JsonPathParseItem *makeItemType(JsonPathItemType type);
static JsonPathParseItem *makeItemString(JsonPathString *s);
@@ -593,13 +578,3 @@ jspConvertRegexFlags(uint32 xflags)
return cflags;
}
-
-/*
- * jsonpath_scan.l is compiled as part of jsonpath_gram.y. Currently, this is
- * unavoidable because jsonpath_gram does not create a .h file to export its
- * token symbols. If these files ever grow large enough to be worth compiling
- * separately, that could be fixed; but for now it seems like useless
- * complication.
- */
-
-#include "jsonpath_scan.c"
diff --git a/src/backend/utils/adt/jsonpath_internal.h b/src/backend/utils/adt/jsonpath_internal.h
new file mode 100644
index 0000000000..edfc6191a0
--- /dev/null
+++ b/src/backend/utils/adt/jsonpath_internal.h
@@ -0,0 +1,32 @@
+/*-------------------------------------------------------------------------
+ *
+ * jsonpath_internal.h
+ * Private definitions for jsonpath scanner & parser
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/utils/adt/jsonpath_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef JSONPATH_INTERNAL_H
+#define JSONPATH_INTERNAL_H
+
+/* struct JsonPathString is shared between scan and gram */
+typedef struct JsonPathString
+{
+ char *val;
+ int len;
+ int total;
+} JsonPathString;
+
+#include "utils/jsonpath.h"
+#include "jsonpath_gram.h"
+
+extern int jsonpath_yylex(YYSTYPE *yylval_param);
+extern int jsonpath_yyparse(JsonPathParseResult **result);
+extern void jsonpath_yyerror(JsonPathParseResult **result, const char *message);
+
+#endif /* JSONPATH_INTERNAL_H */
diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l
index 4351f6ec98..ea824bae73 100644
--- a/src/backend/utils/adt/jsonpath_scan.l
+++ b/src/backend/utils/adt/jsonpath_scan.l
@@ -1,4 +1,4 @@
-%{
+%top{
/*-------------------------------------------------------------------------
*
* jsonpath_scan.l
@@ -17,9 +17,18 @@
#include "postgres.h"
+/*
+ * NB: include jsonpath_gram.h only AFTER including jsonpath_internal.h,
+ * because jsonpath_internal.h contains the declaration for JsonPathString.
+ */
+#include "jsonpath_internal.h"
+#include "jsonpath_gram.h"
+
#include "mb/pg_wchar.h"
#include "nodes/pg_list.h"
+}
+%{
static JsonPathString scanstring;
/* Handles to the buffer that the lexer uses internally */
@@ -142,9 +151,9 @@ hex_fail \\x{hex_dig}{0,1}
<xnq,xq,xvq>{hex_char} { parseHexChar(yytext); }
-<xnq,xq,xvq>{unicode}*{unicodefail} { yyerror(NULL, "invalid unicode sequence"); }
+<xnq,xq,xvq>{unicode}*{unicodefail} { jsonpath_yyerror(NULL, "invalid unicode sequence"); }
-<xnq,xq,xvq>{hex_fail} { yyerror(NULL, "invalid hex character sequence"); }
+<xnq,xq,xvq>{hex_fail} { jsonpath_yyerror(NULL, "invalid hex character sequence"); }
<xnq,xq,xvq>{unicode}+\\ {
/* throw back the \\, and treat as unicode */
@@ -154,9 +163,9 @@ hex_fail \\x{hex_dig}{0,1}
<xnq,xq,xvq>\\. { addchar(false, yytext[1]); }
-<xnq,xq,xvq>\\ { yyerror(NULL, "unexpected end after backslash"); }
+<xnq,xq,xvq>\\ { jsonpath_yyerror(NULL, "unexpected end after backslash"); }
-<xq,xvq><<EOF>> { yyerror(NULL, "unexpected end of quoted string"); }
+<xq,xvq><<EOF>> { jsonpath_yyerror(NULL, "unexpected end of quoted string"); }
<xq>\" {
yylval->str = scanstring;
@@ -178,7 +187,7 @@ hex_fail \\x{hex_dig}{0,1}
<xc>\* { }
-<xc><<EOF>> { yyerror(NULL, "unexpected end of comment"); }
+<xc><<EOF>> { jsonpath_yyerror(NULL, "unexpected end of comment"); }
\&\& { return AND_P; }
@@ -244,10 +253,10 @@ hex_fail \\x{hex_dig}{0,1}
return INT_P;
}
-{realfail} { yyerror(NULL, "invalid numeric literal"); }
-{integer_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
-{decimal_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
-{real_junk} { yyerror(NULL, "trailing junk after numeric literal"); }
+{realfail} { jsonpath_yyerror(NULL, "invalid numeric literal"); }
+{integer_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
+{decimal_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
+{real_junk} { jsonpath_yyerror(NULL, "trailing junk after numeric literal"); }
\" {
addchar(true, '\0');
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 7022ac6c39..1db0096758 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -124,6 +124,7 @@ do
test "$f" = src/backend/bootstrap/bootparse.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
+ test "$f" = src/backend/utils/adt/jsonpath_gram.h && continue
test "$f" = src/test/isolation/specparse.h && continue
test "$f" = src/bin/pgbench/exprparse.h && continue
test "$f" = src/pl/plpgsql/src/pl_gram.h && continue
--
2.36.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-08-18 08:00 John Naylor <[email protected]>
parent: John Naylor <[email protected]>
1 sibling, 0 replies; 27+ messages in thread
From: John Naylor @ 2022-08-18 08:00 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
I wrote
> [v4]
This piece is a leftover from the last version, and forgot to remove
it, will fix:
diff --git a/contrib/cube/cubeparse.y b/contrib/cube/cubeparse.y
index 7577c4515c..e3b750b695 100644
--- a/contrib/cube/cubeparse.y
+++ b/contrib/cube/cubeparse.y
@@ -7,6 +7,7 @@
#include "postgres.h"
#include "cubedata.h"
+#include "cube_internal.h"
#include "utils/float.h"
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-02 18:29 Andres Freund <[email protected]>
parent: John Naylor <[email protected]>
1 sibling, 2 replies; 27+ messages in thread
From: Andres Freund @ 2022-09-02 18:29 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
Hi John,
Are you planning to press ahead with these?
> Subject: [PATCH v4 01/11] Preparatory refactoring for compiling guc-file.c
> standalone
> Subject: [PATCH v4 02/11] Move private declarations shared between guc.c and
> guc-file.l to new header
> Subject: [PATCH v4 03/11] Build guc-file.c standalone
01-03 are a bit more complicated, but still look not far off. There's a FIXME
about failing headercheck.
> Subject: [PATCH v4 04/11] Build bootscanner.c standalone
> Subject: [PATCH v4 05/11] Build repl_scanner.c standalone
> Subject: [PATCH v4 06/11] Build syncrep_scanner.c standalone
> Subject: [PATCH v4 07/11] Build specscanner.c standalone
> Subject: [PATCH v4 08/11] Build exprscan.c standalone
LGTM
> Subject: [PATCH v4 09/11] Build cubescan.c standalone
>
> Pass scanbuflen as a parameter to yyparse rather than
> resorting to a global variable.
Nice.
> Subject: [PATCH v4 10/11] Build segscan.c standalone
> Subject: [PATCH v4 11/11] Build jsonpath_scan.c standalone
LGTM.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-03 03:03 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 0 replies; 27+ messages in thread
From: John Naylor @ 2022-09-03 03:03 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
On Sat, Sep 3, 2022 at 1:29 AM Andres Freund <[email protected]> wrote:
>
> Hi John,
>
> Are you planning to press ahead with these?
I was waiting for feedback on the latest set, so tomorrow I'll see
about the FIXME and remove the leftover bogus include. I was thinking
of applying the guc-file patches separately and then squashing the
rest since they're *mostly* mechanical:
> > Subject: [PATCH v4 01/11] Preparatory refactoring for compiling guc-file.c
> > standalone
> > Subject: [PATCH v4 02/11] Move private declarations shared between guc.c and
> > guc-file.l to new header
> > Subject: [PATCH v4 03/11] Build guc-file.c standalone
>
> 01-03 are a bit more complicated, but still look not far off. There's a FIXME
> about failing headercheck.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-04 05:16 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-09-04 05:16 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers
On Sat, Sep 3, 2022 at 1:29 AM Andres Freund <[email protected]> wrote:
> > Subject: [PATCH v4 01/11] Preparatory refactoring for compiling guc-file.c
> > standalone
> > Subject: [PATCH v4 02/11] Move private declarations shared between guc.c and
> > guc-file.l to new header
> > Subject: [PATCH v4 03/11] Build guc-file.c standalone
>
> 01-03 are a bit more complicated, but still look not far off. There's a FIXME
> about failing headercheck.
Fixed by adding utils/guc.h to the new internal header, which now
lives in the same directory as guc.c and guc-file.l, similar to how I
did json path in the last patch. Also removed the bogus include from
v4 to . Pushed 01 and 02 separately, then squashed and pushed the
rest.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-04 18:17 Andres Freund <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 2 replies; 27+ messages in thread
From: Andres Freund @ 2022-09-04 18:17 UTC (permalink / raw)
To: John Naylor <[email protected]>; Steve Singer <[email protected]>; [email protected]; +Cc: Tom Lane <[email protected]>; pgsql-hackers
Hi,
On 2022-09-04 12:16:10 +0700, John Naylor wrote:
> Pushed 01 and 02 separately, then squashed and pushed the rest.
Thanks a lot! It does look a good bit cleaner to me now.
I think, as a followup improvement, we should move gramparse.h to
src/backend/parser, and stop installing gram.h, gramparse.h. gramparse.h
already had this note:
* NOTE: this file is only meant to be included in the core parsing files,
* i.e., parser.c, gram.y, and scan.l.
* Definitions that are needed outside the core parser should be in parser.h.
What do you think?
I looked for projects including gramparse.h ([1], and found libpg-query, pgpool,
slony1 and oracfe:
- libpg-query, pgpool are partial copies of our code so will catch up when
they sync up,
- slony1's [2] is a configure check, one that long seems outdated, because it's
grepping for standard_conforming strings, which was moved out in 6566e37e027
in 2009.
- As far as I can tell oracfe's include in sqlscan.l is vistigial, it compiles
without it. And the include in parse_keywords.c is just required because it
needs to include parser/scanner.h.
Greetings,
Andres Freund
[1] https://codesearch.debian.net/search?q=gramparse.h&literal=1&perpkg=1
[2] https://git.postgresql.org/gitweb/?p=slony1-engine.git;a=blob;f=config/acx_libpq.m4;h=7653357c0a731e...
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-06 08:03 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 0 replies; 27+ messages in thread
From: John Naylor @ 2022-09-06 08:03 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Steve Singer <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Mon, Sep 5, 2022 at 1:18 AM Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2022-09-04 12:16:10 +0700, John Naylor wrote:
> > Pushed 01 and 02 separately, then squashed and pushed the rest.
>
> Thanks a lot! It does look a good bit cleaner to me now.
>
> I think, as a followup improvement, we should move gramparse.h to
> src/backend/parser, and stop installing gram.h, gramparse.h. gramparse.h
> already had this note:
>
> * NOTE: this file is only meant to be included in the core parsing files,
> * i.e., parser.c, gram.y, and scan.l.
> * Definitions that are needed outside the core parser should be in parser.h.
>
> What do you think?
+1 for the concept, but haven't looked at the details.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-07 09:27 Peter Eisentraut <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 27+ messages in thread
From: Peter Eisentraut @ 2022-09-07 09:27 UTC (permalink / raw)
To: Andres Freund <[email protected]>; John Naylor <[email protected]>; Steve Singer <[email protected]>; [email protected]; +Cc: Tom Lane <[email protected]>; pgsql-hackers
On 04.09.22 20:17, Andres Freund wrote:
> I think, as a followup improvement, we should move gramparse.h to
> src/backend/parser, and stop installing gram.h, gramparse.h. gramparse.h
> already had this note:
>
> * NOTE: this file is only meant to be included in the core parsing files,
> * i.e., parser.c, gram.y, and scan.l.
> * Definitions that are needed outside the core parser should be in parser.h.
>
> What do you think?
I found in my notes:
* maybe gram.h and gramparse.h should not be installed
So, yeah. ;-)
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-09 05:18 John Naylor <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-09-09 05:18 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Andres Freund <[email protected]>; Steve Singer <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, Sep 7, 2022 at 4:27 PM Peter Eisentraut
<[email protected]> wrote:
>
> On 04.09.22 20:17, Andres Freund wrote:
> > I think, as a followup improvement, we should move gramparse.h to
> > src/backend/parser, and stop installing gram.h, gramparse.h. gramparse.h
> > already had this note:
> >
> > * NOTE: this file is only meant to be included in the core parsing files,
> > * i.e., parser.c, gram.y, and scan.l.
> > * Definitions that are needed outside the core parser should be in parser.h.
> >
> > What do you think?
>
> I found in my notes:
>
> * maybe gram.h and gramparse.h should not be installed
>
> So, yeah. ;-)
It seems gramparse.h isn't installed now? In any case, here's a patch
to move gramparse to the backend dir and stop symlinking/ installing
gram.h. Confusingly, MSVC didn't seem to copy gram.h to src/include,
so I'm not yet sure how it still managed to build...
--
John Naylor
EDB: http://www.enterprisedb.com
Attachments:
[text/x-patch] v1-0001-Move-gramparse.h-to-src-backend-parser.patch (7.0K, ../../CAFBsxsFKVze-keZpwihb74Lqd8zV004ZfiHqwYGLRs6Nr=AYxg@mail.gmail.com/2-v1-0001-Move-gramparse.h-to-src-backend-parser.patch)
download | inline diff:
From ff89180d7b0fe4c95a470a657ee465c5384d6ecc Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 9 Sep 2022 11:57:39 +0700
Subject: [PATCH v1] Move gramparse.h to src/backend/parser
This allows removal of makefile rules that symlink gram.h to
src/include/parser. While at it, stop installing gram.h. This
seems unnecessary at present time.
Idea from Andres Freund and Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/20220904181759.px6uosll6zbxcum5%40awork3.anarazel.de
---
src/backend/Makefile | 7 +------
src/backend/parser/gram.y | 2 +-
src/{include => backend}/parser/gramparse.h | 4 ++--
src/backend/parser/parser.c | 2 +-
src/backend/parser/scan.l | 2 +-
src/include/Makefile | 4 ++--
src/include/parser/.gitignore | 1 -
src/tools/msvc/Install.pm | 4 ----
src/tools/pginclude/cpluspluscheck | 1 -
src/tools/pginclude/headerscheck | 1 -
10 files changed, 8 insertions(+), 20 deletions(-)
rename src/{include => backend}/parser/gramparse.h (97%)
delete mode 100644 src/include/parser/.gitignore
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 5c4772298d..42cc3bda1d 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -153,12 +153,7 @@ submake-utils-headers:
.PHONY: generated-headers
-generated-headers: $(top_builddir)/src/include/parser/gram.h $(top_builddir)/src/include/storage/lwlocknames.h submake-catalog-headers submake-nodes-headers submake-utils-headers
-
-$(top_builddir)/src/include/parser/gram.h: parser/gram.h
- prereqdir=`cd '$(dir $<)' >/dev/null && pwd` && \
- cd '$(dir $@)' && rm -f $(notdir $@) && \
- $(LN_S) "$$prereqdir/$(notdir $<)" .
+generated-headers: $(top_builddir)/src/include/storage/lwlocknames.h submake-catalog-headers submake-nodes-headers submake-utils-headers
$(top_builddir)/src/include/storage/lwlocknames.h: storage/lmgr/lwlocknames.h
prereqdir=`cd '$(dir $<)' >/dev/null && pwd` && \
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0492ff9a66..668ad3fa8e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -55,9 +55,9 @@
#include "catalog/pg_trigger.h"
#include "commands/defrem.h"
#include "commands/trigger.h"
+#include "gramparse.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
-#include "parser/gramparse.h"
#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/date.h"
diff --git a/src/include/parser/gramparse.h b/src/backend/parser/gramparse.h
similarity index 97%
rename from src/include/parser/gramparse.h
rename to src/backend/parser/gramparse.h
index 41b753a96c..c4726c618d 100644
--- a/src/include/parser/gramparse.h
+++ b/src/backend/parser/gramparse.h
@@ -11,7 +11,7 @@
* Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * src/include/parser/gramparse.h
+ * src/backend/parser/gramparse.h
*
*-------------------------------------------------------------------------
*/
@@ -26,7 +26,7 @@
* NB: include gram.h only AFTER including scanner.h, because scanner.h
* is what #defines YYLTYPE.
*/
-#include "parser/gram.h"
+#include "gram.h"
/*
* The YY_EXTRA data that a flex scanner allows us to pass around. Private
diff --git a/src/backend/parser/parser.c b/src/backend/parser/parser.c
index 50227cc098..ef85d3bb68 100644
--- a/src/backend/parser/parser.c
+++ b/src/backend/parser/parser.c
@@ -22,7 +22,7 @@
#include "postgres.h"
#include "mb/pg_wchar.h"
-#include "parser/gramparse.h"
+#include "gramparse.h"
#include "parser/parser.h"
#include "parser/scansup.h"
diff --git a/src/backend/parser/scan.l b/src/backend/parser/scan.l
index 882e081aae..db8b0fe8eb 100644
--- a/src/backend/parser/scan.l
+++ b/src/backend/parser/scan.l
@@ -36,7 +36,7 @@
#include <unistd.h>
#include "common/string.h"
-#include "parser/gramparse.h"
+#include "gramparse.h"
#include "parser/parser.h" /* only needed for GUC variables */
#include "parser/scansup.h"
#include "port/pg_bitutils.h"
diff --git a/src/include/Makefile b/src/include/Makefile
index 0b4cab9bb1..2858fdb4c8 100644
--- a/src/include/Makefile
+++ b/src/include/Makefile
@@ -55,7 +55,7 @@ install: all installdirs
cp $(srcdir)/$$dir/*.h '$(DESTDIR)$(includedir_server)'/$$dir/ || exit; \
done
ifeq ($(vpath_build),yes)
- for file in catalog/schemapg.h catalog/system_fk_info.h catalog/pg_*_d.h parser/gram.h storage/lwlocknames.h utils/probes.h; do \
+ for file in catalog/schemapg.h catalog/system_fk_info.h catalog/pg_*_d.h storage/lwlocknames.h utils/probes.h; do \
cp $$file '$(DESTDIR)$(includedir_server)'/$$file || exit; \
done
endif
@@ -79,7 +79,7 @@ uninstall:
clean:
rm -f utils/fmgroids.h utils/fmgrprotos.h utils/errcodes.h utils/header-stamp
- rm -f parser/gram.h storage/lwlocknames.h utils/probes.h
+ rm -f storage/lwlocknames.h utils/probes.h
rm -f catalog/schemapg.h catalog/system_fk_info.h
rm -f catalog/pg_*_d.h catalog/header-stamp
rm -f nodes/nodetags.h nodes/header-stamp
diff --git a/src/include/parser/.gitignore b/src/include/parser/.gitignore
deleted file mode 100644
index 19ea9554a0..0000000000
--- a/src/include/parser/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-/gram.h
diff --git a/src/tools/msvc/Install.pm b/src/tools/msvc/Install.pm
index 5da299476e..e20661ba6e 100644
--- a/src/tools/msvc/Install.pm
+++ b/src/tools/msvc/Install.pm
@@ -618,10 +618,6 @@ sub CopyIncludeFiles
'Server headers',
$target . '/include/server/',
'src/include/', 'pg_config.h', 'pg_config_ext.h', 'pg_config_os.h');
- CopyFiles(
- 'Grammar header',
- $target . '/include/server/parser/',
- 'src/backend/parser/', 'gram.h');
CopySetOfFiles(
'',
[ glob("src\\include\\*.h") ],
diff --git a/src/tools/pginclude/cpluspluscheck b/src/tools/pginclude/cpluspluscheck
index 547f9dc5c4..78253cfbae 100755
--- a/src/tools/pginclude/cpluspluscheck
+++ b/src/tools/pginclude/cpluspluscheck
@@ -125,7 +125,6 @@ do
test "$f" = contrib/cube/cubeparse.h && continue
test "$f" = contrib/seg/segparse.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
- test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 158c99d6af..1addc52002 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -120,7 +120,6 @@ do
test "$f" = contrib/cube/cubeparse.h && continue
test "$f" = contrib/seg/segparse.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
- test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
--
2.36.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-12 07:49 John Naylor <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: John Naylor @ 2022-09-12 07:49 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Andres Freund <[email protected]>; Steve Singer <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Fri, Sep 9, 2022 at 12:18 PM John Naylor
<[email protected]> wrote:
>
> It seems gramparse.h isn't installed now? In any case, here's a patch
> to move gramparse to the backend dir and stop symlinking/ installing
> gram.h.
Looking more closely at src/include/Makefile, this is incorrect -- all
files in SUBDIRS are copied over. So moving gramparse.h to the backend
will automatically not install it. The explicit install rule for
gram.h was for vpath builds.
CI builds fine. For v2 I only adjusted the commit message. I'll push
in a couple days unless there are objections.
--
John Naylor
EDB: http://www.enterprisedb.com
Attachments:
[text/x-patch] v2-0001-Move-gramparse.h-to-src-backend-parser.patch (7.2K, ../../CAFBsxsGmdbPwp9vsW3ECpsatqmYXhgUMAXSjuYNzVbHr3k7i3g@mail.gmail.com/2-v2-0001-Move-gramparse.h-to-src-backend-parser.patch)
download | inline diff:
From dcfe0976e6118a2c385b4473ceab76a742a6f1ff Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Fri, 9 Sep 2022 11:57:39 +0700
Subject: [PATCH v2] Move gramparse.h to src/backend/parser
This header is semi-private, being used only in files related to
raw parsing, so move to the backend directory where those files
live. This allows removal of makefile rules that symlink gram.h to
src/include/parser, since gramparse.h can now include gram.h from within
the same directory. While at it, stop installing gram.h.
Per suggestion from Andres Freund and Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/20220904181759.px6uosll6zbxcum5%40awork3.anarazel.de
---
src/backend/Makefile | 7 +------
src/backend/parser/gram.y | 2 +-
src/{include => backend}/parser/gramparse.h | 4 ++--
src/backend/parser/parser.c | 2 +-
src/backend/parser/scan.l | 2 +-
src/include/Makefile | 4 ++--
src/include/parser/.gitignore | 1 -
src/tools/msvc/Install.pm | 4 ----
src/tools/pginclude/cpluspluscheck | 1 -
src/tools/pginclude/headerscheck | 1 -
10 files changed, 8 insertions(+), 20 deletions(-)
rename src/{include => backend}/parser/gramparse.h (97%)
delete mode 100644 src/include/parser/.gitignore
diff --git a/src/backend/Makefile b/src/backend/Makefile
index d0d34821d5..181c217fae 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -153,12 +153,7 @@ submake-utils-headers:
.PHONY: generated-headers
-generated-headers: $(top_builddir)/src/include/parser/gram.h $(top_builddir)/src/include/storage/lwlocknames.h submake-catalog-headers submake-nodes-headers submake-utils-headers
-
-$(top_builddir)/src/include/parser/gram.h: parser/gram.h
- prereqdir=`cd '$(dir $<)' >/dev/null && pwd` && \
- cd '$(dir $@)' && rm -f $(notdir $@) && \
- $(LN_S) "$$prereqdir/$(notdir $<)" .
+generated-headers: $(top_builddir)/src/include/storage/lwlocknames.h submake-catalog-headers submake-nodes-headers submake-utils-headers
$(top_builddir)/src/include/storage/lwlocknames.h: storage/lmgr/lwlocknames.h
prereqdir=`cd '$(dir $<)' >/dev/null && pwd` && \
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index ea33784316..82f03fc9c9 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -55,9 +55,9 @@
#include "catalog/pg_trigger.h"
#include "commands/defrem.h"
#include "commands/trigger.h"
+#include "gramparse.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
-#include "parser/gramparse.h"
#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/date.h"
diff --git a/src/include/parser/gramparse.h b/src/backend/parser/gramparse.h
similarity index 97%
rename from src/include/parser/gramparse.h
rename to src/backend/parser/gramparse.h
index 41b753a96c..c4726c618d 100644
--- a/src/include/parser/gramparse.h
+++ b/src/backend/parser/gramparse.h
@@ -11,7 +11,7 @@
* Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * src/include/parser/gramparse.h
+ * src/backend/parser/gramparse.h
*
*-------------------------------------------------------------------------
*/
@@ -26,7 +26,7 @@
* NB: include gram.h only AFTER including scanner.h, because scanner.h
* is what #defines YYLTYPE.
*/
-#include "parser/gram.h"
+#include "gram.h"
/*
* The YY_EXTRA data that a flex scanner allows us to pass around. Private
diff --git a/src/backend/parser/parser.c b/src/backend/parser/parser.c
index 50227cc098..ef85d3bb68 100644
--- a/src/backend/parser/parser.c
+++ b/src/backend/parser/parser.c
@@ -22,7 +22,7 @@
#include "postgres.h"
#include "mb/pg_wchar.h"
-#include "parser/gramparse.h"
+#include "gramparse.h"
#include "parser/parser.h"
#include "parser/scansup.h"
diff --git a/src/backend/parser/scan.l b/src/backend/parser/scan.l
index 882e081aae..db8b0fe8eb 100644
--- a/src/backend/parser/scan.l
+++ b/src/backend/parser/scan.l
@@ -36,7 +36,7 @@
#include <unistd.h>
#include "common/string.h"
-#include "parser/gramparse.h"
+#include "gramparse.h"
#include "parser/parser.h" /* only needed for GUC variables */
#include "parser/scansup.h"
#include "port/pg_bitutils.h"
diff --git a/src/include/Makefile b/src/include/Makefile
index 0b4cab9bb1..2858fdb4c8 100644
--- a/src/include/Makefile
+++ b/src/include/Makefile
@@ -55,7 +55,7 @@ install: all installdirs
cp $(srcdir)/$$dir/*.h '$(DESTDIR)$(includedir_server)'/$$dir/ || exit; \
done
ifeq ($(vpath_build),yes)
- for file in catalog/schemapg.h catalog/system_fk_info.h catalog/pg_*_d.h parser/gram.h storage/lwlocknames.h utils/probes.h; do \
+ for file in catalog/schemapg.h catalog/system_fk_info.h catalog/pg_*_d.h storage/lwlocknames.h utils/probes.h; do \
cp $$file '$(DESTDIR)$(includedir_server)'/$$file || exit; \
done
endif
@@ -79,7 +79,7 @@ uninstall:
clean:
rm -f utils/fmgroids.h utils/fmgrprotos.h utils/errcodes.h utils/header-stamp
- rm -f parser/gram.h storage/lwlocknames.h utils/probes.h
+ rm -f storage/lwlocknames.h utils/probes.h
rm -f catalog/schemapg.h catalog/system_fk_info.h
rm -f catalog/pg_*_d.h catalog/header-stamp
rm -f nodes/nodetags.h nodes/header-stamp
diff --git a/src/include/parser/.gitignore b/src/include/parser/.gitignore
deleted file mode 100644
index 19ea9554a0..0000000000
--- a/src/include/parser/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-/gram.h
diff --git a/src/tools/msvc/Install.pm b/src/tools/msvc/Install.pm
index 5da299476e..e20661ba6e 100644
--- a/src/tools/msvc/Install.pm
+++ b/src/tools/msvc/Install.pm
@@ -618,10 +618,6 @@ sub CopyIncludeFiles
'Server headers',
$target . '/include/server/',
'src/include/', 'pg_config.h', 'pg_config_ext.h', 'pg_config_os.h');
- CopyFiles(
- 'Grammar header',
- $target . '/include/server/parser/',
- 'src/backend/parser/', 'gram.h');
CopySetOfFiles(
'',
[ glob("src\\include\\*.h") ],
diff --git a/src/tools/pginclude/cpluspluscheck b/src/tools/pginclude/cpluspluscheck
index 547f9dc5c4..78253cfbae 100755
--- a/src/tools/pginclude/cpluspluscheck
+++ b/src/tools/pginclude/cpluspluscheck
@@ -125,7 +125,6 @@ do
test "$f" = contrib/cube/cubeparse.h && continue
test "$f" = contrib/seg/segparse.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
- test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 158c99d6af..1addc52002 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -120,7 +120,6 @@ do
test "$f" = contrib/cube/cubeparse.h && continue
test "$f" = contrib/seg/segparse.h && continue
test "$f" = src/backend/bootstrap/bootparse.h && continue
- test "$f" = src/include/parser/gram.h && continue
test "$f" = src/backend/parser/gram.h && continue
test "$f" = src/backend/replication/repl_gram.h && continue
test "$f" = src/backend/replication/syncrep_gram.h && continue
--
2.36.1
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-13 23:10 Andres Freund <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 1 reply; 27+ messages in thread
From: Andres Freund @ 2022-09-13 23:10 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Steve Singer <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2022-09-12 14:49:50 +0700, John Naylor wrote:
> CI builds fine. For v2 I only adjusted the commit message. I'll push
> in a couple days unless there are objections.
Makes sense to me. Thanks for working on it!
^ permalink raw reply [nested|flat] 27+ messages in thread
* Re: build remaining Flex files standalone
@ 2022-09-14 04:27 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 0 replies; 27+ messages in thread
From: John Naylor @ 2022-09-14 04:27 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Steve Singer <[email protected]>; Pavel Stehule <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, Sep 14, 2022 at 6:10 AM Andres Freund <[email protected]> wrote:
>
> On 2022-09-12 14:49:50 +0700, John Naylor wrote:
> > CI builds fine. For v2 I only adjusted the commit message. I'll push
> > in a couple days unless there are objections.
>
> Makes sense to me. Thanks for working on it!
This is done.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 27+ messages in thread
end of thread, other threads:[~2022-09-14 04:27 UTC | newest]
Thread overview: 27+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-08-10 17:19 Re: [RFC] building postgres with meson Andres Freund <[email protected]>
2022-08-11 03:33 ` John Naylor <[email protected]>
2022-08-11 03:37 ` Tom Lane <[email protected]>
2022-08-11 03:45 ` Tom Lane <[email protected]>
2022-08-11 03:57 ` John Naylor <[email protected]>
2022-08-11 04:07 ` Andres Freund <[email protected]>
2022-08-12 06:01 ` build remaining Flex files standalone John Naylor <[email protected]>
2022-08-13 08:39 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-08-15 18:11 ` Re: build remaining Flex files standalone Andres Freund <[email protected]>
2022-08-16 10:41 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-08-17 01:14 ` Re: build remaining Flex files standalone Andres Freund <[email protected]>
2022-08-17 01:47 ` Re: build remaining Flex files standalone Tom Lane <[email protected]>
2022-08-17 02:53 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-08-18 07:40 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-08-18 07:43 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-08-18 08:00 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-09-02 18:29 ` Re: build remaining Flex files standalone Andres Freund <[email protected]>
2022-09-03 03:03 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-09-04 05:16 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-09-04 18:17 ` Re: build remaining Flex files standalone Andres Freund <[email protected]>
2022-09-06 08:03 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-09-07 09:27 ` Re: build remaining Flex files standalone Peter Eisentraut <[email protected]>
2022-09-09 05:18 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-09-12 07:49 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-09-13 23:10 ` Re: build remaining Flex files standalone Andres Freund <[email protected]>
2022-09-14 04:27 ` Re: build remaining Flex files standalone John Naylor <[email protected]>
2022-08-11 04:32 ` Tom Lane <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox