pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #16982: pg_rewind cannot deal with configuration files (postgresql.conf) located outside data directory
2+ messages / 2 participants
[nested] [flat]
* BUG #16982: pg_rewind cannot deal with configuration files (postgresql.conf) located outside data directory
@ 2021-04-23 13:36 PG Bug reporting form <noreply@postgresql.org>
2021-04-28 05:22 ` Re: BUG #16982: pg_rewind cannot deal with configuration files (postgresql.conf) located outside data directory Neil Chen <carpenter.nail.cz@gmail.com>
0 siblings, 1 reply; 2+ messages in thread
From: PG Bug reporting form @ 2021-04-23 13:36 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: edwin.polkerman@splendiddata.com
The following bug has been logged on the website:
Bug reference: 16982
Logged by: Edwin Polkerman
Email address: edwin.polkerman@splendiddata.com
PostgreSQL version: 13.2
Operating system: Centos7/8, RHEL7/8, SUSE12/15
Description:
In the used PostgreSQL setup the configuration files (postgresql.conf,
pg_hha.conf and pg_ident.conf) are located in another location than the data
directory: /etc/database/postgres/13/data
In postgresql.conf following locations are updated:
#------------------------------------------------------------------------------
# FILE LOCATIONS
#------------------------------------------------------------------------------
# The default values of these variables are driven from the -D
command-line
# option or PGDATA environment variable, represented here as ConfigDir.
data_directory = '/var/database/postgres/13/data'
hba_file = '/etc/database/postgres/13/data/pg_hba.conf'
ident_file = '/etc/database/postgres/13/data/pg_ident.conf'
The cluster can be started with pg_ctl and use the path to postgresql.conf
in -D:
> pg_ctl -D /etc/database/postgres/13/data start
Trying to use pg_rewind to synchronize a PostgreSQL data directory (e.g.
bringing back failed master)
> pg_rewind --target-pgdata=/etc/database/postgres/13/data -P -R -c
--source-server='host=<ip-address source> port=5432 user=rewind_user
password=rewind_user dbname=postgres'
Observed result:
==========
pg_rewind: connected to server
pg_rewind: fatal: could not open file
"/etc/database/postgres/13/data/global/pg_control" for reading: No such file
or directory
Change --target-pgdata to point to data directory:
> pg_rewind --target-pgdata=/var/database/postgres/13/data -P -R -c
--source-server='host=<ip-address source> port=5432 user=rewind_user
password=rewind_user dbname=postgres'
Now the result is:
pg_rewind: fatal: restore_command is not set in the target cluster
Which is another proof that pg_rewind does not look at
/etc/database/postgres/13/data/postgresql.conf (in which restore_command was
set) but to the one located in the data directory.
Improvement:
It would be nice to have pg_rewind support postgresql.conf being located
somewhere else than in the data directory, just like pg_ctl does.
Of course it is possible to workaround the issue but it takes more steps
than would be necessary (with increased error risk of copying and moving
configuration files around).
Not only applicable for PostgreSQL 13, but also for the other versions as
well
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: BUG #16982: pg_rewind cannot deal with configuration files (postgresql.conf) located outside data directory
2021-04-23 13:36 BUG #16982: pg_rewind cannot deal with configuration files (postgresql.conf) located outside data directory PG Bug reporting form <noreply@postgresql.org>
@ 2021-04-28 05:22 ` Neil Chen <carpenter.nail.cz@gmail.com>
0 siblings, 0 replies; 2+ messages in thread
From: Neil Chen @ 2021-04-28 05:22 UTC (permalink / raw)
To: edwin.polkerman@splendiddata.com; pgsql-bugs@lists.postgresql.org
Greetings,
On Wed, Apr 28, 2021 at 11:38 AM PG Bug reporting form <
noreply@postgresql.org> wrote:
>
>
> In the used PostgreSQL setup the configuration files (postgresql.conf,
> pg_hha.conf and pg_ident.conf) are located in another location than the
> data
> directory: /etc/database/postgres/13/data
>
> In postgresql.conf following locations are updated:
>
> #------------------------------------------------------------------------------
> # FILE LOCATIONS
>
> #------------------------------------------------------------------------------
>
> # The default values of these variables are driven from the -D
> command-line
> # option or PGDATA environment variable, represented here as ConfigDir.
> data_directory = '/var/database/postgres/13/data'
> hba_file = '/etc/database/postgres/13/data/pg_hba.conf'
>
>
>
> ident_file = '/etc/database/postgres/13/data/pg_ident.conf'
>
> The cluster can be started with pg_ctl and use the path to postgresql.conf
> in -D:
> > pg_ctl -D /etc/database/postgres/13/data start
>
>
>
> Trying to use pg_rewind to synchronize a PostgreSQL data directory (e.g.
> bringing back failed master)
> > pg_rewind --target-pgdata=/etc/database/postgres/13/data -P -R -c
> --source-server='host=<ip-address source> port=5432 user=rewind_user
> password=rewind_user dbname=postgres'
>
> Observed result:
> ==========
> pg_rewind: connected to server
> pg_rewind: fatal: could not open file
> "/etc/database/postgres/13/data/global/pg_control" for reading: No such
> file
> or directory
>
> Improvement:
> It would be nice to have pg_rewind support postgresql.conf being located
> somewhere else than in the data directory, just like pg_ctl does.
>
> Of course it is possible to workaround the issue but it takes more steps
> than would be necessary (with increased error risk of copying and moving
> configuration files around).
>
> Not only applicable for PostgreSQL 13, but also for the other versions as
> well
>
>
Yes, I think in general, pg_rewind should also support this function. To
support it, you can copy the relevant code directly from pg_ctl, here is
the patch.
But of course, I hope to listen to other people's opinions, so that we can
understand why we didn't support this function before, and whether there
are any risks...
Thanks.
--
There is no royal road to learning.
HighGo Software Co.
Attachments:
0001-pg_rewind-patch.patch (application/octet-stream, 4.6K) download
From c69fe2c18d62f3dd64b5153c0acfa0a6e9aaf4e1 Mon Sep 17 00:00:00 2001
From: Neil Chen <chenze@highgo.com>
Date: Wed, 28 Apr 2021 13:13:24 +0800
Subject: [PATCH] pg_rewind patch
---
src/bin/pg_rewind/pg_rewind.c | 102 +++++++++++++++++++++++++++++++++-
1 file changed, 101 insertions(+), 1 deletion(-)
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index 5157f59cf7..89e56ea9b7 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -47,6 +47,8 @@ static void sanityChecks(void);
static void findCommonAncestorTimeline(XLogRecPtr *recptr, int *tliIndex);
static void ensureCleanShutdown(const char *argv0);
static void disconnect_atexit(void);
+static void adjust_data_dir(const char *argv0);
+static char *find_other_exec_or_die(const char *argv0, const char *target, const char *versionstr);
static ControlFileData ControlFile_target;
static ControlFileData ControlFile_source;
@@ -57,6 +59,7 @@ int WalSegSz;
/* Configuration options */
char *datadir_target = NULL;
+char *config_target = NULL;
char *datadir_source = NULL;
char *connstr_source = NULL;
char *restore_command = NULL;
@@ -191,6 +194,7 @@ main(int argc, char **argv)
case 'D': /* -D or --target-pgdata */
datadir_target = pg_strdup(optarg);
+ config_target = pg_strdup(optarg);
break;
case 1: /* --source-pgdata */
@@ -228,6 +232,9 @@ main(int argc, char **argv)
exit(1);
}
+ /* -D might point at config-only directory; if so find the real PGDATA */
+ adjust_data_dir(argv[0]);
+
if (writerecoveryconf && connstr_source == NULL)
{
pg_log_error("no source server information (--source-server) specified for --write-recovery-conf");
@@ -1125,7 +1132,7 @@ ensureCleanShutdown(const char *argv0)
* is synced at the end anyway.
*/
snprintf(cmd, MAXCMDLEN, "\"%s\" --single -F -D \"%s\" template1 < \"%s\"",
- exec_path, datadir_target, DEVNULL);
+ exec_path, config_target, DEVNULL);
if (system(cmd) != 0)
{
@@ -1140,3 +1147,96 @@ disconnect_atexit(void)
if (conn != NULL)
PQfinish(conn);
}
+
+/*
+ * adjust_data_dir
+ *
+ * If a configuration-only directory was specified, find the real data dir.
+ */
+static void
+adjust_data_dir(const char *argv0)
+{
+ char *pgdata_opt;
+ char cmd[MAXPGPATH],
+ filename[MAXPGPATH],
+ *my_exec_path;
+ FILE *fd;
+
+ /* do nothing if we're working without knowledge of data dir */
+ if (config_target == NULL)
+ return;
+
+ canonicalize_path(config_target);
+ pgdata_opt = psprintf("-D \"%s\" ", config_target);
+
+ /* If there is no postgresql.conf, it can't be a config-only dir */
+ snprintf(filename, sizeof(filename), "%s/postgresql.conf", config_target);
+ if ((fd = fopen(filename, "r")) == NULL)
+ return;
+ fclose(fd);
+
+ /* If PG_VERSION exists, it can't be a config-only dir */
+ snprintf(filename, sizeof(filename), "%s/PG_VERSION", config_target);
+ if ((fd = fopen(filename, "r")) != NULL)
+ {
+ fclose(fd);
+ return;
+ }
+
+ /* Must be a configuration directory, so find the data directory */
+
+ /* we use a private my_exec_path to avoid interfering with later uses */
+ my_exec_path = find_other_exec_or_die(argv0, "postgres", PG_BACKEND_VERSIONSTR);
+
+ /* it's important for -C to be the first option, see main.c */
+ snprintf(cmd, MAXPGPATH, "\"%s\" -C data_directory %s",
+ my_exec_path,
+ pgdata_opt ? pgdata_opt : "");
+
+ fd = popen(cmd, "r");
+ if (fd == NULL || fgets(filename, sizeof(filename), fd) == NULL)
+ {
+ pg_log_error(_("%s: could not determine the data directory using command \"%s\"\n"), progname, cmd);
+ exit(1);
+ }
+ pclose(fd);
+ free(my_exec_path);
+
+ /* strip trailing newline and carriage return */
+ (void) pg_strip_crlf(filename);
+
+ datadir_target = pg_strdup(filename);
+ canonicalize_path(datadir_target);
+}
+
+static char *
+find_other_exec_or_die(const char *argv0, const char *target, const char *versionstr)
+{
+ int ret;
+ char *found_path;
+
+ found_path = pg_malloc(MAXPGPATH);
+
+ if ((ret = find_other_exec(argv0, target, versionstr, found_path)) < 0)
+ {
+ char full_path[MAXPGPATH];
+
+ if (find_my_exec(argv0, full_path) < 0)
+ strlcpy(full_path, progname, sizeof(full_path));
+
+ if (ret == -1)
+ pg_log_error(_("The program \"%s\" is needed by %s but was not found in the\n"
+ "same directory as \"%s\".\n"
+ "Check your installation.\n"),
+ target, progname, full_path);
+ else
+ pg_log_error(_("The program \"%s\" was found by \"%s\"\n"
+ "but was not the same version as %s.\n"
+ "Check your installation.\n"),
+ target, full_path, progname);
+ exit(1);
+ }
+
+ return found_path;
+}
+
--
2.25.1
end of thread, other threads:[~2021-04-28 05:22 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-04-23 13:36 BUG #16982: pg_rewind cannot deal with configuration files (postgresql.conf) located outside data directory PG Bug reporting form <noreply@postgresql.org>
2021-04-28 05:22 ` Neil Chen <carpenter.nail.cz@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox