($INBOX_DIR/description missing)
help / color / mirror / Atom feedRe: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
33+ messages / 7 participants
[nested] [flat]
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
@ 2017-12-13 02:50 Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
0 siblings, 1 reply; 33+ messages in thread
From: Haribabu Kommi @ 2017-12-13 02:50 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Robert Haas <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Wed, Nov 29, 2017 at 4:25 PM, Michael Paquier <[email protected]>
wrote:
> On Wed, Nov 8, 2017 at 8:50 AM, Haribabu Kommi <[email protected]>
> wrote:
> > Ok. Removed the documentation changes that it cannot be used for normal
> > scenarios, and also added a Note section explaining the problem of using
> > the dump with pg_restore command with --clean and --create options.
>
> Hari, the documentation portion of the patch does not apply. Could you
> rebase? For now I am moving it to next CF as this did not get any
> reviews, and the status is switched to "waiting on author".
>
Rebased patch attached that fixes the documentation build problem.
Regards,
Hari Babu
Fujitsu Australia
Attachments:
[application/octet-stream] pg_dump-and-pg_dumpall-database-handling-refactoring_v11.patch (49.8K, ../../CAJrrPGe=U_ZbLEdKezpr6O_ybHV2VivE-coUh7+mER4k=gp9WQ@mail.gmail.com/3-pg_dump-and-pg_dumpall-database-handling-refactoring_v11.patch)
download | inline diff:
From ace860260e0787a60270703a4debdccb242eccb1 Mon Sep 17 00:00:00 2001
From: Hari Babu Kommi <[email protected]>
Date: Mon, 21 Aug 2017 14:29:58 +1000
Subject: [PATCH] pg_dump and pg_dumpall database handling refactoring
The Database handling especially with ACL's is not proper
between pg_dump and pg_dumpall. So to avoid these problems,
Now the entire database handling is moved to pg_dump and
the pg_dumpall just invokes pg_dump to print the database
attributes.
---
doc/src/sgml/ref/pg_dump.sgml | 33 ++-
doc/src/sgml/ref/pg_dumpall.sgml | 4 +
src/bin/pg_dump/Makefile | 4 +-
src/bin/pg_dump/dumputils.c | 67 +++++
src/bin/pg_dump/dumputils.h | 7 +
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_backup_archiver.c | 2 +
src/bin/pg_dump/pg_backup_utils.c | 3 +
src/bin/pg_dump/pg_backup_utils.h | 1 +
src/bin/pg_dump/pg_dump.c | 308 +++++++++++++++++++---
src/bin/pg_dump/pg_dumpall.c | 488 ++---------------------------------
src/bin/pg_dump/t/002_pg_dump.pl | 4 +-
src/bin/pg_upgrade/dump.c | 12 +-
src/bin/pg_upgrade/pg_upgrade.c | 23 +-
14 files changed, 421 insertions(+), 537 deletions(-)
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 08cad68199..0efa022269 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -192,8 +192,11 @@ PostgreSQL documentation
database itself and reconnect to the created database. (With a
script of this form, it doesn't matter which database in the
destination installation you connect to before running the script.)
- If <option>--clean</option> is also specified, the script drops and
- recreates the target database before reconnecting to it.
+ It also sets all the database-level properties such as ownership,
+ ACLs, <command>ALTER DATABASE ... SET</command> commands, and
+ <command>ALTER ROLE IN DATABASE ... SET</command> commands.
+ If <option>--clean</option> is also specified, the script drops
+ and recreates the target database before reconnecting to it.
</para>
<para>
@@ -706,6 +709,28 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--set-db-properties</option></term>
+ <listitem>
+ <para>
+ This option is to skip create database command without ignoring the rest
+ of the DATABASE specific commands and it is used when the database already
+ exists in the restore location. This option cannot be used with either of
+ <option>-C/--create</option> or <option>-c/--clean</option> options.
+ </para>
+
+ <note>
+ <para>
+ The Dump file generated with this option cannot be restored using
+ <option>-c/--clean</option> and <option>-C/--create</option> options of
+ <application>pg_restore</application> command. This is due to not available
+ of create database command in the dump can raise errors when the database
+ is dropped.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--enable-row-security</option></term>
<listitem>
@@ -1231,10 +1256,6 @@ CREATE DATABASE foo WITH TEMPLATE template0;
<command>ANALYZE</command> after restoring from a dump file
to ensure optimal performance; see <xref linkend="vacuum-for-statistics"/>
and <xref linkend="autovacuum"/> for more information.
- The dump file also does not
- contain any <command>ALTER DATABASE ... SET</command> commands;
- these settings are dumped by <xref linkend="app-pg-dumpall"/>,
- along with database users and other installation-wide settings.
</para>
<para>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5196a211b1..cb3957d669 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -129,6 +129,10 @@ PostgreSQL documentation
<listitem>
<para>
Dump only global objects (roles and tablespaces), no databases.
+ Any database role configuration settings are not dumped with
+ this option. User needs to use <command>pg_dump</command> with
+ <option>--create</option> option to dump
+ <command>ALTER ROLE IN DATABASE ... SET</command> commands.
</para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 3700884720..9709246a19 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -31,8 +31,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o | submake-libpq submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o pg_backup_utils.o | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o pg_backup_utils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/dumputils.c b/src/bin/pg_dump/dumputils.c
index 12290a1aae..def740cd57 100644
--- a/src/bin/pg_dump/dumputils.c
+++ b/src/bin/pg_dump/dumputils.c
@@ -807,3 +807,70 @@ buildACLQueries(PQExpBuffer acl_subquery, PQExpBuffer racl_subquery,
printfPQExpBuffer(init_racl_subquery, "NULL");
}
}
+
+/*
+ * Helper function for dumpXXXConfig().
+ *
+ * Frame the ALTER ".." SET ".." commands and fill it in buf.
+ */
+void
+makeAlterConfigCommand(PGconn *conn, const char *configitem,
+ const char *type, const char *name,
+ const char *type2, const char *name2,
+ PQExpBuffer buf)
+{
+ char *pos;
+ char *mine;
+
+ mine = pg_strdup(configitem);
+ pos = strchr(mine, '=');
+ if (pos == NULL)
+ {
+ pg_free(mine);
+ return;
+ }
+
+ *pos++ = '\0';
+ appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
+ if (type2 != NULL && name2 != NULL)
+ appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
+ appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be quoted.
+ */
+ if (pg_strcasecmp(mine, "DateStyle") == 0
+ || pg_strcasecmp(mine, "search_path") == 0)
+ appendPQExpBufferStr(buf, pos);
+ else
+ appendStringLiteralConn(buf, pos, conn);
+ appendPQExpBufferStr(buf, ";\n");
+
+ pg_free(mine);
+}
+
+/*
+ * Run a query, return the results, exit program on failure.
+ */
+PGresult *
+executeQuery(PGconn *conn, const char *query)
+{
+ PGresult *res;
+
+ if (g_verbose)
+ fprintf(stderr, _("%s: executing %s\n"), progname, query);
+
+ res = PQexec(conn, query);
+ if (!res ||
+ PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ fprintf(stderr, _("%s: query failed: %s"),
+ progname, PQerrorMessage(conn));
+ fprintf(stderr, _("%s: query was: %s\n"),
+ progname, query);
+ PQfinish(conn);
+ exit_nicely(1);
+ }
+
+ return res;
+}
diff --git a/src/bin/pg_dump/dumputils.h b/src/bin/pg_dump/dumputils.h
index fe364dd8c4..c46ac58763 100644
--- a/src/bin/pg_dump/dumputils.h
+++ b/src/bin/pg_dump/dumputils.h
@@ -17,6 +17,7 @@
#include "libpq-fe.h"
#include "pqexpbuffer.h"
+#include "pg_backup_utils.h"
/*
* Preferred strftime(3) format specifier for printing timestamps in pg_dump
@@ -56,4 +57,10 @@ extern void buildACLQueries(PQExpBuffer acl_subquery, PQExpBuffer racl_subquery,
const char *acl_column, const char *acl_owner,
const char *obj_kind, bool binary_upgrade);
+extern void makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
+ const char *type, const char *name, const char *type2,
+ const char *name2, PQExpBuffer buf);
+
+extern PGresult *executeQuery(PGconn *conn, const char *query);
+
#endif /* DUMPUTILS_H */
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..597c4f70e6 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -117,6 +117,7 @@ typedef struct _restoreOptions
int suppressDumpWarnings; /* Suppress output of WARNING entries
* to stderr */
bool single_txn;
+ int set_db_properties;
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
@@ -158,6 +159,7 @@ typedef struct _dumpOptions
int use_setsessauth;
int enable_row_security;
int load_via_partition_root;
+ int set_db_properties;
/* default, if no "inclusion" switches appear, is to dump everything */
bool include_everything;
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 41741aefbc..ea5fba830f 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -831,6 +831,8 @@ restore_toc_entry(ArchiveHandle *AH, TocEntry *te, bool is_parallel)
ahlog(AH, 1, "connecting to new database \"%s\"\n", te->tag);
_reconnectToDB(AH, te->tag);
+
+ ahprintf(AH, "SET default_transaction_read_only = off;\n\n");
ropt->dbname = connstr.data;
}
}
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index 066121449e..a51898fc88 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -19,6 +19,9 @@
/* Globals exported by this file */
const char *progname = NULL;
+/* User wants verbose narration of our activities */
+bool g_verbose = false;
+
#define MAX_ON_EXIT_NICELY 20
static struct
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 14661a1160..1be5e058b7 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -26,6 +26,7 @@ typedef enum /* bits returned by set_dump_section */
typedef void (*on_exit_nicely_callback) (int code, void *arg);
extern const char *progname;
+extern bool g_verbose;
extern void set_dump_section(const char *arg, int *dumpSections);
extern void write_msg(const char *modulename, const char *fmt,...) pg_attribute_printf(2, 3);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e6701aaa78..1523795f60 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -88,9 +88,6 @@ typedef enum OidOptions
zeroAsNone = 8
} OidOptions;
-/* global decls */
-bool g_verbose; /* User wants verbose narration of our
- * activities. */
static bool dosync = true; /* Issue fsync() to make dump durable on disk. */
/* subquery used to convert user ID (eg, datdba) to user name */
@@ -273,6 +270,8 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
static char *get_synchronized_snapshot(Archive *fout);
static void setupDumpWorker(Archive *AHX);
static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void dumpDatabaseConfig(Archive *fout, PQExpBuffer creaQry, const char *dbname);
+static void dumpDbRoleConfig(Archive *AH, PQExpBuffer creaQry);
int
@@ -362,6 +361,7 @@ main(int argc, char **argv)
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
{"no-sync", no_argument, NULL, 7},
+ {"set-db-properties", no_argument, &dopt.set_db_properties, 1},
{NULL, 0, NULL, 0}
};
@@ -581,6 +581,18 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.set_db_properties && dopt.outputCreateDB)
+ {
+ write_msg(NULL, "option --set-db-properties and -C/--create cannot be used together\n");
+ exit_nicely(1);
+ }
+
+ if (dopt.set_db_properties && dopt.outputClean)
+ {
+ write_msg(NULL, "option --set-db-properties and -c/--clean cannot be used together\n");
+ exit_nicely(1);
+ }
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -860,7 +872,11 @@ main(int argc, char **argv)
ropt->dumpSections = dopt.dumpSections;
ropt->aclsSkip = dopt.aclsSkip;
ropt->superuser = dopt.outputSuperuser;
- ropt->createDB = dopt.outputCreateDB;
+ /*
+ * treat --set-db-properties are also as DATABASE commands
+ * like -C/--create
+ */
+ ropt->createDB = dopt.outputCreateDB || dopt.set_db_properties;
ropt->noOwner = dopt.outputNoOwner;
ropt->noTablespace = dopt.outputNoTablespaces;
ropt->disable_triggers = dopt.disable_triggers;
@@ -875,6 +891,7 @@ main(int argc, char **argv)
ropt->enable_row_security = dopt.enable_row_security;
ropt->sequence_data = dopt.sequence_data;
ropt->binary_upgrade = dopt.binary_upgrade;
+ ropt->set_db_properties = dopt.set_db_properties;
if (compressLevel == -1)
ropt->compression = 0;
@@ -973,6 +990,7 @@ help(const char *progname)
printf(_(" --use-set-session-authorization\n"
" use SET SESSION AUTHORIZATION commands instead of\n"
" ALTER OWNER commands to set ownership\n"));
+ printf(_(" --set-db-properties dump db properties, for use by upgrade utilities only\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=DBNAME database to dump\n"));
@@ -2528,25 +2546,34 @@ dumpDatabase(Archive *fout)
PQExpBuffer dbQry = createPQExpBuffer();
PQExpBuffer delQry = createPQExpBuffer();
PQExpBuffer creaQry = createPQExpBuffer();
+ PQExpBuffer aclQry = createPQExpBuffer();
PGconn *conn = GetConnection(fout);
PGresult *res;
int i_tableoid,
i_oid,
i_dba,
+ i_dbacl,
+ i_rdbacl,
i_encoding,
i_collate,
i_ctype,
i_frozenxid,
i_minmxid,
- i_tablespace;
+ i_tablespace,
+ i_dbistemplate,
+ i_dbconnlimit;
CatalogId dbCatId;
DumpId dbDumpId;
const char *datname,
*dba,
+ *dbacl,
+ *rdbacl,
*encoding,
*collate,
*ctype,
- *tablespace;
+ *tablespace,
+ *dbistemplate,
+ *dbconnlimit;
uint32 frozenxid,
minmxid;
@@ -2558,13 +2585,49 @@ dumpDatabase(Archive *fout)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- /* Get the database owner and parameters from pg_database */
- if (fout->remoteVersion >= 90300)
+ /*
+ * Now collect all the information about databases to dump.
+ *
+ * For the database ACLs, as of 9.6, we extract both the positive (as
+ * datacl) and negative (as rdatacl) ACLs, relative to the default ACL for
+ * databases, which are then passed to buildACLCommands() below.
+ *
+ * See buildACLQueries() and buildACLCommands().
+ *
+ * Note that we do not support initial privileges (pg_init_privs) on
+ * databases.
+ */
+
+ if (fout->remoteVersion >= 90600)
+ {
+ appendPQExpBuffer(dbQry,
+ "SELECT tableoid, oid, "
+ "(%s datdba) AS dba, "
+ "pg_encoding_to_char(encoding) AS encoding, "
+ "datcollate, datctype, datfrozenxid, datminmxid, datistemplate, "
+ "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
+ " SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba))) AS acl "
+ " EXCEPT SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba))) as datacls)"
+ " AS datacl, "
+ "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
+ " SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba)) AS acl "
+ " EXCEPT SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba)))) as rdatacls)"
+ " AS rdatacl, "
+ "datconnlimit, "
+ "(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
+ "shobj_description(oid, 'pg_database') AS description "
+
+ "FROM pg_database "
+ "WHERE datname = ",
+ username_subquery);
+ appendStringLiteralAH(dbQry, datname, fout);
+ }
+ else if (fout->remoteVersion >= 90300)
{
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "datcollate, datctype, datfrozenxid, datminmxid, "
+ "datcollate, datctype, datfrozenxid, datminmxid, datistemplate, datacl,'' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2578,7 +2641,7 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
+ "datcollate, datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl, '' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2592,7 +2655,7 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl,'' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2606,7 +2669,8 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl,'' as rdatacl,"
+ "-1 as datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace "
"FROM pg_database "
"WHERE datname = ",
@@ -2625,6 +2689,10 @@ dumpDatabase(Archive *fout)
i_frozenxid = PQfnumber(res, "datfrozenxid");
i_minmxid = PQfnumber(res, "datminmxid");
i_tablespace = PQfnumber(res, "tablespace");
+ i_dbacl = PQfnumber(res, "datacl");
+ i_rdbacl = PQfnumber(res, "rdatacl");
+ i_dbistemplate = PQfnumber(res, "datistemplate");
+ i_dbconnlimit = PQfnumber(res, "datconnlimit");
dbCatId.tableoid = atooid(PQgetvalue(res, 0, i_tableoid));
dbCatId.oid = atooid(PQgetvalue(res, 0, i_oid));
@@ -2635,40 +2703,63 @@ dumpDatabase(Archive *fout)
frozenxid = atooid(PQgetvalue(res, 0, i_frozenxid));
minmxid = atooid(PQgetvalue(res, 0, i_minmxid));
tablespace = PQgetvalue(res, 0, i_tablespace);
+ dbacl = PQgetvalue(res, 0, i_dbacl);
+ rdbacl = PQgetvalue(res, 0, i_rdbacl);
+ dbistemplate = PQgetvalue(res, 0, i_dbistemplate);
+ dbconnlimit = PQgetvalue(res, 0, i_dbconnlimit);
- appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
- fmtId(datname));
- if (strlen(encoding) > 0)
- {
- appendPQExpBufferStr(creaQry, " ENCODING = ");
- appendStringLiteralAH(creaQry, encoding, fout);
- }
- if (strlen(collate) > 0)
+ /*
+ * Skip the CREATE DATABASE commands --set-db-properties is enabled,
+ * since they are presumably already there in the destination cluster.
+ * We do want to emit their ACLs and config options if any, however.
+ */
+ if (!dopt->set_db_properties)
{
- appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
- appendStringLiteralAH(creaQry, collate, fout);
+ appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
+ fmtId(datname));
+
+ if (strlen(encoding) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " ENCODING = ");
+ appendStringLiteralAH(creaQry, encoding, fout);
+ }
+ if (strlen(collate) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
+ appendStringLiteralAH(creaQry, collate, fout);
+ }
+ if (strlen(ctype) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
+ appendStringLiteralAH(creaQry, ctype, fout);
+ }
+ if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
+ !dopt->outputNoTablespaces)
+ {
+ appendPQExpBuffer(creaQry, " TABLESPACE = %s", fmtId(tablespace));
+ }
+
+ if (strlen(dbistemplate) > 0 && strcmp(dbistemplate, "t") == 0)
+ appendPQExpBuffer(creaQry, " IS_TEMPLATE = true");
+
+ if (strlen(dbconnlimit) > 0 && strcmp(dbconnlimit, "-1") != 0)
+ appendPQExpBuffer(creaQry, " CONNECTION LIMIT = %s",
+ dbconnlimit);
+
+ appendPQExpBufferStr(creaQry, ";\n");
}
- if (strlen(ctype) > 0)
+ else if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 && !dopt->outputNoTablespaces)
{
- appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
- appendStringLiteralAH(creaQry, ctype, fout);
- }
- if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
- !dopt->outputNoTablespaces)
- appendPQExpBuffer(creaQry, " TABLESPACE = %s",
- fmtId(tablespace));
- appendPQExpBufferStr(creaQry, ";\n");
+ if (strcmp(datname, "postgres") == 0)
+ appendPQExpBuffer(creaQry, "\\connect template1\n");
+ else if (strcmp(datname, "template1") == 0)
+ appendPQExpBuffer(creaQry, "\\connect postgres\n");
- if (dopt->binary_upgrade)
- {
- appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
- "SET datfrozenxid = '%u', datminmxid = '%u'\n"
- "WHERE datname = ",
- frozenxid, minmxid);
- appendStringLiteralAH(creaQry, datname, fout);
- appendPQExpBufferStr(creaQry, ";\n");
+ appendPQExpBuffer(creaQry, "ALTER DATABASE %s SET TABLESPACE %s;\n",
+ datname, fmtId(tablespace));
+ /* connect to original database */
+ appendPsqlMetaConnect(creaQry, datname);
}
appendPQExpBuffer(delQry, "DROP DATABASE %s;\n",
@@ -2694,6 +2785,58 @@ dumpDatabase(Archive *fout)
NULL, /* Dumper */
NULL); /* Dumper Arg */
+ resetPQExpBuffer(creaQry);
+
+ if (dopt->binary_upgrade)
+ {
+ appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
+ appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
+ "SET datfrozenxid = '%u', datminmxid = '%u'\n"
+ "WHERE datname = ",
+ frozenxid, minmxid);
+ appendStringLiteralAH(creaQry, datname, fout);
+ appendPQExpBufferStr(creaQry, ";\n");
+
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ creaQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+ resetPQExpBuffer(creaQry);
+ }
+
+ if (!dopt->aclsSkip &&
+ !buildACLCommands(datname, NULL, "DATABASE",
+ dbacl, rdbacl, dba,
+ "", fout->remoteVersion, aclQry))
+ {
+ exit_horribly(NULL, _("%s: could not parse ACL list (%s) for database \"%s\"\n"), progname, dbacl, datname);
+ }
+
+ if (strlen(aclQry->data))
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ aclQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ /* Dump database specific configuration */
+ dumpDatabaseConfig(fout, creaQry, datname);
+
+ /* Dump user and database specific configuration */
+ if (fout->remoteVersion >= 90000)
+ dumpDbRoleConfig(fout, creaQry);
+
+ if (strlen(creaQry->data))
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ creaQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
/*
* pg_largeobject and pg_largeobject_metadata come from the old system
* intact, so set their relfrozenxids and relminmxids.
@@ -2847,6 +2990,7 @@ dumpDatabase(Archive *fout)
destroyPQExpBuffer(dbQry);
destroyPQExpBuffer(delQry);
destroyPQExpBuffer(creaQry);
+ destroyPQExpBuffer(aclQry);
}
/*
@@ -18091,3 +18235,87 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
if (!res)
write_msg(NULL, "WARNING: could not parse reloptions array\n");
}
+
+/*
+ * Dump database-specific configuration
+ */
+static void
+dumpDatabaseConfig(Archive *AH, PQExpBuffer creaQry, const char *dbname)
+{
+ PGconn *conn = GetConnection(AH);
+ PQExpBuffer buf = createPQExpBuffer();
+ int count = 1;
+
+ for (;;)
+ {
+ PGresult *res;
+
+ if (AH->remoteVersion >= 90000)
+ printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting WHERE "
+ "setrole = 0 AND setdatabase = (SELECT oid FROM pg_database WHERE datname = ", count);
+ else
+ printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE datname = ", count);
+
+ appendStringLiteralConn(buf, dbname, conn);
+
+ if (AH->remoteVersion >= 90000)
+ appendPQExpBufferChar(buf, ')');
+
+ res = executeQuery(conn, buf->data);
+ resetPQExpBuffer(buf);
+
+ if (PQntuples(res) == 1 &&
+ !PQgetisnull(res, 0, 0))
+ {
+ makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
+ "DATABASE", dbname, NULL, NULL, buf);
+
+ appendPQExpBuffer(creaQry, "%s", buf->data);
+ PQclear(res);
+ count++;
+ }
+ else
+ {
+ PQclear(res);
+ break;
+ }
+ }
+
+ destroyPQExpBuffer(buf);
+}
+
+/*
+ * Dump user-and-database-specific configuration
+ */
+static void
+dumpDbRoleConfig(Archive *AH, PQExpBuffer creaQry)
+{
+ PGconn *conn = GetConnection(AH);
+ PQExpBuffer buf = createPQExpBuffer();
+ PGresult *res;
+ int i;
+
+ printfPQExpBuffer(buf, "SELECT rolname, datname, unnest(setconfig) "
+ "FROM pg_db_role_setting, pg_roles, pg_database "
+ "WHERE setrole = pg_roles.oid AND setdatabase = pg_database.oid");
+ res = executeQuery(conn, buf->data);
+
+ if (PQntuples(res) > 0)
+ {
+ appendPQExpBufferStr(creaQry, "\n\n--\n-- Per-Database Role Settings \n--\n\n");
+
+ resetPQExpBuffer(buf);
+ for (i = 0; i < PQntuples(res); i++)
+ {
+ makeAlterConfigCommand(conn, PQgetvalue(res, i, 2),
+ "ROLE", PQgetvalue(res, i, 0),
+ "DATABASE", PQgetvalue(res, i, 1), buf);
+ }
+
+ appendPQExpBuffer(creaQry, "%s", buf->data);
+ appendPQExpBufferStr(creaQry, "\n\n");
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(buf);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 41c5ff89b7..1f468e0cdb 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -38,16 +38,9 @@ static void dumpGroups(PGconn *conn);
static void dropTablespaces(PGconn *conn);
static void dumpTablespaces(PGconn *conn);
static void dropDBs(PGconn *conn);
-static void dumpCreateDB(PGconn *conn);
-static void dumpDatabaseConfig(PGconn *conn, const char *dbname);
static void dumpUserConfig(PGconn *conn, const char *username);
-static void dumpDbRoleConfig(PGconn *conn);
-static void makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name, const char *type2,
- const char *name2);
static void dumpDatabases(PGconn *conn);
static void dumpTimestamp(const char *msg);
-
static int runPgDump(const char *dbname);
static void buildShSecLabels(PGconn *conn, const char *catalog_name,
uint32 objectId, PQExpBuffer buffer,
@@ -55,15 +48,12 @@ static void buildShSecLabels(PGconn *conn, const char *catalog_name,
static PGconn *connectDatabase(const char *dbname, const char *connstr, const char *pghost, const char *pgport,
const char *pguser, trivalue prompt_password, bool fail_on_error);
static char *constructConnStr(const char **keywords, const char **values);
-static PGresult *executeQuery(PGconn *conn, const char *query);
static void executeCommand(PGconn *conn, const char *query);
static char pg_dump_bin[MAXPGPATH];
-static const char *progname;
static PQExpBuffer pgdumpopts;
static char *connstr = "";
static bool skip_acls = false;
-static bool verbose = false;
static bool dosync = true;
static int binary_upgrade = 0;
@@ -281,7 +271,7 @@ main(int argc, char *argv[])
break;
case 'v':
- verbose = true;
+ g_verbose = true;
appendPQExpBufferStr(pgdumpopts, " -v");
break;
@@ -498,7 +488,7 @@ main(int argc, char *argv[])
executeCommand(conn, "SET quote_all_identifiers = true");
fprintf(OPF, "--\n-- PostgreSQL database cluster dump\n--\n\n");
- if (verbose)
+ if (g_verbose)
dumpTimestamp("Started on");
/*
@@ -558,17 +548,6 @@ main(int argc, char *argv[])
/* Dump tablespaces */
if (!roles_only && !no_tablespaces)
dumpTablespaces(conn);
-
- /* Dump CREATE DATABASE commands */
- if (binary_upgrade || (!globals_only && !roles_only && !tablespaces_only))
- dumpCreateDB(conn);
-
- /* Dump role/database settings */
- if (!tablespaces_only && !roles_only)
- {
- if (server_version >= 90000)
- dumpDbRoleConfig(conn);
- }
}
if (!globals_only && !roles_only && !tablespaces_only)
@@ -576,7 +555,7 @@ main(int argc, char *argv[])
PQfinish(conn);
- if (verbose)
+ if (g_verbose)
dumpTimestamp("Completed on");
fprintf(OPF, "--\n-- PostgreSQL database cluster dump complete\n--\n\n");
@@ -1302,323 +1281,6 @@ dropDBs(PGconn *conn)
fprintf(OPF, "\n\n");
}
-/*
- * Dump commands to create each database.
- *
- * To minimize the number of reconnections (and possibly ensuing
- * password prompts) required by the output script, we emit all CREATE
- * DATABASE commands during the initial phase of the script, and then
- * run pg_dump for each database to dump the contents of that
- * database. We skip databases marked not datallowconn, since we'd be
- * unable to connect to them anyway (and besides, we don't want to
- * dump template0).
- */
-static void
-dumpCreateDB(PGconn *conn)
-{
- PQExpBuffer buf = createPQExpBuffer();
- char *default_encoding = NULL;
- char *default_collate = NULL;
- char *default_ctype = NULL;
- PGresult *res;
- int i;
-
- fprintf(OPF, "--\n-- Database creation\n--\n\n");
-
- /*
- * First, get the installation's default encoding and locale information.
- * We will dump encoding and locale specifications in the CREATE DATABASE
- * commands for just those databases with values different from defaults.
- *
- * We consider template0's encoding and locale to define the installation
- * default. Pre-8.4 installations do not have per-database locale
- * settings; for them, every database must necessarily be using the
- * installation default, so there's no need to do anything.
- */
- if (server_version >= 80400)
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "datcollate, datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
- else
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "null::text AS datcollate, null::text AS datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
-
- /* If for some reason the template DB isn't there, treat as unknown */
- if (PQntuples(res) > 0)
- {
- if (!PQgetisnull(res, 0, 0))
- default_encoding = pg_strdup(PQgetvalue(res, 0, 0));
- if (!PQgetisnull(res, 0, 1))
- default_collate = pg_strdup(PQgetvalue(res, 0, 1));
- if (!PQgetisnull(res, 0, 2))
- default_ctype = pg_strdup(PQgetvalue(res, 0, 2));
- }
-
- PQclear(res);
-
-
- /*
- * Now collect all the information about databases to dump.
- *
- * For the database ACLs, as of 9.6, we extract both the positive (as
- * datacl) and negative (as rdatacl) ACLs, relative to the default ACL for
- * databases, which are then passed to buildACLCommands() below.
- *
- * See buildACLQueries() and buildACLCommands().
- *
- * Note that we do not support initial privileges (pg_init_privs) on
- * databases.
- */
- if (server_version >= 90600)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba))) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba))) as datacls)"
- "AS datacl, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba)) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba)))) as rdatacls)"
- "AS rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 90300)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80400)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80100)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(usename, (select usename from pg_shadow where usesysid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "-1 as datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN pg_shadow u ON (datdba = usesysid) "
- "WHERE datallowconn ORDER BY 1");
-
- res = executeQuery(conn, buf->data);
-
- for (i = 0; i < PQntuples(res); i++)
- {
- char *dbname = PQgetvalue(res, i, 0);
- char *dbowner = PQgetvalue(res, i, 1);
- char *dbencoding = PQgetvalue(res, i, 2);
- char *dbcollate = PQgetvalue(res, i, 3);
- char *dbctype = PQgetvalue(res, i, 4);
- uint32 dbfrozenxid = atooid(PQgetvalue(res, i, 5));
- uint32 dbminmxid = atooid(PQgetvalue(res, i, 6));
- char *dbistemplate = PQgetvalue(res, i, 7);
- char *dbacl = PQgetvalue(res, i, 8);
- char *rdbacl = PQgetvalue(res, i, 9);
- char *dbconnlimit = PQgetvalue(res, i, 10);
- char *dbtablespace = PQgetvalue(res, i, 11);
- char *fdbname;
-
- fdbname = pg_strdup(fmtId(dbname));
-
- resetPQExpBuffer(buf);
-
- /*
- * Skip the CREATE DATABASE commands for "template1" and "postgres",
- * since they are presumably already there in the destination cluster.
- * We do want to emit their ACLs and config options if any, however.
- */
- if (strcmp(dbname, "template1") != 0 &&
- strcmp(dbname, "postgres") != 0)
- {
- appendPQExpBuffer(buf, "CREATE DATABASE %s", fdbname);
-
- appendPQExpBufferStr(buf, " WITH TEMPLATE = template0");
-
- if (strlen(dbowner) != 0)
- appendPQExpBuffer(buf, " OWNER = %s", fmtId(dbowner));
-
- if (default_encoding && strcmp(dbencoding, default_encoding) != 0)
- {
- appendPQExpBufferStr(buf, " ENCODING = ");
- appendStringLiteralConn(buf, dbencoding, conn);
- }
-
- if (default_collate && strcmp(dbcollate, default_collate) != 0)
- {
- appendPQExpBufferStr(buf, " LC_COLLATE = ");
- appendStringLiteralConn(buf, dbcollate, conn);
- }
-
- if (default_ctype && strcmp(dbctype, default_ctype) != 0)
- {
- appendPQExpBufferStr(buf, " LC_CTYPE = ");
- appendStringLiteralConn(buf, dbctype, conn);
- }
-
- /*
- * Output tablespace if it isn't the default. For default, it
- * uses the default from the template database. If tablespace is
- * specified and tablespace creation failed earlier, (e.g. no such
- * directory), the database creation will fail too. One solution
- * would be to use 'SET default_tablespace' like we do in pg_dump
- * for setting non-default database locations.
- */
- if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- appendPQExpBuffer(buf, " TABLESPACE = %s",
- fmtId(dbtablespace));
-
- if (strcmp(dbistemplate, "t") == 0)
- appendPQExpBuffer(buf, " IS_TEMPLATE = true");
-
- if (strcmp(dbconnlimit, "-1") != 0)
- appendPQExpBuffer(buf, " CONNECTION LIMIT = %s",
- dbconnlimit);
-
- appendPQExpBufferStr(buf, ";\n");
- }
- else if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- {
- /*
- * Cannot change tablespace of the database we're connected to, so
- * to move "postgres" to another tablespace, we connect to
- * "template1", and vice versa.
- */
- if (strcmp(dbname, "postgres") == 0)
- appendPQExpBuffer(buf, "\\connect template1\n");
- else
- appendPQExpBuffer(buf, "\\connect postgres\n");
-
- appendPQExpBuffer(buf, "ALTER DATABASE %s SET TABLESPACE %s;\n",
- fdbname, fmtId(dbtablespace));
-
- /* connect to original database */
- appendPsqlMetaConnect(buf, dbname);
- }
-
- if (binary_upgrade)
- {
- appendPQExpBufferStr(buf, "-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(buf, "UPDATE pg_catalog.pg_database "
- "SET datfrozenxid = '%u', datminmxid = '%u' "
- "WHERE datname = ",
- dbfrozenxid, dbminmxid);
- appendStringLiteralConn(buf, dbname, conn);
- appendPQExpBufferStr(buf, ";\n");
- }
-
- if (!skip_acls &&
- !buildACLCommands(fdbname, NULL, "DATABASE",
- dbacl, rdbacl, dbowner,
- "", server_version, buf))
- {
- fprintf(stderr, _("%s: could not parse ACL list (%s) for database \"%s\"\n"),
- progname, dbacl, fdbname);
- PQfinish(conn);
- exit_nicely(1);
- }
-
- fprintf(OPF, "%s", buf->data);
-
- dumpDatabaseConfig(conn, dbname);
-
- free(fdbname);
- }
-
- if (default_encoding)
- free(default_encoding);
- if (default_collate)
- free(default_collate);
- if (default_ctype)
- free(default_ctype);
-
- PQclear(res);
- destroyPQExpBuffer(buf);
-
- fprintf(OPF, "\n\n");
-}
-
-
-/*
- * Dump database-specific configuration
- */
-static void
-dumpDatabaseConfig(PGconn *conn, const char *dbname)
-{
- PQExpBuffer buf = createPQExpBuffer();
- int count = 1;
-
- for (;;)
- {
- PGresult *res;
-
- if (server_version >= 90000)
- printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting WHERE "
- "setrole = 0 AND setdatabase = (SELECT oid FROM pg_database WHERE datname = ", count);
- else
- printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE datname = ", count);
- appendStringLiteralConn(buf, dbname, conn);
-
- if (server_version >= 90000)
- appendPQExpBufferChar(buf, ')');
-
- res = executeQuery(conn, buf->data);
- if (PQntuples(res) == 1 &&
- !PQgetisnull(res, 0, 0))
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
- "DATABASE", dbname, NULL, NULL);
- PQclear(res);
- count++;
- }
- else
- {
- PQclear(res);
- break;
- }
- }
-
- destroyPQExpBuffer(buf);
-}
-
-
/*
* Dump user-specific configuration
@@ -1646,11 +1308,14 @@ dumpUserConfig(PGconn *conn, const char *username)
appendPQExpBufferChar(buf, ')');
res = executeQuery(conn, buf->data);
+ resetPQExpBuffer(buf);
+
if (PQntuples(res) == 1 &&
!PQgetisnull(res, 0, 0))
{
makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
- "ROLE", username, NULL, NULL);
+ "ROLE", username, NULL, NULL, buf);
+ fprintf(OPF, "%s", buf->data);
PQclear(res);
count++;
}
@@ -1665,85 +1330,6 @@ dumpUserConfig(PGconn *conn, const char *username)
}
-/*
- * Dump user-and-database-specific configuration
- */
-static void
-dumpDbRoleConfig(PGconn *conn)
-{
- PQExpBuffer buf = createPQExpBuffer();
- PGresult *res;
- int i;
-
- printfPQExpBuffer(buf, "SELECT rolname, datname, unnest(setconfig) "
- "FROM pg_db_role_setting, %s u, pg_database "
- "WHERE setrole = u.oid AND setdatabase = pg_database.oid", role_catalog);
- res = executeQuery(conn, buf->data);
-
- if (PQntuples(res) > 0)
- {
- fprintf(OPF, "--\n-- Per-Database Role Settings \n--\n\n");
-
- for (i = 0; i < PQntuples(res); i++)
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, i, 2),
- "ROLE", PQgetvalue(res, i, 0),
- "DATABASE", PQgetvalue(res, i, 1));
- }
-
- fprintf(OPF, "\n\n");
- }
-
- PQclear(res);
- destroyPQExpBuffer(buf);
-}
-
-
-/*
- * Helper function for dumpXXXConfig().
- */
-static void
-makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name,
- const char *type2, const char *name2)
-{
- char *pos;
- char *mine;
- PQExpBuffer buf;
-
- mine = pg_strdup(arrayitem);
- pos = strchr(mine, '=');
- if (pos == NULL)
- {
- free(mine);
- return;
- }
-
- buf = createPQExpBuffer();
-
- *pos = 0;
- appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
- if (type2 != NULL && name2 != NULL)
- appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
- appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
-
- /*
- * Some GUC variable names are 'LIST' type and hence must not be quoted.
- */
- if (pg_strcasecmp(mine, "DateStyle") == 0
- || pg_strcasecmp(mine, "search_path") == 0)
- appendPQExpBufferStr(buf, pos + 1);
- else
- appendStringLiteralConn(buf, pos + 1, conn);
- appendPQExpBufferStr(buf, ";\n");
-
- fprintf(OPF, "%s", buf->data);
- destroyPQExpBuffer(buf);
- free(mine);
-}
-
-
-
/*
* Dump contents of databases.
*/
@@ -1758,29 +1344,11 @@ dumpDatabases(PGconn *conn)
for (i = 0; i < PQntuples(res); i++)
{
int ret;
-
char *dbname = PQgetvalue(res, i, 0);
- PQExpBufferData connectbuf;
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: dumping database \"%s\"...\n"), progname, dbname);
- initPQExpBuffer(&connectbuf);
- appendPsqlMetaConnect(&connectbuf, dbname);
- fprintf(OPF, "%s\n", connectbuf.data);
- termPQExpBuffer(&connectbuf);
-
- /*
- * Restore will need to write to the target cluster. This connection
- * setting is emitted for pg_dumpall rather than in the code also used
- * by pg_dump, so that a cluster with databases or users which have
- * this flag turned on can still be replicated through pg_dumpall
- * without editing the file or stream. With pg_dump there are many
- * other ways to allow the file to be used, and leaving it out allows
- * users to protect databases from being accidental restore targets.
- */
- fprintf(OPF, "SET default_transaction_read_only = off;\n\n");
-
if (filename)
fclose(OPF);
@@ -1822,6 +1390,18 @@ runPgDump(const char *dbname)
appendPQExpBuffer(cmd, "\"%s\" %s", pg_dump_bin,
pgdumpopts->data);
+
+ /*
+ * The create individual database dump, pass the the create database option
+ * to pg_dump. But skip generating create database commands for default
+ * databases (template1, postgres).
+ */
+ if (strcmp(dbname, "template1") == 0 || strcmp(dbname, "postgres") == 0)
+ appendPQExpBufferStr(cmd, " --set-db-properties ");
+ else
+ appendPQExpBufferStr(cmd, " -C ");
+
+
/*
* If we have a filename, use the undocumented plain-append pg_dump
* format.
@@ -1840,7 +1420,7 @@ runPgDump(const char *dbname)
appendShellString(cmd, connstrbuf->data);
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: running \"%s\"\n"), progname, cmd->data);
fflush(stdout);
@@ -2133,31 +1713,7 @@ constructConnStr(const char **keywords, const char **values)
return connstr;
}
-/*
- * Run a query, return the results, exit program on failure.
- */
-static PGresult *
-executeQuery(PGconn *conn, const char *query)
-{
- PGresult *res;
- if (verbose)
- fprintf(stderr, _("%s: executing %s\n"), progname, query);
-
- res = PQexec(conn, query);
- if (!res ||
- PQresultStatus(res) != PGRES_TUPLES_OK)
- {
- fprintf(stderr, _("%s: query failed: %s"),
- progname, PQerrorMessage(conn));
- fprintf(stderr, _("%s: query was: %s\n"),
- progname, query);
- PQfinish(conn);
- exit_nicely(1);
- }
-
- return res;
-}
/*
* As above for a SQL command (which returns nothing).
@@ -2167,7 +1723,7 @@ executeCommand(PGconn *conn, const char *query)
{
PGresult *res;
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: executing %s\n"), progname, query);
res = PQexec(conn, query);
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 7cf9bdadb2..cbc92fdd05 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2680,7 +2680,7 @@ qr/CREATE CAST \(timestamp with time zone AS interval\) WITH FUNCTION pg_catalog
'CREATE DATABASE postgres' => {
all_runs => 1,
regexp => qr/^
- \QCREATE DATABASE postgres WITH TEMPLATE = template0 \E
+ \QCREATE DATABASE postgres WITH TEMPLATE = template0\E
.*;/xm,
like => { createdb => 1, },
unlike => {
@@ -2714,7 +2714,7 @@ qr/CREATE CAST \(timestamp with time zone AS interval\) WITH FUNCTION pg_catalog
create_order => 47,
create_sql => 'CREATE DATABASE dump_test;',
regexp => qr/^
- \QCREATE DATABASE dump_test WITH TEMPLATE = template0 \E
+ \QCREATE DATABASE dump_test WITH TEMPLATE = template0\E
.*;/xm,
like => { pg_dumpall_dbprivs => 1, },
unlike => {
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 77c03ac05b..de07eaf7d1 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -48,6 +48,7 @@ generate_old_dump(void)
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
PQExpBufferData connstr,
escaped_connstr;
+ PQExpBuffer create_opts = createPQExpBuffer();
initPQExpBuffer(&connstr);
appendPQExpBuffer(&connstr, "dbname=");
@@ -60,13 +61,20 @@ generate_old_dump(void)
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
+ if (strcmp(old_db->db_name, "template1") == 0 || strcmp(old_db->db_name, "postgres") == 0)
+ appendPQExpBufferStr(create_opts, " --set-db-properties ");
+ else
+ appendPQExpBufferStr(create_opts, " -C ");
+
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade %s "
+ "--format=custom %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
- log_opts.verbose ? "--verbose" : "",
+ create_opts->data, log_opts.verbose ? "--verbose" : "",
sql_file_name, escaped_connstr.data);
+ termPQExpBuffer(create_opts);
termPQExpBuffer(&escaped_connstr);
}
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index c10103f0bf..f1438a81b9 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -46,7 +46,7 @@
#endif
static void prepare_new_cluster(void);
-static void prepare_new_databases(void);
+static void prepare_new_globals(void);
static void create_new_objects(void);
static void copy_xact_xlog_xid(void);
static void set_frozenxids(bool minmxid_only);
@@ -124,7 +124,7 @@ main(int argc, char **argv)
/* -- NEW -- */
start_postmaster(&new_cluster, true);
- prepare_new_databases();
+ prepare_new_globals();
create_new_objects();
@@ -271,7 +271,7 @@ prepare_new_cluster(void)
static void
-prepare_new_databases(void)
+prepare_new_globals(void)
{
/*
* We set autovacuum_freeze_max_age to its maximum value so autovacuum
@@ -294,9 +294,6 @@ prepare_new_databases(void)
new_cluster.bindir, cluster_conn_opts(&new_cluster),
GLOBALS_DUMP_FILE);
check_ok();
-
- /* we load this to get a current list of databases */
- get_db_and_rel_infos(&new_cluster);
}
@@ -312,15 +309,6 @@ create_new_objects(void)
char sql_file_name[MAXPGPATH],
log_file_name[MAXPGPATH];
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
- PQExpBufferData connstr,
- escaped_connstr;
-
- initPQExpBuffer(&connstr);
- appendPQExpBuffer(&connstr, "dbname=");
- appendConnStrVal(&connstr, old_db->db_name);
- initPQExpBuffer(&escaped_connstr);
- appendShellString(&escaped_connstr, connstr.data);
- termPQExpBuffer(&connstr);
pg_log(PG_STATUS, "%s", old_db->db_name);
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
@@ -332,13 +320,10 @@ create_new_objects(void)
*/
parallel_exec_prog(log_file_name,
NULL,
- "\"%s/pg_restore\" %s --exit-on-error --verbose --dbname %s \"%s\"",
+ "\"%s/pg_restore\" %s --create --exit-on-error --verbose --dbname template1 \"%s\"",
new_cluster.bindir,
cluster_conn_opts(&new_cluster),
- escaped_connstr.data,
sql_file_name);
-
- termPQExpBuffer(&escaped_connstr);
}
/* reap all children */
--
2.15.0.windows.1
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
@ 2018-01-05 03:54 ` Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
0 siblings, 1 reply; 33+ messages in thread
From: Vaishnavi Prabakaran @ 2018-01-05 03:54 UTC (permalink / raw)
To: Haribabu Kommi <[email protected]>; +Cc: Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Wed, Dec 13, 2017 at 1:50 PM, Haribabu Kommi <[email protected]>
wrote:
> Rebased patch attached that fixes the documentation build problem.
>
> Latest patch applies without noise. And here are some minor comments
+ <varlistentry>
+ <term><option>--set-db-properties</option></term>
+ <listitem>
+ <para>
+ This option is to skip create database command without ignoring
the rest
+ of the DATABASE specific commands and it is used when the database
already
+ exists in the restore location. This option cannot be used with
either of
+ <option>-C/--create</option> or <option>-c/--clean</option>
options.
+ </para>
I see the description is framed based on previous flag "
--skip-default-create-db" , and I think this needs to be re-phrased to
describe what it does , e.g: This option is to set database properties for
already existing database without issuing create database command ....".
+ printf(_(" --set-db-properties dump db properties, for use by
upgrade utilities only\n"));
After the discussion about its usage up-thread, this needs to be corrected.
Thanks & Regards,
Vaishnavi,
Fujitsu Australia.
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
@ 2018-01-05 05:32 ` Haribabu Kommi <[email protected]>
2018-01-05 06:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
0 siblings, 2 replies; 33+ messages in thread
From: Haribabu Kommi @ 2018-01-05 05:32 UTC (permalink / raw)
To: Vaishnavi Prabakaran <[email protected]>; +Cc: Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Fri, Jan 5, 2018 at 2:54 PM, Vaishnavi Prabakaran <
[email protected]> wrote:
>
> On Wed, Dec 13, 2017 at 1:50 PM, Haribabu Kommi <[email protected]>
> wrote:
>
>> Rebased patch attached that fixes the documentation build problem.
>>
>> Latest patch applies without noise. And here are some minor comments
>
Thanks for the review.
> + <varlistentry>
> + <term><option>--set-db-properties</option></term>
> + <listitem>
> + <para>
> + This option is to skip create database command without ignoring
> the rest
> + of the DATABASE specific commands and it is used when the
> database already
> + exists in the restore location. This option cannot be used with
> either of
> + <option>-C/--create</option> or <option>-c/--clean</option>
> options.
> + </para>
>
> I see the description is framed based on previous flag "
> --skip-default-create-db" , and I think this needs to be re-phrased to
> describe what it does , e.g: This option is to set database properties for
> already existing database without issuing create database command ....".
>
Updated with more details.
+ printf(_(" --set-db-properties dump db properties, for use by
> upgrade utilities only\n"));
> After the discussion about its usage up-thread, this needs to be corrected.
>
Corrected.
Updated patch attached.
Regards,
Hari Babu
Fujitsu Australia
Attachments:
[application/octet-stream] pg_dump-and-pg_dumpall-database-handling-refactoring_v12.patch (49.8K, ../../CAJrrPGedLZDH4enjJvySie9wtfR+85Kqs7E-SuLe710VL0dhcw@mail.gmail.com/3-pg_dump-and-pg_dumpall-database-handling-refactoring_v12.patch)
download | inline diff:
From b9eb77bc0dc2887968c5c183eba4c9c23d09a371 Mon Sep 17 00:00:00 2001
From: Hari Babu Kommi <[email protected]>
Date: Mon, 21 Aug 2017 14:29:58 +1000
Subject: [PATCH] pg_dump and pg_dumpall database handling refactoring
The Database handling especially with ACL's is not proper
between pg_dump and pg_dumpall. So to avoid these problems,
Now the entire database handling is moved to pg_dump and
the pg_dumpall just invokes pg_dump to print the database
attributes.
---
doc/src/sgml/ref/pg_dump.sgml | 33 ++-
doc/src/sgml/ref/pg_dumpall.sgml | 4 +
src/bin/pg_dump/Makefile | 4 +-
src/bin/pg_dump/dumputils.c | 67 +++++
src/bin/pg_dump/dumputils.h | 7 +
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_backup_archiver.c | 2 +
src/bin/pg_dump/pg_backup_utils.c | 3 +
src/bin/pg_dump/pg_backup_utils.h | 1 +
src/bin/pg_dump/pg_dump.c | 309 +++++++++++++++++++---
src/bin/pg_dump/pg_dumpall.c | 488 ++---------------------------------
src/bin/pg_dump/t/002_pg_dump.pl | 4 +-
src/bin/pg_upgrade/dump.c | 12 +-
src/bin/pg_upgrade/pg_upgrade.c | 23 +-
14 files changed, 422 insertions(+), 537 deletions(-)
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 08cad68199..63934a75ec 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -192,8 +192,11 @@ PostgreSQL documentation
database itself and reconnect to the created database. (With a
script of this form, it doesn't matter which database in the
destination installation you connect to before running the script.)
- If <option>--clean</option> is also specified, the script drops and
- recreates the target database before reconnecting to it.
+ It also sets all the database-level properties such as ownership,
+ ACLs, <command>ALTER DATABASE ... SET</command> commands, and
+ <command>ALTER ROLE IN DATABASE ... SET</command> commands.
+ If <option>--clean</option> is also specified, the script drops
+ and recreates the target database before reconnecting to it.
</para>
<para>
@@ -706,6 +709,28 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--set-db-properties</option></term>
+ <listitem>
+ <para>
+ This option is to dump the database properties without issuing the
+ <command>CREATE DATABASE</command> command. The generated dump is
+ used to restore the databases on a target location where the database
+ is already exists. This option cannot be used with either of
+ <option>-C/--create</option> or <option>-c/--clean</option> options.
+ </para>
+
+ <note>
+ <para>
+ The Dump file generated with this option cannot be restored using
+ <option>-c/--clean</option> and <option>-C/--create</option> options of
+ <application>pg_restore</application> command. This is due to not available
+ of <command>CREATE DATABASE</command> command in the dump file.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--enable-row-security</option></term>
<listitem>
@@ -1231,10 +1256,6 @@ CREATE DATABASE foo WITH TEMPLATE template0;
<command>ANALYZE</command> after restoring from a dump file
to ensure optimal performance; see <xref linkend="vacuum-for-statistics"/>
and <xref linkend="autovacuum"/> for more information.
- The dump file also does not
- contain any <command>ALTER DATABASE ... SET</command> commands;
- these settings are dumped by <xref linkend="app-pg-dumpall"/>,
- along with database users and other installation-wide settings.
</para>
<para>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5196a211b1..cb3957d669 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -129,6 +129,10 @@ PostgreSQL documentation
<listitem>
<para>
Dump only global objects (roles and tablespaces), no databases.
+ Any database role configuration settings are not dumped with
+ this option. User needs to use <command>pg_dump</command> with
+ <option>--create</option> option to dump
+ <command>ALTER ROLE IN DATABASE ... SET</command> commands.
</para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index e3bfc95f16..99387ddd99 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -31,8 +31,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o | submake-libpq submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o pg_backup_utils.o | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o pg_backup_utils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/dumputils.c b/src/bin/pg_dump/dumputils.c
index 32ad600fd0..9ed4171b5d 100644
--- a/src/bin/pg_dump/dumputils.c
+++ b/src/bin/pg_dump/dumputils.c
@@ -807,3 +807,70 @@ buildACLQueries(PQExpBuffer acl_subquery, PQExpBuffer racl_subquery,
printfPQExpBuffer(init_racl_subquery, "NULL");
}
}
+
+/*
+ * Helper function for dumpXXXConfig().
+ *
+ * Frame the ALTER ".." SET ".." commands and fill it in buf.
+ */
+void
+makeAlterConfigCommand(PGconn *conn, const char *configitem,
+ const char *type, const char *name,
+ const char *type2, const char *name2,
+ PQExpBuffer buf)
+{
+ char *pos;
+ char *mine;
+
+ mine = pg_strdup(configitem);
+ pos = strchr(mine, '=');
+ if (pos == NULL)
+ {
+ pg_free(mine);
+ return;
+ }
+
+ *pos++ = '\0';
+ appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
+ if (type2 != NULL && name2 != NULL)
+ appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
+ appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be quoted.
+ */
+ if (pg_strcasecmp(mine, "DateStyle") == 0
+ || pg_strcasecmp(mine, "search_path") == 0)
+ appendPQExpBufferStr(buf, pos);
+ else
+ appendStringLiteralConn(buf, pos, conn);
+ appendPQExpBufferStr(buf, ";\n");
+
+ pg_free(mine);
+}
+
+/*
+ * Run a query, return the results, exit program on failure.
+ */
+PGresult *
+executeQuery(PGconn *conn, const char *query)
+{
+ PGresult *res;
+
+ if (g_verbose)
+ fprintf(stderr, _("%s: executing %s\n"), progname, query);
+
+ res = PQexec(conn, query);
+ if (!res ||
+ PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ fprintf(stderr, _("%s: query failed: %s"),
+ progname, PQerrorMessage(conn));
+ fprintf(stderr, _("%s: query was: %s\n"),
+ progname, query);
+ PQfinish(conn);
+ exit_nicely(1);
+ }
+
+ return res;
+}
diff --git a/src/bin/pg_dump/dumputils.h b/src/bin/pg_dump/dumputils.h
index d5f150dfa0..28b84f50d3 100644
--- a/src/bin/pg_dump/dumputils.h
+++ b/src/bin/pg_dump/dumputils.h
@@ -17,6 +17,7 @@
#include "libpq-fe.h"
#include "pqexpbuffer.h"
+#include "pg_backup_utils.h"
/*
* Preferred strftime(3) format specifier for printing timestamps in pg_dump
@@ -56,4 +57,10 @@ extern void buildACLQueries(PQExpBuffer acl_subquery, PQExpBuffer racl_subquery,
const char *acl_column, const char *acl_owner,
const char *obj_kind, bool binary_upgrade);
+extern void makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
+ const char *type, const char *name, const char *type2,
+ const char *name2, PQExpBuffer buf);
+
+extern PGresult *executeQuery(PGconn *conn, const char *query);
+
#endif /* DUMPUTILS_H */
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..597c4f70e6 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -117,6 +117,7 @@ typedef struct _restoreOptions
int suppressDumpWarnings; /* Suppress output of WARNING entries
* to stderr */
bool single_txn;
+ int set_db_properties;
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
@@ -158,6 +159,7 @@ typedef struct _dumpOptions
int use_setsessauth;
int enable_row_security;
int load_via_partition_root;
+ int set_db_properties;
/* default, if no "inclusion" switches appear, is to dump everything */
bool include_everything;
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 41741aefbc..ea5fba830f 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -831,6 +831,8 @@ restore_toc_entry(ArchiveHandle *AH, TocEntry *te, bool is_parallel)
ahlog(AH, 1, "connecting to new database \"%s\"\n", te->tag);
_reconnectToDB(AH, te->tag);
+
+ ahprintf(AH, "SET default_transaction_read_only = off;\n\n");
ropt->dbname = connstr.data;
}
}
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index df5fe708ba..07f2a43e35 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -19,6 +19,9 @@
/* Globals exported by this file */
const char *progname = NULL;
+/* User wants verbose narration of our activities */
+bool g_verbose = false;
+
#define MAX_ON_EXIT_NICELY 20
static struct
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 6eaf302fc7..b55fe5e8c3 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -26,6 +26,7 @@ typedef enum /* bits returned by set_dump_section */
typedef void (*on_exit_nicely_callback) (int code, void *arg);
extern const char *progname;
+extern bool g_verbose;
extern void set_dump_section(const char *arg, int *dumpSections);
extern void write_msg(const char *modulename, const char *fmt,...) pg_attribute_printf(2, 3);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 27628a397c..2f607abd1c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -88,9 +88,6 @@ typedef enum OidOptions
zeroAsNone = 8
} OidOptions;
-/* global decls */
-bool g_verbose; /* User wants verbose narration of our
- * activities. */
static bool dosync = true; /* Issue fsync() to make dump durable on disk. */
/* subquery used to convert user ID (eg, datdba) to user name */
@@ -273,6 +270,8 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
static char *get_synchronized_snapshot(Archive *fout);
static void setupDumpWorker(Archive *AHX);
static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void dumpDatabaseConfig(Archive *fout, PQExpBuffer creaQry, const char *dbname);
+static void dumpDbRoleConfig(Archive *AH, PQExpBuffer creaQry);
int
@@ -362,6 +361,7 @@ main(int argc, char **argv)
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
{"no-sync", no_argument, NULL, 7},
+ {"set-db-properties", no_argument, &dopt.set_db_properties, 1},
{NULL, 0, NULL, 0}
};
@@ -581,6 +581,18 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.set_db_properties && dopt.outputCreateDB)
+ {
+ write_msg(NULL, "option --set-db-properties and -C/--create cannot be used together\n");
+ exit_nicely(1);
+ }
+
+ if (dopt.set_db_properties && dopt.outputClean)
+ {
+ write_msg(NULL, "option --set-db-properties and -c/--clean cannot be used together\n");
+ exit_nicely(1);
+ }
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -860,7 +872,12 @@ main(int argc, char **argv)
ropt->dumpSections = dopt.dumpSections;
ropt->aclsSkip = dopt.aclsSkip;
ropt->superuser = dopt.outputSuperuser;
- ropt->createDB = dopt.outputCreateDB;
+
+ /*
+ * treat --set-db-properties are also as DATABASE commands like
+ * -C/--create
+ */
+ ropt->createDB = dopt.outputCreateDB || dopt.set_db_properties;
ropt->noOwner = dopt.outputNoOwner;
ropt->noTablespace = dopt.outputNoTablespaces;
ropt->disable_triggers = dopt.disable_triggers;
@@ -875,6 +892,7 @@ main(int argc, char **argv)
ropt->enable_row_security = dopt.enable_row_security;
ropt->sequence_data = dopt.sequence_data;
ropt->binary_upgrade = dopt.binary_upgrade;
+ ropt->set_db_properties = dopt.set_db_properties;
if (compressLevel == -1)
ropt->compression = 0;
@@ -973,6 +991,7 @@ help(const char *progname)
printf(_(" --use-set-session-authorization\n"
" use SET SESSION AUTHORIZATION commands instead of\n"
" ALTER OWNER commands to set ownership\n"));
+ printf(_(" --set-db-properties dump database properties without CREATE DATABASE command\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=DBNAME database to dump\n"));
@@ -2528,25 +2547,34 @@ dumpDatabase(Archive *fout)
PQExpBuffer dbQry = createPQExpBuffer();
PQExpBuffer delQry = createPQExpBuffer();
PQExpBuffer creaQry = createPQExpBuffer();
+ PQExpBuffer aclQry = createPQExpBuffer();
PGconn *conn = GetConnection(fout);
PGresult *res;
int i_tableoid,
i_oid,
i_dba,
+ i_dbacl,
+ i_rdbacl,
i_encoding,
i_collate,
i_ctype,
i_frozenxid,
i_minmxid,
- i_tablespace;
+ i_tablespace,
+ i_dbistemplate,
+ i_dbconnlimit;
CatalogId dbCatId;
DumpId dbDumpId;
const char *datname,
*dba,
+ *dbacl,
+ *rdbacl,
*encoding,
*collate,
*ctype,
- *tablespace;
+ *tablespace,
+ *dbistemplate,
+ *dbconnlimit;
uint32 frozenxid,
minmxid;
@@ -2558,13 +2586,49 @@ dumpDatabase(Archive *fout)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- /* Get the database owner and parameters from pg_database */
- if (fout->remoteVersion >= 90300)
+ /*
+ * Now collect all the information about databases to dump.
+ *
+ * For the database ACLs, as of 9.6, we extract both the positive (as
+ * datacl) and negative (as rdatacl) ACLs, relative to the default ACL for
+ * databases, which are then passed to buildACLCommands() below.
+ *
+ * See buildACLQueries() and buildACLCommands().
+ *
+ * Note that we do not support initial privileges (pg_init_privs) on
+ * databases.
+ */
+
+ if (fout->remoteVersion >= 90600)
+ {
+ appendPQExpBuffer(dbQry,
+ "SELECT tableoid, oid, "
+ "(%s datdba) AS dba, "
+ "pg_encoding_to_char(encoding) AS encoding, "
+ "datcollate, datctype, datfrozenxid, datminmxid, datistemplate, "
+ "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
+ " SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba))) AS acl "
+ " EXCEPT SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba))) as datacls)"
+ " AS datacl, "
+ "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
+ " SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba)) AS acl "
+ " EXCEPT SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba)))) as rdatacls)"
+ " AS rdatacl, "
+ "datconnlimit, "
+ "(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
+ "shobj_description(oid, 'pg_database') AS description "
+
+ "FROM pg_database "
+ "WHERE datname = ",
+ username_subquery);
+ appendStringLiteralAH(dbQry, datname, fout);
+ }
+ else if (fout->remoteVersion >= 90300)
{
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "datcollate, datctype, datfrozenxid, datminmxid, "
+ "datcollate, datctype, datfrozenxid, datminmxid, datistemplate, datacl,'' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2578,7 +2642,7 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
+ "datcollate, datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl, '' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2592,7 +2656,7 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl,'' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2606,7 +2670,8 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl,'' as rdatacl,"
+ "-1 as datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace "
"FROM pg_database "
"WHERE datname = ",
@@ -2625,6 +2690,10 @@ dumpDatabase(Archive *fout)
i_frozenxid = PQfnumber(res, "datfrozenxid");
i_minmxid = PQfnumber(res, "datminmxid");
i_tablespace = PQfnumber(res, "tablespace");
+ i_dbacl = PQfnumber(res, "datacl");
+ i_rdbacl = PQfnumber(res, "rdatacl");
+ i_dbistemplate = PQfnumber(res, "datistemplate");
+ i_dbconnlimit = PQfnumber(res, "datconnlimit");
dbCatId.tableoid = atooid(PQgetvalue(res, 0, i_tableoid));
dbCatId.oid = atooid(PQgetvalue(res, 0, i_oid));
@@ -2635,40 +2704,63 @@ dumpDatabase(Archive *fout)
frozenxid = atooid(PQgetvalue(res, 0, i_frozenxid));
minmxid = atooid(PQgetvalue(res, 0, i_minmxid));
tablespace = PQgetvalue(res, 0, i_tablespace);
+ dbacl = PQgetvalue(res, 0, i_dbacl);
+ rdbacl = PQgetvalue(res, 0, i_rdbacl);
+ dbistemplate = PQgetvalue(res, 0, i_dbistemplate);
+ dbconnlimit = PQgetvalue(res, 0, i_dbconnlimit);
- appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
- fmtId(datname));
- if (strlen(encoding) > 0)
- {
- appendPQExpBufferStr(creaQry, " ENCODING = ");
- appendStringLiteralAH(creaQry, encoding, fout);
- }
- if (strlen(collate) > 0)
+ /*
+ * Skip the CREATE DATABASE commands --set-db-properties is enabled, since
+ * they are presumably already there in the destination cluster. We do
+ * want to emit their ACLs and config options if any, however.
+ */
+ if (!dopt->set_db_properties)
{
- appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
- appendStringLiteralAH(creaQry, collate, fout);
+ appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
+ fmtId(datname));
+
+ if (strlen(encoding) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " ENCODING = ");
+ appendStringLiteralAH(creaQry, encoding, fout);
+ }
+ if (strlen(collate) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
+ appendStringLiteralAH(creaQry, collate, fout);
+ }
+ if (strlen(ctype) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
+ appendStringLiteralAH(creaQry, ctype, fout);
+ }
+ if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
+ !dopt->outputNoTablespaces)
+ {
+ appendPQExpBuffer(creaQry, " TABLESPACE = %s", fmtId(tablespace));
+ }
+
+ if (strlen(dbistemplate) > 0 && strcmp(dbistemplate, "t") == 0)
+ appendPQExpBuffer(creaQry, " IS_TEMPLATE = true");
+
+ if (strlen(dbconnlimit) > 0 && strcmp(dbconnlimit, "-1") != 0)
+ appendPQExpBuffer(creaQry, " CONNECTION LIMIT = %s",
+ dbconnlimit);
+
+ appendPQExpBufferStr(creaQry, ";\n");
}
- if (strlen(ctype) > 0)
+ else if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 && !dopt->outputNoTablespaces)
{
- appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
- appendStringLiteralAH(creaQry, ctype, fout);
- }
- if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
- !dopt->outputNoTablespaces)
- appendPQExpBuffer(creaQry, " TABLESPACE = %s",
- fmtId(tablespace));
- appendPQExpBufferStr(creaQry, ";\n");
+ if (strcmp(datname, "postgres") == 0)
+ appendPQExpBuffer(creaQry, "\\connect template1\n");
+ else if (strcmp(datname, "template1") == 0)
+ appendPQExpBuffer(creaQry, "\\connect postgres\n");
- if (dopt->binary_upgrade)
- {
- appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
- "SET datfrozenxid = '%u', datminmxid = '%u'\n"
- "WHERE datname = ",
- frozenxid, minmxid);
- appendStringLiteralAH(creaQry, datname, fout);
- appendPQExpBufferStr(creaQry, ";\n");
+ appendPQExpBuffer(creaQry, "ALTER DATABASE %s SET TABLESPACE %s;\n",
+ datname, fmtId(tablespace));
+ /* connect to original database */
+ appendPsqlMetaConnect(creaQry, datname);
}
appendPQExpBuffer(delQry, "DROP DATABASE %s;\n",
@@ -2694,6 +2786,58 @@ dumpDatabase(Archive *fout)
NULL, /* Dumper */
NULL); /* Dumper Arg */
+ resetPQExpBuffer(creaQry);
+
+ if (dopt->binary_upgrade)
+ {
+ appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
+ appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
+ "SET datfrozenxid = '%u', datminmxid = '%u'\n"
+ "WHERE datname = ",
+ frozenxid, minmxid);
+ appendStringLiteralAH(creaQry, datname, fout);
+ appendPQExpBufferStr(creaQry, ";\n");
+
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ creaQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+ resetPQExpBuffer(creaQry);
+ }
+
+ if (!dopt->aclsSkip &&
+ !buildACLCommands(datname, NULL, "DATABASE",
+ dbacl, rdbacl, dba,
+ "", fout->remoteVersion, aclQry))
+ {
+ exit_horribly(NULL, _("%s: could not parse ACL list (%s) for database \"%s\"\n"), progname, dbacl, datname);
+ }
+
+ if (strlen(aclQry->data))
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ aclQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ /* Dump database specific configuration */
+ dumpDatabaseConfig(fout, creaQry, datname);
+
+ /* Dump user and database specific configuration */
+ if (fout->remoteVersion >= 90000)
+ dumpDbRoleConfig(fout, creaQry);
+
+ if (strlen(creaQry->data))
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ creaQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
/*
* pg_largeobject and pg_largeobject_metadata come from the old system
* intact, so set their relfrozenxids and relminmxids.
@@ -2847,6 +2991,7 @@ dumpDatabase(Archive *fout)
destroyPQExpBuffer(dbQry);
destroyPQExpBuffer(delQry);
destroyPQExpBuffer(creaQry);
+ destroyPQExpBuffer(aclQry);
}
/*
@@ -18091,3 +18236,87 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
if (!res)
write_msg(NULL, "WARNING: could not parse reloptions array\n");
}
+
+/*
+ * Dump database-specific configuration
+ */
+static void
+dumpDatabaseConfig(Archive *AH, PQExpBuffer creaQry, const char *dbname)
+{
+ PGconn *conn = GetConnection(AH);
+ PQExpBuffer buf = createPQExpBuffer();
+ int count = 1;
+
+ for (;;)
+ {
+ PGresult *res;
+
+ if (AH->remoteVersion >= 90000)
+ printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting WHERE "
+ "setrole = 0 AND setdatabase = (SELECT oid FROM pg_database WHERE datname = ", count);
+ else
+ printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE datname = ", count);
+
+ appendStringLiteralConn(buf, dbname, conn);
+
+ if (AH->remoteVersion >= 90000)
+ appendPQExpBufferChar(buf, ')');
+
+ res = executeQuery(conn, buf->data);
+ resetPQExpBuffer(buf);
+
+ if (PQntuples(res) == 1 &&
+ !PQgetisnull(res, 0, 0))
+ {
+ makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
+ "DATABASE", dbname, NULL, NULL, buf);
+
+ appendPQExpBuffer(creaQry, "%s", buf->data);
+ PQclear(res);
+ count++;
+ }
+ else
+ {
+ PQclear(res);
+ break;
+ }
+ }
+
+ destroyPQExpBuffer(buf);
+}
+
+/*
+ * Dump user-and-database-specific configuration
+ */
+static void
+dumpDbRoleConfig(Archive *AH, PQExpBuffer creaQry)
+{
+ PGconn *conn = GetConnection(AH);
+ PQExpBuffer buf = createPQExpBuffer();
+ PGresult *res;
+ int i;
+
+ printfPQExpBuffer(buf, "SELECT rolname, datname, unnest(setconfig) "
+ "FROM pg_db_role_setting, pg_roles, pg_database "
+ "WHERE setrole = pg_roles.oid AND setdatabase = pg_database.oid");
+ res = executeQuery(conn, buf->data);
+
+ if (PQntuples(res) > 0)
+ {
+ appendPQExpBufferStr(creaQry, "\n\n--\n-- Per-Database Role Settings \n--\n\n");
+
+ resetPQExpBuffer(buf);
+ for (i = 0; i < PQntuples(res); i++)
+ {
+ makeAlterConfigCommand(conn, PQgetvalue(res, i, 2),
+ "ROLE", PQgetvalue(res, i, 0),
+ "DATABASE", PQgetvalue(res, i, 1), buf);
+ }
+
+ appendPQExpBuffer(creaQry, "%s", buf->data);
+ appendPQExpBufferStr(creaQry, "\n\n");
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(buf);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 3dd2c3871e..ae2c5fa17a 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -38,16 +38,9 @@ static void dumpGroups(PGconn *conn);
static void dropTablespaces(PGconn *conn);
static void dumpTablespaces(PGconn *conn);
static void dropDBs(PGconn *conn);
-static void dumpCreateDB(PGconn *conn);
-static void dumpDatabaseConfig(PGconn *conn, const char *dbname);
static void dumpUserConfig(PGconn *conn, const char *username);
-static void dumpDbRoleConfig(PGconn *conn);
-static void makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name, const char *type2,
- const char *name2);
static void dumpDatabases(PGconn *conn);
static void dumpTimestamp(const char *msg);
-
static int runPgDump(const char *dbname);
static void buildShSecLabels(PGconn *conn, const char *catalog_name,
uint32 objectId, PQExpBuffer buffer,
@@ -55,15 +48,12 @@ static void buildShSecLabels(PGconn *conn, const char *catalog_name,
static PGconn *connectDatabase(const char *dbname, const char *connstr, const char *pghost, const char *pgport,
const char *pguser, trivalue prompt_password, bool fail_on_error);
static char *constructConnStr(const char **keywords, const char **values);
-static PGresult *executeQuery(PGconn *conn, const char *query);
static void executeCommand(PGconn *conn, const char *query);
static char pg_dump_bin[MAXPGPATH];
-static const char *progname;
static PQExpBuffer pgdumpopts;
static char *connstr = "";
static bool skip_acls = false;
-static bool verbose = false;
static bool dosync = true;
static int binary_upgrade = 0;
@@ -281,7 +271,7 @@ main(int argc, char *argv[])
break;
case 'v':
- verbose = true;
+ g_verbose = true;
appendPQExpBufferStr(pgdumpopts, " -v");
break;
@@ -498,7 +488,7 @@ main(int argc, char *argv[])
executeCommand(conn, "SET quote_all_identifiers = true");
fprintf(OPF, "--\n-- PostgreSQL database cluster dump\n--\n\n");
- if (verbose)
+ if (g_verbose)
dumpTimestamp("Started on");
/*
@@ -558,17 +548,6 @@ main(int argc, char *argv[])
/* Dump tablespaces */
if (!roles_only && !no_tablespaces)
dumpTablespaces(conn);
-
- /* Dump CREATE DATABASE commands */
- if (binary_upgrade || (!globals_only && !roles_only && !tablespaces_only))
- dumpCreateDB(conn);
-
- /* Dump role/database settings */
- if (!tablespaces_only && !roles_only)
- {
- if (server_version >= 90000)
- dumpDbRoleConfig(conn);
- }
}
if (!globals_only && !roles_only && !tablespaces_only)
@@ -576,7 +555,7 @@ main(int argc, char *argv[])
PQfinish(conn);
- if (verbose)
+ if (g_verbose)
dumpTimestamp("Completed on");
fprintf(OPF, "--\n-- PostgreSQL database cluster dump complete\n--\n\n");
@@ -1302,323 +1281,6 @@ dropDBs(PGconn *conn)
fprintf(OPF, "\n\n");
}
-/*
- * Dump commands to create each database.
- *
- * To minimize the number of reconnections (and possibly ensuing
- * password prompts) required by the output script, we emit all CREATE
- * DATABASE commands during the initial phase of the script, and then
- * run pg_dump for each database to dump the contents of that
- * database. We skip databases marked not datallowconn, since we'd be
- * unable to connect to them anyway (and besides, we don't want to
- * dump template0).
- */
-static void
-dumpCreateDB(PGconn *conn)
-{
- PQExpBuffer buf = createPQExpBuffer();
- char *default_encoding = NULL;
- char *default_collate = NULL;
- char *default_ctype = NULL;
- PGresult *res;
- int i;
-
- fprintf(OPF, "--\n-- Database creation\n--\n\n");
-
- /*
- * First, get the installation's default encoding and locale information.
- * We will dump encoding and locale specifications in the CREATE DATABASE
- * commands for just those databases with values different from defaults.
- *
- * We consider template0's encoding and locale to define the installation
- * default. Pre-8.4 installations do not have per-database locale
- * settings; for them, every database must necessarily be using the
- * installation default, so there's no need to do anything.
- */
- if (server_version >= 80400)
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "datcollate, datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
- else
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "null::text AS datcollate, null::text AS datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
-
- /* If for some reason the template DB isn't there, treat as unknown */
- if (PQntuples(res) > 0)
- {
- if (!PQgetisnull(res, 0, 0))
- default_encoding = pg_strdup(PQgetvalue(res, 0, 0));
- if (!PQgetisnull(res, 0, 1))
- default_collate = pg_strdup(PQgetvalue(res, 0, 1));
- if (!PQgetisnull(res, 0, 2))
- default_ctype = pg_strdup(PQgetvalue(res, 0, 2));
- }
-
- PQclear(res);
-
-
- /*
- * Now collect all the information about databases to dump.
- *
- * For the database ACLs, as of 9.6, we extract both the positive (as
- * datacl) and negative (as rdatacl) ACLs, relative to the default ACL for
- * databases, which are then passed to buildACLCommands() below.
- *
- * See buildACLQueries() and buildACLCommands().
- *
- * Note that we do not support initial privileges (pg_init_privs) on
- * databases.
- */
- if (server_version >= 90600)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba))) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba))) as datacls)"
- "AS datacl, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba)) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba)))) as rdatacls)"
- "AS rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 90300)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80400)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80100)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(usename, (select usename from pg_shadow where usesysid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "-1 as datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN pg_shadow u ON (datdba = usesysid) "
- "WHERE datallowconn ORDER BY 1");
-
- res = executeQuery(conn, buf->data);
-
- for (i = 0; i < PQntuples(res); i++)
- {
- char *dbname = PQgetvalue(res, i, 0);
- char *dbowner = PQgetvalue(res, i, 1);
- char *dbencoding = PQgetvalue(res, i, 2);
- char *dbcollate = PQgetvalue(res, i, 3);
- char *dbctype = PQgetvalue(res, i, 4);
- uint32 dbfrozenxid = atooid(PQgetvalue(res, i, 5));
- uint32 dbminmxid = atooid(PQgetvalue(res, i, 6));
- char *dbistemplate = PQgetvalue(res, i, 7);
- char *dbacl = PQgetvalue(res, i, 8);
- char *rdbacl = PQgetvalue(res, i, 9);
- char *dbconnlimit = PQgetvalue(res, i, 10);
- char *dbtablespace = PQgetvalue(res, i, 11);
- char *fdbname;
-
- fdbname = pg_strdup(fmtId(dbname));
-
- resetPQExpBuffer(buf);
-
- /*
- * Skip the CREATE DATABASE commands for "template1" and "postgres",
- * since they are presumably already there in the destination cluster.
- * We do want to emit their ACLs and config options if any, however.
- */
- if (strcmp(dbname, "template1") != 0 &&
- strcmp(dbname, "postgres") != 0)
- {
- appendPQExpBuffer(buf, "CREATE DATABASE %s", fdbname);
-
- appendPQExpBufferStr(buf, " WITH TEMPLATE = template0");
-
- if (strlen(dbowner) != 0)
- appendPQExpBuffer(buf, " OWNER = %s", fmtId(dbowner));
-
- if (default_encoding && strcmp(dbencoding, default_encoding) != 0)
- {
- appendPQExpBufferStr(buf, " ENCODING = ");
- appendStringLiteralConn(buf, dbencoding, conn);
- }
-
- if (default_collate && strcmp(dbcollate, default_collate) != 0)
- {
- appendPQExpBufferStr(buf, " LC_COLLATE = ");
- appendStringLiteralConn(buf, dbcollate, conn);
- }
-
- if (default_ctype && strcmp(dbctype, default_ctype) != 0)
- {
- appendPQExpBufferStr(buf, " LC_CTYPE = ");
- appendStringLiteralConn(buf, dbctype, conn);
- }
-
- /*
- * Output tablespace if it isn't the default. For default, it
- * uses the default from the template database. If tablespace is
- * specified and tablespace creation failed earlier, (e.g. no such
- * directory), the database creation will fail too. One solution
- * would be to use 'SET default_tablespace' like we do in pg_dump
- * for setting non-default database locations.
- */
- if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- appendPQExpBuffer(buf, " TABLESPACE = %s",
- fmtId(dbtablespace));
-
- if (strcmp(dbistemplate, "t") == 0)
- appendPQExpBuffer(buf, " IS_TEMPLATE = true");
-
- if (strcmp(dbconnlimit, "-1") != 0)
- appendPQExpBuffer(buf, " CONNECTION LIMIT = %s",
- dbconnlimit);
-
- appendPQExpBufferStr(buf, ";\n");
- }
- else if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- {
- /*
- * Cannot change tablespace of the database we're connected to, so
- * to move "postgres" to another tablespace, we connect to
- * "template1", and vice versa.
- */
- if (strcmp(dbname, "postgres") == 0)
- appendPQExpBuffer(buf, "\\connect template1\n");
- else
- appendPQExpBuffer(buf, "\\connect postgres\n");
-
- appendPQExpBuffer(buf, "ALTER DATABASE %s SET TABLESPACE %s;\n",
- fdbname, fmtId(dbtablespace));
-
- /* connect to original database */
- appendPsqlMetaConnect(buf, dbname);
- }
-
- if (binary_upgrade)
- {
- appendPQExpBufferStr(buf, "-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(buf, "UPDATE pg_catalog.pg_database "
- "SET datfrozenxid = '%u', datminmxid = '%u' "
- "WHERE datname = ",
- dbfrozenxid, dbminmxid);
- appendStringLiteralConn(buf, dbname, conn);
- appendPQExpBufferStr(buf, ";\n");
- }
-
- if (!skip_acls &&
- !buildACLCommands(fdbname, NULL, "DATABASE",
- dbacl, rdbacl, dbowner,
- "", server_version, buf))
- {
- fprintf(stderr, _("%s: could not parse ACL list (%s) for database \"%s\"\n"),
- progname, dbacl, fdbname);
- PQfinish(conn);
- exit_nicely(1);
- }
-
- fprintf(OPF, "%s", buf->data);
-
- dumpDatabaseConfig(conn, dbname);
-
- free(fdbname);
- }
-
- if (default_encoding)
- free(default_encoding);
- if (default_collate)
- free(default_collate);
- if (default_ctype)
- free(default_ctype);
-
- PQclear(res);
- destroyPQExpBuffer(buf);
-
- fprintf(OPF, "\n\n");
-}
-
-
-/*
- * Dump database-specific configuration
- */
-static void
-dumpDatabaseConfig(PGconn *conn, const char *dbname)
-{
- PQExpBuffer buf = createPQExpBuffer();
- int count = 1;
-
- for (;;)
- {
- PGresult *res;
-
- if (server_version >= 90000)
- printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting WHERE "
- "setrole = 0 AND setdatabase = (SELECT oid FROM pg_database WHERE datname = ", count);
- else
- printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE datname = ", count);
- appendStringLiteralConn(buf, dbname, conn);
-
- if (server_version >= 90000)
- appendPQExpBufferChar(buf, ')');
-
- res = executeQuery(conn, buf->data);
- if (PQntuples(res) == 1 &&
- !PQgetisnull(res, 0, 0))
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
- "DATABASE", dbname, NULL, NULL);
- PQclear(res);
- count++;
- }
- else
- {
- PQclear(res);
- break;
- }
- }
-
- destroyPQExpBuffer(buf);
-}
-
-
/*
* Dump user-specific configuration
@@ -1646,11 +1308,14 @@ dumpUserConfig(PGconn *conn, const char *username)
appendPQExpBufferChar(buf, ')');
res = executeQuery(conn, buf->data);
+ resetPQExpBuffer(buf);
+
if (PQntuples(res) == 1 &&
!PQgetisnull(res, 0, 0))
{
makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
- "ROLE", username, NULL, NULL);
+ "ROLE", username, NULL, NULL, buf);
+ fprintf(OPF, "%s", buf->data);
PQclear(res);
count++;
}
@@ -1665,85 +1330,6 @@ dumpUserConfig(PGconn *conn, const char *username)
}
-/*
- * Dump user-and-database-specific configuration
- */
-static void
-dumpDbRoleConfig(PGconn *conn)
-{
- PQExpBuffer buf = createPQExpBuffer();
- PGresult *res;
- int i;
-
- printfPQExpBuffer(buf, "SELECT rolname, datname, unnest(setconfig) "
- "FROM pg_db_role_setting, %s u, pg_database "
- "WHERE setrole = u.oid AND setdatabase = pg_database.oid", role_catalog);
- res = executeQuery(conn, buf->data);
-
- if (PQntuples(res) > 0)
- {
- fprintf(OPF, "--\n-- Per-Database Role Settings \n--\n\n");
-
- for (i = 0; i < PQntuples(res); i++)
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, i, 2),
- "ROLE", PQgetvalue(res, i, 0),
- "DATABASE", PQgetvalue(res, i, 1));
- }
-
- fprintf(OPF, "\n\n");
- }
-
- PQclear(res);
- destroyPQExpBuffer(buf);
-}
-
-
-/*
- * Helper function for dumpXXXConfig().
- */
-static void
-makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name,
- const char *type2, const char *name2)
-{
- char *pos;
- char *mine;
- PQExpBuffer buf;
-
- mine = pg_strdup(arrayitem);
- pos = strchr(mine, '=');
- if (pos == NULL)
- {
- free(mine);
- return;
- }
-
- buf = createPQExpBuffer();
-
- *pos = 0;
- appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
- if (type2 != NULL && name2 != NULL)
- appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
- appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
-
- /*
- * Some GUC variable names are 'LIST' type and hence must not be quoted.
- */
- if (pg_strcasecmp(mine, "DateStyle") == 0
- || pg_strcasecmp(mine, "search_path") == 0)
- appendPQExpBufferStr(buf, pos + 1);
- else
- appendStringLiteralConn(buf, pos + 1, conn);
- appendPQExpBufferStr(buf, ";\n");
-
- fprintf(OPF, "%s", buf->data);
- destroyPQExpBuffer(buf);
- free(mine);
-}
-
-
-
/*
* Dump contents of databases.
*/
@@ -1758,29 +1344,11 @@ dumpDatabases(PGconn *conn)
for (i = 0; i < PQntuples(res); i++)
{
int ret;
-
char *dbname = PQgetvalue(res, i, 0);
- PQExpBufferData connectbuf;
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: dumping database \"%s\"...\n"), progname, dbname);
- initPQExpBuffer(&connectbuf);
- appendPsqlMetaConnect(&connectbuf, dbname);
- fprintf(OPF, "%s\n", connectbuf.data);
- termPQExpBuffer(&connectbuf);
-
- /*
- * Restore will need to write to the target cluster. This connection
- * setting is emitted for pg_dumpall rather than in the code also used
- * by pg_dump, so that a cluster with databases or users which have
- * this flag turned on can still be replicated through pg_dumpall
- * without editing the file or stream. With pg_dump there are many
- * other ways to allow the file to be used, and leaving it out allows
- * users to protect databases from being accidental restore targets.
- */
- fprintf(OPF, "SET default_transaction_read_only = off;\n\n");
-
if (filename)
fclose(OPF);
@@ -1822,6 +1390,18 @@ runPgDump(const char *dbname)
appendPQExpBuffer(cmd, "\"%s\" %s", pg_dump_bin,
pgdumpopts->data);
+
+ /*
+ * The create individual database dump, pass the the create database option
+ * to pg_dump. But skip generating create database commands for default
+ * databases (template1, postgres).
+ */
+ if (strcmp(dbname, "template1") == 0 || strcmp(dbname, "postgres") == 0)
+ appendPQExpBufferStr(cmd, " --set-db-properties ");
+ else
+ appendPQExpBufferStr(cmd, " -C ");
+
+
/*
* If we have a filename, use the undocumented plain-append pg_dump
* format.
@@ -1840,7 +1420,7 @@ runPgDump(const char *dbname)
appendShellString(cmd, connstrbuf->data);
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: running \"%s\"\n"), progname, cmd->data);
fflush(stdout);
@@ -2133,31 +1713,7 @@ constructConnStr(const char **keywords, const char **values)
return connstr;
}
-/*
- * Run a query, return the results, exit program on failure.
- */
-static PGresult *
-executeQuery(PGconn *conn, const char *query)
-{
- PGresult *res;
- if (verbose)
- fprintf(stderr, _("%s: executing %s\n"), progname, query);
-
- res = PQexec(conn, query);
- if (!res ||
- PQresultStatus(res) != PGRES_TUPLES_OK)
- {
- fprintf(stderr, _("%s: query failed: %s"),
- progname, PQerrorMessage(conn));
- fprintf(stderr, _("%s: query was: %s\n"),
- progname, query);
- PQfinish(conn);
- exit_nicely(1);
- }
-
- return res;
-}
/*
* As above for a SQL command (which returns nothing).
@@ -2167,7 +1723,7 @@ executeCommand(PGconn *conn, const char *query)
{
PGresult *res;
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: executing %s\n"), progname, query);
res = PQexec(conn, query);
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 7cf9bdadb2..cbc92fdd05 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2680,7 +2680,7 @@ qr/CREATE CAST \(timestamp with time zone AS interval\) WITH FUNCTION pg_catalog
'CREATE DATABASE postgres' => {
all_runs => 1,
regexp => qr/^
- \QCREATE DATABASE postgres WITH TEMPLATE = template0 \E
+ \QCREATE DATABASE postgres WITH TEMPLATE = template0\E
.*;/xm,
like => { createdb => 1, },
unlike => {
@@ -2714,7 +2714,7 @@ qr/CREATE CAST \(timestamp with time zone AS interval\) WITH FUNCTION pg_catalog
create_order => 47,
create_sql => 'CREATE DATABASE dump_test;',
regexp => qr/^
- \QCREATE DATABASE dump_test WITH TEMPLATE = template0 \E
+ \QCREATE DATABASE dump_test WITH TEMPLATE = template0\E
.*;/xm,
like => { pg_dumpall_dbprivs => 1, },
unlike => {
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 5ed6b786e2..81af68b868 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -48,6 +48,7 @@ generate_old_dump(void)
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
PQExpBufferData connstr,
escaped_connstr;
+ PQExpBuffer create_opts = createPQExpBuffer();
initPQExpBuffer(&connstr);
appendPQExpBuffer(&connstr, "dbname=");
@@ -60,13 +61,20 @@ generate_old_dump(void)
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
+ if (strcmp(old_db->db_name, "template1") == 0 || strcmp(old_db->db_name, "postgres") == 0)
+ appendPQExpBufferStr(create_opts, " --set-db-properties ");
+ else
+ appendPQExpBufferStr(create_opts, " -C ");
+
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade %s "
+ "--format=custom %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
- log_opts.verbose ? "--verbose" : "",
+ create_opts->data, log_opts.verbose ? "--verbose" : "",
sql_file_name, escaped_connstr.data);
+ termPQExpBuffer(create_opts);
termPQExpBuffer(&escaped_connstr);
}
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 2b7da529e5..625740d5d0 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -46,7 +46,7 @@
#endif
static void prepare_new_cluster(void);
-static void prepare_new_databases(void);
+static void prepare_new_globals(void);
static void create_new_objects(void);
static void copy_xact_xlog_xid(void);
static void set_frozenxids(bool minmxid_only);
@@ -124,7 +124,7 @@ main(int argc, char **argv)
/* -- NEW -- */
start_postmaster(&new_cluster, true);
- prepare_new_databases();
+ prepare_new_globals();
create_new_objects();
@@ -271,7 +271,7 @@ prepare_new_cluster(void)
static void
-prepare_new_databases(void)
+prepare_new_globals(void)
{
/*
* We set autovacuum_freeze_max_age to its maximum value so autovacuum
@@ -294,9 +294,6 @@ prepare_new_databases(void)
new_cluster.bindir, cluster_conn_opts(&new_cluster),
GLOBALS_DUMP_FILE);
check_ok();
-
- /* we load this to get a current list of databases */
- get_db_and_rel_infos(&new_cluster);
}
@@ -312,15 +309,6 @@ create_new_objects(void)
char sql_file_name[MAXPGPATH],
log_file_name[MAXPGPATH];
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
- PQExpBufferData connstr,
- escaped_connstr;
-
- initPQExpBuffer(&connstr);
- appendPQExpBuffer(&connstr, "dbname=");
- appendConnStrVal(&connstr, old_db->db_name);
- initPQExpBuffer(&escaped_connstr);
- appendShellString(&escaped_connstr, connstr.data);
- termPQExpBuffer(&connstr);
pg_log(PG_STATUS, "%s", old_db->db_name);
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
@@ -332,13 +320,10 @@ create_new_objects(void)
*/
parallel_exec_prog(log_file_name,
NULL,
- "\"%s/pg_restore\" %s --exit-on-error --verbose --dbname %s \"%s\"",
+ "\"%s/pg_restore\" %s --create --exit-on-error --verbose --dbname template1 \"%s\"",
new_cluster.bindir,
cluster_conn_opts(&new_cluster),
- escaped_connstr.data,
sql_file_name);
-
- termPQExpBuffer(&escaped_connstr);
}
/* reap all children */
--
2.15.0.windows.1
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
@ 2018-01-05 06:00 ` Vaishnavi Prabakaran <[email protected]>
1 sibling, 0 replies; 33+ messages in thread
From: Vaishnavi Prabakaran @ 2018-01-05 06:00 UTC (permalink / raw)
To: Haribabu Kommi <[email protected]>; +Cc: Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Fri, Jan 5, 2018 at 4:32 PM, Haribabu Kommi <[email protected]>
wrote:
>
>
> Corrected.
> Updated patch attached.
>
> Moved this CF item to "Ready for committer"
Regards,
Vaishnavi,
Fujitsu Australia.
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
@ 2018-01-17 21:25 ` Tom Lane <[email protected]>
2018-01-17 23:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
1 sibling, 2 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-17 21:25 UTC (permalink / raw)
To: Haribabu Kommi <[email protected]>; +Cc: Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Haribabu Kommi <[email protected]> writes:
> [ pg_dump-and-pg_dumpall-database-handling-refactoring_v12.patch ]
I started to look through this, and almost immediately found that the
diff in t/002_pg_dump.pl doesn't seem necessary --- the test passes
for me without applying that hunk. Is that a leftover from a previous
patch iteration, or is there some platform dependency in the test?
If it's not necessary, I'd be inclined to leave it as it was.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-17 23:32 ` Haribabu Kommi <[email protected]>
1 sibling, 0 replies; 33+ messages in thread
From: Haribabu Kommi @ 2018-01-17 23:32 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Thu, Jan 18, 2018 at 8:25 AM, Tom Lane <[email protected]> wrote:
> Haribabu Kommi <[email protected]> writes:
> > [ pg_dump-and-pg_dumpall-database-handling-refactoring_v12.patch ]
>
> I started to look through this, and almost immediately found that the
> diff in t/002_pg_dump.pl doesn't seem necessary --- the test passes
> for me without applying that hunk. Is that a leftover from a previous
> patch iteration, or is there some platform dependency in the test?
> If it's not necessary, I'd be inclined to leave it as it was.
>
Thanks for the review.
Yes, it is a left over from previous patch iteration. The test used to fail
with this
patch earlier. Now there is no problem in test even after removing the hunk
from
my side also. There is no platform dependency.
Attached is an updated patch after removing the test changes.
Regards,
Hari Babu
Fujitsu Australia
Attachments:
[application/octet-stream] pg_dump-and-pg_dumpall-database-handling-refactoring_v13.patch (48.8K, ../../CAJrrPGfo5voeNGDgn=Hqyi7bMZJhQJAADWfKJ_w+Z8szrCeNDg@mail.gmail.com/3-pg_dump-and-pg_dumpall-database-handling-refactoring_v13.patch)
download | inline diff:
From 98bac6a151e53d93d7a6bcaca9786fa325a0b2d3 Mon Sep 17 00:00:00 2001
From: Hari Babu <[email protected]>
Date: Thu, 18 Jan 2018 10:10:20 +1100
Subject: [PATCH] pg_dump and pg_dumpall database handling refactoring
The Database handling especially with ACL's is not proper
between pg_dump and pg_dumpall. So to avoid these problems,
Now the entire database handling is moved to pg_dump and
the pg_dumpall just invokes pg_dump to print the database
attributes.
---
doc/src/sgml/ref/pg_dump.sgml | 33 ++-
doc/src/sgml/ref/pg_dumpall.sgml | 4 +
src/bin/pg_dump/Makefile | 4 +-
src/bin/pg_dump/dumputils.c | 67 +++++
src/bin/pg_dump/dumputils.h | 7 +
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_backup_archiver.c | 2 +
src/bin/pg_dump/pg_backup_utils.c | 3 +
src/bin/pg_dump/pg_backup_utils.h | 1 +
src/bin/pg_dump/pg_dump.c | 309 +++++++++++++++++++---
src/bin/pg_dump/pg_dumpall.c | 488 ++---------------------------------
src/bin/pg_upgrade/dump.c | 12 +-
src/bin/pg_upgrade/pg_upgrade.c | 23 +-
13 files changed, 420 insertions(+), 535 deletions(-)
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 08cad68199..63934a75ec 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -192,8 +192,11 @@ PostgreSQL documentation
database itself and reconnect to the created database. (With a
script of this form, it doesn't matter which database in the
destination installation you connect to before running the script.)
- If <option>--clean</option> is also specified, the script drops and
- recreates the target database before reconnecting to it.
+ It also sets all the database-level properties such as ownership,
+ ACLs, <command>ALTER DATABASE ... SET</command> commands, and
+ <command>ALTER ROLE IN DATABASE ... SET</command> commands.
+ If <option>--clean</option> is also specified, the script drops
+ and recreates the target database before reconnecting to it.
</para>
<para>
@@ -706,6 +709,28 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--set-db-properties</option></term>
+ <listitem>
+ <para>
+ This option is to dump the database properties without issuing the
+ <command>CREATE DATABASE</command> command. The generated dump is
+ used to restore the databases on a target location where the database
+ is already exists. This option cannot be used with either of
+ <option>-C/--create</option> or <option>-c/--clean</option> options.
+ </para>
+
+ <note>
+ <para>
+ The Dump file generated with this option cannot be restored using
+ <option>-c/--clean</option> and <option>-C/--create</option> options of
+ <application>pg_restore</application> command. This is due to not available
+ of <command>CREATE DATABASE</command> command in the dump file.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--enable-row-security</option></term>
<listitem>
@@ -1231,10 +1256,6 @@ CREATE DATABASE foo WITH TEMPLATE template0;
<command>ANALYZE</command> after restoring from a dump file
to ensure optimal performance; see <xref linkend="vacuum-for-statistics"/>
and <xref linkend="autovacuum"/> for more information.
- The dump file also does not
- contain any <command>ALTER DATABASE ... SET</command> commands;
- these settings are dumped by <xref linkend="app-pg-dumpall"/>,
- along with database users and other installation-wide settings.
</para>
<para>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5196a211b1..cb3957d669 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -129,6 +129,10 @@ PostgreSQL documentation
<listitem>
<para>
Dump only global objects (roles and tablespaces), no databases.
+ Any database role configuration settings are not dumped with
+ this option. User needs to use <command>pg_dump</command> with
+ <option>--create</option> option to dump
+ <command>ALTER ROLE IN DATABASE ... SET</command> commands.
</para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index e3bfc95f16..99387ddd99 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -31,8 +31,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o $(OBJS) | submake-libpq submake-libpg
pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o | submake-libpq submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o pg_backup_utils.o | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o pg_backup_utils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff --git a/src/bin/pg_dump/dumputils.c b/src/bin/pg_dump/dumputils.c
index 32ad600fd0..9ed4171b5d 100644
--- a/src/bin/pg_dump/dumputils.c
+++ b/src/bin/pg_dump/dumputils.c
@@ -807,3 +807,70 @@ buildACLQueries(PQExpBuffer acl_subquery, PQExpBuffer racl_subquery,
printfPQExpBuffer(init_racl_subquery, "NULL");
}
}
+
+/*
+ * Helper function for dumpXXXConfig().
+ *
+ * Frame the ALTER ".." SET ".." commands and fill it in buf.
+ */
+void
+makeAlterConfigCommand(PGconn *conn, const char *configitem,
+ const char *type, const char *name,
+ const char *type2, const char *name2,
+ PQExpBuffer buf)
+{
+ char *pos;
+ char *mine;
+
+ mine = pg_strdup(configitem);
+ pos = strchr(mine, '=');
+ if (pos == NULL)
+ {
+ pg_free(mine);
+ return;
+ }
+
+ *pos++ = '\0';
+ appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
+ if (type2 != NULL && name2 != NULL)
+ appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
+ appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be quoted.
+ */
+ if (pg_strcasecmp(mine, "DateStyle") == 0
+ || pg_strcasecmp(mine, "search_path") == 0)
+ appendPQExpBufferStr(buf, pos);
+ else
+ appendStringLiteralConn(buf, pos, conn);
+ appendPQExpBufferStr(buf, ";\n");
+
+ pg_free(mine);
+}
+
+/*
+ * Run a query, return the results, exit program on failure.
+ */
+PGresult *
+executeQuery(PGconn *conn, const char *query)
+{
+ PGresult *res;
+
+ if (g_verbose)
+ fprintf(stderr, _("%s: executing %s\n"), progname, query);
+
+ res = PQexec(conn, query);
+ if (!res ||
+ PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ fprintf(stderr, _("%s: query failed: %s"),
+ progname, PQerrorMessage(conn));
+ fprintf(stderr, _("%s: query was: %s\n"),
+ progname, query);
+ PQfinish(conn);
+ exit_nicely(1);
+ }
+
+ return res;
+}
diff --git a/src/bin/pg_dump/dumputils.h b/src/bin/pg_dump/dumputils.h
index d5f150dfa0..28b84f50d3 100644
--- a/src/bin/pg_dump/dumputils.h
+++ b/src/bin/pg_dump/dumputils.h
@@ -17,6 +17,7 @@
#include "libpq-fe.h"
#include "pqexpbuffer.h"
+#include "pg_backup_utils.h"
/*
* Preferred strftime(3) format specifier for printing timestamps in pg_dump
@@ -56,4 +57,10 @@ extern void buildACLQueries(PQExpBuffer acl_subquery, PQExpBuffer racl_subquery,
const char *acl_column, const char *acl_owner,
const char *obj_kind, bool binary_upgrade);
+extern void makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
+ const char *type, const char *name, const char *type2,
+ const char *name2, PQExpBuffer buf);
+
+extern PGresult *executeQuery(PGconn *conn, const char *query);
+
#endif /* DUMPUTILS_H */
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..597c4f70e6 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -117,6 +117,7 @@ typedef struct _restoreOptions
int suppressDumpWarnings; /* Suppress output of WARNING entries
* to stderr */
bool single_txn;
+ int set_db_properties;
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
@@ -158,6 +159,7 @@ typedef struct _dumpOptions
int use_setsessauth;
int enable_row_security;
int load_via_partition_root;
+ int set_db_properties;
/* default, if no "inclusion" switches appear, is to dump everything */
bool include_everything;
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 41741aefbc..ea5fba830f 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -831,6 +831,8 @@ restore_toc_entry(ArchiveHandle *AH, TocEntry *te, bool is_parallel)
ahlog(AH, 1, "connecting to new database \"%s\"\n", te->tag);
_reconnectToDB(AH, te->tag);
+
+ ahprintf(AH, "SET default_transaction_read_only = off;\n\n");
ropt->dbname = connstr.data;
}
}
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index df5fe708ba..07f2a43e35 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -19,6 +19,9 @@
/* Globals exported by this file */
const char *progname = NULL;
+/* User wants verbose narration of our activities */
+bool g_verbose = false;
+
#define MAX_ON_EXIT_NICELY 20
static struct
diff --git a/src/bin/pg_dump/pg_backup_utils.h b/src/bin/pg_dump/pg_backup_utils.h
index 6eaf302fc7..b55fe5e8c3 100644
--- a/src/bin/pg_dump/pg_backup_utils.h
+++ b/src/bin/pg_dump/pg_backup_utils.h
@@ -26,6 +26,7 @@ typedef enum /* bits returned by set_dump_section */
typedef void (*on_exit_nicely_callback) (int code, void *arg);
extern const char *progname;
+extern bool g_verbose;
extern void set_dump_section(const char *arg, int *dumpSections);
extern void write_msg(const char *modulename, const char *fmt,...) pg_attribute_printf(2, 3);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 27628a397c..2f607abd1c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -88,9 +88,6 @@ typedef enum OidOptions
zeroAsNone = 8
} OidOptions;
-/* global decls */
-bool g_verbose; /* User wants verbose narration of our
- * activities. */
static bool dosync = true; /* Issue fsync() to make dump durable on disk. */
/* subquery used to convert user ID (eg, datdba) to user name */
@@ -273,6 +270,8 @@ static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
static char *get_synchronized_snapshot(Archive *fout);
static void setupDumpWorker(Archive *AHX);
static TableInfo *getRootTableInfo(TableInfo *tbinfo);
+static void dumpDatabaseConfig(Archive *fout, PQExpBuffer creaQry, const char *dbname);
+static void dumpDbRoleConfig(Archive *AH, PQExpBuffer creaQry);
int
@@ -362,6 +361,7 @@ main(int argc, char **argv)
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
{"no-sync", no_argument, NULL, 7},
+ {"set-db-properties", no_argument, &dopt.set_db_properties, 1},
{NULL, 0, NULL, 0}
};
@@ -581,6 +581,18 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.set_db_properties && dopt.outputCreateDB)
+ {
+ write_msg(NULL, "option --set-db-properties and -C/--create cannot be used together\n");
+ exit_nicely(1);
+ }
+
+ if (dopt.set_db_properties && dopt.outputClean)
+ {
+ write_msg(NULL, "option --set-db-properties and -c/--clean cannot be used together\n");
+ exit_nicely(1);
+ }
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -860,7 +872,12 @@ main(int argc, char **argv)
ropt->dumpSections = dopt.dumpSections;
ropt->aclsSkip = dopt.aclsSkip;
ropt->superuser = dopt.outputSuperuser;
- ropt->createDB = dopt.outputCreateDB;
+
+ /*
+ * treat --set-db-properties are also as DATABASE commands like
+ * -C/--create
+ */
+ ropt->createDB = dopt.outputCreateDB || dopt.set_db_properties;
ropt->noOwner = dopt.outputNoOwner;
ropt->noTablespace = dopt.outputNoTablespaces;
ropt->disable_triggers = dopt.disable_triggers;
@@ -875,6 +892,7 @@ main(int argc, char **argv)
ropt->enable_row_security = dopt.enable_row_security;
ropt->sequence_data = dopt.sequence_data;
ropt->binary_upgrade = dopt.binary_upgrade;
+ ropt->set_db_properties = dopt.set_db_properties;
if (compressLevel == -1)
ropt->compression = 0;
@@ -973,6 +991,7 @@ help(const char *progname)
printf(_(" --use-set-session-authorization\n"
" use SET SESSION AUTHORIZATION commands instead of\n"
" ALTER OWNER commands to set ownership\n"));
+ printf(_(" --set-db-properties dump database properties without CREATE DATABASE command\n"));
printf(_("\nConnection options:\n"));
printf(_(" -d, --dbname=DBNAME database to dump\n"));
@@ -2528,25 +2547,34 @@ dumpDatabase(Archive *fout)
PQExpBuffer dbQry = createPQExpBuffer();
PQExpBuffer delQry = createPQExpBuffer();
PQExpBuffer creaQry = createPQExpBuffer();
+ PQExpBuffer aclQry = createPQExpBuffer();
PGconn *conn = GetConnection(fout);
PGresult *res;
int i_tableoid,
i_oid,
i_dba,
+ i_dbacl,
+ i_rdbacl,
i_encoding,
i_collate,
i_ctype,
i_frozenxid,
i_minmxid,
- i_tablespace;
+ i_tablespace,
+ i_dbistemplate,
+ i_dbconnlimit;
CatalogId dbCatId;
DumpId dbDumpId;
const char *datname,
*dba,
+ *dbacl,
+ *rdbacl,
*encoding,
*collate,
*ctype,
- *tablespace;
+ *tablespace,
+ *dbistemplate,
+ *dbconnlimit;
uint32 frozenxid,
minmxid;
@@ -2558,13 +2586,49 @@ dumpDatabase(Archive *fout)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- /* Get the database owner and parameters from pg_database */
- if (fout->remoteVersion >= 90300)
+ /*
+ * Now collect all the information about databases to dump.
+ *
+ * For the database ACLs, as of 9.6, we extract both the positive (as
+ * datacl) and negative (as rdatacl) ACLs, relative to the default ACL for
+ * databases, which are then passed to buildACLCommands() below.
+ *
+ * See buildACLQueries() and buildACLCommands().
+ *
+ * Note that we do not support initial privileges (pg_init_privs) on
+ * databases.
+ */
+
+ if (fout->remoteVersion >= 90600)
+ {
+ appendPQExpBuffer(dbQry,
+ "SELECT tableoid, oid, "
+ "(%s datdba) AS dba, "
+ "pg_encoding_to_char(encoding) AS encoding, "
+ "datcollate, datctype, datfrozenxid, datminmxid, datistemplate, "
+ "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
+ " SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba))) AS acl "
+ " EXCEPT SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba))) as datacls)"
+ " AS datacl, "
+ "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
+ " SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba)) AS acl "
+ " EXCEPT SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba)))) as rdatacls)"
+ " AS rdatacl, "
+ "datconnlimit, "
+ "(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
+ "shobj_description(oid, 'pg_database') AS description "
+
+ "FROM pg_database "
+ "WHERE datname = ",
+ username_subquery);
+ appendStringLiteralAH(dbQry, datname, fout);
+ }
+ else if (fout->remoteVersion >= 90300)
{
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "datcollate, datctype, datfrozenxid, datminmxid, "
+ "datcollate, datctype, datfrozenxid, datminmxid, datistemplate, datacl,'' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2578,7 +2642,7 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
+ "datcollate, datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl, '' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2592,7 +2656,7 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl,'' as rdatacl, datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
@@ -2606,7 +2670,8 @@ dumpDatabase(Archive *fout)
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
- "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, datistemplate, datacl,'' as rdatacl,"
+ "-1 as datconnlimit,"
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace "
"FROM pg_database "
"WHERE datname = ",
@@ -2625,6 +2690,10 @@ dumpDatabase(Archive *fout)
i_frozenxid = PQfnumber(res, "datfrozenxid");
i_minmxid = PQfnumber(res, "datminmxid");
i_tablespace = PQfnumber(res, "tablespace");
+ i_dbacl = PQfnumber(res, "datacl");
+ i_rdbacl = PQfnumber(res, "rdatacl");
+ i_dbistemplate = PQfnumber(res, "datistemplate");
+ i_dbconnlimit = PQfnumber(res, "datconnlimit");
dbCatId.tableoid = atooid(PQgetvalue(res, 0, i_tableoid));
dbCatId.oid = atooid(PQgetvalue(res, 0, i_oid));
@@ -2635,40 +2704,63 @@ dumpDatabase(Archive *fout)
frozenxid = atooid(PQgetvalue(res, 0, i_frozenxid));
minmxid = atooid(PQgetvalue(res, 0, i_minmxid));
tablespace = PQgetvalue(res, 0, i_tablespace);
+ dbacl = PQgetvalue(res, 0, i_dbacl);
+ rdbacl = PQgetvalue(res, 0, i_rdbacl);
+ dbistemplate = PQgetvalue(res, 0, i_dbistemplate);
+ dbconnlimit = PQgetvalue(res, 0, i_dbconnlimit);
- appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
- fmtId(datname));
- if (strlen(encoding) > 0)
- {
- appendPQExpBufferStr(creaQry, " ENCODING = ");
- appendStringLiteralAH(creaQry, encoding, fout);
- }
- if (strlen(collate) > 0)
+ /*
+ * Skip the CREATE DATABASE commands --set-db-properties is enabled, since
+ * they are presumably already there in the destination cluster. We do
+ * want to emit their ACLs and config options if any, however.
+ */
+ if (!dopt->set_db_properties)
{
- appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
- appendStringLiteralAH(creaQry, collate, fout);
+ appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
+ fmtId(datname));
+
+ if (strlen(encoding) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " ENCODING = ");
+ appendStringLiteralAH(creaQry, encoding, fout);
+ }
+ if (strlen(collate) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
+ appendStringLiteralAH(creaQry, collate, fout);
+ }
+ if (strlen(ctype) > 0)
+ {
+ appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
+ appendStringLiteralAH(creaQry, ctype, fout);
+ }
+ if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
+ !dopt->outputNoTablespaces)
+ {
+ appendPQExpBuffer(creaQry, " TABLESPACE = %s", fmtId(tablespace));
+ }
+
+ if (strlen(dbistemplate) > 0 && strcmp(dbistemplate, "t") == 0)
+ appendPQExpBuffer(creaQry, " IS_TEMPLATE = true");
+
+ if (strlen(dbconnlimit) > 0 && strcmp(dbconnlimit, "-1") != 0)
+ appendPQExpBuffer(creaQry, " CONNECTION LIMIT = %s",
+ dbconnlimit);
+
+ appendPQExpBufferStr(creaQry, ";\n");
}
- if (strlen(ctype) > 0)
+ else if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 && !dopt->outputNoTablespaces)
{
- appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
- appendStringLiteralAH(creaQry, ctype, fout);
- }
- if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
- !dopt->outputNoTablespaces)
- appendPQExpBuffer(creaQry, " TABLESPACE = %s",
- fmtId(tablespace));
- appendPQExpBufferStr(creaQry, ";\n");
+ if (strcmp(datname, "postgres") == 0)
+ appendPQExpBuffer(creaQry, "\\connect template1\n");
+ else if (strcmp(datname, "template1") == 0)
+ appendPQExpBuffer(creaQry, "\\connect postgres\n");
- if (dopt->binary_upgrade)
- {
- appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
- "SET datfrozenxid = '%u', datminmxid = '%u'\n"
- "WHERE datname = ",
- frozenxid, minmxid);
- appendStringLiteralAH(creaQry, datname, fout);
- appendPQExpBufferStr(creaQry, ";\n");
+ appendPQExpBuffer(creaQry, "ALTER DATABASE %s SET TABLESPACE %s;\n",
+ datname, fmtId(tablespace));
+ /* connect to original database */
+ appendPsqlMetaConnect(creaQry, datname);
}
appendPQExpBuffer(delQry, "DROP DATABASE %s;\n",
@@ -2694,6 +2786,58 @@ dumpDatabase(Archive *fout)
NULL, /* Dumper */
NULL); /* Dumper Arg */
+ resetPQExpBuffer(creaQry);
+
+ if (dopt->binary_upgrade)
+ {
+ appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
+ appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
+ "SET datfrozenxid = '%u', datminmxid = '%u'\n"
+ "WHERE datname = ",
+ frozenxid, minmxid);
+ appendStringLiteralAH(creaQry, datname, fout);
+ appendPQExpBufferStr(creaQry, ";\n");
+
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ creaQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+ resetPQExpBuffer(creaQry);
+ }
+
+ if (!dopt->aclsSkip &&
+ !buildACLCommands(datname, NULL, "DATABASE",
+ dbacl, rdbacl, dba,
+ "", fout->remoteVersion, aclQry))
+ {
+ exit_horribly(NULL, _("%s: could not parse ACL list (%s) for database \"%s\"\n"), progname, dbacl, datname);
+ }
+
+ if (strlen(aclQry->data))
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ aclQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ /* Dump database specific configuration */
+ dumpDatabaseConfig(fout, creaQry, datname);
+
+ /* Dump user and database specific configuration */
+ if (fout->remoteVersion >= 90000)
+ dumpDbRoleConfig(fout, creaQry);
+
+ if (strlen(creaQry->data))
+ ArchiveEntry(fout, dbCatId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE", SECTION_PRE_DATA,
+ creaQry->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
/*
* pg_largeobject and pg_largeobject_metadata come from the old system
* intact, so set their relfrozenxids and relminmxids.
@@ -2847,6 +2991,7 @@ dumpDatabase(Archive *fout)
destroyPQExpBuffer(dbQry);
destroyPQExpBuffer(delQry);
destroyPQExpBuffer(creaQry);
+ destroyPQExpBuffer(aclQry);
}
/*
@@ -18091,3 +18236,87 @@ appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
if (!res)
write_msg(NULL, "WARNING: could not parse reloptions array\n");
}
+
+/*
+ * Dump database-specific configuration
+ */
+static void
+dumpDatabaseConfig(Archive *AH, PQExpBuffer creaQry, const char *dbname)
+{
+ PGconn *conn = GetConnection(AH);
+ PQExpBuffer buf = createPQExpBuffer();
+ int count = 1;
+
+ for (;;)
+ {
+ PGresult *res;
+
+ if (AH->remoteVersion >= 90000)
+ printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting WHERE "
+ "setrole = 0 AND setdatabase = (SELECT oid FROM pg_database WHERE datname = ", count);
+ else
+ printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE datname = ", count);
+
+ appendStringLiteralConn(buf, dbname, conn);
+
+ if (AH->remoteVersion >= 90000)
+ appendPQExpBufferChar(buf, ')');
+
+ res = executeQuery(conn, buf->data);
+ resetPQExpBuffer(buf);
+
+ if (PQntuples(res) == 1 &&
+ !PQgetisnull(res, 0, 0))
+ {
+ makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
+ "DATABASE", dbname, NULL, NULL, buf);
+
+ appendPQExpBuffer(creaQry, "%s", buf->data);
+ PQclear(res);
+ count++;
+ }
+ else
+ {
+ PQclear(res);
+ break;
+ }
+ }
+
+ destroyPQExpBuffer(buf);
+}
+
+/*
+ * Dump user-and-database-specific configuration
+ */
+static void
+dumpDbRoleConfig(Archive *AH, PQExpBuffer creaQry)
+{
+ PGconn *conn = GetConnection(AH);
+ PQExpBuffer buf = createPQExpBuffer();
+ PGresult *res;
+ int i;
+
+ printfPQExpBuffer(buf, "SELECT rolname, datname, unnest(setconfig) "
+ "FROM pg_db_role_setting, pg_roles, pg_database "
+ "WHERE setrole = pg_roles.oid AND setdatabase = pg_database.oid");
+ res = executeQuery(conn, buf->data);
+
+ if (PQntuples(res) > 0)
+ {
+ appendPQExpBufferStr(creaQry, "\n\n--\n-- Per-Database Role Settings \n--\n\n");
+
+ resetPQExpBuffer(buf);
+ for (i = 0; i < PQntuples(res); i++)
+ {
+ makeAlterConfigCommand(conn, PQgetvalue(res, i, 2),
+ "ROLE", PQgetvalue(res, i, 0),
+ "DATABASE", PQgetvalue(res, i, 1), buf);
+ }
+
+ appendPQExpBuffer(creaQry, "%s", buf->data);
+ appendPQExpBufferStr(creaQry, "\n\n");
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(buf);
+}
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 3dd2c3871e..ae2c5fa17a 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -38,16 +38,9 @@ static void dumpGroups(PGconn *conn);
static void dropTablespaces(PGconn *conn);
static void dumpTablespaces(PGconn *conn);
static void dropDBs(PGconn *conn);
-static void dumpCreateDB(PGconn *conn);
-static void dumpDatabaseConfig(PGconn *conn, const char *dbname);
static void dumpUserConfig(PGconn *conn, const char *username);
-static void dumpDbRoleConfig(PGconn *conn);
-static void makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name, const char *type2,
- const char *name2);
static void dumpDatabases(PGconn *conn);
static void dumpTimestamp(const char *msg);
-
static int runPgDump(const char *dbname);
static void buildShSecLabels(PGconn *conn, const char *catalog_name,
uint32 objectId, PQExpBuffer buffer,
@@ -55,15 +48,12 @@ static void buildShSecLabels(PGconn *conn, const char *catalog_name,
static PGconn *connectDatabase(const char *dbname, const char *connstr, const char *pghost, const char *pgport,
const char *pguser, trivalue prompt_password, bool fail_on_error);
static char *constructConnStr(const char **keywords, const char **values);
-static PGresult *executeQuery(PGconn *conn, const char *query);
static void executeCommand(PGconn *conn, const char *query);
static char pg_dump_bin[MAXPGPATH];
-static const char *progname;
static PQExpBuffer pgdumpopts;
static char *connstr = "";
static bool skip_acls = false;
-static bool verbose = false;
static bool dosync = true;
static int binary_upgrade = 0;
@@ -281,7 +271,7 @@ main(int argc, char *argv[])
break;
case 'v':
- verbose = true;
+ g_verbose = true;
appendPQExpBufferStr(pgdumpopts, " -v");
break;
@@ -498,7 +488,7 @@ main(int argc, char *argv[])
executeCommand(conn, "SET quote_all_identifiers = true");
fprintf(OPF, "--\n-- PostgreSQL database cluster dump\n--\n\n");
- if (verbose)
+ if (g_verbose)
dumpTimestamp("Started on");
/*
@@ -558,17 +548,6 @@ main(int argc, char *argv[])
/* Dump tablespaces */
if (!roles_only && !no_tablespaces)
dumpTablespaces(conn);
-
- /* Dump CREATE DATABASE commands */
- if (binary_upgrade || (!globals_only && !roles_only && !tablespaces_only))
- dumpCreateDB(conn);
-
- /* Dump role/database settings */
- if (!tablespaces_only && !roles_only)
- {
- if (server_version >= 90000)
- dumpDbRoleConfig(conn);
- }
}
if (!globals_only && !roles_only && !tablespaces_only)
@@ -576,7 +555,7 @@ main(int argc, char *argv[])
PQfinish(conn);
- if (verbose)
+ if (g_verbose)
dumpTimestamp("Completed on");
fprintf(OPF, "--\n-- PostgreSQL database cluster dump complete\n--\n\n");
@@ -1302,323 +1281,6 @@ dropDBs(PGconn *conn)
fprintf(OPF, "\n\n");
}
-/*
- * Dump commands to create each database.
- *
- * To minimize the number of reconnections (and possibly ensuing
- * password prompts) required by the output script, we emit all CREATE
- * DATABASE commands during the initial phase of the script, and then
- * run pg_dump for each database to dump the contents of that
- * database. We skip databases marked not datallowconn, since we'd be
- * unable to connect to them anyway (and besides, we don't want to
- * dump template0).
- */
-static void
-dumpCreateDB(PGconn *conn)
-{
- PQExpBuffer buf = createPQExpBuffer();
- char *default_encoding = NULL;
- char *default_collate = NULL;
- char *default_ctype = NULL;
- PGresult *res;
- int i;
-
- fprintf(OPF, "--\n-- Database creation\n--\n\n");
-
- /*
- * First, get the installation's default encoding and locale information.
- * We will dump encoding and locale specifications in the CREATE DATABASE
- * commands for just those databases with values different from defaults.
- *
- * We consider template0's encoding and locale to define the installation
- * default. Pre-8.4 installations do not have per-database locale
- * settings; for them, every database must necessarily be using the
- * installation default, so there's no need to do anything.
- */
- if (server_version >= 80400)
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "datcollate, datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
- else
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "null::text AS datcollate, null::text AS datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
-
- /* If for some reason the template DB isn't there, treat as unknown */
- if (PQntuples(res) > 0)
- {
- if (!PQgetisnull(res, 0, 0))
- default_encoding = pg_strdup(PQgetvalue(res, 0, 0));
- if (!PQgetisnull(res, 0, 1))
- default_collate = pg_strdup(PQgetvalue(res, 0, 1));
- if (!PQgetisnull(res, 0, 2))
- default_ctype = pg_strdup(PQgetvalue(res, 0, 2));
- }
-
- PQclear(res);
-
-
- /*
- * Now collect all the information about databases to dump.
- *
- * For the database ACLs, as of 9.6, we extract both the positive (as
- * datacl) and negative (as rdatacl) ACLs, relative to the default ACL for
- * databases, which are then passed to buildACLCommands() below.
- *
- * See buildACLQueries() and buildACLCommands().
- *
- * Note that we do not support initial privileges (pg_init_privs) on
- * databases.
- */
- if (server_version >= 90600)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba))) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba))) as datacls)"
- "AS datacl, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba)) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba)))) as rdatacls)"
- "AS rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 90300)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80400)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80100)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(usename, (select usename from pg_shadow where usesysid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "-1 as datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN pg_shadow u ON (datdba = usesysid) "
- "WHERE datallowconn ORDER BY 1");
-
- res = executeQuery(conn, buf->data);
-
- for (i = 0; i < PQntuples(res); i++)
- {
- char *dbname = PQgetvalue(res, i, 0);
- char *dbowner = PQgetvalue(res, i, 1);
- char *dbencoding = PQgetvalue(res, i, 2);
- char *dbcollate = PQgetvalue(res, i, 3);
- char *dbctype = PQgetvalue(res, i, 4);
- uint32 dbfrozenxid = atooid(PQgetvalue(res, i, 5));
- uint32 dbminmxid = atooid(PQgetvalue(res, i, 6));
- char *dbistemplate = PQgetvalue(res, i, 7);
- char *dbacl = PQgetvalue(res, i, 8);
- char *rdbacl = PQgetvalue(res, i, 9);
- char *dbconnlimit = PQgetvalue(res, i, 10);
- char *dbtablespace = PQgetvalue(res, i, 11);
- char *fdbname;
-
- fdbname = pg_strdup(fmtId(dbname));
-
- resetPQExpBuffer(buf);
-
- /*
- * Skip the CREATE DATABASE commands for "template1" and "postgres",
- * since they are presumably already there in the destination cluster.
- * We do want to emit their ACLs and config options if any, however.
- */
- if (strcmp(dbname, "template1") != 0 &&
- strcmp(dbname, "postgres") != 0)
- {
- appendPQExpBuffer(buf, "CREATE DATABASE %s", fdbname);
-
- appendPQExpBufferStr(buf, " WITH TEMPLATE = template0");
-
- if (strlen(dbowner) != 0)
- appendPQExpBuffer(buf, " OWNER = %s", fmtId(dbowner));
-
- if (default_encoding && strcmp(dbencoding, default_encoding) != 0)
- {
- appendPQExpBufferStr(buf, " ENCODING = ");
- appendStringLiteralConn(buf, dbencoding, conn);
- }
-
- if (default_collate && strcmp(dbcollate, default_collate) != 0)
- {
- appendPQExpBufferStr(buf, " LC_COLLATE = ");
- appendStringLiteralConn(buf, dbcollate, conn);
- }
-
- if (default_ctype && strcmp(dbctype, default_ctype) != 0)
- {
- appendPQExpBufferStr(buf, " LC_CTYPE = ");
- appendStringLiteralConn(buf, dbctype, conn);
- }
-
- /*
- * Output tablespace if it isn't the default. For default, it
- * uses the default from the template database. If tablespace is
- * specified and tablespace creation failed earlier, (e.g. no such
- * directory), the database creation will fail too. One solution
- * would be to use 'SET default_tablespace' like we do in pg_dump
- * for setting non-default database locations.
- */
- if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- appendPQExpBuffer(buf, " TABLESPACE = %s",
- fmtId(dbtablespace));
-
- if (strcmp(dbistemplate, "t") == 0)
- appendPQExpBuffer(buf, " IS_TEMPLATE = true");
-
- if (strcmp(dbconnlimit, "-1") != 0)
- appendPQExpBuffer(buf, " CONNECTION LIMIT = %s",
- dbconnlimit);
-
- appendPQExpBufferStr(buf, ";\n");
- }
- else if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- {
- /*
- * Cannot change tablespace of the database we're connected to, so
- * to move "postgres" to another tablespace, we connect to
- * "template1", and vice versa.
- */
- if (strcmp(dbname, "postgres") == 0)
- appendPQExpBuffer(buf, "\\connect template1\n");
- else
- appendPQExpBuffer(buf, "\\connect postgres\n");
-
- appendPQExpBuffer(buf, "ALTER DATABASE %s SET TABLESPACE %s;\n",
- fdbname, fmtId(dbtablespace));
-
- /* connect to original database */
- appendPsqlMetaConnect(buf, dbname);
- }
-
- if (binary_upgrade)
- {
- appendPQExpBufferStr(buf, "-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(buf, "UPDATE pg_catalog.pg_database "
- "SET datfrozenxid = '%u', datminmxid = '%u' "
- "WHERE datname = ",
- dbfrozenxid, dbminmxid);
- appendStringLiteralConn(buf, dbname, conn);
- appendPQExpBufferStr(buf, ";\n");
- }
-
- if (!skip_acls &&
- !buildACLCommands(fdbname, NULL, "DATABASE",
- dbacl, rdbacl, dbowner,
- "", server_version, buf))
- {
- fprintf(stderr, _("%s: could not parse ACL list (%s) for database \"%s\"\n"),
- progname, dbacl, fdbname);
- PQfinish(conn);
- exit_nicely(1);
- }
-
- fprintf(OPF, "%s", buf->data);
-
- dumpDatabaseConfig(conn, dbname);
-
- free(fdbname);
- }
-
- if (default_encoding)
- free(default_encoding);
- if (default_collate)
- free(default_collate);
- if (default_ctype)
- free(default_ctype);
-
- PQclear(res);
- destroyPQExpBuffer(buf);
-
- fprintf(OPF, "\n\n");
-}
-
-
-/*
- * Dump database-specific configuration
- */
-static void
-dumpDatabaseConfig(PGconn *conn, const char *dbname)
-{
- PQExpBuffer buf = createPQExpBuffer();
- int count = 1;
-
- for (;;)
- {
- PGresult *res;
-
- if (server_version >= 90000)
- printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting WHERE "
- "setrole = 0 AND setdatabase = (SELECT oid FROM pg_database WHERE datname = ", count);
- else
- printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE datname = ", count);
- appendStringLiteralConn(buf, dbname, conn);
-
- if (server_version >= 90000)
- appendPQExpBufferChar(buf, ')');
-
- res = executeQuery(conn, buf->data);
- if (PQntuples(res) == 1 &&
- !PQgetisnull(res, 0, 0))
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
- "DATABASE", dbname, NULL, NULL);
- PQclear(res);
- count++;
- }
- else
- {
- PQclear(res);
- break;
- }
- }
-
- destroyPQExpBuffer(buf);
-}
-
-
/*
* Dump user-specific configuration
@@ -1646,11 +1308,14 @@ dumpUserConfig(PGconn *conn, const char *username)
appendPQExpBufferChar(buf, ')');
res = executeQuery(conn, buf->data);
+ resetPQExpBuffer(buf);
+
if (PQntuples(res) == 1 &&
!PQgetisnull(res, 0, 0))
{
makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
- "ROLE", username, NULL, NULL);
+ "ROLE", username, NULL, NULL, buf);
+ fprintf(OPF, "%s", buf->data);
PQclear(res);
count++;
}
@@ -1665,85 +1330,6 @@ dumpUserConfig(PGconn *conn, const char *username)
}
-/*
- * Dump user-and-database-specific configuration
- */
-static void
-dumpDbRoleConfig(PGconn *conn)
-{
- PQExpBuffer buf = createPQExpBuffer();
- PGresult *res;
- int i;
-
- printfPQExpBuffer(buf, "SELECT rolname, datname, unnest(setconfig) "
- "FROM pg_db_role_setting, %s u, pg_database "
- "WHERE setrole = u.oid AND setdatabase = pg_database.oid", role_catalog);
- res = executeQuery(conn, buf->data);
-
- if (PQntuples(res) > 0)
- {
- fprintf(OPF, "--\n-- Per-Database Role Settings \n--\n\n");
-
- for (i = 0; i < PQntuples(res); i++)
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, i, 2),
- "ROLE", PQgetvalue(res, i, 0),
- "DATABASE", PQgetvalue(res, i, 1));
- }
-
- fprintf(OPF, "\n\n");
- }
-
- PQclear(res);
- destroyPQExpBuffer(buf);
-}
-
-
-/*
- * Helper function for dumpXXXConfig().
- */
-static void
-makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name,
- const char *type2, const char *name2)
-{
- char *pos;
- char *mine;
- PQExpBuffer buf;
-
- mine = pg_strdup(arrayitem);
- pos = strchr(mine, '=');
- if (pos == NULL)
- {
- free(mine);
- return;
- }
-
- buf = createPQExpBuffer();
-
- *pos = 0;
- appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
- if (type2 != NULL && name2 != NULL)
- appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
- appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
-
- /*
- * Some GUC variable names are 'LIST' type and hence must not be quoted.
- */
- if (pg_strcasecmp(mine, "DateStyle") == 0
- || pg_strcasecmp(mine, "search_path") == 0)
- appendPQExpBufferStr(buf, pos + 1);
- else
- appendStringLiteralConn(buf, pos + 1, conn);
- appendPQExpBufferStr(buf, ";\n");
-
- fprintf(OPF, "%s", buf->data);
- destroyPQExpBuffer(buf);
- free(mine);
-}
-
-
-
/*
* Dump contents of databases.
*/
@@ -1758,29 +1344,11 @@ dumpDatabases(PGconn *conn)
for (i = 0; i < PQntuples(res); i++)
{
int ret;
-
char *dbname = PQgetvalue(res, i, 0);
- PQExpBufferData connectbuf;
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: dumping database \"%s\"...\n"), progname, dbname);
- initPQExpBuffer(&connectbuf);
- appendPsqlMetaConnect(&connectbuf, dbname);
- fprintf(OPF, "%s\n", connectbuf.data);
- termPQExpBuffer(&connectbuf);
-
- /*
- * Restore will need to write to the target cluster. This connection
- * setting is emitted for pg_dumpall rather than in the code also used
- * by pg_dump, so that a cluster with databases or users which have
- * this flag turned on can still be replicated through pg_dumpall
- * without editing the file or stream. With pg_dump there are many
- * other ways to allow the file to be used, and leaving it out allows
- * users to protect databases from being accidental restore targets.
- */
- fprintf(OPF, "SET default_transaction_read_only = off;\n\n");
-
if (filename)
fclose(OPF);
@@ -1822,6 +1390,18 @@ runPgDump(const char *dbname)
appendPQExpBuffer(cmd, "\"%s\" %s", pg_dump_bin,
pgdumpopts->data);
+
+ /*
+ * The create individual database dump, pass the the create database option
+ * to pg_dump. But skip generating create database commands for default
+ * databases (template1, postgres).
+ */
+ if (strcmp(dbname, "template1") == 0 || strcmp(dbname, "postgres") == 0)
+ appendPQExpBufferStr(cmd, " --set-db-properties ");
+ else
+ appendPQExpBufferStr(cmd, " -C ");
+
+
/*
* If we have a filename, use the undocumented plain-append pg_dump
* format.
@@ -1840,7 +1420,7 @@ runPgDump(const char *dbname)
appendShellString(cmd, connstrbuf->data);
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: running \"%s\"\n"), progname, cmd->data);
fflush(stdout);
@@ -2133,31 +1713,7 @@ constructConnStr(const char **keywords, const char **values)
return connstr;
}
-/*
- * Run a query, return the results, exit program on failure.
- */
-static PGresult *
-executeQuery(PGconn *conn, const char *query)
-{
- PGresult *res;
- if (verbose)
- fprintf(stderr, _("%s: executing %s\n"), progname, query);
-
- res = PQexec(conn, query);
- if (!res ||
- PQresultStatus(res) != PGRES_TUPLES_OK)
- {
- fprintf(stderr, _("%s: query failed: %s"),
- progname, PQerrorMessage(conn));
- fprintf(stderr, _("%s: query was: %s\n"),
- progname, query);
- PQfinish(conn);
- exit_nicely(1);
- }
-
- return res;
-}
/*
* As above for a SQL command (which returns nothing).
@@ -2167,7 +1723,7 @@ executeCommand(PGconn *conn, const char *query)
{
PGresult *res;
- if (verbose)
+ if (g_verbose)
fprintf(stderr, _("%s: executing %s\n"), progname, query);
res = PQexec(conn, query);
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 8a662e9865..276262475c 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -48,6 +48,7 @@ generate_old_dump(void)
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
PQExpBufferData connstr,
escaped_connstr;
+ PQExpBuffer create_opts = createPQExpBuffer();
initPQExpBuffer(&connstr);
appendPQExpBuffer(&connstr, "dbname=");
@@ -60,13 +61,20 @@ generate_old_dump(void)
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
+ if (strcmp(old_db->db_name, "template1") == 0 || strcmp(old_db->db_name, "postgres") == 0)
+ appendPQExpBufferStr(create_opts, " --set-db-properties ");
+ else
+ appendPQExpBufferStr(create_opts, " -C ");
+
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade %s "
+ "--format=custom %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
- log_opts.verbose ? "--verbose" : "",
+ create_opts->data, log_opts.verbose ? "--verbose" : "",
sql_file_name, escaped_connstr.data);
+ termPQExpBuffer(create_opts);
termPQExpBuffer(&escaped_connstr);
}
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 872621489f..c233e68dbd 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -46,7 +46,7 @@
#endif
static void prepare_new_cluster(void);
-static void prepare_new_databases(void);
+static void prepare_new_globals(void);
static void create_new_objects(void);
static void copy_xact_xlog_xid(void);
static void set_frozenxids(bool minmxid_only);
@@ -124,7 +124,7 @@ main(int argc, char **argv)
/* -- NEW -- */
start_postmaster(&new_cluster, true);
- prepare_new_databases();
+ prepare_new_globals();
create_new_objects();
@@ -271,7 +271,7 @@ prepare_new_cluster(void)
static void
-prepare_new_databases(void)
+prepare_new_globals(void)
{
/*
* We set autovacuum_freeze_max_age to its maximum value so autovacuum
@@ -294,9 +294,6 @@ prepare_new_databases(void)
new_cluster.bindir, cluster_conn_opts(&new_cluster),
GLOBALS_DUMP_FILE);
check_ok();
-
- /* we load this to get a current list of databases */
- get_db_and_rel_infos(&new_cluster);
}
@@ -312,15 +309,6 @@ create_new_objects(void)
char sql_file_name[MAXPGPATH],
log_file_name[MAXPGPATH];
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
- PQExpBufferData connstr,
- escaped_connstr;
-
- initPQExpBuffer(&connstr);
- appendPQExpBuffer(&connstr, "dbname=");
- appendConnStrVal(&connstr, old_db->db_name);
- initPQExpBuffer(&escaped_connstr);
- appendShellString(&escaped_connstr, connstr.data);
- termPQExpBuffer(&connstr);
pg_log(PG_STATUS, "%s", old_db->db_name);
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
@@ -332,13 +320,10 @@ create_new_objects(void)
*/
parallel_exec_prog(log_file_name,
NULL,
- "\"%s/pg_restore\" %s --exit-on-error --verbose --dbname %s \"%s\"",
+ "\"%s/pg_restore\" %s --create --exit-on-error --verbose --dbname template1 \"%s\"",
new_cluster.bindir,
cluster_conn_opts(&new_cluster),
- escaped_connstr.data,
sql_file_name);
-
- termPQExpBuffer(&escaped_connstr);
}
/* reap all children */
--
2.15.0.windows.1
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-17 23:50 ` Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
1 sibling, 1 reply; 33+ messages in thread
From: Tom Lane @ 2018-01-17 23:50 UTC (permalink / raw)
To: Haribabu Kommi <[email protected]>; +Cc: Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Haribabu Kommi <[email protected]> writes:
> [ pg_dump-and-pg_dumpall-database-handling-refactoring_v12.patch ]
I've gone through this in a once-over-lightly fashion. Since there was
quite a bit of debate upthread about how things should work, I'd like
to start by summarizing the decisions this patch has made, in case
anyone still wants to object to them:
* pg_dump will now be responsible for dumping all per-database properties,
including "ALTER ROLE IN DATABASE SET" settings. To get that behavior,
pg_dumpall will invoke pg_dump using -C, or using the new switch
--set-db-properties when working on "template1" or "postgres" databases,
since those should already exist in the target installation.
* pg_dumpall still won't dump template0 (or any other not-datallowconn
database). This means that it will no longer propagate any non-default
per-database properties for such databases. I think this is fine for
template0: if you've messed with that at all, you are fooling with non
user-serviceable parts. It's a bit less good for other DBs maybe, but on
the whole it seems more consistent to simply treat nonconnectable DBs
as not existing. (That could stand to be documented though.)
* "pg_dumpall -g" will now produce only role- and tablespace-related output.
* There isn't any direct way to ask pg_dump for "just the DB properties
and nothing else" (although I suppose you can mostly fake it by adding
"--schema nosuchschema" or some such). This makes it inconvenient to
exactly duplicate the old behavior of "pg_dumpall -g", if someone wanted
to do that to fit into their existing backup procedures. I'm not sure
how important that is. Personally I'm content to lose it, but if there's
enough pushback maybe we could invent a "--db-properties-only" switch?
There are some other points that haven't been debated:
* As the patch stands, --set-db-properties is implicit when you specify
-C, and in fact the patch goes to the trouble of throwing an error if you
try to specify both switches. I'm inclined to think this might be a bad
idea. What about saying that -C enables emitting CREATE DATABASE and
reconnecting to that DB, and independently of that, --set-db-properties
enables emitting the additional per-database properties? This seems
simpler to understand, more flexible, and less of a change from the
previous behavior of -C. On the other hand you could argue that people
would always want --set-db-properties with -C and so we're merely
promoting carpal tunnel (and errors of omission) if we do it like that.
If so, maybe we could say "-C implies --set-db-properties by default, but
if you really don't want that, you can say -C --no-set-db-properties".
Perhaps the only application of this is to reproduce pg_dump's historical
behavior, but that's probably of some value.
* The patch fails to make any provision for deciding at pg_restore time
whether to emit DB properties. Considering that you can use -C at restore
time, I think it's pretty awful that you can't use --set-db-properties
then. Moreover, not only has the patch not added a separate "DATABASE
PROPERTIES" TOC entry type as was originally proposed, what it's actually
doing is emitting two identically labeled "DATABASE" TOC entries, one
with the CREATE and the other with the rest. That's simply horrid.
I think we need to do this properly and emit two distinguishable TOC
entries, and control which of them get printed on the restore side of the
logic, not the TOC entry construction side. (I'm not sure at this point
if we need an archive version bump to do that, but perhaps not --- in the
past we've added new TOC types without a version bump.)
* Some years ago, commit 4bd371f6f caused pg_dumpall to emit "SET
default_transaction_read_only = off;" in its scripts, so that it could
successfully handle DBs that have "SET default_transaction_read_only = on"
as a database property. This was intentionally NOT done to pg_dump,
arguing that doing so would greatly weaken such a setting as a defense
against stupid errors (i.e. restoring into your production database).
The patch currently ignores that reasoning and moves the setting into
pg_dump anyway. Haribabu mentioned this point upthread and got basically
no response, but I'm concerned about it.
I think however that we might be able to dodge that issue as a byproduct
of fixing the previous problem. If the CREATE DATABASE is done as one
TOC entry, and we reconnect to the target DB after processing that entry,
and then we process the DATABASE PROPERTIES entry, then any DB property
settings we've installed will not apply in our existing session. So I
think we can just leave out the "SET default_transaction_read_only = on"
altogether. Obviously this puts a premium on not reconnecting, but
reconnecting would break other more important features like
--single-transaction, so I am not worried about that.
* An issue in connection with that is that because you can't issue
ALTER DATABASE SET TABLESPACE against the current DB, pg_dumpall
currently takes the trouble to explicitly reconnect to a different DB
and then back to the target DB when issuing that command. That doesn't
play nice with my suggestion above, nor with --single-transaction.
I'm still thinking about how to fix that, but probably the best fix
involves handling non-default tablespaces by including them in the
initial CREATE DATABASE command, plus emitting an ALTER DATABASE SET
TABLESPACE in a separate new "DATABASE TABLESPACE" TOC object, which
we would have to teach the pg_restore logic to handle correctly.
At minimum it'd need to be aware about the interaction with
--single-transaction; since we have to have that anyway, we could also
make it do the extra-reconnections dance instead of hard-wiring \connect
commands into the text of the entry, and/or skip the whole thing if it
knows it emitted the CREATE DATABASE command.
Comments? If there's not objections I plan to push forward on this.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-18 21:34 ` Robert Haas <[email protected]>
2018-01-18 22:09 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
0 siblings, 2 replies; 33+ messages in thread
From: Robert Haas @ 2018-01-18 21:34 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Wed, Jan 17, 2018 at 6:50 PM, Tom Lane <[email protected]> wrote:
> * As the patch stands, --set-db-properties is implicit when you specify
> -C, and in fact the patch goes to the trouble of throwing an error if you
> try to specify both switches. I'm inclined to think this might be a bad
> idea. What about saying that -C enables emitting CREATE DATABASE and
> reconnecting to that DB, and independently of that, --set-db-properties
> enables emitting the additional per-database properties? This seems
> simpler to understand, more flexible, and less of a change from the
> previous behavior of -C. On the other hand you could argue that people
> would always want --set-db-properties with -C and so we're merely
> promoting carpal tunnel (and errors of omission) if we do it like that.
I would vigorously agree with that latter argument. I think the
chances of errors of omission would be very high even if the carpal
tunnel dangers were ameliorated by giving --set-db-properties a short
option name.
> If so, maybe we could say "-C implies --set-db-properties by default, but
> if you really don't want that, you can say -C --no-set-db-properties".
That seems like a better idea.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
@ 2018-01-18 22:09 ` Tom Lane <[email protected]>
1 sibling, 0 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-18 22:09 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Robert Haas <[email protected]> writes:
> On Wed, Jan 17, 2018 at 6:50 PM, Tom Lane <[email protected]> wrote:
>> * As the patch stands, --set-db-properties is implicit when you specify
>> -C, and in fact the patch goes to the trouble of throwing an error if you
>> try to specify both switches. I'm inclined to think this might be a bad
>> idea. What about saying that -C enables emitting CREATE DATABASE and
>> reconnecting to that DB, and independently of that, --set-db-properties
>> enables emitting the additional per-database properties? This seems
>> simpler to understand, more flexible, and less of a change from the
>> previous behavior of -C. On the other hand you could argue that people
>> would always want --set-db-properties with -C and so we're merely
>> promoting carpal tunnel (and errors of omission) if we do it like that.
> I would vigorously agree with that latter argument. I think the
> chances of errors of omission would be very high even if the carpal
> tunnel dangers were ameliorated by giving --set-db-properties a short
> option name.
Fair enough. We'll keep the behavioral change then.
>> If so, maybe we could say "-C implies --set-db-properties by default, but
>> if you really don't want that, you can say -C --no-set-db-properties".
> That seems like a better idea.
What I think I'll do for the moment is make them independent options so
far as the implementation is concerned, but have the command line switch
processing do
/* --create implies --set-db-properties, for now anyway */
if (dopt.outputCreateDB)
dopt.set_db_properties = 1;
If somebody actually asks for --no-set-db-properties, we can add that
later.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
@ 2018-01-18 23:15 ` Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
1 sibling, 1 reply; 33+ messages in thread
From: Tom Lane @ 2018-01-18 23:15 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
... okay, so the reason why --set-db-properties isn't as general-purpose
a switch as you might think finally penetrated my thick skull :-(
The problem is that, as the patch is currently constituted, that will
result in emitting a lot of "ALTER DATABASE foo" type commands. If
somebody tries to load the pg_dump output into a database not named foo,
mayhem ensues. This is exactly the same problem people have already
noted with respect to COMMENT ON DATABASE, but now we're propagating it to
other operations that have (much) higher downsides for getting it wrong.
What we need, therefore, is ALTER DATABASE CURRENT_DATABASE, which
I see is part of the pending patch for fixing the COMMENT ON DATABASE
problem. That one is stuck in Waiting on Author, but I'm inclined to
go see if I can push it across the finish line, and then come back
to this one.
Even with that, there's a small problem: the backend cannot reasonably
support ALTER DATABASE CURRENT_DATABASE SET TABLESPACE, except perhaps
in the no-op case where the specified tablespace is already the right
one. So this puts a serious hole in my thoughts about allowing the
tablespace to be adjusted during --set-db-properties without --create.
What I think we should do for the time being is to have pg_dump treat
database tablespace as a property it can't adjust after creation, just
as it can't adjust locale or encoding. That's a loss of functionality
for pg_dumpall/pg_upgrade compared to where we are today, in that if
you've set up the template1 or postgres DBs with nondefault tablespace
then that won't propagate to the new cluster. But the same can already
be said about their locale and encoding, and I find it hard to believe
that many people are trying to give those two DBs tablespace settings
different from the cluster default, anyway.
The only way around that that I can see is to give pg_dump some
additional switch along the lines of --i-promise-i-wont-restore-
this-into-a-database-with-a-different-name, and then it could emit
the same sort of
\connect template1
alter database postgres set tablespace ...;
\connect postgres
dance that pg_dumpall is using today. But that sort of ugliness
is exactly what we said we didn't want in this patch.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-18 23:35 ` Tom Lane <[email protected]>
2018-01-19 00:09 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-19 14:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
0 siblings, 3 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-18 23:35 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
I wrote:
> What I think we should do for the time being is to have pg_dump treat
> database tablespace as a property it can't adjust after creation, just
> as it can't adjust locale or encoding. That's a loss of functionality
> for pg_dumpall/pg_upgrade compared to where we are today, in that if
> you've set up the template1 or postgres DBs with nondefault tablespace
> then that won't propagate to the new cluster. But the same can already
> be said about their locale and encoding, and I find it hard to believe
> that many people are trying to give those two DBs tablespace settings
> different from the cluster default, anyway.
Hm ... actually, there is more than one way to skin this cat.
Let me offer a modest proposal: pg_dumpall/pg_upgrade should simply DROP
postgres and template1 in the target cluster, and then re-create them
(from template0 of course). With that, we'd not only cope with preserving
their tablespace settings, but we'd gain the ability to preserve their
locale and encoding, even if the target cluster had been initialized with
some other default.
If we did it like that, the rationale for an actual --set-db-properties
switch would vanish, at least so far as pg_dumpall is concerned -- we
could just make all that behavior an integral part of --create. And
this wouldn't need to be conditional on getting ALTER DATABASE
CURRENT_DATABASE done.
Comments?
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 00:09 ` Haribabu Kommi <[email protected]>
2 siblings, 0 replies; 33+ messages in thread
From: Haribabu Kommi @ 2018-01-19 00:09 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Fri, Jan 19, 2018 at 10:35 AM, Tom Lane <[email protected]> wrote:
> I wrote:
> > What I think we should do for the time being is to have pg_dump treat
> > database tablespace as a property it can't adjust after creation, just
> > as it can't adjust locale or encoding. That's a loss of functionality
> > for pg_dumpall/pg_upgrade compared to where we are today, in that if
> > you've set up the template1 or postgres DBs with nondefault tablespace
> > then that won't propagate to the new cluster. But the same can already
> > be said about their locale and encoding, and I find it hard to believe
> > that many people are trying to give those two DBs tablespace settings
> > different from the cluster default, anyway.
>
> Hm ... actually, there is more than one way to skin this cat.
>
> Let me offer a modest proposal: pg_dumpall/pg_upgrade should simply DROP
> postgres and template1 in the target cluster, and then re-create them
> (from template0 of course). With that, we'd not only cope with preserving
> their tablespace settings, but we'd gain the ability to preserve their
> locale and encoding, even if the target cluster had been initialized with
> some other default.
>
Yes, I agree that this may be simple change to handle this problem.
Already pg_upgrade doesn't work if there is any encoding difference
between source and target databases. Most probably User will create
with same encoding.
Regards,
Hari Babu
Fujitsu Australia
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 14:28 ` Robert Haas <[email protected]>
2 siblings, 0 replies; 33+ messages in thread
From: Robert Haas @ 2018-01-19 14:28 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Thu, Jan 18, 2018 at 6:35 PM, Tom Lane <[email protected]> wrote:
> If we did it like that, the rationale for an actual --set-db-properties
> switch would vanish, at least so far as pg_dumpall is concerned -- we
> could just make all that behavior an integral part of --create. And
> this wouldn't need to be conditional on getting ALTER DATABASE
> CURRENT_DATABASE done.
Unfortunately, I have a feeling that --set-db-properties might not be
the only thing that would vanish. I think users are accustomed by now
to the idea that if you restore into an existing database, the
existing contents are preserved and the new stuff from the dump is
added (possibly with some errors and messiness). With this design,
the existing database contents will instead vanish, and that is
probably going to make somebody unhappy.
I agree with you that making ALTER DATABASE accept CURRENT_DATABASE is
a good idea. I don't have a great idea what to do about the SET
TABLESPACE problem. It's always seemed to me to be sort of weird that
you have to have a database in order to create, drop, etc. another
database, or even get a list of databases that exist. As a new user,
I remember being quite frustrated that connecting to a database that
didn't exist gave no hint of how to find out what databases did exist.
Eventually I discovered psql -l and all was well, but I had no idea
how it worked under the hood. Even though I now understand the
architectural considerations that have gotten us to where we are, I
still think it would be more intuitive to users if there were a
command-shell unrelated to any database that would let you perform
operations on databases. I realize, however, that this patch isn't
going to create such a thing.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 14:45 ` Tom Lane <[email protected]>
2018-01-19 14:52 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2 siblings, 2 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-19 14:45 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Robert Haas <[email protected]> writes:
> On Thu, Jan 18, 2018 at 6:35 PM, Tom Lane <[email protected]> wrote:
>> If we did it like that, the rationale for an actual --set-db-properties
>> switch would vanish, at least so far as pg_dumpall is concerned -- we
>> could just make all that behavior an integral part of --create. And
>> this wouldn't need to be conditional on getting ALTER DATABASE
>> CURRENT_DATABASE done.
> Unfortunately, I have a feeling that --set-db-properties might not be
> the only thing that would vanish. I think users are accustomed by now
> to the idea that if you restore into an existing database, the
> existing contents are preserved and the new stuff from the dump is
> added (possibly with some errors and messiness). With this design,
> the existing database contents will instead vanish, and that is
> probably going to make somebody unhappy.
Well, we could say that the properties of template1 and postgres
are only restored if you use --clean.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 14:52 ` Robert Haas <[email protected]>
1 sibling, 0 replies; 33+ messages in thread
From: Robert Haas @ 2018-01-19 14:52 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Fri, Jan 19, 2018 at 9:45 AM, Tom Lane <[email protected]> wrote:
> Robert Haas <[email protected]> writes:
>> On Thu, Jan 18, 2018 at 6:35 PM, Tom Lane <[email protected]> wrote:
>>> If we did it like that, the rationale for an actual --set-db-properties
>>> switch would vanish, at least so far as pg_dumpall is concerned -- we
>>> could just make all that behavior an integral part of --create. And
>>> this wouldn't need to be conditional on getting ALTER DATABASE
>>> CURRENT_DATABASE done.
>
>> Unfortunately, I have a feeling that --set-db-properties might not be
>> the only thing that would vanish. I think users are accustomed by now
>> to the idea that if you restore into an existing database, the
>> existing contents are preserved and the new stuff from the dump is
>> added (possibly with some errors and messiness). With this design,
>> the existing database contents will instead vanish, and that is
>> probably going to make somebody unhappy.
>
> Well, we could say that the properties of template1 and postgres
> are only restored if you use --clean.
True. Would that be a POLA violation, do you think?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 15:00 ` Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
1 sibling, 1 reply; 33+ messages in thread
From: Tom Lane @ 2018-01-19 15:00 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Robert Haas <[email protected]> writes:
> On Fri, Jan 19, 2018 at 9:45 AM, Tom Lane <[email protected]> wrote:
>> Well, we could say that the properties of template1 and postgres
>> are only restored if you use --clean.
> True. Would that be a POLA violation, do you think?
It seems a bit non-orthogonal. Also, while your point that people
expect "merge" behavior from pg_dump is certainly true, I'm not
convinced that anybody would be relying on that for pg_dumpall.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 15:13 ` Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
0 siblings, 1 reply; 33+ messages in thread
From: Robert Haas @ 2018-01-19 15:13 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Fri, Jan 19, 2018 at 10:00 AM, Tom Lane <[email protected]> wrote:
> Robert Haas <[email protected]> writes:
>> On Fri, Jan 19, 2018 at 9:45 AM, Tom Lane <[email protected]> wrote:
>>> Well, we could say that the properties of template1 and postgres
>>> are only restored if you use --clean.
>
>> True. Would that be a POLA violation, do you think?
>
> It seems a bit non-orthogonal. Also, while your point that people
> expect "merge" behavior from pg_dump is certainly true, I'm not
> convinced that anybody would be relying on that for pg_dumpall.
What I expect typically happens is that someone restores into an
existing cluster and then realizes their mistake and goes and removes
all of the extra stuff that got created. This is a bit like when you
run tar -zxf expecting it to create a subdirectory, but it turns out
to extract in the current directory instead. You then curse your fate
and remove all of the files it created. It's annoying, but not that
bad. If it nuked the contents of the currently directory first,
though, you'd be much sadder. If somebody changed tar to have that
behavior, people would probably *eventually* become aware of the risk
and adjust their behavior to avoid catastrophe, but it would probably
take 1-2 disasters per individual before they got used to the new way,
and that's a lot of disasters considering how many people use tar.
Or, maybe people wouldn't get used to it and they'd just go after
whoever made the change with pitchforks. Anyway, that kind of thing
seems like a danger here.
The other possibility that comes to mind is that somebody might be
doing this is working around a failure of something like CREATE
LANGUAGE. Say the name of the .so has changed, so CREATE LANGUAGE
will fail or do the wrong thing. Instead of editing the dump, they
pre-install the language with the correct definition and then restore
over it, ignoring errors. I guess that would only work with pg_dump,
not pg_dumpall, unless the database that had the problem was one of
the pre-created ones. Still, it's worth thinking over these kinds of
scenarios, I think. People do a lot of ugly things in the real world
that we as developers would never do, mostly to work around the
problems we fail to foresee.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
@ 2018-01-19 16:28 ` Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
0 siblings, 1 reply; 33+ messages in thread
From: Tom Lane @ 2018-01-19 16:28 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Robert Haas <[email protected]> writes:
> On Fri, Jan 19, 2018 at 10:00 AM, Tom Lane <[email protected]> wrote:
>> Robert Haas <[email protected]> writes:
>>> On Fri, Jan 19, 2018 at 9:45 AM, Tom Lane <[email protected]> wrote:
>>>> Well, we could say that the properties of template1 and postgres
>>>> are only restored if you use --clean.
>>> True. Would that be a POLA violation, do you think?
>> It seems a bit non-orthogonal. Also, while your point that people
>> expect "merge" behavior from pg_dump is certainly true, I'm not
>> convinced that anybody would be relying on that for pg_dumpall.
> [ assorted examples ]
> Still, it's worth thinking over these kinds of
> scenarios, I think. People do a lot of ugly things in the real world
> that we as developers would never do, mostly to work around the
> problems we fail to foresee.
Unless someone has a better idea, I'll go with the semantics stated
above: DB-level properties of the two standard databases are only
transferred to pg_dumpall's target cluster if you authorize dropping
their old contents by saying --clean. (pg_upgrade, of course, will
do exactly that.)
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 19:54 ` Tom Lane <[email protected]>
2018-01-19 20:43 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-21 15:17 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Bruce Momjian <[email protected]>
0 siblings, 2 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-19 19:54 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Hmm ... so there's a small problem with this idea of dropping and
recreating template1:
pg_restore: connecting to database for restore
pg_restore: dropping DATABASE template1
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 3024; 1262 1 DATABASE template1
postgres
pg_restore: [archiver (db)] could not execute query: ERROR: cannot drop a templ
ate database
Command was: DROP DATABASE "template1";
Now in principle we could hack around that by issuing "ALTER DATABASE
... IS_TEMPLATE false" first, but it turns out to be harder than you
might think to wedge that into the pg_dump infrastructure. (The
natural way to do it, trying to add this into the dropCmd for the
database TOC entry, fails because (a) DROP DATABASE then ends up as
one part of an implicit transaction block, and (b) it confuses the heck
out of pg_restore's --if-exists kluge.)
You can actually exhibit this in current releases if you try "pg_dump
--clean --create" on a user-created template database, so it's not
solely the fault of this patch.
What do people think of just removing this DROP DATABASE restriction?
Arguably, superusers should know better than to drop template1 anyway.
Maybe we should replace it with a hard-wired check against dropping
template0 (matched by name) just to stave off the worst-case scenario.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-19 20:43 ` Robert Haas <[email protected]>
2018-01-19 21:07 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
1 sibling, 1 reply; 33+ messages in thread
From: Robert Haas @ 2018-01-19 20:43 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Fri, Jan 19, 2018 at 2:54 PM, Tom Lane <[email protected]> wrote:
> Hmm ... so there's a small problem with this idea of dropping and
> recreating template1:
>
> pg_restore: connecting to database for restore
> pg_restore: dropping DATABASE template1
> pg_restore: [archiver (db)] Error while PROCESSING TOC:
> pg_restore: [archiver (db)] Error from TOC entry 3024; 1262 1 DATABASE template1
> postgres
> pg_restore: [archiver (db)] could not execute query: ERROR: cannot drop a templ
> ate database
> Command was: DROP DATABASE "template1";
>
> Now in principle we could hack around that by issuing "ALTER DATABASE
> ... IS_TEMPLATE false" first, but it turns out to be harder than you
> might think to wedge that into the pg_dump infrastructure. (The
> natural way to do it, trying to add this into the dropCmd for the
> database TOC entry, fails because (a) DROP DATABASE then ends up as
> one part of an implicit transaction block, and (b) it confuses the heck
> out of pg_restore's --if-exists kluge.)
>
> You can actually exhibit this in current releases if you try "pg_dump
> --clean --create" on a user-created template database, so it's not
> solely the fault of this patch.
>
> What do people think of just removing this DROP DATABASE restriction?
> Arguably, superusers should know better than to drop template1 anyway.
> Maybe we should replace it with a hard-wired check against dropping
> template0 (matched by name) just to stave off the worst-case scenario.
I think it's a little scary. The tail might be wagging the dog at this point.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 20:43 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
@ 2018-01-19 21:07 ` Tom Lane <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-19 21:07 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Robert Haas <[email protected]> writes:
> On Fri, Jan 19, 2018 at 2:54 PM, Tom Lane <[email protected]> wrote:
>> What do people think of just removing this DROP DATABASE restriction?
>> Arguably, superusers should know better than to drop template1 anyway.
>> Maybe we should replace it with a hard-wired check against dropping
>> template0 (matched by name) just to stave off the worst-case scenario.
> I think it's a little scary. The tail might be wagging the dog at this point.
True, and having to make assumptions about which server version we're
restoring to is not very nice either.
After further reflection I've found a way to do this on the pg_dump
end that's not as ugly as I feared -- no uglier than most of the other
hacks there, anyway. So nevermind ...
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-21 15:17 ` Bruce Momjian <[email protected]>
2018-01-21 16:02 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
1 sibling, 1 reply; 33+ messages in thread
From: Bruce Momjian @ 2018-01-21 15:17 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Fri, Jan 19, 2018 at 02:54:25PM -0500, Tom Lane wrote:
> Hmm ... so there's a small problem with this idea of dropping and
> recreating template1:
>
> pg_restore: connecting to database for restore
> pg_restore: dropping DATABASE template1
> pg_restore: [archiver (db)] Error while PROCESSING TOC:
> pg_restore: [archiver (db)] Error from TOC entry 3024; 1262 1 DATABASE template1
> postgres
> pg_restore: [archiver (db)] could not execute query: ERROR: cannot drop a templ
> ate database
> Command was: DROP DATABASE "template1";
Uh, the oid of the template1 database is 1, and I assume we would want
to preserve that too.
--
Bruce Momjian <[email protected]> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-21 15:17 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Bruce Momjian <[email protected]>
@ 2018-01-21 16:02 ` Tom Lane <[email protected]>
2018-01-21 17:42 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-23 15:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Bruce Momjian <[email protected]>
0 siblings, 2 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-21 16:02 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Robert Haas <[email protected]>; Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Bruce Momjian <[email protected]> writes:
> On Fri, Jan 19, 2018 at 02:54:25PM -0500, Tom Lane wrote:
>> Command was: DROP DATABASE "template1";
> Uh, the oid of the template1 database is 1, and I assume we would want
> to preserve that too.
I don't feel any huge attachment to that. In the first place, under
this proposal recreating template1 is something you would only need to do
if you weren't satisfied with its default properties as set by initdb.
Which ought to be a minority of users. In the second place, if you are
changing those properties from the way initdb set it up, it's not really
virgin template1 anymore, so why shouldn't it have a new OID?
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-21 15:17 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Bruce Momjian <[email protected]>
2018-01-21 16:02 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-21 17:42 ` Tom Lane <[email protected]>
1 sibling, 0 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-21 17:42 UTC (permalink / raw)
To: Haribabu Kommi <[email protected]>; +Cc: Bruce Momjian <[email protected]>; Robert Haas <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
I've been hacking away at this patch, and attached is what I've got
so far. I think this is committable, but if anyone wants to do
further review and testing, that'd be fine.
Per discussion, I got rid of the separate --set-db-properties switch:
additional database properties are now applied if and only if --create is
specified. But the DATABASE TOC entry itself still contains only CREATE
DATABASE; the additional commands are carried in a "DATABASE PROPERTIES"
TOC entry so that they will be issued after reconnecting to the new DB.
This dodges the "ALTER DATABASE SET default_transaction_read_only"
problem. (Furthermore, it turns out that we have to do it like that
because pg_restore issues any one TOC entry's contents as a single PQexec.
If you try to cram anything else in with the CREATE DATABASE, then the
server spits up because CREATE DATABASE becomes part of an implicit
transaction block.)
I also fixed it so that a database's comment, security label, and ACL are
restored only when saying --create. This is different from the previous
behavior for DB comments and security labels, but it seems a great deal
more useful and consistent. In no other case would pg_dump/pg_restore
attempt to restore a comment, security label, or ACL for an object it
hadn't created, so why should it act differently for databases?
Worth noting here is that if we accept that behavior, the problem for
which "COMMENT ON CURRENT_DATABASE" was proposed goes away, because
there's no longer a risk of trying to apply COMMENT ON DATABASE to the
wrong database. We might still want that patch as a standalone feature,
but pg_dump no longer needs it.
Another point worth commenting on (this change was also in the v13 patch)
is that pg_dumpall formerly took some pains to omit the encoding and
locale specifications in its CREATE DATABASE commands, for databases whose
settings matched the cluster default. This new implementation won't do
that. We could change it to do so, but then a standalone "pg_dump
--create" would also act that way, which is not really easy to defend.
IIRC, the argument for pg_dumpall doing it like that was to make it
less painful to migrate a cluster to a new machine that might not have the
identical set of locales, or if one wanted to migrate everything to a new
encoding. But those are not mainstream use-cases, and if you really need
to do that you can still get there by dumping databases individually
without using --create. On the other hand, there are obvious gotchas
involved in letting a dump/reload silently change to another locale or
encoding. So I think this is an improvement overall, but it bears
noting as a behavioral change.
Another point to note is that I dithered about whether to bump the
pg_dump archive version number, which would have the effect of preventing
pre-v11 versions of pg_restore from processing dumps made by this version.
The argument for breaking archive compatibility is that older pg_restore
versions will not know that it'd be a good idea to skip DATABASE
PROPERTIES TOC entries and database ACL entries if not using --create.
However, in default cases there won't be a DATABASE PROPERTIES entry to
skip. Moreover, applying these entries unconditionally isn't that much
different conceptually from applying database comments or sec labels
unconditionally, as such older pg_restore versions would do anyway.
It also seems like if your back were against the wall, being able to
read a newer archive file with an older pg_restore is better than being
locked out of it completely. So I'm leaning to no version bump, but
it could still be discussed.
One thing we could possibly use here is more regression test cases.
The only existing regression test that's affected by this patch is
002_pg_dump.pl's expectations about which cases will print a database
comment, so it seems like we're missing some behavioral coverage.
Not sure what that should look like though.
This patch has to be applied over the patches proposed in
https://www.postgresql.org/message-id/21714.1516553459%40sss.pgh.pa.us
Aside from touching some of the same code, this is dependent on
the changes made there to make comment, seclabel, and ACL entries
reliably identifiable.
As far as notable code changes go, I got rid of the previous patch's
move of executeQuery() into dumputils.c. That had some undesirable
knock-on effects in terms of creating even more coupling between
different modules (through the g_verbose global), and it was basically
misguided anyway. pg_dump executes queries via ExecuteSqlQuery; we
do not need a few of its functions to be using low-level routines with
behavior different from that.
If anyone wants to do further review on this, let me know; otherwise
I'll push it in a day or so.
regards, tom lane
Attachments:
[text/x-diff] pg_dump-and-pg_dumpall-refactor-v14.patch (66.0K, ../../[email protected]/2-pg_dump-and-pg_dumpall-refactor-v14.patch)
download | inline diff:
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 08cad68..11582dd 100644
*** a/doc/src/sgml/ref/pg_dump.sgml
--- b/doc/src/sgml/ref/pg_dump.sgml
*************** PostgreSQL documentation
*** 46,54 ****
</para>
<para>
! <application>pg_dump</application> only dumps a single database. To backup
! global objects that are common to all databases in a cluster, such as roles
! and tablespaces, use <xref linkend="app-pg-dumpall"/>.
</para>
<para>
--- 46,55 ----
</para>
<para>
! <application>pg_dump</application> only dumps a single database.
! To back up an entire cluster, or to back up global objects that are
! common to all databases in a cluster (such as roles and tablespaces),
! use <xref linkend="app-pg-dumpall"/>.
</para>
<para>
*************** PostgreSQL documentation
*** 142,148 ****
switch is therefore only useful to add large objects to dumps
where a specific schema or table has been requested. Note that
blobs are considered data and therefore will be included when
! --data-only is used, but not when --schema-only is.
</para>
</listitem>
</varlistentry>
--- 143,150 ----
switch is therefore only useful to add large objects to dumps
where a specific schema or table has been requested. Note that
blobs are considered data and therefore will be included when
! <option>--data-only</option> is used, but not
! when <option>--schema-only</option> is.
</para>
</listitem>
</varlistentry>
*************** PostgreSQL documentation
*** 197,202 ****
--- 199,215 ----
</para>
<para>
+ With <option>--create</option>, the output also includes the
+ database's comment if any, and any configuration variable settings
+ that are specific to this database, that is,
+ any <command>ALTER DATABASE ... SET ...</command>
+ and <command>ALTER ROLE ... IN DATABASE ... SET ...</command>
+ commands that mention this database.
+ Access privileges for the database itself are also dumped,
+ unless <option>--no-acl</option> is specified.
+ </para>
+
+ <para>
This option is only meaningful for the plain-text format. For
the archive formats, you can specify the option when you
call <command>pg_restore</command>.
*************** CREATE DATABASE foo WITH TEMPLATE templa
*** 1231,1240 ****
<command>ANALYZE</command> after restoring from a dump file
to ensure optimal performance; see <xref linkend="vacuum-for-statistics"/>
and <xref linkend="autovacuum"/> for more information.
- The dump file also does not
- contain any <command>ALTER DATABASE ... SET</command> commands;
- these settings are dumped by <xref linkend="app-pg-dumpall"/>,
- along with database users and other installation-wide settings.
</para>
<para>
--- 1244,1249 ----
*************** CREATE DATABASE foo WITH TEMPLATE templa
*** 1326,1331 ****
--- 1335,1349 ----
</para>
<para>
+ To reload an archive file into the same database it was dumped from,
+ discarding the current contents of that database:
+
+ <screen>
+ <prompt>$</prompt> <userinput>pg_restore -d postgres --clean --create db.dump</userinput>
+ </screen>
+ </para>
+
+ <para>
To dump a single table named <literal>mytab</literal>:
<screen>
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 5196a21..4a639f2 100644
*** a/doc/src/sgml/ref/pg_dumpall.sgml
--- b/doc/src/sgml/ref/pg_dumpall.sgml
*************** PostgreSQL documentation
*** 36,48 ****
of a cluster into one script file. The script file contains
<acronym>SQL</acronym> commands that can be used as input to <xref
linkend="app-psql"/> to restore the databases. It does this by
! calling <xref linkend="app-pgdump"/> for each database in a cluster.
<application>pg_dumpall</application> also dumps global objects
! that are common to all databases.
(<application>pg_dump</application> does not save these objects.)
- This currently includes information about database users and
- groups, tablespaces, and properties such as access permissions
- that apply to databases as a whole.
</para>
<para>
--- 36,45 ----
of a cluster into one script file. The script file contains
<acronym>SQL</acronym> commands that can be used as input to <xref
linkend="app-psql"/> to restore the databases. It does this by
! calling <xref linkend="app-pgdump"/> for each database in the cluster.
<application>pg_dumpall</application> also dumps global objects
! that are common to all databases, that is, database roles and tablespaces.
(<application>pg_dump</application> does not save these objects.)
</para>
<para>
*************** PostgreSQL documentation
*** 50,56 ****
databases you will most likely have to connect as a database
superuser in order to produce a complete dump. Also you will need
superuser privileges to execute the saved script in order to be
! allowed to add users and groups, and to create databases.
</para>
<para>
--- 47,53 ----
databases you will most likely have to connect as a database
superuser in order to produce a complete dump. Also you will need
superuser privileges to execute the saved script in order to be
! allowed to add roles and create databases.
</para>
<para>
*************** PostgreSQL documentation
*** 308,314 ****
<listitem>
<para>
Use conditional commands (i.e. add an <literal>IF EXISTS</literal>
! clause) to clean databases and other objects. This option is not valid
unless <option>--clean</option> is also specified.
</para>
</listitem>
--- 305,311 ----
<listitem>
<para>
Use conditional commands (i.e. add an <literal>IF EXISTS</literal>
! clause) to drop databases and other objects. This option is not valid
unless <option>--clean</option> is also specified.
</para>
</listitem>
*************** PostgreSQL documentation
*** 500,509 ****
<para>
The option is called <literal>--dbname</literal> for consistency with other
client applications, but because <application>pg_dumpall</application>
! needs to connect to many databases, database name in the connection
! string will be ignored. Use <literal>-l</literal> option to specify
! the name of the database used to dump global objects and to discover
! what other databases should be dumped.
</para>
</listitem>
</varlistentry>
--- 497,507 ----
<para>
The option is called <literal>--dbname</literal> for consistency with other
client applications, but because <application>pg_dumpall</application>
! needs to connect to many databases, the database name in the
! connection string will be ignored. Use the <literal>-l</literal>
! option to specify the name of the database used for the initial
! connection, which will dump global objects and discover what other
! databases should be dumped.
</para>
</listitem>
</varlistentry>
*************** PostgreSQL documentation
*** 658,663 ****
--- 656,672 ----
</para>
<para>
+ The <option>--clean</option> option can be useful even when your
+ intention is to restore the dump script into a fresh cluster. Use of
+ <option>--clean</option> authorizes the script to drop and re-create the
+ built-in <literal>postgres</literal> and <literal>template1</literal>
+ databases, ensuring that those databases will retain the same properties
+ (for instance, locale and encoding) that they had in the source cluster.
+ Without the option, those databases will retain their existing
+ database-level properties, as well as any pre-existing contents.
+ </para>
+
+ <para>
Once restored, it is wise to run <command>ANALYZE</command> on each
database so the optimizer has useful statistics. You
can also run <command>vacuumdb -a -z</command> to analyze all
*************** PostgreSQL documentation
*** 665,670 ****
--- 674,691 ----
</para>
<para>
+ The dump script should not be expected to run completely without errors.
+ In particular, because the script will issue <command>CREATE ROLE</command>
+ for every role existing in the source cluster, it is certain to get a
+ <quote>role already exists</quote> error for the bootstrap superuser,
+ unless the destination cluster was initialized with a different bootstrap
+ superuser name. This error is harmless and should be ignored. Use of
+ the <option>--clean</option> option is likely to produce additional
+ harmless error messages about non-existent objects, although you can
+ minimize those by adding <option>--if-exists</option>.
+ </para>
+
+ <para>
<application>pg_dumpall</application> requires all needed
tablespace directories to exist before the restore; otherwise,
database creation will fail for databases in non-default
*************** PostgreSQL documentation
*** 688,697 ****
<screen>
<prompt>$</prompt> <userinput>psql -f db.out postgres</userinput>
</screen>
! (It is not important to which database you connect here since the
script file created by <application>pg_dumpall</application> will
contain the appropriate commands to create and connect to the saved
! databases.)
</para>
</refsect1>
--- 709,721 ----
<screen>
<prompt>$</prompt> <userinput>psql -f db.out postgres</userinput>
</screen>
! It is not important to which database you connect here since the
script file created by <application>pg_dumpall</application> will
contain the appropriate commands to create and connect to the saved
! databases. An exception is that if you specified <option>--clean</option>,
! you must connect to the <literal>postgres</literal> database initially;
! the script will attempt to drop other databases immediately, and that
! will fail for the database you are connected to.
</para>
</refsect1>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 9946b94..a2ebf75 100644
*** a/doc/src/sgml/ref/pg_restore.sgml
--- b/doc/src/sgml/ref/pg_restore.sgml
***************
*** 127,132 ****
--- 127,143 ----
</para>
<para>
+ With <option>--create</option>, <application>pg_restore</application>
+ also restores the database's comment if any, and any configuration
+ variable settings that are specific to this database, that is,
+ any <command>ALTER DATABASE ... SET ...</command>
+ and <command>ALTER ROLE ... IN DATABASE ... SET ...</command>
+ commands that mention this database.
+ Access privileges for the database itself are also restored,
+ unless <option>--no-acl</option> is specified.
+ </para>
+
+ <para>
When this option is used, the database named with <option>-d</option>
is used only to issue the initial <command>DROP DATABASE</command> and
<command>CREATE DATABASE</command> commands. All data is restored into the
diff --git a/src/bin/pg_dump/dumputils.c b/src/bin/pg_dump/dumputils.c
index 32ad600..7afddc3 100644
*** a/src/bin/pg_dump/dumputils.c
--- b/src/bin/pg_dump/dumputils.c
*************** buildACLQueries(PQExpBuffer acl_subquery
*** 807,809 ****
--- 807,860 ----
printfPQExpBuffer(init_racl_subquery, "NULL");
}
}
+
+ /*
+ * Helper function for dumping "ALTER DATABASE/ROLE SET ..." commands.
+ *
+ * Parse the contents of configitem (a "name=value" string), wrap it in
+ * a complete ALTER command, and append it to buf.
+ *
+ * type is DATABASE or ROLE, and name is the name of the database or role.
+ * If we need an "IN" clause, type2 and name2 similarly define what to put
+ * there; otherwise they should be NULL.
+ * conn is used only to determine string-literal quoting conventions.
+ */
+ void
+ makeAlterConfigCommand(PGconn *conn, const char *configitem,
+ const char *type, const char *name,
+ const char *type2, const char *name2,
+ PQExpBuffer buf)
+ {
+ char *mine;
+ char *pos;
+
+ /* Parse the configitem. If we can't find an "=", silently do nothing. */
+ mine = pg_strdup(configitem);
+ pos = strchr(mine, '=');
+ if (pos == NULL)
+ {
+ pg_free(mine);
+ return;
+ }
+ *pos++ = '\0';
+
+ /* Build the command, with suitable quoting for everything. */
+ appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
+ if (type2 != NULL && name2 != NULL)
+ appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
+ appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be quoted.
+ * XXX this list is incomplete ...
+ */
+ if (pg_strcasecmp(mine, "DateStyle") == 0
+ || pg_strcasecmp(mine, "search_path") == 0)
+ appendPQExpBufferStr(buf, pos);
+ else
+ appendStringLiteralConn(buf, pos, conn);
+
+ appendPQExpBufferStr(buf, ";\n");
+
+ pg_free(mine);
+ }
diff --git a/src/bin/pg_dump/dumputils.h b/src/bin/pg_dump/dumputils.h
index d5f150d..23a0645 100644
*** a/src/bin/pg_dump/dumputils.h
--- b/src/bin/pg_dump/dumputils.h
*************** extern void buildACLQueries(PQExpBuffer
*** 56,59 ****
--- 56,64 ----
const char *acl_column, const char *acl_owner,
const char *obj_kind, bool binary_upgrade);
+ extern void makeAlterConfigCommand(PGconn *conn, const char *configitem,
+ const char *type, const char *name,
+ const char *type2, const char *name2,
+ PQExpBuffer buf);
+
#endif /* DUMPUTILS_H */
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 41741ae..9dabbf6 100644
*** a/src/bin/pg_dump/pg_backup_archiver.c
--- b/src/bin/pg_dump/pg_backup_archiver.c
*************** RestoreArchive(Archive *AHX)
*** 489,504 ****
* whole. Issuing drops against anything else would be wrong,
* because at this point we're connected to the wrong database.
* Conversely, if we're not in createDB mode, we'd better not
! * issue a DROP against the database at all.
*/
if (ropt->createDB)
{
! if (strcmp(te->desc, "DATABASE") != 0)
continue;
}
else
{
! if (strcmp(te->desc, "DATABASE") == 0)
continue;
}
--- 489,507 ----
* whole. Issuing drops against anything else would be wrong,
* because at this point we're connected to the wrong database.
* Conversely, if we're not in createDB mode, we'd better not
! * issue a DROP against the database at all. (The DATABASE
! * PROPERTIES entry, if any, works like the DATABASE entry.)
*/
if (ropt->createDB)
{
! if (strcmp(te->desc, "DATABASE") != 0 &&
! strcmp(te->desc, "DATABASE PROPERTIES") != 0)
continue;
}
else
{
! if (strcmp(te->desc, "DATABASE") == 0 ||
! strcmp(te->desc, "DATABASE PROPERTIES") == 0)
continue;
}
*************** RestoreArchive(Archive *AHX)
*** 558,563 ****
--- 561,568 ----
* we simply emit the original command for DEFAULT
* objects (modulo the adjustment made above).
*
+ * Likewise, don't mess with DATABASE PROPERTIES.
+ *
* If we used CREATE OR REPLACE VIEW as a means of
* quasi-dropping an ON SELECT rule, that should
* be emitted unchanged as well.
*************** RestoreArchive(Archive *AHX)
*** 570,575 ****
--- 575,581 ----
* search for hardcoded "DROP CONSTRAINT" instead.
*/
if (strcmp(te->desc, "DEFAULT") == 0 ||
+ strcmp(te->desc, "DATABASE PROPERTIES") == 0 ||
strncmp(dropStmt, "CREATE OR REPLACE VIEW", 22) == 0)
appendPQExpBufferStr(ftStmt, dropStmt);
else
*************** restore_toc_entry(ArchiveHandle *AH, Toc
*** 750,760 ****
reqs = te->reqs;
/*
! * Ignore DATABASE entry unless we should create it. We must check this
! * here, not in _tocEntryRequired, because the createDB option should not
! * affect emitting a DATABASE entry to an archive file.
*/
! if (!ropt->createDB && strcmp(te->desc, "DATABASE") == 0)
reqs = 0;
/* Dump any relevant dump warnings to stderr */
--- 756,774 ----
reqs = te->reqs;
/*
! * Ignore DATABASE and related entries unless createDB is specified. We
! * must check this here, not in _tocEntryRequired, because !createDB
! * should not prevent emitting these entries to an archive file.
*/
! if (!ropt->createDB &&
! (strcmp(te->desc, "DATABASE") == 0 ||
! strcmp(te->desc, "DATABASE PROPERTIES") == 0 ||
! (strcmp(te->desc, "ACL") == 0 &&
! strncmp(te->tag, "DATABASE ", 9) == 0) ||
! (strcmp(te->desc, "COMMENT") == 0 &&
! strncmp(te->tag, "DATABASE ", 9) == 0) ||
! (strcmp(te->desc, "SECURITY LABEL") == 0 &&
! strncmp(te->tag, "DATABASE ", 9) == 0)))
reqs = 0;
/* Dump any relevant dump warnings to stderr */
*************** _tocEntryRequired(TocEntry *te, teSectio
*** 2917,2924 ****
* Special Case: If 'SEQUENCE SET' or anything to do with BLOBs, then
* it is considered a data entry. We don't need to check for the
* BLOBS entry or old-style BLOB COMMENTS, because they will have
! * hadDumper = true ... but we do need to check new-style BLOB
! * comments.
*/
if (strcmp(te->desc, "SEQUENCE SET") == 0 ||
strcmp(te->desc, "BLOB") == 0 ||
--- 2931,2938 ----
* Special Case: If 'SEQUENCE SET' or anything to do with BLOBs, then
* it is considered a data entry. We don't need to check for the
* BLOBS entry or old-style BLOB COMMENTS, because they will have
! * hadDumper = true ... but we do need to check new-style BLOB ACLs,
! * comments, etc.
*/
if (strcmp(te->desc, "SEQUENCE SET") == 0 ||
strcmp(te->desc, "BLOB") == 0 ||
*************** _printTocEntry(ArchiveHandle *AH, TocEnt
*** 3590,3595 ****
--- 3604,3610 ----
else if (strcmp(te->desc, "CAST") == 0 ||
strcmp(te->desc, "CHECK CONSTRAINT") == 0 ||
strcmp(te->desc, "CONSTRAINT") == 0 ||
+ strcmp(te->desc, "DATABASE PROPERTIES") == 0 ||
strcmp(te->desc, "DEFAULT") == 0 ||
strcmp(te->desc, "FK CONSTRAINT") == 0 ||
strcmp(te->desc, "INDEX") == 0 ||
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index db65552..6473e58 100644
*** a/src/bin/pg_dump/pg_dump.c
--- b/src/bin/pg_dump/pg_dump.c
*************** static void dumpPublication(Archive *fou
*** 252,257 ****
--- 252,259 ----
static void dumpPublicationTable(Archive *fout, PublicationRelInfo *pubrinfo);
static void dumpSubscription(Archive *fout, SubscriptionInfo *subinfo);
static void dumpDatabase(Archive *AH);
+ static void dumpDatabaseConfig(Archive *AH, PQExpBuffer outbuf,
+ const char *dbname, Oid dboid);
static void dumpEncoding(Archive *AH);
static void dumpStdStrings(Archive *AH);
static void binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
*************** main(int argc, char **argv)
*** 838,844 ****
dumpEncoding(fout);
dumpStdStrings(fout);
! /* The database item is always next, unless we don't want it at all */
if (dopt.include_everything && !dopt.dataOnly)
dumpDatabase(fout);
--- 840,846 ----
dumpEncoding(fout);
dumpStdStrings(fout);
! /* The database items are always next, unless we don't want them at all */
if (dopt.include_everything && !dopt.dataOnly)
dumpDatabase(fout);
*************** dumpDatabase(Archive *fout)
*** 2540,2545 ****
--- 2542,2551 ----
i_ctype,
i_frozenxid,
i_minmxid,
+ i_datacl,
+ i_rdatacl,
+ i_datistemplate,
+ i_datconnlimit,
i_tablespace;
CatalogId dbCatId;
DumpId dbDumpId;
*************** dumpDatabase(Archive *fout)
*** 2548,2558 ****
--- 2554,2570 ----
*encoding,
*collate,
*ctype,
+ *datacl,
+ *rdatacl,
+ *datistemplate,
+ *datconnlimit,
*tablespace;
uint32 frozenxid,
minmxid;
+ char *qdatname;
datname = PQdb(conn);
+ qdatname = pg_strdup(fmtId(datname));
if (g_verbose)
write_msg(NULL, "saving database definition\n");
*************** dumpDatabase(Archive *fout)
*** 2560,2572 ****
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
! /* Get the database owner and parameters from pg_database */
! if (fout->remoteVersion >= 90300)
{
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
"datcollate, datctype, datfrozenxid, datminmxid, "
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
--- 2572,2608 ----
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
! /* Fetch the database-level properties for this database */
! if (fout->remoteVersion >= 90600)
! {
! appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
! "(%s datdba) AS dba, "
! "pg_encoding_to_char(encoding) AS encoding, "
! "datcollate, datctype, datfrozenxid, datminmxid, "
! "(SELECT array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
! " SELECT unnest(coalesce(datacl,acldefault('d',datdba))) AS acl "
! " EXCEPT SELECT unnest(acldefault('d',datdba))) as datacls)"
! " AS datacl, "
! "(SELECT array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
! " SELECT unnest(acldefault('d',datdba)) AS acl "
! " EXCEPT SELECT unnest(coalesce(datacl,acldefault('d',datdba)))) as rdatacls)"
! " AS rdatacl, "
! "datistemplate, datconnlimit, "
! "(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
! "shobj_description(oid, 'pg_database') AS description "
!
! "FROM pg_database "
! "WHERE datname = ",
! username_subquery);
! appendStringLiteralAH(dbQry, datname, fout);
! }
! else if (fout->remoteVersion >= 90300)
{
appendPQExpBuffer(dbQry, "SELECT tableoid, oid, "
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
"datcollate, datctype, datfrozenxid, datminmxid, "
+ "datacl, '' as rdatacl, datistemplate, datconnlimit, "
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
*************** dumpDatabase(Archive *fout)
*** 2581,2586 ****
--- 2617,2623 ----
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
"datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
+ "datacl, '' as rdatacl, datistemplate, datconnlimit, "
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
*************** dumpDatabase(Archive *fout)
*** 2595,2600 ****
--- 2632,2638 ----
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
"NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "datacl, '' as rdatacl, datistemplate, datconnlimit, "
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace, "
"shobj_description(oid, 'pg_database') AS description "
*************** dumpDatabase(Archive *fout)
*** 2609,2614 ****
--- 2647,2654 ----
"(%s datdba) AS dba, "
"pg_encoding_to_char(encoding) AS encoding, "
"NULL AS datcollate, NULL AS datctype, datfrozenxid, 0 AS datminmxid, "
+ "datacl, '' as rdatacl, datistemplate, "
+ "-1 as datconnlimit, "
"(SELECT spcname FROM pg_tablespace t WHERE t.oid = dattablespace) AS tablespace "
"FROM pg_database "
"WHERE datname = ",
*************** dumpDatabase(Archive *fout)
*** 2626,2631 ****
--- 2666,2675 ----
i_ctype = PQfnumber(res, "datctype");
i_frozenxid = PQfnumber(res, "datfrozenxid");
i_minmxid = PQfnumber(res, "datminmxid");
+ i_datacl = PQfnumber(res, "datacl");
+ i_rdatacl = PQfnumber(res, "rdatacl");
+ i_datistemplate = PQfnumber(res, "datistemplate");
+ i_datconnlimit = PQfnumber(res, "datconnlimit");
i_tablespace = PQfnumber(res, "tablespace");
dbCatId.tableoid = atooid(PQgetvalue(res, 0, i_tableoid));
*************** dumpDatabase(Archive *fout)
*** 2636,2645 ****
ctype = PQgetvalue(res, 0, i_ctype);
frozenxid = atooid(PQgetvalue(res, 0, i_frozenxid));
minmxid = atooid(PQgetvalue(res, 0, i_minmxid));
tablespace = PQgetvalue(res, 0, i_tablespace);
appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
! fmtId(datname));
if (strlen(encoding) > 0)
{
appendPQExpBufferStr(creaQry, " ENCODING = ");
--- 2680,2699 ----
ctype = PQgetvalue(res, 0, i_ctype);
frozenxid = atooid(PQgetvalue(res, 0, i_frozenxid));
minmxid = atooid(PQgetvalue(res, 0, i_minmxid));
+ datacl = PQgetvalue(res, 0, i_datacl);
+ rdatacl = PQgetvalue(res, 0, i_rdatacl);
+ datistemplate = PQgetvalue(res, 0, i_datistemplate);
+ datconnlimit = PQgetvalue(res, 0, i_datconnlimit);
tablespace = PQgetvalue(res, 0, i_tablespace);
+ /*
+ * Prepare the CREATE DATABASE command. We must specify encoding, locale,
+ * and tablespace since those can't be altered later. Other DB properties
+ * are left to the DATABASE PROPERTIES entry, so that they can be applied
+ * after reconnecting to the target DB.
+ */
appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0",
! qdatname);
if (strlen(encoding) > 0)
{
appendPQExpBufferStr(creaQry, " ENCODING = ");
*************** dumpDatabase(Archive *fout)
*** 2655,2680 ****
appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
appendStringLiteralAH(creaQry, ctype, fout);
}
if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
!dopt->outputNoTablespaces)
appendPQExpBuffer(creaQry, " TABLESPACE = %s",
fmtId(tablespace));
appendPQExpBufferStr(creaQry, ";\n");
- if (dopt->binary_upgrade)
- {
- appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
- "SET datfrozenxid = '%u', datminmxid = '%u'\n"
- "WHERE datname = ",
- frozenxid, minmxid);
- appendStringLiteralAH(creaQry, datname, fout);
- appendPQExpBufferStr(creaQry, ";\n");
-
- }
-
appendPQExpBuffer(delQry, "DROP DATABASE %s;\n",
! fmtId(datname));
dbDumpId = createDumpId();
--- 2709,2731 ----
appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
appendStringLiteralAH(creaQry, ctype, fout);
}
+
+ /*
+ * Note: looking at dopt->outputNoTablespaces here is completely the wrong
+ * thing; the decision whether to specify a tablespace should be left till
+ * pg_restore, so that pg_restore --no-tablespaces applies. Ideally we'd
+ * label the DATABASE entry with the tablespace and let the normal
+ * tablespace selection logic work ... but CREATE DATABASE doesn't pay
+ * attention to default_tablespace, so that won't work.
+ */
if (strlen(tablespace) > 0 && strcmp(tablespace, "pg_default") != 0 &&
!dopt->outputNoTablespaces)
appendPQExpBuffer(creaQry, " TABLESPACE = %s",
fmtId(tablespace));
appendPQExpBufferStr(creaQry, ";\n");
appendPQExpBuffer(delQry, "DROP DATABASE %s;\n",
! qdatname);
dbDumpId = createDumpId();
*************** dumpDatabase(Archive *fout)
*** 2697,2703 ****
NULL); /* Dumper Arg */
/* Compute correct tag for comments etc */
! appendPQExpBuffer(labelq, "DATABASE %s", fmtId(datname));
/* Dump DB comment if any */
if (fout->remoteVersion >= 80200)
--- 2748,2754 ----
NULL); /* Dumper Arg */
/* Compute correct tag for comments etc */
! appendPQExpBuffer(labelq, "DATABASE %s", qdatname);
/* Dump DB comment if any */
if (fout->remoteVersion >= 80200)
*************** dumpDatabase(Archive *fout)
*** 2717,2723 ****
* Generates warning when loaded into a differently-named
* database.
*/
! appendPQExpBuffer(dbQry, "COMMENT ON DATABASE %s IS ", fmtId(datname));
appendStringLiteralAH(dbQry, comment, fout);
appendPQExpBufferStr(dbQry, ";\n");
--- 2768,2774 ----
* Generates warning when loaded into a differently-named
* database.
*/
! appendPQExpBuffer(dbQry, "COMMENT ON DATABASE %s IS ", qdatname);
appendStringLiteralAH(dbQry, comment, fout);
appendPQExpBufferStr(dbQry, ";\n");
*************** dumpDatabase(Archive *fout)
*** 2759,2764 ****
--- 2810,2882 ----
}
/*
+ * Dump ACL if any. Note that we do not support initial privileges
+ * (pg_init_privs) on databases.
+ */
+ dumpACL(fout, dbCatId, dbDumpId, "DATABASE",
+ qdatname, NULL, labelq->data, NULL,
+ dba, datacl, rdatacl, "", "");
+
+ /*
+ * Now construct a DATABASE PROPERTIES archive entry to restore any
+ * non-default database-level properties. We want to do this after
+ * reconnecting so that these properties won't apply during the restore
+ * session. In this way, restoring works even if there is, say, an ALTER
+ * DATABASE SET that turns on default_transaction_read_only.
+ */
+ resetPQExpBuffer(creaQry);
+ resetPQExpBuffer(delQry);
+
+ if (strlen(datconnlimit) > 0 && strcmp(datconnlimit, "-1") != 0)
+ appendPQExpBuffer(creaQry, "ALTER DATABASE %s CONNECTION LIMIT = %s;\n",
+ qdatname, datconnlimit);
+
+ if (strcmp(datistemplate, "t") == 0)
+ {
+ appendPQExpBuffer(creaQry, "ALTER DATABASE %s IS_TEMPLATE = true;\n",
+ qdatname);
+
+ /*
+ * The backend won't accept DROP DATABASE on a template database. We
+ * can deal with that by removing the template marking before the DROP
+ * gets issued. We'd prefer to use ALTER DATABASE IF EXISTS here, but
+ * since no such command is currently supported, fake it with a direct
+ * UPDATE on pg_database.
+ */
+ appendPQExpBufferStr(delQry, "UPDATE pg_catalog.pg_database "
+ "SET datistemplate = false WHERE datname = ");
+ appendStringLiteralAH(delQry, datname, fout);
+ appendPQExpBufferStr(delQry, ";\n");
+ }
+
+ /* Add database-specific SET options */
+ dumpDatabaseConfig(fout, creaQry, datname, dbCatId.oid);
+
+ /*
+ * We stick this binary-upgrade query into the DATABASE PROPERTIES archive
+ * entry, too. It can't go into the DATABASE entry because that would
+ * result in an implicit transaction block around the CREATE DATABASE.
+ */
+ if (dopt->binary_upgrade)
+ {
+ appendPQExpBufferStr(creaQry, "\n-- For binary upgrade, set datfrozenxid and datminmxid.\n");
+ appendPQExpBuffer(creaQry, "UPDATE pg_catalog.pg_database\n"
+ "SET datfrozenxid = '%u', datminmxid = '%u'\n"
+ "WHERE datname = ",
+ frozenxid, minmxid);
+ appendStringLiteralAH(creaQry, datname, fout);
+ appendPQExpBufferStr(creaQry, ";\n");
+ }
+
+ if (creaQry->len > 0)
+ ArchiveEntry(fout, nilCatalogId, createDumpId(),
+ datname, NULL, NULL, dba,
+ false, "DATABASE PROPERTIES", SECTION_PRE_DATA,
+ creaQry->data, delQry->data, NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ /*
* pg_largeobject and pg_largeobject_metadata come from the old system
* intact, so set their relfrozenxids and relminmxids.
*/
*************** dumpDatabase(Archive *fout)
*** 2793,2800 ****
appendPQExpBuffer(loOutQry, "UPDATE pg_catalog.pg_class\n"
"SET relfrozenxid = '%u', relminmxid = '%u'\n"
"WHERE oid = %u;\n",
! atoi(PQgetvalue(lo_res, 0, i_relfrozenxid)),
! atoi(PQgetvalue(lo_res, 0, i_relminmxid)),
LargeObjectRelationId);
ArchiveEntry(fout, nilCatalogId, createDumpId(),
"pg_largeobject", NULL, NULL, "",
--- 2911,2918 ----
appendPQExpBuffer(loOutQry, "UPDATE pg_catalog.pg_class\n"
"SET relfrozenxid = '%u', relminmxid = '%u'\n"
"WHERE oid = %u;\n",
! atooid(PQgetvalue(lo_res, 0, i_relfrozenxid)),
! atooid(PQgetvalue(lo_res, 0, i_relminmxid)),
LargeObjectRelationId);
ArchiveEntry(fout, nilCatalogId, createDumpId(),
"pg_largeobject", NULL, NULL, "",
*************** dumpDatabase(Archive *fout)
*** 2833,2840 ****
appendPQExpBuffer(loOutQry, "UPDATE pg_catalog.pg_class\n"
"SET relfrozenxid = '%u', relminmxid = '%u'\n"
"WHERE oid = %u;\n",
! atoi(PQgetvalue(lo_res, 0, i_relfrozenxid)),
! atoi(PQgetvalue(lo_res, 0, i_relminmxid)),
LargeObjectMetadataRelationId);
ArchiveEntry(fout, nilCatalogId, createDumpId(),
"pg_largeobject_metadata", NULL, NULL, "",
--- 2951,2958 ----
appendPQExpBuffer(loOutQry, "UPDATE pg_catalog.pg_class\n"
"SET relfrozenxid = '%u', relminmxid = '%u'\n"
"WHERE oid = %u;\n",
! atooid(PQgetvalue(lo_res, 0, i_relfrozenxid)),
! atooid(PQgetvalue(lo_res, 0, i_relminmxid)),
LargeObjectMetadataRelationId);
ArchiveEntry(fout, nilCatalogId, createDumpId(),
"pg_largeobject_metadata", NULL, NULL, "",
*************** dumpDatabase(Archive *fout)
*** 2852,2857 ****
--- 2970,2976 ----
PQclear(res);
+ free(qdatname);
destroyPQExpBuffer(dbQry);
destroyPQExpBuffer(delQry);
destroyPQExpBuffer(creaQry);
*************** dumpDatabase(Archive *fout)
*** 2859,2864 ****
--- 2978,3055 ----
}
/*
+ * Collect any database-specific or role-and-database-specific SET options
+ * for this database, and append them to outbuf.
+ */
+ static void
+ dumpDatabaseConfig(Archive *AH, PQExpBuffer outbuf,
+ const char *dbname, Oid dboid)
+ {
+ PGconn *conn = GetConnection(AH);
+ PQExpBuffer buf = createPQExpBuffer();
+ PGresult *res;
+ int count = 1;
+
+ /*
+ * First collect database-specific options. Old server versions lack
+ * unnest(), so we do that the hard way by querying once per subscript.
+ */
+ for (;;)
+ {
+ if (AH->remoteVersion >= 90000)
+ printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting "
+ "WHERE setrole = 0 AND setdatabase = '%u'::oid",
+ count, dboid);
+ else
+ printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE oid = '%u'::oid", count, dboid);
+
+ res = ExecuteSqlQuery(AH, buf->data, PGRES_TUPLES_OK);
+
+ if (PQntuples(res) == 1 &&
+ !PQgetisnull(res, 0, 0))
+ {
+ makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
+ "DATABASE", dbname, NULL, NULL,
+ outbuf);
+ PQclear(res);
+ count++;
+ }
+ else
+ {
+ PQclear(res);
+ break;
+ }
+ }
+
+ /* Now look for role-and-database-specific options */
+ if (AH->remoteVersion >= 90000)
+ {
+ /* Here we can assume we have unnest() */
+ printfPQExpBuffer(buf, "SELECT rolname, unnest(setconfig) "
+ "FROM pg_db_role_setting s, pg_roles r "
+ "WHERE setrole = r.oid AND setdatabase = '%u'::oid",
+ dboid);
+
+ res = ExecuteSqlQuery(AH, buf->data, PGRES_TUPLES_OK);
+
+ if (PQntuples(res) > 0)
+ {
+ int i;
+
+ for (i = 0; i < PQntuples(res); i++)
+ makeAlterConfigCommand(conn, PQgetvalue(res, i, 1),
+ "ROLE", PQgetvalue(res, i, 0),
+ "DATABASE", dbname,
+ outbuf);
+ }
+
+ PQclear(res);
+ }
+
+ destroyPQExpBuffer(buf);
+ }
+
+ /*
* dumpEncoding: put the correct encoding into the archive
*/
static void
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 3dd2c38..2fd5a02 100644
*** a/src/bin/pg_dump/pg_dumpall.c
--- b/src/bin/pg_dump/pg_dumpall.c
*************** static void dumpGroups(PGconn *conn);
*** 38,54 ****
static void dropTablespaces(PGconn *conn);
static void dumpTablespaces(PGconn *conn);
static void dropDBs(PGconn *conn);
- static void dumpCreateDB(PGconn *conn);
- static void dumpDatabaseConfig(PGconn *conn, const char *dbname);
static void dumpUserConfig(PGconn *conn, const char *username);
- static void dumpDbRoleConfig(PGconn *conn);
- static void makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name, const char *type2,
- const char *name2);
static void dumpDatabases(PGconn *conn);
static void dumpTimestamp(const char *msg);
!
! static int runPgDump(const char *dbname);
static void buildShSecLabels(PGconn *conn, const char *catalog_name,
uint32 objectId, PQExpBuffer buffer,
const char *target, const char *objname);
--- 38,47 ----
static void dropTablespaces(PGconn *conn);
static void dumpTablespaces(PGconn *conn);
static void dropDBs(PGconn *conn);
static void dumpUserConfig(PGconn *conn, const char *username);
static void dumpDatabases(PGconn *conn);
static void dumpTimestamp(const char *msg);
! static int runPgDump(const char *dbname, const char *create_opts);
static void buildShSecLabels(PGconn *conn, const char *catalog_name,
uint32 objectId, PQExpBuffer buffer,
const char *target, const char *objname);
*************** static char pg_dump_bin[MAXPGPATH];
*** 62,67 ****
--- 55,61 ----
static const char *progname;
static PQExpBuffer pgdumpopts;
static char *connstr = "";
+ static bool output_clean = false;
static bool skip_acls = false;
static bool verbose = false;
static bool dosync = true;
*************** main(int argc, char *argv[])
*** 152,158 ****
trivalue prompt_password = TRI_DEFAULT;
bool data_only = false;
bool globals_only = false;
- bool output_clean = false;
bool roles_only = false;
bool tablespaces_only = false;
PGconn *conn;
--- 146,151 ----
*************** main(int argc, char *argv[])
*** 558,574 ****
/* Dump tablespaces */
if (!roles_only && !no_tablespaces)
dumpTablespaces(conn);
-
- /* Dump CREATE DATABASE commands */
- if (binary_upgrade || (!globals_only && !roles_only && !tablespaces_only))
- dumpCreateDB(conn);
-
- /* Dump role/database settings */
- if (!tablespaces_only && !roles_only)
- {
- if (server_version >= 90000)
- dumpDbRoleConfig(conn);
- }
}
if (!globals_only && !roles_only && !tablespaces_only)
--- 551,556 ----
*************** dumpTablespaces(PGconn *conn)
*** 1262,1269 ****
/*
* Dump commands to drop each database.
- *
- * This should match the set of databases targeted by dumpCreateDB().
*/
static void
dropDBs(PGconn *conn)
--- 1244,1249 ----
*************** dropDBs(PGconn *conn)
*** 1271,1294 ****
PGresult *res;
int i;
res = executeQuery(conn,
"SELECT datname "
"FROM pg_database d "
! "WHERE datallowconn ORDER BY 1");
if (PQntuples(res) > 0)
! fprintf(OPF, "--\n-- Drop databases\n--\n\n");
for (i = 0; i < PQntuples(res); i++)
{
char *dbname = PQgetvalue(res, i, 0);
/*
! * Skip "template1" and "postgres"; the restore script is almost
! * certainly going to be run in one or the other, and we don't know
! * which. This must agree with dumpCreateDB's choices!
*/
if (strcmp(dbname, "template1") != 0 &&
strcmp(dbname, "postgres") != 0)
{
fprintf(OPF, "DROP DATABASE %s%s;\n",
--- 1251,1280 ----
PGresult *res;
int i;
+ /*
+ * Skip databases marked not datallowconn, since we'd be unable to connect
+ * to them anyway. This must agree with dumpDatabases().
+ */
res = executeQuery(conn,
"SELECT datname "
"FROM pg_database d "
! "WHERE datallowconn "
! "ORDER BY datname");
if (PQntuples(res) > 0)
! fprintf(OPF, "--\n-- Drop databases (except postgres and template1)\n--\n\n");
for (i = 0; i < PQntuples(res); i++)
{
char *dbname = PQgetvalue(res, i, 0);
/*
! * Skip "postgres" and "template1"; dumpDatabases() will deal with
! * them specially. Also, be sure to skip "template0", even if for
! * some reason it's not marked !datallowconn.
*/
if (strcmp(dbname, "template1") != 0 &&
+ strcmp(dbname, "template0") != 0 &&
strcmp(dbname, "postgres") != 0)
{
fprintf(OPF, "DROP DATABASE %s%s;\n",
*************** dropDBs(PGconn *conn)
*** 1302,1624 ****
fprintf(OPF, "\n\n");
}
- /*
- * Dump commands to create each database.
- *
- * To minimize the number of reconnections (and possibly ensuing
- * password prompts) required by the output script, we emit all CREATE
- * DATABASE commands during the initial phase of the script, and then
- * run pg_dump for each database to dump the contents of that
- * database. We skip databases marked not datallowconn, since we'd be
- * unable to connect to them anyway (and besides, we don't want to
- * dump template0).
- */
- static void
- dumpCreateDB(PGconn *conn)
- {
- PQExpBuffer buf = createPQExpBuffer();
- char *default_encoding = NULL;
- char *default_collate = NULL;
- char *default_ctype = NULL;
- PGresult *res;
- int i;
-
- fprintf(OPF, "--\n-- Database creation\n--\n\n");
-
- /*
- * First, get the installation's default encoding and locale information.
- * We will dump encoding and locale specifications in the CREATE DATABASE
- * commands for just those databases with values different from defaults.
- *
- * We consider template0's encoding and locale to define the installation
- * default. Pre-8.4 installations do not have per-database locale
- * settings; for them, every database must necessarily be using the
- * installation default, so there's no need to do anything.
- */
- if (server_version >= 80400)
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "datcollate, datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
- else
- res = executeQuery(conn,
- "SELECT pg_encoding_to_char(encoding), "
- "null::text AS datcollate, null::text AS datctype "
- "FROM pg_database "
- "WHERE datname = 'template0'");
-
- /* If for some reason the template DB isn't there, treat as unknown */
- if (PQntuples(res) > 0)
- {
- if (!PQgetisnull(res, 0, 0))
- default_encoding = pg_strdup(PQgetvalue(res, 0, 0));
- if (!PQgetisnull(res, 0, 1))
- default_collate = pg_strdup(PQgetvalue(res, 0, 1));
- if (!PQgetisnull(res, 0, 2))
- default_ctype = pg_strdup(PQgetvalue(res, 0, 2));
- }
-
- PQclear(res);
-
-
- /*
- * Now collect all the information about databases to dump.
- *
- * For the database ACLs, as of 9.6, we extract both the positive (as
- * datacl) and negative (as rdatacl) ACLs, relative to the default ACL for
- * databases, which are then passed to buildACLCommands() below.
- *
- * See buildACLQueries() and buildACLCommands().
- *
- * Note that we do not support initial privileges (pg_init_privs) on
- * databases.
- */
- if (server_version >= 90600)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba))) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba))) as datacls)"
- "AS datacl, "
- "(SELECT pg_catalog.array_agg(acl ORDER BY acl::text COLLATE \"C\") FROM ( "
- " SELECT pg_catalog.unnest(pg_catalog.acldefault('d',datdba)) AS acl "
- " EXCEPT SELECT pg_catalog.unnest(coalesce(datacl,pg_catalog.acldefault('d',datdba)))) as rdatacls)"
- "AS rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 90300)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80400)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "datcollate, datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else if (server_version >= 80100)
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(rolname, (select rolname from %s where oid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN %s u ON (datdba = u.oid) "
- "WHERE datallowconn ORDER BY 1", role_catalog, role_catalog);
- else
- printfPQExpBuffer(buf,
- "SELECT datname, "
- "coalesce(usename, (select usename from pg_shadow where usesysid=(select datdba from pg_database where datname='template0'))), "
- "pg_encoding_to_char(d.encoding), "
- "null::text AS datcollate, null::text AS datctype, datfrozenxid, 0 AS datminmxid, "
- "datistemplate, datacl, '' as rdatacl, "
- "-1 as datconnlimit, "
- "(SELECT spcname FROM pg_tablespace t WHERE t.oid = d.dattablespace) AS dattablespace "
- "FROM pg_database d LEFT JOIN pg_shadow u ON (datdba = usesysid) "
- "WHERE datallowconn ORDER BY 1");
-
- res = executeQuery(conn, buf->data);
-
- for (i = 0; i < PQntuples(res); i++)
- {
- char *dbname = PQgetvalue(res, i, 0);
- char *dbowner = PQgetvalue(res, i, 1);
- char *dbencoding = PQgetvalue(res, i, 2);
- char *dbcollate = PQgetvalue(res, i, 3);
- char *dbctype = PQgetvalue(res, i, 4);
- uint32 dbfrozenxid = atooid(PQgetvalue(res, i, 5));
- uint32 dbminmxid = atooid(PQgetvalue(res, i, 6));
- char *dbistemplate = PQgetvalue(res, i, 7);
- char *dbacl = PQgetvalue(res, i, 8);
- char *rdbacl = PQgetvalue(res, i, 9);
- char *dbconnlimit = PQgetvalue(res, i, 10);
- char *dbtablespace = PQgetvalue(res, i, 11);
- char *fdbname;
-
- fdbname = pg_strdup(fmtId(dbname));
-
- resetPQExpBuffer(buf);
-
- /*
- * Skip the CREATE DATABASE commands for "template1" and "postgres",
- * since they are presumably already there in the destination cluster.
- * We do want to emit their ACLs and config options if any, however.
- */
- if (strcmp(dbname, "template1") != 0 &&
- strcmp(dbname, "postgres") != 0)
- {
- appendPQExpBuffer(buf, "CREATE DATABASE %s", fdbname);
-
- appendPQExpBufferStr(buf, " WITH TEMPLATE = template0");
-
- if (strlen(dbowner) != 0)
- appendPQExpBuffer(buf, " OWNER = %s", fmtId(dbowner));
-
- if (default_encoding && strcmp(dbencoding, default_encoding) != 0)
- {
- appendPQExpBufferStr(buf, " ENCODING = ");
- appendStringLiteralConn(buf, dbencoding, conn);
- }
-
- if (default_collate && strcmp(dbcollate, default_collate) != 0)
- {
- appendPQExpBufferStr(buf, " LC_COLLATE = ");
- appendStringLiteralConn(buf, dbcollate, conn);
- }
-
- if (default_ctype && strcmp(dbctype, default_ctype) != 0)
- {
- appendPQExpBufferStr(buf, " LC_CTYPE = ");
- appendStringLiteralConn(buf, dbctype, conn);
- }
-
- /*
- * Output tablespace if it isn't the default. For default, it
- * uses the default from the template database. If tablespace is
- * specified and tablespace creation failed earlier, (e.g. no such
- * directory), the database creation will fail too. One solution
- * would be to use 'SET default_tablespace' like we do in pg_dump
- * for setting non-default database locations.
- */
- if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- appendPQExpBuffer(buf, " TABLESPACE = %s",
- fmtId(dbtablespace));
-
- if (strcmp(dbistemplate, "t") == 0)
- appendPQExpBuffer(buf, " IS_TEMPLATE = true");
-
- if (strcmp(dbconnlimit, "-1") != 0)
- appendPQExpBuffer(buf, " CONNECTION LIMIT = %s",
- dbconnlimit);
-
- appendPQExpBufferStr(buf, ";\n");
- }
- else if (strcmp(dbtablespace, "pg_default") != 0 && !no_tablespaces)
- {
- /*
- * Cannot change tablespace of the database we're connected to, so
- * to move "postgres" to another tablespace, we connect to
- * "template1", and vice versa.
- */
- if (strcmp(dbname, "postgres") == 0)
- appendPQExpBuffer(buf, "\\connect template1\n");
- else
- appendPQExpBuffer(buf, "\\connect postgres\n");
-
- appendPQExpBuffer(buf, "ALTER DATABASE %s SET TABLESPACE %s;\n",
- fdbname, fmtId(dbtablespace));
-
- /* connect to original database */
- appendPsqlMetaConnect(buf, dbname);
- }
-
- if (binary_upgrade)
- {
- appendPQExpBufferStr(buf, "-- For binary upgrade, set datfrozenxid and datminmxid.\n");
- appendPQExpBuffer(buf, "UPDATE pg_catalog.pg_database "
- "SET datfrozenxid = '%u', datminmxid = '%u' "
- "WHERE datname = ",
- dbfrozenxid, dbminmxid);
- appendStringLiteralConn(buf, dbname, conn);
- appendPQExpBufferStr(buf, ";\n");
- }
-
- if (!skip_acls &&
- !buildACLCommands(fdbname, NULL, "DATABASE",
- dbacl, rdbacl, dbowner,
- "", server_version, buf))
- {
- fprintf(stderr, _("%s: could not parse ACL list (%s) for database \"%s\"\n"),
- progname, dbacl, fdbname);
- PQfinish(conn);
- exit_nicely(1);
- }
-
- fprintf(OPF, "%s", buf->data);
-
- dumpDatabaseConfig(conn, dbname);
-
- free(fdbname);
- }
-
- if (default_encoding)
- free(default_encoding);
- if (default_collate)
- free(default_collate);
- if (default_ctype)
- free(default_ctype);
-
- PQclear(res);
- destroyPQExpBuffer(buf);
-
- fprintf(OPF, "\n\n");
- }
-
-
- /*
- * Dump database-specific configuration
- */
- static void
- dumpDatabaseConfig(PGconn *conn, const char *dbname)
- {
- PQExpBuffer buf = createPQExpBuffer();
- int count = 1;
-
- for (;;)
- {
- PGresult *res;
-
- if (server_version >= 90000)
- printfPQExpBuffer(buf, "SELECT setconfig[%d] FROM pg_db_role_setting WHERE "
- "setrole = 0 AND setdatabase = (SELECT oid FROM pg_database WHERE datname = ", count);
- else
- printfPQExpBuffer(buf, "SELECT datconfig[%d] FROM pg_database WHERE datname = ", count);
- appendStringLiteralConn(buf, dbname, conn);
-
- if (server_version >= 90000)
- appendPQExpBufferChar(buf, ')');
-
- res = executeQuery(conn, buf->data);
- if (PQntuples(res) == 1 &&
- !PQgetisnull(res, 0, 0))
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
- "DATABASE", dbname, NULL, NULL);
- PQclear(res);
- count++;
- }
- else
- {
- PQclear(res);
- break;
- }
- }
-
- destroyPQExpBuffer(buf);
- }
-
-
/*
* Dump user-specific configuration
--- 1288,1293 ----
*************** dumpUserConfig(PGconn *conn, const char
*** 1649,1656 ****
if (PQntuples(res) == 1 &&
!PQgetisnull(res, 0, 0))
{
makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
! "ROLE", username, NULL, NULL);
PQclear(res);
count++;
}
--- 1318,1328 ----
if (PQntuples(res) == 1 &&
!PQgetisnull(res, 0, 0))
{
+ resetPQExpBuffer(buf);
makeAlterConfigCommand(conn, PQgetvalue(res, 0, 0),
! "ROLE", username, NULL, NULL,
! buf);
! fprintf(OPF, "%s", buf->data);
PQclear(res);
count++;
}
*************** dumpUserConfig(PGconn *conn, const char
*** 1666,1750 ****
/*
- * Dump user-and-database-specific configuration
- */
- static void
- dumpDbRoleConfig(PGconn *conn)
- {
- PQExpBuffer buf = createPQExpBuffer();
- PGresult *res;
- int i;
-
- printfPQExpBuffer(buf, "SELECT rolname, datname, unnest(setconfig) "
- "FROM pg_db_role_setting, %s u, pg_database "
- "WHERE setrole = u.oid AND setdatabase = pg_database.oid", role_catalog);
- res = executeQuery(conn, buf->data);
-
- if (PQntuples(res) > 0)
- {
- fprintf(OPF, "--\n-- Per-Database Role Settings \n--\n\n");
-
- for (i = 0; i < PQntuples(res); i++)
- {
- makeAlterConfigCommand(conn, PQgetvalue(res, i, 2),
- "ROLE", PQgetvalue(res, i, 0),
- "DATABASE", PQgetvalue(res, i, 1));
- }
-
- fprintf(OPF, "\n\n");
- }
-
- PQclear(res);
- destroyPQExpBuffer(buf);
- }
-
-
- /*
- * Helper function for dumpXXXConfig().
- */
- static void
- makeAlterConfigCommand(PGconn *conn, const char *arrayitem,
- const char *type, const char *name,
- const char *type2, const char *name2)
- {
- char *pos;
- char *mine;
- PQExpBuffer buf;
-
- mine = pg_strdup(arrayitem);
- pos = strchr(mine, '=');
- if (pos == NULL)
- {
- free(mine);
- return;
- }
-
- buf = createPQExpBuffer();
-
- *pos = 0;
- appendPQExpBuffer(buf, "ALTER %s %s ", type, fmtId(name));
- if (type2 != NULL && name2 != NULL)
- appendPQExpBuffer(buf, "IN %s %s ", type2, fmtId(name2));
- appendPQExpBuffer(buf, "SET %s TO ", fmtId(mine));
-
- /*
- * Some GUC variable names are 'LIST' type and hence must not be quoted.
- */
- if (pg_strcasecmp(mine, "DateStyle") == 0
- || pg_strcasecmp(mine, "search_path") == 0)
- appendPQExpBufferStr(buf, pos + 1);
- else
- appendStringLiteralConn(buf, pos + 1, conn);
- appendPQExpBufferStr(buf, ";\n");
-
- fprintf(OPF, "%s", buf->data);
- destroyPQExpBuffer(buf);
- free(mine);
- }
-
-
-
- /*
* Dump contents of databases.
*/
static void
--- 1338,1343 ----
*************** dumpDatabases(PGconn *conn)
*** 1753,1790 ****
PGresult *res;
int i;
! res = executeQuery(conn, "SELECT datname FROM pg_database WHERE datallowconn ORDER BY 1");
for (i = 0; i < PQntuples(res); i++)
{
int ret;
! char *dbname = PQgetvalue(res, i, 0);
! PQExpBufferData connectbuf;
if (verbose)
fprintf(stderr, _("%s: dumping database \"%s\"...\n"), progname, dbname);
- initPQExpBuffer(&connectbuf);
- appendPsqlMetaConnect(&connectbuf, dbname);
- fprintf(OPF, "%s\n", connectbuf.data);
- termPQExpBuffer(&connectbuf);
-
/*
! * Restore will need to write to the target cluster. This connection
! * setting is emitted for pg_dumpall rather than in the code also used
! * by pg_dump, so that a cluster with databases or users which have
! * this flag turned on can still be replicated through pg_dumpall
! * without editing the file or stream. With pg_dump there are many
! * other ways to allow the file to be used, and leaving it out allows
! * users to protect databases from being accidental restore targets.
*/
! fprintf(OPF, "SET default_transaction_read_only = off;\n\n");
if (filename)
fclose(OPF);
! ret = runPgDump(dbname);
if (ret != 0)
{
fprintf(stderr, _("%s: pg_dump failed on database \"%s\", exiting\n"), progname, dbname);
--- 1346,1407 ----
PGresult *res;
int i;
! /*
! * Skip databases marked not datallowconn, since we'd be unable to connect
! * to them anyway. This must agree with dropDBs().
! *
! * We arrange for template1 to be processed first, then we process other
! * DBs in alphabetical order. If we just did them all alphabetically, we
! * might find ourselves trying to drop the "postgres" database while still
! * connected to it. This makes trying to run the restore script while
! * connected to "template1" a bad idea, but there's no fixed order that
! * doesn't have some failure mode with --clean.
! */
! res = executeQuery(conn,
! "SELECT datname "
! "FROM pg_database d "
! "WHERE datallowconn "
! "ORDER BY (datname <> 'template1'), datname");
for (i = 0; i < PQntuples(res); i++)
{
+ char *dbname = PQgetvalue(res, i, 0);
+ const char *create_opts;
int ret;
! /* Skip template0, even if it's not marked !datallowconn. */
! if (strcmp(dbname, "template0") == 0)
! continue;
if (verbose)
fprintf(stderr, _("%s: dumping database \"%s\"...\n"), progname, dbname);
/*
! * We assume that "template1" and "postgres" already exist in the
! * target installation. dropDBs() won't have removed them, for fear
! * of removing the DB the restore script is initially connected to. If
! * --clean was specified, tell pg_dump to drop and recreate them;
! * otherwise we'll merely restore their contents. Other databases
! * should simply be created.
*/
! if (strcmp(dbname, "template1") == 0 || strcmp(dbname, "postgres") == 0)
! {
! if (output_clean)
! create_opts = "--clean --create";
! else
! {
! create_opts = "";
! /* Since pg_dump won't emit a \connect command, we must */
! fprintf(OPF, "\\connect %s\n\n", dbname);
! }
! }
! else
! create_opts = "--create";
if (filename)
fclose(OPF);
! ret = runPgDump(dbname, create_opts);
if (ret != 0)
{
fprintf(stderr, _("%s: pg_dump failed on database \"%s\", exiting\n"), progname, dbname);
*************** dumpDatabases(PGconn *conn)
*** 1810,1826 ****
/*
! * Run pg_dump on dbname.
*/
static int
! runPgDump(const char *dbname)
{
PQExpBuffer connstrbuf = createPQExpBuffer();
PQExpBuffer cmd = createPQExpBuffer();
int ret;
! appendPQExpBuffer(cmd, "\"%s\" %s", pg_dump_bin,
! pgdumpopts->data);
/*
* If we have a filename, use the undocumented plain-append pg_dump
--- 1427,1443 ----
/*
! * Run pg_dump on dbname, with specified options.
*/
static int
! runPgDump(const char *dbname, const char *create_opts)
{
PQExpBuffer connstrbuf = createPQExpBuffer();
PQExpBuffer cmd = createPQExpBuffer();
int ret;
! appendPQExpBuffer(cmd, "\"%s\" %s %s", pg_dump_bin,
! pgdumpopts->data, create_opts);
/*
* If we have a filename, use the undocumented plain-append pg_dump
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index fce1465..74730bf 100644
*** a/src/bin/pg_dump/t/002_pg_dump.pl
--- b/src/bin/pg_dump/t/002_pg_dump.pl
*************** qr/^ALTER (?!EVENT TRIGGER|LARGE OBJECT|
*** 1516,1526 ****
all_runs => 1,
catch_all => 'COMMENT commands',
regexp => qr/^COMMENT ON DATABASE postgres IS .*;/m,
! like => {
binary_upgrade => 1,
clean => 1,
clean_if_exists => 1,
! createdb => 1,
defaults => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
--- 1516,1529 ----
all_runs => 1,
catch_all => 'COMMENT commands',
regexp => qr/^COMMENT ON DATABASE postgres IS .*;/m,
! # Should appear in the same tests as "CREATE DATABASE postgres"
! like => { createdb => 1, },
! unlike => {
binary_upgrade => 1,
clean => 1,
clean_if_exists => 1,
! column_inserts => 1,
! data_only => 1,
defaults => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
*************** qr/^ALTER (?!EVENT TRIGGER|LARGE OBJECT|
*** 1528,1545 ****
no_blobs => 1,
no_privs => 1,
no_owner => 1,
pg_dumpall_dbprivs => 1,
schema_only => 1,
section_pre_data => 1,
! with_oids => 1, },
! unlike => {
! column_inserts => 1,
! data_only => 1,
! only_dump_test_schema => 1,
! only_dump_test_table => 1,
! role => 1,
! section_post_data => 1,
! test_schema_plus_blobs => 1, }, },
'COMMENT ON EXTENSION plpgsql' => {
all_runs => 1,
--- 1531,1548 ----
no_blobs => 1,
no_privs => 1,
no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
pg_dumpall_dbprivs => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
schema_only => 1,
section_pre_data => 1,
! section_data => 1,
! section_post_data => 1,
! test_schema_plus_blobs => 1,
! with_oids => 1, }, },
'COMMENT ON EXTENSION plpgsql' => {
all_runs => 1,
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 8726214..a67e484 100644
*** a/src/bin/pg_upgrade/pg_upgrade.c
--- b/src/bin/pg_upgrade/pg_upgrade.c
***************
*** 46,52 ****
#endif
static void prepare_new_cluster(void);
! static void prepare_new_databases(void);
static void create_new_objects(void);
static void copy_xact_xlog_xid(void);
static void set_frozenxids(bool minmxid_only);
--- 46,52 ----
#endif
static void prepare_new_cluster(void);
! static void prepare_new_globals(void);
static void create_new_objects(void);
static void copy_xact_xlog_xid(void);
static void set_frozenxids(bool minmxid_only);
*************** main(int argc, char **argv)
*** 124,130 ****
/* -- NEW -- */
start_postmaster(&new_cluster, true);
! prepare_new_databases();
create_new_objects();
--- 124,130 ----
/* -- NEW -- */
start_postmaster(&new_cluster, true);
! prepare_new_globals();
create_new_objects();
*************** prepare_new_cluster(void)
*** 271,277 ****
static void
! prepare_new_databases(void)
{
/*
* We set autovacuum_freeze_max_age to its maximum value so autovacuum
--- 271,277 ----
static void
! prepare_new_globals(void)
{
/*
* We set autovacuum_freeze_max_age to its maximum value so autovacuum
*************** prepare_new_databases(void)
*** 283,302 ****
prep_status("Restoring global objects in the new cluster");
- /*
- * We have to create the databases first so we can install support
- * functions in all the other databases. Ideally we could create the
- * support functions in template1 but pg_dumpall creates database using
- * the template0 template.
- */
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
"\"%s/psql\" " EXEC_PSQL_ARGS " %s -f \"%s\"",
new_cluster.bindir, cluster_conn_opts(&new_cluster),
GLOBALS_DUMP_FILE);
check_ok();
-
- /* we load this to get a current list of databases */
- get_db_and_rel_infos(&new_cluster);
}
--- 283,293 ----
*************** create_new_objects(void)
*** 312,344 ****
char sql_file_name[MAXPGPATH],
log_file_name[MAXPGPATH];
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
! PQExpBufferData connstr,
! escaped_connstr;
!
! initPQExpBuffer(&connstr);
! appendPQExpBuffer(&connstr, "dbname=");
! appendConnStrVal(&connstr, old_db->db_name);
! initPQExpBuffer(&escaped_connstr);
! appendShellString(&escaped_connstr, connstr.data);
! termPQExpBuffer(&connstr);
pg_log(PG_STATUS, "%s", old_db->db_name);
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
/*
! * pg_dump only produces its output at the end, so there is little
! * parallelism if using the pipe.
*/
parallel_exec_prog(log_file_name,
NULL,
! "\"%s/pg_restore\" %s --exit-on-error --verbose --dbname %s \"%s\"",
new_cluster.bindir,
cluster_conn_opts(&new_cluster),
! escaped_connstr.data,
sql_file_name);
-
- termPQExpBuffer(&escaped_connstr);
}
/* reap all children */
--- 303,342 ----
char sql_file_name[MAXPGPATH],
log_file_name[MAXPGPATH];
DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
! const char *create_opts;
! const char *starting_db;
pg_log(PG_STATUS, "%s", old_db->db_name);
snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
/*
! * template1 and postgres databases will already exist in the target
! * installation, so tell pg_restore to drop and recreate them;
! * otherwise we would fail to propagate their database-level
! * properties.
*/
+ if (strcmp(old_db->db_name, "template1") == 0 ||
+ strcmp(old_db->db_name, "postgres") == 0)
+ create_opts = "--clean --create";
+ else
+ create_opts = "--create";
+
+ /* When processing template1, we can't connect there to start with */
+ if (strcmp(old_db->db_name, "template1") == 0)
+ starting_db = "postgres";
+ else
+ starting_db = "template1";
+
parallel_exec_prog(log_file_name,
NULL,
! "\"%s/pg_restore\" %s %s --exit-on-error --verbose "
! "--dbname %s \"%s\"",
new_cluster.bindir,
cluster_conn_opts(&new_cluster),
! create_opts,
! starting_db,
sql_file_name);
}
/* reap all children */
*************** create_new_objects(void)
*** 355,361 ****
if (GET_MAJOR_VERSION(old_cluster.major_version) < 903)
set_frozenxids(true);
! /* regenerate now that we have objects in the databases */
get_db_and_rel_infos(&new_cluster);
}
--- 353,359 ----
if (GET_MAJOR_VERSION(old_cluster.major_version) < 903)
set_frozenxids(true);
! /* update new_cluster info now that we have objects in the databases */
get_db_and_rel_infos(&new_cluster);
}
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-21 15:17 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Bruce Momjian <[email protected]>
2018-01-21 16:02 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
@ 2018-01-23 15:45 ` Bruce Momjian <[email protected]>
2018-01-23 16:46 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
1 sibling, 1 reply; 33+ messages in thread
From: Bruce Momjian @ 2018-01-23 15:45 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
On Sun, Jan 21, 2018 at 11:02:29AM -0500, Tom Lane wrote:
> Bruce Momjian <[email protected]> writes:
> > On Fri, Jan 19, 2018 at 02:54:25PM -0500, Tom Lane wrote:
> >> Command was: DROP DATABASE "template1";
>
> > Uh, the oid of the template1 database is 1, and I assume we would want
> > to preserve that too.
>
> I don't feel any huge attachment to that. In the first place, under
> this proposal recreating template1 is something you would only need to do
> if you weren't satisfied with its default properties as set by initdb.
> Which ought to be a minority of users. In the second place, if you are
> changing those properties from the way initdb set it up, it's not really
> virgin template1 anymore, so why shouldn't it have a new OID?
Oh, I see what you mean. I was just worried that some code might expect
template1 to always have an oid of 1, but we can just call that code
broken.
--
Bruce Momjian <[email protected]> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-17 21:25 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-17 23:50 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 21:34 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-18 23:15 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-18 23:35 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 14:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:00 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 15:13 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Robert Haas <[email protected]>
2018-01-19 16:28 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-19 19:54 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-21 15:17 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Bruce Momjian <[email protected]>
2018-01-21 16:02 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Tom Lane <[email protected]>
2018-01-23 15:45 ` Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Bruce Momjian <[email protected]>
@ 2018-01-23 16:46 ` Tom Lane <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Tom Lane @ 2018-01-23 16:46 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Robert Haas <[email protected]>; Haribabu Kommi <[email protected]>; Vaishnavi Prabakaran <[email protected]>; Michael Paquier <[email protected]>; Thomas Munro <[email protected]>; Peter Eisentraut <[email protected]>; Andreas Karlsson <[email protected]>; pgsql-hackers
Bruce Momjian <[email protected]> writes:
> Oh, I see what you mean. I was just worried that some code might expect
> template1 to always have an oid of 1, but we can just call that code
> broken.
Ever since we invented template0, it's been possible to drop and recreate
template1, so I'd say any expectation that it must have OID 1 has been
wrong for a long time. This change will just make the situation more
common.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
* [PATCH v5 1/4] Add a parser_hook hook.
@ 2021-04-21 14:47 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Julien Rouhaud @ 2021-04-21 14:47 UTC (permalink / raw)
This does nothing but allow third-party plugins to implement a different
syntax, and fallback on the core parser if they don't implement a superset of
the supported core syntax.
---
src/backend/tcop/postgres.c | 16 ++++++++++++++--
src/include/tcop/tcopprot.h | 5 +++++
2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 3f9ed549f9..66ee58a4b1 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -99,6 +99,9 @@ int log_statement = LOGSTMT_NONE;
/* GUC variable for maximum stack depth (measured in kilobytes) */
int max_stack_depth = 100;
+/* Hook for plugins to get control in pg_parse_query() */
+parser_hook_type parser_hook = NULL;
+
/* wait N seconds to allow attach from a debugger */
int PostAuthDelay = 0;
@@ -589,18 +592,27 @@ ProcessClientWriteInterrupt(bool blocked)
* database tables. So, we rely on the raw parser to determine whether
* we've seen a COMMIT or ABORT command; when we are in abort state, other
* commands are not processed any further than the raw parse stage.
+ *
+ * To support loadable plugins that monitor the parsing or implements SQL
+ * syntactic sugar we provide a hook variable that lets a plugin get control
+ * before and after the standard parsing process. If the plugin only implement
+ * a subset of postgres supported syntax, it's its duty to call raw_parser (or
+ * the previous hook if any) for the statements it doesn't understand.
*/
List *
pg_parse_query(const char *query_string)
{
- List *raw_parsetree_list;
+ List *raw_parsetree_list = NIL;
TRACE_POSTGRESQL_QUERY_PARSE_START(query_string);
if (log_parser_stats)
ResetUsage();
- raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
+ if (parser_hook)
+ raw_parsetree_list = (*parser_hook) (query_string, RAW_PARSE_DEFAULT);
+ else
+ raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
if (log_parser_stats)
ShowUsage("PARSER STATISTICS");
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 968345404e..131dc2b22e 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -17,6 +17,7 @@
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "nodes/plannodes.h"
+#include "parser/parser.h"
#include "storage/procsignal.h"
#include "utils/guc.h"
#include "utils/queryenvironment.h"
@@ -43,6 +44,10 @@ typedef enum
extern PGDLLIMPORT int log_statement;
+/* Hook for plugins to get control in pg_parse_query() */
+typedef List *(*parser_hook_type) (const char *str, RawParseMode mode);
+extern PGDLLIMPORT parser_hook_type parser_hook;
+
extern List *pg_parse_query(const char *query_string);
extern List *pg_rewrite_query(Query *query);
extern List *pg_analyze_and_rewrite(RawStmt *parsetree,
--
2.32.0
--n4pdl6wolj3mjhvd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v5-0002-Add-a-sqlol-parser.patch"
^ permalink raw reply [nested|flat] 33+ messages in thread
* [PATCH v3 1/4] Add a parser_hook hook.
@ 2021-04-21 14:47 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Julien Rouhaud @ 2021-04-21 14:47 UTC (permalink / raw)
This does nothing but allow third-party plugins to implement a different
syntax, and fallback on the core parser if they don't implement a superset of
the supported core syntax.
---
src/backend/tcop/postgres.c | 16 ++++++++++++++--
src/include/tcop/tcopprot.h | 5 +++++
2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..e941b59b85 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -99,6 +99,9 @@ int log_statement = LOGSTMT_NONE;
/* GUC variable for maximum stack depth (measured in kilobytes) */
int max_stack_depth = 100;
+/* Hook for plugins to get control in pg_parse_query() */
+parser_hook_type parser_hook = NULL;
+
/* wait N seconds to allow attach from a debugger */
int PostAuthDelay = 0;
@@ -589,18 +592,27 @@ ProcessClientWriteInterrupt(bool blocked)
* database tables. So, we rely on the raw parser to determine whether
* we've seen a COMMIT or ABORT command; when we are in abort state, other
* commands are not processed any further than the raw parse stage.
+ *
+ * To support loadable plugins that monitor the parsing or implements SQL
+ * syntactic sugar we provide a hook variable that lets a plugin get control
+ * before and after the standard parsing process. If the plugin only implement
+ * a subset of postgres supported syntax, it's its duty to call raw_parser (or
+ * the previous hook if any) for the statements it doesn't understand.
*/
List *
pg_parse_query(const char *query_string)
{
- List *raw_parsetree_list;
+ List *raw_parsetree_list = NIL;
TRACE_POSTGRESQL_QUERY_PARSE_START(query_string);
if (log_parser_stats)
ResetUsage();
- raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
+ if (parser_hook)
+ raw_parsetree_list = (*parser_hook) (query_string, RAW_PARSE_DEFAULT);
+ else
+ raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
if (log_parser_stats)
ShowUsage("PARSER STATISTICS");
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 968345404e..131dc2b22e 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -17,6 +17,7 @@
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "nodes/plannodes.h"
+#include "parser/parser.h"
#include "storage/procsignal.h"
#include "utils/guc.h"
#include "utils/queryenvironment.h"
@@ -43,6 +44,10 @@ typedef enum
extern PGDLLIMPORT int log_statement;
+/* Hook for plugins to get control in pg_parse_query() */
+typedef List *(*parser_hook_type) (const char *str, RawParseMode mode);
+extern PGDLLIMPORT parser_hook_type parser_hook;
+
extern List *pg_parse_query(const char *query_string);
extern List *pg_rewrite_query(Query *query);
extern List *pg_analyze_and_rewrite(RawStmt *parsetree,
--
2.31.1
--v6yspzozp6k6x4hy
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v3-0002-Add-a-sqlol-parser.patch"
^ permalink raw reply [nested|flat] 33+ messages in thread
* [PATCH v1 1/4] Add a parser_hook hook.
@ 2021-04-21 14:47 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Julien Rouhaud @ 2021-04-21 14:47 UTC (permalink / raw)
This does nothing but allow third-party plugins to implement a different
syntax, and fallback on the core parser if they don't implement a superset of
the supported core syntax.
---
src/backend/tcop/postgres.c | 16 ++++++++++++++--
src/include/tcop/tcopprot.h | 5 +++++
2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 2d6d145ecc..e91db69830 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -99,6 +99,9 @@ int log_statement = LOGSTMT_NONE;
/* GUC variable for maximum stack depth (measured in kilobytes) */
int max_stack_depth = 100;
+/* Hook for plugins to get control in pg_parse_query() */
+parser_hook_type parser_hook = NULL;
+
/* wait N seconds to allow attach from a debugger */
int PostAuthDelay = 0;
@@ -589,18 +592,27 @@ ProcessClientWriteInterrupt(bool blocked)
* database tables. So, we rely on the raw parser to determine whether
* we've seen a COMMIT or ABORT command; when we are in abort state, other
* commands are not processed any further than the raw parse stage.
+ *
+ * To support loadable plugins that monitor the parsing or implements SQL
+ * syntactic sugar we provide a hook variable that lets a plugin get control
+ * before and after the standard parsing process. If the plugin only implement
+ * a subset of postgres supported syntax, it's its duty to call raw_parser (or
+ * the previous hook if any) for the statements it doesn't understand.
*/
List *
pg_parse_query(const char *query_string)
{
- List *raw_parsetree_list;
+ List *raw_parsetree_list = NIL;
TRACE_POSTGRESQL_QUERY_PARSE_START(query_string);
if (log_parser_stats)
ResetUsage();
- raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
+ if (parser_hook)
+ raw_parsetree_list = (*parser_hook) (query_string, RAW_PARSE_DEFAULT);
+ else
+ raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
if (log_parser_stats)
ShowUsage("PARSER STATISTICS");
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 968345404e..131dc2b22e 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -17,6 +17,7 @@
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "nodes/plannodes.h"
+#include "parser/parser.h"
#include "storage/procsignal.h"
#include "utils/guc.h"
#include "utils/queryenvironment.h"
@@ -43,6 +44,10 @@ typedef enum
extern PGDLLIMPORT int log_statement;
+/* Hook for plugins to get control in pg_parse_query() */
+typedef List *(*parser_hook_type) (const char *str, RawParseMode mode);
+extern PGDLLIMPORT parser_hook_type parser_hook;
+
extern List *pg_parse_query(const char *query_string);
extern List *pg_rewrite_query(Query *query);
extern List *pg_analyze_and_rewrite(RawStmt *parsetree,
--
2.30.1
--ymfuxy2befenx45f
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v1-0002-Add-a-sqlol-parser.patch"
^ permalink raw reply [nested|flat] 33+ messages in thread
* [PATCH v4 1/4] Add a parser_hook hook.
@ 2021-04-21 14:47 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Julien Rouhaud @ 2021-04-21 14:47 UTC (permalink / raw)
This does nothing but allow third-party plugins to implement a different
syntax, and fallback on the core parser if they don't implement a superset of
the supported core syntax.
---
src/backend/tcop/postgres.c | 16 ++++++++++++++--
src/include/tcop/tcopprot.h | 5 +++++
2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..e941b59b85 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -99,6 +99,9 @@ int log_statement = LOGSTMT_NONE;
/* GUC variable for maximum stack depth (measured in kilobytes) */
int max_stack_depth = 100;
+/* Hook for plugins to get control in pg_parse_query() */
+parser_hook_type parser_hook = NULL;
+
/* wait N seconds to allow attach from a debugger */
int PostAuthDelay = 0;
@@ -589,18 +592,27 @@ ProcessClientWriteInterrupt(bool blocked)
* database tables. So, we rely on the raw parser to determine whether
* we've seen a COMMIT or ABORT command; when we are in abort state, other
* commands are not processed any further than the raw parse stage.
+ *
+ * To support loadable plugins that monitor the parsing or implements SQL
+ * syntactic sugar we provide a hook variable that lets a plugin get control
+ * before and after the standard parsing process. If the plugin only implement
+ * a subset of postgres supported syntax, it's its duty to call raw_parser (or
+ * the previous hook if any) for the statements it doesn't understand.
*/
List *
pg_parse_query(const char *query_string)
{
- List *raw_parsetree_list;
+ List *raw_parsetree_list = NIL;
TRACE_POSTGRESQL_QUERY_PARSE_START(query_string);
if (log_parser_stats)
ResetUsage();
- raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
+ if (parser_hook)
+ raw_parsetree_list = (*parser_hook) (query_string, RAW_PARSE_DEFAULT);
+ else
+ raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
if (log_parser_stats)
ShowUsage("PARSER STATISTICS");
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 968345404e..131dc2b22e 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -17,6 +17,7 @@
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "nodes/plannodes.h"
+#include "parser/parser.h"
#include "storage/procsignal.h"
#include "utils/guc.h"
#include "utils/queryenvironment.h"
@@ -43,6 +44,10 @@ typedef enum
extern PGDLLIMPORT int log_statement;
+/* Hook for plugins to get control in pg_parse_query() */
+typedef List *(*parser_hook_type) (const char *str, RawParseMode mode);
+extern PGDLLIMPORT parser_hook_type parser_hook;
+
extern List *pg_parse_query(const char *query_string);
extern List *pg_rewrite_query(Query *query);
extern List *pg_analyze_and_rewrite(RawStmt *parsetree,
--
2.31.1
--j3ft3uyq57t2xkwr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v4-0002-Add-a-sqlol-parser.patch"
^ permalink raw reply [nested|flat] 33+ messages in thread
* [PATCH v2 1/4] Add a parser_hook hook.
@ 2021-04-21 14:47 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Julien Rouhaud @ 2021-04-21 14:47 UTC (permalink / raw)
This does nothing but allow third-party plugins to implement a different
syntax, and fallback on the core parser if they don't implement a superset of
the supported core syntax.
---
src/backend/tcop/postgres.c | 16 ++++++++++++++--
src/include/tcop/tcopprot.h | 5 +++++
2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..e941b59b85 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -99,6 +99,9 @@ int log_statement = LOGSTMT_NONE;
/* GUC variable for maximum stack depth (measured in kilobytes) */
int max_stack_depth = 100;
+/* Hook for plugins to get control in pg_parse_query() */
+parser_hook_type parser_hook = NULL;
+
/* wait N seconds to allow attach from a debugger */
int PostAuthDelay = 0;
@@ -589,18 +592,27 @@ ProcessClientWriteInterrupt(bool blocked)
* database tables. So, we rely on the raw parser to determine whether
* we've seen a COMMIT or ABORT command; when we are in abort state, other
* commands are not processed any further than the raw parse stage.
+ *
+ * To support loadable plugins that monitor the parsing or implements SQL
+ * syntactic sugar we provide a hook variable that lets a plugin get control
+ * before and after the standard parsing process. If the plugin only implement
+ * a subset of postgres supported syntax, it's its duty to call raw_parser (or
+ * the previous hook if any) for the statements it doesn't understand.
*/
List *
pg_parse_query(const char *query_string)
{
- List *raw_parsetree_list;
+ List *raw_parsetree_list = NIL;
TRACE_POSTGRESQL_QUERY_PARSE_START(query_string);
if (log_parser_stats)
ResetUsage();
- raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
+ if (parser_hook)
+ raw_parsetree_list = (*parser_hook) (query_string, RAW_PARSE_DEFAULT);
+ else
+ raw_parsetree_list = raw_parser(query_string, RAW_PARSE_DEFAULT);
if (log_parser_stats)
ShowUsage("PARSER STATISTICS");
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 968345404e..131dc2b22e 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -17,6 +17,7 @@
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "nodes/plannodes.h"
+#include "parser/parser.h"
#include "storage/procsignal.h"
#include "utils/guc.h"
#include "utils/queryenvironment.h"
@@ -43,6 +44,10 @@ typedef enum
extern PGDLLIMPORT int log_statement;
+/* Hook for plugins to get control in pg_parse_query() */
+typedef List *(*parser_hook_type) (const char *str, RawParseMode mode);
+extern PGDLLIMPORT parser_hook_type parser_hook;
+
extern List *pg_parse_query(const char *query_string);
extern List *pg_rewrite_query(Query *query);
extern List *pg_analyze_and_rewrite(RawStmt *parsetree,
--
2.31.1
--p53pz3r7l3llx7gx
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v2-0002-Add-a-sqlol-parser.patch"
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: First draft of back-branch release notes is done
@ 2023-02-05 20:37 Thomas Munro <[email protected]>
2023-02-05 20:44 ` Re: First draft of back-branch release notes is done Tom Lane <[email protected]>
0 siblings, 1 reply; 33+ messages in thread
From: Thomas Munro @ 2023-02-05 20:37 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; [email protected]
On Mon, Feb 6, 2023 at 8:57 AM Tom Lane <[email protected]> wrote:
> Alvaro Herrera <[email protected]> writes:
> > On 2023-Feb-03, Tom Lane wrote:
> >> Fix edge-case data corruption in shared tuplestores (Dmitry Astapov)
>
> > I think this sounds really scary, because people are going to think that
> > their stored data can get corrupted -- they don't necessarily know what
> > a "shared tuplestore" is. Maybe "Avoid query failures in parallel hash
> > joins" as headline?
>
> Hmmm ... are we sure it *can't* lead to corruption of stored data,
> if this happens during an INSERT or UPDATE plan? I'll grant that
> such a case seems pretty unlikely though, as the bogus data
> retrieved from the tuplestore would have to not cause a failure
> within the query before it can get written out.
Agreed. I think you have to be quite unlucky to hit this in the first
place (very large tuples with very particular alignment), and then
you'd be highly likely to fail in some way due to corrupted tuple
size, making permanent corruption extremely unlikely.
> Also, aren't shared tuplestores used in more places than just
> parallel hash join? I mentioned that as an example, not an
> exhaustive list of trouble spots.
Shared file sets (= a directory of temp files with automatic cleanup)
are used by more things, but shared tuplestores (= a shared file set
with a tuple-oriented interface on top, to support partial scan) are
currently only used by PHJ.
^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: First draft of back-branch release notes is done
2023-02-05 20:37 Re: First draft of back-branch release notes is done Thomas Munro <[email protected]>
@ 2023-02-05 20:44 ` Tom Lane <[email protected]>
0 siblings, 0 replies; 33+ messages in thread
From: Tom Lane @ 2023-02-05 20:44 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; [email protected]
Thomas Munro <[email protected]> writes:
> On Mon, Feb 6, 2023 at 8:57 AM Tom Lane <[email protected]> wrote:
>> Also, aren't shared tuplestores used in more places than just
>> parallel hash join? I mentioned that as an example, not an
>> exhaustive list of trouble spots.
> Shared file sets (= a directory of temp files with automatic cleanup)
> are used by more things, but shared tuplestores (= a shared file set
> with a tuple-oriented interface on top, to support partial scan) are
> currently only used by PHJ.
Oh, okay. I'll change it to say "corruption within parallel hash
join", then, and not use the word "tuplestore" at all.
regards, tom lane
^ permalink raw reply [nested|flat] 33+ messages in thread
end of thread, other threads:[~2023-02-05 20:44 UTC | newest]
Thread overview: 33+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-12-13 02:50 Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall Haribabu Kommi <[email protected]>
2018-01-05 03:54 ` Vaishnavi Prabakaran <[email protected]>
2018-01-05 05:32 ` Haribabu Kommi <[email protected]>
2018-01-05 06:00 ` Vaishnavi Prabakaran <[email protected]>
2018-01-17 21:25 ` Tom Lane <[email protected]>
2018-01-17 23:32 ` Haribabu Kommi <[email protected]>
2018-01-17 23:50 ` Tom Lane <[email protected]>
2018-01-18 21:34 ` Robert Haas <[email protected]>
2018-01-18 22:09 ` Tom Lane <[email protected]>
2018-01-18 23:15 ` Tom Lane <[email protected]>
2018-01-18 23:35 ` Tom Lane <[email protected]>
2018-01-19 00:09 ` Haribabu Kommi <[email protected]>
2018-01-19 14:28 ` Robert Haas <[email protected]>
2018-01-19 14:45 ` Tom Lane <[email protected]>
2018-01-19 14:52 ` Robert Haas <[email protected]>
2018-01-19 15:00 ` Tom Lane <[email protected]>
2018-01-19 15:13 ` Robert Haas <[email protected]>
2018-01-19 16:28 ` Tom Lane <[email protected]>
2018-01-19 19:54 ` Tom Lane <[email protected]>
2018-01-19 20:43 ` Robert Haas <[email protected]>
2018-01-19 21:07 ` Tom Lane <[email protected]>
2018-01-21 15:17 ` Bruce Momjian <[email protected]>
2018-01-21 16:02 ` Tom Lane <[email protected]>
2018-01-21 17:42 ` Tom Lane <[email protected]>
2018-01-23 15:45 ` Bruce Momjian <[email protected]>
2018-01-23 16:46 ` Tom Lane <[email protected]>
2021-04-21 14:47 [PATCH v5 1/4] Add a parser_hook hook. Julien Rouhaud <[email protected]>
2021-04-21 14:47 [PATCH v3 1/4] Add a parser_hook hook. Julien Rouhaud <[email protected]>
2021-04-21 14:47 [PATCH v1 1/4] Add a parser_hook hook. Julien Rouhaud <[email protected]>
2021-04-21 14:47 [PATCH v4 1/4] Add a parser_hook hook. Julien Rouhaud <[email protected]>
2021-04-21 14:47 [PATCH v2 1/4] Add a parser_hook hook. Julien Rouhaud <[email protected]>
2023-02-05 20:37 Re: First draft of back-branch release notes is done Thomas Munro <[email protected]>
2023-02-05 20:44 ` Re: First draft of back-branch release notes is done Tom Lane <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox