From: Jesper Pedersen <jesper.pedersen@redhat.com>
To: Michael Paquier <michael@paquier.xyz>
To: Jamison, Kirk <k.jamison@jp.fujitsu.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: fabriziomello@gmail.com <fabriziomello@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: pgsql-hackers@lists.postgresql.org <pgsql-hackers@lists.postgresql.org>
Cc: Bruce Momjian <bruce@momjian.us>
Subject: Re: pg_upgrade: Pass -j down to vacuumdb
Date: Tue, 29 Jan 2019 15:19:28 -0500
Message-ID: <37538daf-5f5b-afed-b3f5-a64e154e6433@redhat.com> (raw)
In-Reply-To: <20190129050830.GE3121@paquier.xyz>
References: <5b7d95ba-8920-357d-1ec6-1084ce34a9e6@redhat.com>
<D09B13F772D2274BB348A310EE3027C64137E4@g01jpexmbkw24>
<c448acf6-7c38-3d6d-c663-5387ccffb906@redhat.com>
<D09B13F772D2274BB348A310EE3027C6413F1A@g01jpexmbkw24>
<388de5bc-858b-4686-d2e5-d0b21981dd72@redhat.com>
<CAFcNs+qCPbzErgcmTDqW2qzkt7kzarjWqD48VimKD4mXM0nutA@mail.gmail.com>
<4ce565f9-d21f-039f-8f5a-2e16e6aa1ae4@redhat.com>
<e103a8a1-a6f1-5709-6ac6-95684fd5bff2@2ndquadrant.com>
<be5cd0aa-304f-1b9f-68b4-0c747c67f73b@redhat.com>
<D09B13F772D2274BB348A310EE3027C64146AF@g01jpexmbkw24>
<20190129050830.GE3121@paquier.xyz>
Hi,
On 1/29/19 12:08 AM, Michael Paquier wrote:
> On Tue, Jan 29, 2019 at 12:35:30AM +0000, Jamison, Kirk wrote:
>> I just checked the patch.
>> As per advice, you removed the versioning and specified --jobs.
>> The patch still applies, builds and passed the tests successfully.
>
> I would document the optional VACUUM_OPTS on the page of pg_upgrade.
> If Peter thinks it is fine to not do so, that's fine for me as well.
>
I added most of the documentation back, as requested by Kirk.
> It seems to me that the latest patch sent is incorrect for multiple
> reasons:
> 1) You still enforce -j to use the number of jobs that the caller of
> pg_upgrade provides, and we agreed that both things are separate
> concepts upthread, no? What has been suggested by Alvaro is to add a
> comment so as one can use VACUUM_OPTS with -j optionally, instead of
> suggesting a full-fledged vacuumdb command which depends on what
> pg_upgrade uses. So there is no actual need for the if/else
> complication business.
I think it is ok for the echo command to highlight to the user that
running --analyze-only using the same amount of jobs will give a faster
result.
> 2) Perhaps we need to worry about the second vacuumdb --all command,
> which may want custom options, which are not necessarily the same as
> the options of the first command? I don't think we need to care as it
> applies only to an upgraded cluster using something older 8.4, just
> wondering..
I think that --all --analyze-in-stages is what most people want. And
with the $VACUUMDB_OPTS variable people have an option to put in more,
such as -j X.
Best regards,
Jesper
Attachments:v6_0001-Highlight-that-the-jobs-option-isn-t-passed-down-to-.patch (text/x-patch, 3.1K) download
From b02e1f414e309361595ab3fe84fb6f66bcf63265 Mon Sep 17 00:00:00 2001
From: jesperpedersen <jesper.pedersen@redhat.com>
Date: Fri, 21 Dec 2018 05:05:31 -0500
Subject: [PATCH] Highlight that the --jobs option isn't passed down to
vacuumdb by default.
Author: Jesper Pedersen <jesper.pedersen@redhat.com>
---
doc/src/sgml/ref/pgupgrade.sgml | 6 +++++-
src/bin/pg_upgrade/check.c | 28 ++++++++++++++++++++++------
2 files changed, 27 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgmlindex 7e1afaf0a5..84a7ccb6f1 100644--- a/doc/src/sgml/ref/pgupgrade.sgml+++ b/doc/src/sgml/ref/pgupgrade.sgml@@ -395,7 +395,11 @@ NET STOP postgresql-&majorversion;
in parallel; a good place to start is the maximum of the number of
CPU cores and tablespaces. This option can dramatically reduce the
time to upgrade a multi-database server running on a multiprocessor
- machine.+ machine. Note that this option isn't passed to the+ <application>vacuumdb</application> application by default.+ The system environment variable <literal>VACUUMDB_OPTS</literal> can be+ used to pass extra options to the <application>vacuumdb</application>+ application during the script execution.
</para>
<para>
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.cindex fc5aa7010f..847e604088 100644--- a/src/bin/pg_upgrade/check.c+++ b/src/bin/pg_upgrade/check.c@@ -495,15 +495,31 @@ create_script_for_cluster_analyze(char **analyze_script_file_name)
ECHO_QUOTE, ECHO_QUOTE);
fprintf(script, "echo %sthis script and run:%s\n",
ECHO_QUOTE, ECHO_QUOTE);
- fprintf(script, "echo %s \"%s/vacuumdb\" %s--all %s%s\n", ECHO_QUOTE,- new_cluster.bindir, user_specification.data,- /* Did we copy the free space files? */- (GET_MAJOR_VERSION(old_cluster.major_version) >= 804) ?- "--analyze-only" : "--analyze", ECHO_QUOTE);+ if (user_opts.jobs <= 1)+ {+ fprintf(script, "echo %s \"%s/vacuumdb\" %s--all %s%s\n", ECHO_QUOTE,+ new_cluster.bindir, user_specification.data,+ /* Did we copy the free space files? */+ (GET_MAJOR_VERSION(old_cluster.major_version) >= 804) ?+ "--analyze-only" : "--analyze", ECHO_QUOTE);+ }+ else+ {+ fprintf(script, "echo %s \"%s/vacuumdb\" %s--jobs %d --all %s%s\n", ECHO_QUOTE,+ new_cluster.bindir, user_specification.data, user_opts.jobs,+ /* Did we copy the free space files? */+ (GET_MAJOR_VERSION(old_cluster.major_version) >= 804) ?+ "--analyze-only" : "--analyze", ECHO_QUOTE);+ }
fprintf(script, "echo%s\n\n", ECHO_BLANK);
- fprintf(script, "\"%s/vacuumdb\" %s--all --analyze-in-stages\n",+#ifndef WIN32+ fprintf(script, "\"%s/vacuumdb\" %s$VACUUMDB_OPTS --all --analyze-in-stages\n",
new_cluster.bindir, user_specification.data);
+#else+ fprintf(script, "\"%s\\vacuumdb.exe\" %s%%VACUUMDB_OPTS%% --all --analyze-in-stages\n",+ new_cluster.bindir, user_specification.data);+#endif
/* Did we copy the free space files? */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 804)
fprintf(script, "\"%s/vacuumdb\" %s--all\n", new_cluster.bindir,
--
2.17.2
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: jesper.pedersen@redhat.com, michael@paquier.xyz, k.jamison@jp.fujitsu.com, peter.eisentraut@2ndquadrant.com, fabriziomello@gmail.com, tgl@sss.pgh.pa.us, alvherre@2ndquadrant.com, pgsql-hackers@lists.postgresql.org, bruce@momjian.us
Subject: Re: pg_upgrade: Pass -j down to vacuumdb
In-Reply-To: <37538daf-5f5b-afed-b3f5-a64e154e6433@redhat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox