Reply-To: jesper.pedersen@redhat.com
Subject: Re: pg_upgrade: Pass -j down to vacuumdb
To: "Jamison, Kirk" <k.jamison@jp.fujitsu.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, 'Alvaro Herrera'
 <alvherre@2ndquadrant.com>, 'Michael Paquier' <michael@paquier.xyz>,
 Peter Eisentraut <peter.eisentraut@2ndquadrant.com>,
 "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
 Bruce Momjian <bruce@momjian.us>
References: <11b6b108-6ac0-79a6-785a-f13dfe60ab92@redhat.com>
 <ba90295e-9734-07bc-3453-bfa878948478@2ndquadrant.com>
 <2dc5eaeb-e60d-2a16-696e-7b4d3d3548cb@redhat.com>
 <59f9323a-fee8-88f6-16ab-2319149b00be@2ndquadrant.com>
 <5b7d95ba-8920-357d-1ec6-1084ce34a9e6@redhat.com>
 <D09B13F772D2274BB348A310EE3027C64137E4@g01jpexmbkw24>
 <c448acf6-7c38-3d6d-c663-5387ccffb906@redhat.com>
 <D09B13F772D2274BB348A310EE3027C6413F1A@g01jpexmbkw24>
From: Jesper Pedersen <jesper.pedersen@redhat.com>
Organization: Red Hat
Message-ID: <388de5bc-858b-4686-d2e5-d0b21981dd72@redhat.com>
Date: Mon, 28 Jan 2019 10:15:34 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <D09B13F772D2274BB348A310EE3027C6413F1A@g01jpexmbkw24>
Content-Type: multipart/mixed;
 boundary="------------D73A50015A30B21EBFE6291A"
Content-Language: en-US
Precedence: bulk

This is a multi-part message in MIME format.
--------------D73A50015A30B21EBFE6291A
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 1/27/19 7:50 PM, Jamison, Kirk wrote:
> There were also helpful comments from the developers above,
> pointing it to the right direction.
> In addition to Peter's comment, quoting "...if you want to do this
> as fast as possible, don't use this script.  That comment could be
> enhanced to suggest the use of the -j option.", so I think you
> should also update the following script in create_script_for_cluster_analyze():
>       fprintf(script, "echo %sIf you would like default statistics as quickly as possible, cancel%s\n",
>               ECHO_QUOTE, ECHO_QUOTE);
>

Yes, it will echo the -j option if passed, and the server supports it.

>> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>>> So let's have it write with a $VACUUMDB_OPTS variable, which is by
>>> default defined as empty but with a comment suggesting that maybe the
>>> user wants to add the -j option.  This way, if they have to edit it,
>>> they only have to edit the VACUUMDB_OPTS line instead of each of the
>>> two vacuumdb lines.
>>
> Tom Lane wrote:
>> Even better, allow the script to absorb a value from the environment, so that it needn't be edited at all.
> 

Done.

Attached is v3.

Best regards,
  Jesper

--------------D73A50015A30B21EBFE6291A
Content-Type: text/x-patch;
 name="v3_0001-Highlight-that-the-j-option-isn-t-passed-down-to-vac.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="v3_0001-Highlight-that-the-j-option-isn-t-passed-down-to-vac";
 filename*1=".patch"

From 1cba8299ae328942a4bf623d841a80b1b6043d3a Mon Sep 17 00:00:00 2001
From: jesperpedersen <jesper.pedersen@redhat.com>
Date: Fri, 21 Dec 2018 05:05:31 -0500
Subject: [PATCH] Highlight that the -j option isn't passed down to vacuumdb by
 default.

Author: Jesper Pedersen <jesper.pedersen@redhat.com>
---
 doc/src/sgml/ref/pgupgrade.sgml |  6 +++++-
 src/bin/pg_upgrade/check.c      | 28 ++++++++++++++++++++++------
 2 files changed, 27 insertions(+), 7 deletions(-)

diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 7e1afaf0a5..b1da0cc7a2 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -395,7 +395,11 @@ NET STOP postgresql-&majorversion;
      in parallel;  a good place to start is the maximum of the number of
      CPU cores and tablespaces.  This option can dramatically reduce the
      time to upgrade a multi-database server running on a multiprocessor
-     machine.
+     machine. Note, that this option isn't passed to the
+     <application>vacuumdb</application> application by default.
+     The system environment variable <literal>VACUUMDB_OPTS</literal> can be
+     used to pass extra options to the <application>vacuumdb</application>
+     application during the script execution such as <option>--jobs</option>.
     </para>
 
     <para>
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index fc5aa7010f..5114df01b5 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -495,15 +495,31 @@ create_script_for_cluster_analyze(char **analyze_script_file_name)
 			ECHO_QUOTE, ECHO_QUOTE);
 	fprintf(script, "echo %sthis script and run:%s\n",
 			ECHO_QUOTE, ECHO_QUOTE);
-	fprintf(script, "echo %s    \"%s/vacuumdb\" %s--all %s%s\n", ECHO_QUOTE,
-			new_cluster.bindir, user_specification.data,
-	/* Did we copy the free space files? */
-			(GET_MAJOR_VERSION(old_cluster.major_version) >= 804) ?
-			"--analyze-only" : "--analyze", ECHO_QUOTE);
+	if (user_opts.jobs <= 1 || GET_MAJOR_VERSION(new_cluster.major_version) < 905)
+	{
+		fprintf(script, "echo %s    \"%s/vacuumdb\" %s--all %s%s\n", ECHO_QUOTE,
+				new_cluster.bindir, user_specification.data,
+				/* Did we copy the free space files? */
+				(GET_MAJOR_VERSION(old_cluster.major_version) >= 804) ?
+				"--analyze-only" : "--analyze", ECHO_QUOTE);
+	}
+	else
+	{
+		fprintf(script, "echo %s    \"%s/vacuumdb\" %s-j %d --all %s%s\n", ECHO_QUOTE,
+				new_cluster.bindir, user_specification.data, user_opts.jobs,
+				/* Did we copy the free space files? */
+				(GET_MAJOR_VERSION(old_cluster.major_version) >= 804) ?
+				"--analyze-only" : "--analyze", ECHO_QUOTE);
+	}
 	fprintf(script, "echo%s\n\n", ECHO_BLANK);
 
-	fprintf(script, "\"%s/vacuumdb\" %s--all --analyze-in-stages\n",
+#ifndef WIN32
+	fprintf(script, "\"%s/vacuumdb\" %s$VACUUMDB_OPTS --all --analyze-in-stages\n",
 			new_cluster.bindir, user_specification.data);
+#else
+	fprintf(script, "\"%s\\vacuumdb.exe\" %s%%VACUUMDB_OPTS%% --all --analyze-in-stages\n",
+			new_cluster.bindir, user_specification.data);
+#endif
 	/* Did we copy the free space files? */
 	if (GET_MAJOR_VERSION(old_cluster.major_version) < 804)
 		fprintf(script, "\"%s/vacuumdb\" %s--all\n", new_cluster.bindir,
-- 
2.17.2


--------------D73A50015A30B21EBFE6291A--