Date: Wed, 3 Apr 2019 16:24:34 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Jeff Janes <jeff.janes@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>,
	Tom Lane <tgl@sss.pgh.pa.us>,
	Jesper Pedersen <jesper.pedersen@redhat.com>,
	Robert Haas <robertmhaas@gmail.com>,
	"Jamison, Kirk" <k.jamison@jp.fujitsu.com>,
	Alvaro Herrera <alvherre@2ndquadrant.com>,
	Michael Paquier <michael@paquier.xyz>,
	"fabriziomello@gmail.com" <fabriziomello@gmail.com>,
	"pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
	Bruce Momjian <bruce@momjian.us>
Subject: Re: pg_upgrade: Pass -j down to vacuumdb
Message-ID: <20190403212434.GY17544@telsasoft.com>
References: <201902010958.cf2cbi3d2ule@alvherre.pgsql>
 <bef52417-2416-cb97-afce-a351601eab43@redhat.com>
 <D09B13F772D2274BB348A310EE3027C64659E9@g01jpexmbkw24>
 <CA+TgmobMGTfkMY-isz4pLw_bfywVOqTUDcW+obeAfYSeTBJM7A@mail.gmail.com>
 <03845f77-49e7-4569-7062-8ee1fb460ec2@redhat.com>
 <30470.1553551070@sss.pgh.pa.us>
 <2c054085-0b68-c049-d6d8-8b74852b4c73@2ndquadrant.com>
 <CAMkU=1zcOfThON4p00VTWEO1xdY6R8Xbh9KSt9pZkLHq0gdE=A@mail.gmail.com>
 <da27737d-9f3e-6798-c999-4adaa22c47f5@2ndquadrant.com>
 <CAMkU=1w2Yuvz+xbCU+42pT2U3GCE0xmkaSDXCgM9CR3bt=5Crw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAMkU=1w2Yuvz+xbCU+42pT2U3GCE0xmkaSDXCgM9CR3bt=5Crw@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Precedence: bulk

On Wed, Apr 03, 2019 at 04:42:14PM -0400, Jeff Janes wrote:
> So maybe the first stage could
> be run by pg_upgrade itself, while the new server is still running on a
> linux socket in a private directory.

I think that would take too long.  It would be less of an issue if there was
feedback/progress from pg_upgrade during the analyze.

For our upgrades (which typically take ~15min but several customers take up to
~60min), I only analyze base tables (essentially, those which are neither
parents nor children), then start services, then ANALYZE with default stats
target.  I would want to avoid delaying services restart for more than another
(say) 5 minutes, and I would want to avoid even that unless there was a
progress report indicating that it's projected to take only a few more minutes.

I just did a test on one of our large-but-not-huge customers.  With
stats_target=1, analyzing a 145GB partitioned table looks like it'll take
perhaps an hour; they have ~1TB data, so delaying services during ANALYZE would
nullify the utility of pg_upgrade.  I can restore the essential tables from
backup in 15-30 minutes.

It might be fine if pg_upgrade took an option which enabled analyze, perhaps
instead of outputting analyze_new_cluster.sh.  But actually, a problem with
*that* is that currently pg_upgrade avoids starting the new cluster.  That
seems to be deliberate, since, with --link, that's an irreversible operation:
it's unsafe to start the old cluster afterwards.

Tangent: I have a queued mail from ~15 months ago wherein I proposed adding to
pg_upgrade an option to remove the old data dir (or probably only the files
associated with known relations).  I realized at the time that would be pretty
scary without having first verified that the new cluster at least starts.  I'm
not sure how good an idea that is, but --startnewcluster would be needed there,
too.

Justin