From: Justin Pryzby <pryzby@telsasoft.com>
To: Jeff Janes <jeff.janes@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Jesper Pedersen <jesper.pedersen@redhat.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Jamison, Kirk <k.jamison@jp.fujitsu.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: fabriziomello@gmail.com <fabriziomello@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org <pgsql-hackers@lists.postgresql.org>
Cc: Bruce Momjian <bruce@momjian.us>
Subject: Re: pg_upgrade: Pass -j down to vacuumdb
Date: Wed, 3 Apr 2019 16:24:34 -0500
Message-ID: <20190403212434.GY17544@telsasoft.com> (raw)
In-Reply-To: <CAMkU=1w2Yuvz+xbCU+42pT2U3GCE0xmkaSDXCgM9CR3bt=5Crw@mail.gmail.com>
References: <201902010958.cf2cbi3d2ule@alvherre.pgsql>
<bef52417-2416-cb97-afce-a351601eab43@redhat.com>
<D09B13F772D2274BB348A310EE3027C64659E9@g01jpexmbkw24>
<CA+TgmobMGTfkMY-isz4pLw_bfywVOqTUDcW+obeAfYSeTBJM7A@mail.gmail.com>
<03845f77-49e7-4569-7062-8ee1fb460ec2@redhat.com>
<30470.1553551070@sss.pgh.pa.us>
<2c054085-0b68-c049-d6d8-8b74852b4c73@2ndquadrant.com>
<CAMkU=1zcOfThON4p00VTWEO1xdY6R8Xbh9KSt9pZkLHq0gdE=A@mail.gmail.com>
<da27737d-9f3e-6798-c999-4adaa22c47f5@2ndquadrant.com>
<CAMkU=1w2Yuvz+xbCU+42pT2U3GCE0xmkaSDXCgM9CR3bt=5Crw@mail.gmail.com>
On Wed, Apr 03, 2019 at 04:42:14PM -0400, Jeff Janes wrote:
> So maybe the first stage could
> be run by pg_upgrade itself, while the new server is still running on a
> linux socket in a private directory.
I think that would take too long. It would be less of an issue if there was
feedback/progress from pg_upgrade during the analyze.
For our upgrades (which typically take ~15min but several customers take up to
~60min), I only analyze base tables (essentially, those which are neither
parents nor children), then start services, then ANALYZE with default stats
target. I would want to avoid delaying services restart for more than another
(say) 5 minutes, and I would want to avoid even that unless there was a
progress report indicating that it's projected to take only a few more minutes.
I just did a test on one of our large-but-not-huge customers. With
stats_target=1, analyzing a 145GB partitioned table looks like it'll take
perhaps an hour; they have ~1TB data, so delaying services during ANALYZE would
nullify the utility of pg_upgrade. I can restore the essential tables from
backup in 15-30 minutes.
It might be fine if pg_upgrade took an option which enabled analyze, perhaps
instead of outputting analyze_new_cluster.sh. But actually, a problem with
*that* is that currently pg_upgrade avoids starting the new cluster. That
seems to be deliberate, since, with --link, that's an irreversible operation:
it's unsafe to start the old cluster afterwards.
Tangent: I have a queued mail from ~15 months ago wherein I proposed adding to
pg_upgrade an option to remove the old data dir (or probably only the files
associated with known relations). I realized at the time that would be pretty
scary without having first verified that the new cluster at least starts. I'm
not sure how good an idea that is, but --startnewcluster would be needed there,
too.
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, jeff.janes@gmail.com, peter.eisentraut@2ndquadrant.com, tgl@sss.pgh.pa.us, jesper.pedersen@redhat.com, robertmhaas@gmail.com, k.jamison@jp.fujitsu.com, alvherre@2ndquadrant.com, michael@paquier.xyz, fabriziomello@gmail.com, pgsql-hackers@lists.postgresql.org, bruce@momjian.us
Subject: Re: pg_upgrade: Pass -j down to vacuumdb
In-Reply-To: <20190403212434.GY17544@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox