Date: Tue, 5 Apr 2022 13:10:38 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Laurenz Albe <laurenz.albe@cybertec.at>,
	pgsql-docs@lists.postgresql.org
Subject: Re: Improve documentation for pg_upgrade, standbys and rsync
Message-ID: <20220405171038.GU10577@tamriel.snowman.net>
References: <22f129004bb66cd91e1dfd3345a9787f5039f3ae.camel@cybertec.at>
 <20210519143135.GI20766@tamriel.snowman.net>
 <c176f23cdde4d4ff0010b5e7a517c1b371500279.camel@cybertec.at>
 <20210716131744.GA20766@tamriel.snowman.net>
 <a1a9cf4ebc8c674f4165af376c522f52ab02a3f6.camel@cybertec.at>
 <20210726191126.GW20766@tamriel.snowman.net>
 <CA+TgmobJBezsjg1afKK3jQCz75RX_Fankf65os=0Cuh93oFjPQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="N9nS8Yqueyt4m6+M"
Content-Disposition: inline
In-Reply-To: 
 <CA+TgmobJBezsjg1afKK3jQCz75RX_Fankf65os=0Cuh93oFjPQ@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20220405171038.GU10577%40tamriel.snowman.net>
Precedence: bulk


--N9nS8Yqueyt4m6+M
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Robert Haas (robertmhaas@gmail.com) wrote:
> On Mon, Jul 26, 2021 at 3:11 PM Stephen Frost <sfrost@snowman.net> wrote:
> > * Laurenz Albe (laurenz.albe@cybertec.at) wrote:
> > > Thanks for looking at this!
> >
> > Sure.  Thanks for working on it!
>=20
> Stephen, do you intend to do something about this patch in terms of
> getting it committed? You're the only reviewer but haven't responded
> to the thread for more than 5 months.

I tried to be clear in the last email on the thread, the one which you
just responded to, here:

* Stephen Frost (sfrost@snowman.net) wrote:
> This, of course, all comes back to the original complaint I had about
> documenting this approach, which is that these things should only be
> done by someone extremely familiar with the PG codebase, until and
> unless we write an actual tool to do this.

To be more explicit though- we should write a tool to do this.  We
shouldn't try to document a way to do it because it's hard to get right.
While rsync is very capable, what's needed to really do this goes beyond
what we could reasonably put into any rsync command or really even into
a documented procedure.  I get that we already have it documented (and
I'll note that doing so was against my recommendation..) and that some
folks (likely those who follow this mailing list) have had success using
it, but I'd really rather we just take it out and put it on a wiki
somewhere as a "we need a tool that does this stuff" and hope that
someone finds time to write one.

> I don't feel that I know this area of the documentation well enough to
> feel comfortable passing judgement on whether this change is an
> improvement or not. However I do feel somewhat uncomfortable with
> this:
>=20
> -   <step>
> -    <title>Prepare for standby server upgrades</title>
> -
> -    <para>
> -     If you are upgrading standby servers using methods outlined in
> section <xref
> -     linkend=3D"pgupgrade-step-replicas"/>, verify that the old standby
> -     servers are caught up by running <application>pg_controldata</appli=
cation>
> -     against the old primary and standby clusters.  Verify that the
> -     <quote>Latest checkpoint location</quote> values match in all clust=
ers.
> -     (There will be a mismatch if old standby servers were shut down
> -     before the old primary or if the old standby servers are still runn=
ing.)
> -     Also, make sure <varname>wal_level</varname> is not set to
> -     <literal>minimal</literal> in the
> <filename>postgresql.conf</filename> file on the
> -     new primary cluster.
> -    </para>
> -   </step>
>=20
> Right now, we say that you should stop the standby servers and then
> prepared for standby server upgrades. With this patch, we say that you
> should first prepare for standby server upgrades, and then stop the
> standby servers. But the last part of the text about preparing for
> standby server upgrades now mentions things to be done after carrying
> out the next step where the servers are actually stopped. That seems
> confusing. Perhaps we need two separate steps here, one to be
> performed before stopping both servers and the other after.

It should really be both- things to do on the primary ahead of time
(truncate all unlogged tables, make sure there aren't any orphaned
temporary tables, etc), and then things to do on the replicas after
shutting the primary down (basically, make sure they are fully caught up
with where the primary was at shutdown).  I tried to explain that in my
prior email but perhaps didn't do a very good job.

> Also, let me express my general terror at the idea of anyone actually
> using this procedure.

I mean, yeah, I agree.

Thanks,

Stephen

--N9nS8Yqueyt4m6+M
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJiTHgNAAoJEO1sijiDR2RVBr4QAJ5lbMvc5uC1gCz7EcJbcQCJ
pddHFIabDp/M76Jy6VruQ80PI2zVB99D/MCfHD7cQ3SR8Kou2JAiXZ8HEpBZw6VL
VqFNaQN/Ob4T+sZJh81c4mHJ29SlAOjqLsl7oCHRyAcsSNGdzGCEYTkmUKcoDxJz
XOKwSbF2M6rv3RGsGBxA6TLNmyXHfRY1npkhSZKCN5LJH3geysyUQwWPff/1nfAL
/uXo+8K+hb7rjM8/SheAVOXaoNLVYLfq4yri765NsHCBMZhM1SC78QynB19NdqjH
guVqY7fbvKOkRAwqEkPMBfudvoLFcHhRzKtFMtfrZZK7jyaBwe+7eGBupN1/IpXp
EqUTRZOxbBiT/3DMSFpNdLW1RCG7I40DCtBnBx290+oZdNc2gKD9w40/jwJ7Z03z
BZPTr1Bmb6/felqCp48dnlwfXAitdyUN6nFC5PN0Phc36u7hDlQchYxfMMpQJt08
5YyzmbXYgIUbTwVevlV75tJUXcySkSTv2A/zlYd0Nw7vmiTODSTQ4KRAzYYqQXCX
3sAFI+tb+UOfTeB28q4IphAmMlk6QEdlSP0cQc2j9vOyq8LXFO4XS5CnGwrv3uUu
Om2g5/n6larYynsQTJS/I4cjj/5T/2StBUC05GNhqSZrsmjNzKjU/eC58tpFhhTT
8hhd3n29OjLnAvw0p6h+
=bOCH
-----END PGP SIGNATURE-----

--N9nS8Yqueyt4m6+M--