Date: Thu, 14 Sep 2017 21:21:25 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Michael Paquier <michael.paquier@gmail.com>
Cc: Andreas Joseph Krogh <andreas@visena.com>,
	Bruce Momjian <bruce@momjian.us>,
	Robert Haas <robertmhaas@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: Clarification in pg10's pgupgrade.html step 10
 (upgrading standby servers)
Message-ID: <20170915012124.GI4628@tamriel.snowman.net>
References: <20170914191356.GC6595@momjian.us>
 <VisenaEmail.db.e5ceedc56ec1471.15e82123988@tc7-visena>
 <CAB7nPqS9pL7qwRu8ku8byvYYZOEHHXzUAgSJRbP7AENtHT8E_A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="KODr0wp1DpddtCAq"
Content-Disposition: inline
In-Reply-To: 
 <CAB7nPqS9pL7qwRu8ku8byvYYZOEHHXzUAgSJRbP7AENtHT8E_A@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org


--KODr0wp1DpddtCAq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Michael, all,

* Michael Paquier (michael.paquier@gmail.com) wrote:
> On Fri, Sep 15, 2017 at 8:23 AM, Andreas Joseph Krogh
> <andreas@visena.com> wrote:
> > I tested upgrading from 9.6 to 10 now, using pg_upgrade, and pg_upgrade
> > creates the new data-dir with pg_wal "in it" (just like regular initdb)=
, so
> > pg_upgrade seems not to care about where the old version's pg_xlog was.=
 You
> > have to move (by symlinking) pg_wal to a separate location manually *af=
ter*
> > running pg_upgrade on the master.
>=20
> That's true, this should definitely be mentioned in the documentation.

Uh, this seems like something that should be *fixed*, not documented.
That initdb accepts an alternative location for pg_xlog/pg_wal now
raises that to a first-class feature, in my view, and other tools should
recognize the case and deal with it correctly.

Of course, that having been said, there's some technical challenges
there.  One being that we don't really want to mess with the old
cluster's WAL during pg_upgrade.  Frustratingly, we have a way to deal
with that case today for tablespaces, it was just never applied to WAL
when we started allowing WAL to be stored elsewhere (formally).  That
seems like it was a mistake to me.

Then again, the more I think about this, the more I wonder about putting
more-or-less everything in PGDATA into per-catalog-version directories
and making everything self-contained.  Part of the ugly bit with the
rsync is exactly that we have to go up an extra level for the data
directories themselves, and users can actually put them anywhere so
there might not even *be* a common parent directory to use.

> An improvement could be done as well here for pg_upgrade: when using
> --link, the new PGDATA created could consider as well the source
> pg_wal and create a link to it, and then clean up its contents. I am
> not completely sure if this would be worth doing as people are likely
> used to the current flow though. The documentation needs to outline
> the matter at least.

No, one of the baseline requirements of pg_upgrade is to *not* screw
with the existing cluster.  Removing its WAL or "cleaning it up"
definitely seems like it's violating that principle.

I tend to agree that it'd be good for the documentation to address this,
but this is all really getting to be a bit much for a manpage to be able
to handle, I think..

Thaks!

Stephen

--KODr0wp1DpddtCAq
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJZuysUAAoJEO1sijiDR2RVxH4P/in/MkNnBxk7dMJjLti/6dHL
IvGTiTOzChBXjjqFnvR3ao/C55cwxI2rwgRSONL2xV+DcyZwdd3YuNqf2+CpQdq9
OVWG/klszWWzyF1m6nOX9dulH8Dr9dALB6WubzxROWf3+Mi2V7kQJzQsAh2Tgy41
rP5Xx7teMpmxtfHWyclFaFGB4zhOrQWhn12fFXL1goMylCmHckDsvUPmMunHJyaL
Bb6wfhONvBW9paEduJ4C6KWE1fOvrcbe9PaPZShXcoOKvXW8xzg32wDSTlWHjfyV
RtDhNhxVyO3Bp1/qbUf7TJshM2yPMWc4kqw/3RHLdpgYXqYjOeTLOXCVDoOMzXkB
ChEKn4EgnmE9ibqq5S1GlmLroLezsXr++Y2J+K5/aA73rd/H+dgWQdwXNqx7DwQZ
ZCAnJ0LwRLY/LK5gHsoiIP6WAdEkcom3zPGv/EdETy/9Jpgx4OV7b8grJNt6cxUb
CKiUK/PQO0AKBQI3mZqBDTsgSfvoeRHl0Z6G94HLHkCqbszPf4nwNyAg++m8amJk
yZIrbnERI2eu5gNc9LhddRbYbLBj+7aY59HxjUx3x2XZPS1RrmfQHS3mLyJt3Kvy
jKFv8atVOw3/40YttCbsJlBjSXmMdOX8sVQ7xpWoK0Sl2L9YlORmV3X/1CKDKc74
bBREsUMhPY5xRMy+6YXH
=tTQC
-----END PGP SIGNATURE-----

--KODr0wp1DpddtCAq--