Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jlyYo-00083o-Ga for pgsql-bugs@arkaria.postgresql.org; Thu, 18 Jun 2020 17:42:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jlyYn-0000Ti-8f for pgsql-bugs@arkaria.postgresql.org; Thu, 18 Jun 2020 17:42:45 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jlyYn-0000Tb-2a for pgsql-bugs@lists.postgresql.org; Thu, 18 Jun 2020 17:42:45 +0000 Received: from tamriel.snowman.net ([2001:470:e38f::11]) by makus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jlyYk-0001ZS-Fr for pgsql-bugs@lists.postgresql.org; Thu, 18 Jun 2020 17:42:43 +0000 Received: by tamriel.snowman.net (Postfix, from userid 1000) id C17395F7A0; Thu, 18 Jun 2020 13:42:41 -0400 (EDT) Date: Thu, 18 Jun 2020 13:42:41 -0400 From: Stephen Frost To: Bruce Momjian Cc: Tom Lane , Jeff Janes , Michael Paquier , ram.maurya@lavainternational.in, pgsql-bugs Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match Message-ID: <20200618174241.GD6680@tamriel.snowman.net> References: <16497-38bb1f1bcec44964@postgresql.org> <20200618080509.GH118592@paquier.xyz> <1549966.1592493410@sss.pgh.pa.us> <20200618152911.GF7349@momjian.us> <20200618154152.GB6680@tamriel.snowman.net> <20200618154854.GG7349@momjian.us> <20200618155537.GC6680@tamriel.snowman.net> <20200618163615.GI7349@momjian.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="QTrnGtbQvXh53f+1" Content-Disposition: inline In-Reply-To: <20200618163615.GI7349@momjian.us> User-Agent: Mutt/1.5.24 (2015-08-30) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --QTrnGtbQvXh53f+1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Greetings, * Bruce Momjian (bruce@momjian.us) wrote: > On Thu, Jun 18, 2020 at 11:55:37AM -0400, Stephen Frost wrote: > > * Bruce Momjian (bruce@momjian.us) wrote: > > > On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote: > > > > Sure, most options to initdb need to be the same between the old cl= uster > > > > and the new cluster, but this specific option doesn't have to be, s= ince > > > > we require that it's a cleanly shut down cluster, so why are we > > > > complaining about it if it's different..? > > >=20 > > > Did you not read my previous email that we might have added this so we > > > can upgrade replicas? > >=20 > > I don't see how this option is related to dealing with replicas..? > >=20 > > > I am sure I can dig out the commit that added > > > this and find the original cause, but running pg_upgrade on replicas = is > > > enough of a reason to me. > >=20 > > pg_upgrade can't be run on replicas, so I don't understand what you're > > referring to here.. The 'upgrading replicas' process that involves > > rsync also requires everything to have been shut down cleanly. >=20 > Oh, I forgot we copy the WAL from the primary and don't run initdb on > the standbys, so it might work, but looking at the pg_upgrade code, I > see: >=20 > /* now reset the wal archives in the new cluster */ > prep_status("Resetting WAL archives"); > exec_prog(UTILITY_LOG_FILE, NULL, true, true, > /* use timeline 1 to match controldata and no WAL history file */ > "\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bind= ir, > old_cluster.controldata.nextxlogfile + 8, > new_cluster.pgdata); >=20 > So, while we don't copy over the old WAL, we do assume the WAL will be > at the same segment from the old cluster. I think this is because of > the LSN stored on user data pages. pg_resetwal doesn't seem to care > about that, so maybe pg_upgrade doesn't need to either. I don't know. Yeah, this would need to be adjusted, since we're passing in an actual WAL filename there rather than an LSN. > > > Yeah, we could add a flag to pg_upgrade to > > > allow this if you are not upgrading replicas, but why bother? It mig= ht > > > even work if you create the new replicas with the same WAL segment si= ze, > > > but why add complexity for pg_upgrade, which is already complex enoug= h. > >=20 > > Users already have to deal with various options that need to be > > configured to match up between the primary and replicas, so this really > > seems like it's entirely independent of pg_upgrade and isn't something > > pg_upgrade needs to be worrying about.. >=20 > Do you know why we require this step? >=20 > https://www.postgresql.org/docs/12/pgupgrade.html > =09 > Also, change wal_level to replica in the postgresql.conf file on > the new primary cluster. Well, we'll need wal_level to be at least replica if we're going to have replicas streaming from the primary.. > The other modes don't work? I see this C comment: >=20 > * We unconditionally start/stop the new server because pg_resetwal -= o set > * wal_level to 'minimum'. If the user is upgrading standby servers = using > * the rsync instructions, they will need pg_upgrade to write its fin= al > * WAL record showing wal_level as 'replica'. Yes, the new system needs to have replica (or higher) as the wal_level, but that doesn't depend on the WAL segment size. Thanks, Stephen --QTrnGtbQvXh53f+1 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJe66eRAAoJEO1sijiDR2RVkGcQAKluZViBcSgm7zik/8H0dkLF 62sDgXOk/dCu6/D73ukI6cBilus3VExdqGEwwW5PvCdRfawTCFNVjo6vKKNKMZzK KMWEka3JkeGIowYUcoiprK4k2+7FzPpVTVDXlkpjjQljwwb7WYAR0An0K94/r3Wh xXoJdo95FTX1oOLVWyc2ENiuWaSIjOby6LutbyI3dAitNVqT5XL0+PULtRK7CfuR U7feQS/GZZlXta0lJo26Jg/q+lSbF/3OazUfUDZZNcDnL/64PaG2lY6jIrE7/oDx 1QLJfJqm+wpJ47IwkWSrt1sFVKCm2IJx5a/S9W4R3vslAGAfexIOmgI+ZTitIg9x hCU1keQ2sa7Xr3WQ8gwSLKar/rhECIb8KglsaJEbSv+yRzfjBJlb7+RtzDyMV6nG xoZRMcR5e/y8OpxZH3cTyx/wtqQ80RaufVwYI8fbJZisiWE9JUV/GwFZgxH9Us9C u2Jwj17MRLByaeBBwztBNLorUHpVPypWZhcmRzMvpzmnwWSSEEF9KniiKHR/LEWF 5mNQBsyWznAY5SaGTTHAtdzV++D4wLD0e7y6c+RTMrEsoK7lzsL2a7yCVos/Q+vD HDI1YHsnG4dOeaKxiWweDAi4PeX0dKYhd8JEDImSLERKYHDTS3vMZnaORdX1nBDl ud2iu9z4qIqlHc7Cpxip =XxI2 -----END PGP SIGNATURE----- --QTrnGtbQvXh53f+1--