Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dtKln-0003Hk-8N for pgsql-hackers@arkaria.postgresql.org; Sat, 16 Sep 2017 21:36:59 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1dtKlm-00088D-Ea for pgsql-hackers@arkaria.postgresql.org; Sat, 16 Sep 2017 21:36:58 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1dtKll-000874-1y for pgsql-hackers@postgresql.org; Sat, 16 Sep 2017 21:36:57 +0000 Received: from post.visena.com ([46.226.10.50]) by makus.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1dtKld-0003FF-UR for pgsql-hackers@postgresql.org; Sat, 16 Sep 2017 21:36:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=visena.com; s=20141101.wh; h=Content-Type:MIME-Version:Subject:In-Reply-To:Message-ID:Cc:To:From:Date; bh=TqocSOAfaXUQo9kpujvRMShEuFD2ALrcIBY0Hb84H+0=; b=AhWRbiwyz0Q4BSxqzNkvQMUYJKaZjelvtNc29n0cjvVRf1JcKb0/h03CxkXrnYgulnM08HV7IS1wjp3Fm4gA6FeqArYX6VKifd+UR4kpHClhZo3gUFzkMHhqi59oSIHTtbBOdjiy7MIELGFT50dlmMYTqAtMpGnv3XRcLDQep04=; Received: from [10.0.1.10] (helo=tc7-visena.wh.internal.visena.com) by post.visena.com with esmtp (Exim 4.82) (envelope-from ) id 1dtKlX-0008I3-Rf; Sat, 16 Sep 2017 23:36:45 +0200 Received: from localhost ([127.0.0.1] helo=tc7-visena.wh.internal.visena.com) by tc7-visena.wh.internal.visena.com with esmtp (Exim 4.86_2) (envelope-from ) id 1dtKlU-0002aK-9o; Sat, 16 Sep 2017 23:36:40 +0200 Date: Sat, 16 Sep 2017 23:36:40 +0200 (CEST) From: Andreas Joseph Krogh To: Bruce Momjian Cc: Michael Paquier , Robert Haas , "pgsql-hackers@postgresql.org" , Stephen Frost Message-ID: In-Reply-To: <20170916163451.GE9874@momjian.us> Subject: Re: Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers) MIME-Version: 1.0 X-Mailer: Visena Mail 2.1.0-SNAPSHOT X-Spam-Score: -1.0 X-Spam-Report: SpamAssasin (score=-1.0, required 5.0 ALL_TRUSTED=-1,HTML_MESSAGE=0.001) Content-Type: multipart/alternative; boundary="----=_Part_1557_1560037620.1505597800175" List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org ------=_Part_1555_1670095080.1505597800164 Content-Type: multipart/related; boundary="----=_Part_1556_1670964866.1505597800164" ------=_Part_1556_1670964866.1505597800164 Content-Type: multipart/alternative; boundary="----=_Part_1557_1560037620.1505597800175" ------=_Part_1557_1560037620.1505597800175 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable P=C3=A5 l=C3=B8rdag 16. september 2017 kl. 18:34:51, skrev Bruce Momjian >: On Sat, Sep 16, 2017 at 06:11:17PM +0200, Andreas Joseph Krogh wrote: > I'm a little unsure what scenario we're trying to describe here. Copying= the > pg_wal separately (for which there's no need optimizing for)=C2=A0is onl= y needed=20 if > you've moved it out of $PGDATA=C2=A0_after_ running pg_upgrade, IIUC. So= , I=20 think it No.=C2=A0 If you ran initdb with --waldir on the new primary, you will cre= ate a symbolic link in the PGDATA directory, and a directory outside of PGDATA for storing the WAL.=C2=A0 When you run rsync on the new primary PGDATA directory, you will copy the symlink in the PGDATA directory, but it will point to probably nothing on the standby. =C2=A0 =C2=A0 The misunderstanding here comes from the fact that I used pg_upgradecluster= =20 like this: pg_upgradecluster --method=3Dupgrade --link 9.6 main =C2=A0 and it didn't issue initdb with=C2=A0--waldir on the new cluster (because= =20 pg_upgradecluster issues initdb for you), so pg_wal ended up in $PGDIR beca= use=20 pg_upgradecluster didn't figure out the old cluster was initialized with=20 --xlogdir. This is why I thought i made sense mentioning that one had to mo= ve=20 pg_wal manually. =C2=A0 I know it's debian-stuff, but maybe it's worth mentioning pg_upgradecluster= =20 somewhere and recommend not using it? It seems to start the new cluster=20 automatically and when upgrading standbys section 10 tells you not to do th= at. =C2=A0 =C2=A0 > should be clearly pointed out that copying pg_wal is only needed in those > cases, and that it can be done with whatever network-copying procedure= =20 you're > familiar with, ie. scp/rsync. This step is not similar to the steps requ= ired > for copying tablespaces outside $PGDATA, so it's worth documenting=20 explicitly. > Maybe also telling users to ensure the synlink (in $PGDATA) to pg_wal on > standby points to pg_wal. Why tell them new instructions when the rsync instructions work fine? What is the value? =C2=A0 The rsync instructions mentioned in 10.F all address the --link scenario an= d=20 use "--delete --hard-links --size-only", and "merge 2 source-dirs into one"= ,=20 which isn't relevant when copying pg_wal. =C2=A0 This sentence: "If you have relocated=C2=A0pg_wal=C2=A0outside the data directories,=C2=A0= rsync=C2=A0must be run=20 on those directories too." implies one must follow the rsync pattern elsewhere in 10.F, which isn't=20 really true. Maybe re-wording it to: "If you have relocated=C2=A0pg_wal=C2=A0outside the data directories you mu= st copy it=20 over to the new standby,=C2=A0and ensure the symlink from $PGDATA points to= it" helps? -- Andreas Joseph Krogh ------=_Part_1557_1560037620.1505597800175 Content-Type: text/html;charset=UTF-8 Content-Transfer-Encoding: quoted-printable
P=C3=A5 l=C3=B8rdag 16. september 2017 kl. 18:34:51, skrev Bruce Momji= an <bruce@momjian.us>:
On = Sat, Sep 16, 2017 at 06:11:17PM +0200, Andreas Joseph Krogh wrote:
> I'm a little unsure what scenario we're trying to describe here. Copyi= ng the
> pg_wal separately (for which there's no need optimizing for)=C2=A0is o= nly needed if
> you've moved it out of $PGDATA=C2=A0_after_ running pg_upgrade, IIUC. = So, I think it

No.=C2=A0 If you ran initdb with --waldir on the new primary, you will crea= te
a symbolic link in the PGDATA directory, and a directory outside of
PGDATA for storing the WAL.=C2=A0 When you run rsync on the new primary
PGDATA directory, you will copy the symlink in the PGDATA directory, but it will point to probably nothing on the standby.
=C2= =A0
=C2= =A0
The= misunderstanding here comes from the fact that I used pg_upgradecluster li= ke this:
pg_= upgradecluster --method=3Dupgrade --link 9.6 main
=C2=A0
and= it didn't issue initdb with=C2=A0--waldir on the new cluster (because pg_u= pgradecluster issues initdb for you), so pg_wal ended up in $PGDIR because = pg_upgradecluster didn't figure out the old cluster was initialized with --= xlogdir. This is why I thought i made sense mentioning that one had to move= pg_wal manually.
=C2= =A0
I k= now it's debian-stuff, but maybe it's worth mentioning pg_upgradecluster so= mewhere and recommend not using it? It seems to start the new cluster autom= atically and when upgrading standbys section 10 tells you not to do that.
=C2= =A0
=C2= =A0
>= ; should be clearly pointed out that copying pg_wal is only needed in those=
> cases, and that it can be done with whatever network-copying procedure= you're
> familiar with, ie. scp/rsync. This step is not similar to the steps re= quired
> for copying tablespaces outside $PGDATA, so it's worth documenting exp= licitly.
> Maybe also telling users to ensure the synlink (in $PGDATA) to pg_wal = on
> standby points to pg_wal.

Why tell them new instructions when the rsync instructions work fine?
What is the value?
=C2=A0
The rsync instructions mentioned in 10.F all address the --link scenar= io and use "--delete --hard-links= --size-only", and "merge 2 source-dirs into one", wh= ich isn't relevant when copying pg_wal.
=C2=A0
This sentence:
"If you have relocated=C2=A0pg_wal=C2=A0outside the data directories,=C2=A0rsync= =C2=A0must be run on those directories too."
implies one must follow the rsync pattern elsewhere in 10.F, which isn= 't really true. Maybe re-wording it to:
"If you have relocated=C2=A0pg_wal=C2=A0outside the data directories you must copy it over to the new standb= y,=C2=A0and ensure the symlink from $PGDATA points to it"
helps?

--
Andreas Joseph Krogh
------=_Part_1557_1560037620.1505597800175-- ------=_Part_1556_1670964866.1505597800164-- ------=_Part_1555_1670095080.1505597800164--