Date: Sat, 16 Sep 2017 18:11:17 +0200 (CEST)
From: Andreas Joseph Krogh <andreas@visena.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Michael Paquier <michael.paquier@gmail.com>,
	Robert Haas <robertmhaas@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>,
	Stephen Frost <sfrost@snowman.net>
Message-ID: <VisenaEmail.167.3c69dc1e033531bd.15e8b6a944d@tc7-visena>
In-Reply-To: <20170916152414.GC9874@momjian.us>
Subject: Re: Clarification in pg10's pgupgrade.html
 step 10 (upgrading standby servers)
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_1552_1481587675.1505578277779"
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

------=_Part_1550_1271539427.1505578277765
Content-Type: multipart/related; 
	boundary="----=_Part_1551_321509669.1505578277765"

------=_Part_1551_321509669.1505578277765
Content-Type: multipart/alternative; 
	boundary="----=_Part_1552_1481587675.1505578277779"

------=_Part_1552_1481587675.1505578277779
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

P=C3=A5 l=C3=B8rdag 16. september 2017 kl. 17:24:14, skrev Bruce Momjian <b=
ruce@momjian.us
 <mailto:bruce@momjian.us>>:
On Fri, Sep 15, 2017 at 01:23:45AM +0200, Andreas Joseph Krogh wrote:
 > I tested upgrading from 9.6 to 10=C2=A0now, using pg_upgrade, and pg_upg=
rade=20
creates
 > the new data-dir with pg_wal "in it" (just like regular initdb), so=20
pg_upgrade
 > seems not to care about where the old version's pg_xlog was. You have to=
=20
move
 > (by symlinking)=C2=A0pg_wal to a separate location manually *after* runn=
ing
 > pg_upgrade on the master. No special handling is needed when rsync'ing i=
t=20
over
 > to the standby, so it doesn't need any=C2=A0--hard-links or --size-only,=
 correct?
 > =C2=A0
 > Given the path,=C2=A0on the upgraded=20
primary,=C2=A0to=C2=A0pg_wal=C2=A0is=C2=A0/custom/path/to/pg_wal,
 > the rsync command will be:
 > =C2=A0
 > rsync --archive --delete /custom/path/to/pg_wal standby.example.com:/cus=
tom/
 > path/to/pg_wal
 > =C2=A0
 > I think it's useful to mention this to eliminate any doubt.
 > =C2=A0
 > I also think it's worth mentioning that you have to manually move pg_wal=
 to=20
a
 > custom location after running pg_upgrade as it will not preserve/use the=
 old
 > path.

 Thinking some more, you are right that there is no need to rsync the
 _old_ primary WAL directory since it is the same on the standby old WAL
 directory, and there are no links between the old and new WAL
 directories, so you could just do the new one, or just copy it and not
 even use rsync.

 However, I think it adds complexity to try to optimize the copy of the
 WAL files and we are better just requiring them to use the same steps
 for WAL copy that they _must_ use for the data directory and
 tablespaces because of the links between old and new files there.

 Agreed?
=C2=A0
I'm a little unsure what scenario we're trying to describe here. Copying th=
e=20
pg_wal separately (for which there's no need optimizing for)=C2=A0is only n=
eeded if=20
you've moved it out of $PGDATA=C2=A0_after_ running pg_upgrade, IIUC. So, I=
 think it=20
should be clearly pointed out that copying pg_wal is only needed in those=
=20
cases, and that it can be done with whatever network-copying procedure you'=
re=20
familiar with, ie. scp/rsync. This step is not similar to the steps require=
d=20
for copying tablespaces outside $PGDATA, so it's worth documenting explicit=
ly.=20
Maybe also telling users to ensure the synlink (in $PGDATA) to pg_wal on=20
standby points to pg_wal.
=C2=A0
--
 Andreas Joseph Krogh
=C2=A0


------=_Part_1552_1481587675.1505578277779
Content-Type: text/html;charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div>P=C3=A5 l=C3=B8rdag 16. september 2017 kl. 17:24:14, skrev Bruce Momji=
an &lt;<a href=3D"mailto:bruce@momjian.us">bruce@momjian.us</a>&gt;:</div>

<blockquote style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt=
 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style=3D"display:inline; font-family: monospace; font-size: 12px;">On =
Fri, Sep 15, 2017 at 01:23:45AM +0200, Andreas Joseph Krogh wrote:<br>
&gt; I tested upgrading from 9.6 to 10=C2=A0now, using pg_upgrade, and pg_u=
pgrade creates<br>
&gt; the new data-dir with pg_wal &quot;in it&quot; (just like regular init=
db), so pg_upgrade<br>
&gt; seems not to care about where the old version's pg_xlog was. You have =
to move<br>
&gt; (by symlinking)=C2=A0pg_wal to a separate location manually *after* ru=
nning<br>
&gt; pg_upgrade on the master. No special handling is needed when rsync'ing=
 it over<br>
&gt; to the standby, so it doesn't need any=C2=A0--hard-links or --size-onl=
y, correct?<br>
&gt; =C2=A0<br>
&gt; Given the path,=C2=A0on the upgraded primary,=C2=A0to=C2=A0pg_wal=C2=
=A0is=C2=A0/custom/path/to/pg_wal,<br>
&gt; the rsync command will be:<br>
&gt; =C2=A0<br>
&gt; rsync --archive --delete /custom/path/to/pg_wal standby.example.com:/c=
ustom/<br>
&gt; path/to/pg_wal<br>
&gt; =C2=A0<br>
&gt; I think it's useful to mention this to eliminate any doubt.<br>
&gt; =C2=A0<br>
&gt; I also think it's worth mentioning that you have to manually move pg_w=
al to a<br>
&gt; custom location after running pg_upgrade as it will not preserve/use t=
he old<br>
&gt; path.<br>
<br>
Thinking some more, you are right that there is no need to rsync the<br>
_old_ primary WAL directory since it is the same on the standby old WAL<br>
directory, and there are no links between the old and new WAL<br>
directories, so you could just do the new one, or just copy it and not<br>
even use rsync.<br>
<br>
However, I think it adds complexity to try to optimize the copy of the<br>
WAL files and we are better just requiring them to use the same steps<br>
for WAL copy that they _must_ use for the data directory and<br>
tablespaces because of the links between old and new files there.<br>
<br>
Agreed?</div>
</blockquote>

<div>=C2=A0</div>

<div>I'm a little unsure what scenario we're trying to describe here. Copyi=
ng the pg_wal separately (for which there's no need optimizing for)=C2=A0is=
 only needed if you've moved it out of $PGDATA=C2=A0_after_ running pg_upgr=
ade, IIUC. So, I think it should be clearly pointed out that copying pg_wal=
 is only needed in those cases, and that it can be done with whatever netwo=
rk-copying procedure you're familiar with, ie. scp/rsync. This step is not =
similar to the steps required for copying tablespaces outside $PGDATA, so i=
t's worth documenting explicitly. Maybe also telling users to ensure the sy=
nlink (in $PGDATA) to pg_wal on standby points to pg_wal.</div>

<div>=C2=A0</div>

<div class=3D"origo-email-signature">
<div>
<div>--<br>
Andreas Joseph Krogh</div>

<div>=C2=A0</div>
</div>
</div>
------=_Part_1552_1481587675.1505578277779--

------=_Part_1551_321509669.1505578277765--

------=_Part_1550_1271539427.1505578277765--