Date: Tue, 12 Sep 2017 20:59:05 +0200 (CEST)
From: Andreas Joseph Krogh <andreas@visena.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Michael Paquier <michael.paquier@gmail.com>,
	Robert Haas <robertmhaas@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Message-ID: <VisenaEmail.66.44a2ed6a4130e92c.15e776c27e4@tc7-visena>
In-Reply-To: <20170912171922.GA3705@momjian.us>
Subject: Re: Clarification in pg10's pgupgrade.html step 10
 (upgrading standby servers)
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_442_17225077.1505242745653"
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

------=_Part_440_2037296846.1505242745623
Content-Type: multipart/related; 
	boundary="----=_Part_441_1847304949.1505242745624"

------=_Part_441_1847304949.1505242745624
Content-Type: multipart/alternative; 
	boundary="----=_Part_442_17225077.1505242745653"

------=_Part_442_17225077.1505242745653
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

P=C3=A5 tirsdag 12. september 2017 kl. 19:19:22, skrev Bruce Momjian <
bruce@momjian.us <mailto:bruce@momjian.us>>:
On Thu, Aug=C2=A0 3, 2017 at 11:37:32AM +0200, Michael Paquier wrote:
 > On Mon, Jul 31, 2017 at 6:13 PM, Robert Haas <robertmhaas@gmail.com> wro=
te:
 > > On Fri, Jul 28, 2017 at 10:35 AM, Andreas Joseph Krogh
 > > <andreas@visena.com> wrote:
 > >> I'm reading https://www.postgresql.org/docs/10/static/pgupgrade.html =
to=20
try
 > >> to understand how to upgrade standby-servers using pg_upgrade with pg=
10.
 > >>
 > >> The text in step 10 sais:
 > >> "You will not be running pg_upgrade on the standby servers, but rathe=
r
 > >> rsync", which to me sounds like rsync, in step 10-f, should be issued=
 on=20
the
 > >> standy servers. Is this the case? If so I don't understand how the=20
standby's
 > >> data is upgraded and what "remote_dir" is. If rsync is supposed to be=
=20
issued
 > >> on the primary then I think it should be explicitly mentioned, and st=
ep=20
10-f
 > >> should provide a clarer example with more detailed values for the
 > >> directory-structures involved.
 > >>
 > >> I really think section 10 needs improvement as I'm certainly not=20
comfortable
 > >> upgrading standbys following the existing procedure.
 > >
 > > Yeah, I don't understand it either, and I have never been convinced
 > > that there's any safe way to do it other than recloning the standbys
 > > from the upgraded master.
 >
 > Here are my 2c on the matter. 10-f means that the upgraded node may
 > have generated WAL with wal_level =3D minimal, which, at least it seems
 > to me, that we have a risk of having inconsistent data pages if only a
 > rsync is used on the old standbys. Like Robert, the flow we used in
 > the products I work on is to re-create standbys from scratch after the
 > upgrade using a fresh backup, with a VM cloning. An upgrade here is an
 > in-place process not only linked to Postgres, so standby VMs are made
 > of many services, some are being linked to Postgres. So this choice is
 > mainly decided by those dependencies, still it feels safer anyway.

 I have applied the attached doc patch back to 9.5 to clarify
 pg_upgrade's rsync instructions and explain how it works.

 Improvements?
=C2=A0
=C2=A0
Thanks, that certainly improves things.
But; I still find the rsync-command in f) confusing;
1. Why=C2=A0--size-only? From rsync manual: "skip files that match in size"=
, is=20
this safe??
2. Why is old_pgdata in the rsync-command, why is it needed to sync it?
=C2=A0
There are many ways to do/configure things it seems, resulting in many ifs =
and=20
buts which makes section 10 rather confusing. I really think a complete=20
example, with absolute paths, would be clarifying.
=C2=A0
I'm afraid many will still re-create standbys from scratch without a really=
=20
good and complete example to follow.

 --
 Andreas Joseph Krogh

------=_Part_442_17225077.1505242745653
Content-Type: text/html;charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div>P=C3=A5 tirsdag 12. september 2017 kl. 19:19:22, skrev Bruce Momjian &=
lt;<a href=3D"mailto:bruce@momjian.us">bruce@momjian.us</a>&gt;:</div>

<blockquote style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt=
 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style=3D"display:inline; font-family: monospace; font-size: 12px;">On =
Thu, Aug=C2=A0 3, 2017 at 11:37:32AM +0200, Michael Paquier wrote:<br>
&gt; On Mon, Jul 31, 2017 at 6:13 PM, Robert Haas &lt;robertmhaas@gmail.com=
&gt; wrote:<br>
&gt; &gt; On Fri, Jul 28, 2017 at 10:35 AM, Andreas Joseph Krogh<br>
&gt; &gt; &lt;andreas@visena.com&gt; wrote:<br>
&gt; &gt;&gt; I'm reading https://www.postgresql.org/docs/10/static/pgupgra=
de.html to try<br>
&gt; &gt;&gt; to understand how to upgrade standby-servers using pg_upgrade=
 with pg10.<br>
&gt; &gt;&gt;<br>
&gt; &gt;&gt; The text in step 10 sais:<br>
&gt; &gt;&gt; &quot;You will not be running pg_upgrade on the standby serve=
rs, but rather<br>
&gt; &gt;&gt; rsync&quot;, which to me sounds like rsync, in step 10-f, sho=
uld be issued on the<br>
&gt; &gt;&gt; standy servers. Is this the case? If so I don't understand ho=
w the standby's<br>
&gt; &gt;&gt; data is upgraded and what &quot;remote_dir&quot; is. If rsync=
 is supposed to be issued<br>
&gt; &gt;&gt; on the primary then I think it should be explicitly mentioned=
, and step 10-f<br>
&gt; &gt;&gt; should provide a clarer example with more detailed values for=
 the<br>
&gt; &gt;&gt; directory-structures involved.<br>
&gt; &gt;&gt;<br>
&gt; &gt;&gt; I really think section 10 needs improvement as I'm certainly =
not comfortable<br>
&gt; &gt;&gt; upgrading standbys following the existing procedure.<br>
&gt; &gt;<br>
&gt; &gt; Yeah, I don't understand it either, and I have never been convinc=
ed<br>
&gt; &gt; that there's any safe way to do it other than recloning the stand=
bys<br>
&gt; &gt; from the upgraded master.<br>
&gt;<br>
&gt; Here are my 2c on the matter. 10-f means that the upgraded node may<br=
>
&gt; have generated WAL with wal_level =3D minimal, which, at least it seem=
s<br>
&gt; to me, that we have a risk of having inconsistent data pages if only a=
<br>
&gt; rsync is used on the old standbys. Like Robert, the flow we used in<br=
>
&gt; the products I work on is to re-create standbys from scratch after the=
<br>
&gt; upgrade using a fresh backup, with a VM cloning. An upgrade here is an=
<br>
&gt; in-place process not only linked to Postgres, so standby VMs are made<=
br>
&gt; of many services, some are being linked to Postgres. So this choice is=
<br>
&gt; mainly decided by those dependencies, still it feels safer anyway.<br>
<br>
I have applied the attached doc patch back to 9.5 to clarify<br>
pg_upgrade's rsync instructions and explain how it works.<br>
<br>
Improvements?</div>
</blockquote>

<div>=C2=A0</div>

<div>=C2=A0</div>

<div>Thanks, that certainly improves things.</div>

<div>But; I still find the rsync-command in f) confusing;</div>

<div>1. Why=C2=A0--size-only? From rsync manual: &quot;skip files that matc=
h in size&quot;, is this safe??</div>

<div>2. Why is old_pgdata in the rsync-command, why is it needed to sync it=
?</div>

<div>=C2=A0</div>

<div>There are many ways to do/configure things it seems, resulting in many=
 ifs and buts which makes section 10 rather confusing. I really think a com=
plete example, with absolute paths, would be clarifying.</div>

<div>=C2=A0</div>

<div>I'm afraid many will still re-create standbys from scratch without a r=
eally good and complete example to follow.</div>

<div><br>
--<br>
Andreas Joseph Krogh</div>
------=_Part_442_17225077.1505242745653--

------=_Part_441_1847304949.1505242745624--

------=_Part_440_2037296846.1505242745623--