From: Ronan Dunklau <ronan.dunklau@aiven.io>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
 Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Masahiko Sawada <sawada.mshk@gmail.com>
Subject: Re: pg_receivewal starting position
Date: Thu, 02 Sep 2021 09:02:57 +0200
Message-ID: <3025343.EqsfBnhMdK@aivenronan>
Organization: aiven
In-Reply-To: 
 <CALj2ACUOkq=w4jFZiRxEgh3wBk8B=GvB5KjH0m+HO3b1mBuKwQ@mail.gmail.com>
References: 
 <CALj2ACW6YA0SfqwPm1-FRRdeNj1e9isiSax3jZ8MmyQYYQQWNw@mail.gmail.com>
 <20210902.144554.1303720268994714850.horikyota.ntt@gmail.com>
 <CALj2ACUOkq=w4jFZiRxEgh3wBk8B=GvB5KjH0m+HO3b1mBuKwQ@mail.gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"
Archived-At: 
 <https://www.postgresql.org/message-id/3025343.EqsfBnhMdK%40aivenronan>
Precedence: bulk

Le jeudi 2 septembre 2021, 08:42:22 CEST Bharath Rupireddy a =E9crit :
> On Thu, Sep 2, 2021 at 11:15 AM Kyotaro Horiguchi
>=20
> <horikyota.ntt@gmail.com> wrote:
> > At Wed, 1 Sep 2021 10:30:05 +0530, Bharath Rupireddy
> > <bharath.rupireddyforpostgres@gmail.com> wrote in>=20
> > > On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau <ronan.dunklau@aiven.io=
>=20
wrote:
> > > > > 7) How about we let pg_receivewal use READ_REPLICATION_SLOT as an
> > > > > option?
> > > >=20
> > > > From my point of view, I already expected it to use something like
> > > > that when using a replication slot. Maybe an option to turn it off
> > > > could be useful ?> >=20
> > > IMO, pg_receivewal should have a way to turn off/on using
> > > READ_REPLICATION_SLOT. Imagine if the postgres server doesn't support
> > > READ_REPLICATION_SLOT (a lower version) but for some reasons the
> > > pg_receivewal(running separately) is upgraded to the version that uses
> > > READ_REPLICATION_SLOT, knowing that the server doesn't support
> > > READ_REPLICATION_SLOT why should user let pg_receivewal run an
> > > unnecessary code?
> >=20
> > If I read the patch correctly the situation above is warned by the
> > following message then continue to the next step giving up to identify
> > start position from slot data.
> >=20
> > > "server does not suport fetching the slot's position, resuming from t=
he
> > > current server position instead">=20
> > (The message looks a bit too long, though..)
> >=20
> > However, if the operator doesn't know the server is old, pg_receivewal
> > starts streaming from unexpected position, which is a kind of
> > disaster. So I'm inclined to agree to Bharath, but rather I imagine of
> > an option to explicitly specify how to determine the start position.
> >=20
> > --start-source=3D[server,wal,slot]  specify starting-LSN source, defaul=
t is
> >=20
> >                      trying all of them in the order of wal, slot, serv=
er.
> >=20
> > I don't think the option doesn't need to accept multiple values at once.
>=20
> If --start-source =3D 'wal' fails, then pg_receivewal should show up an
> error saying "cannot find start position from <<user-specified-wal>>
> directory, try with "server" or "slot" for --start-source". We might
> end having similar errors for other options as well. Isn't this going
> to create unnecessary complexity?
>=20
> The existing way the pg_receivewal fetches start pos i.e. first from
> wal directory and then from server start position, isn't known to the
> user at all, no verbose message or anything specified in the docs. Why
> do we need to expose this with the --start-source option? IMO, we can
> keep it that way and we can just have a way to turn off the new
> behaviour that we are proposing here, i.e.fetching the start position
> from the slot's restart_lsn.

Then it should probably be documented. We write in the docs that it is=20
strongly recommended to use a replication slot, but do not mention how we=20
resume from have been already processed.

If someone really cares about having control over how the start position is=
=20
defined instead of relying on the auto detection, it would be wiser to add =
a --
startpos parameter similar to the endpos one, which would override everythi=
ng=20
else, instead of different flags for different behaviours.

Regards,

=2D-=20
Ronan Dunklau