From: Ronan Dunklau <ronan.dunklau@aiven.io>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: pg_receivewal starting position
Date: Wed, 28 Jul 2021 12:57:39 +0200
Message-ID: <15651763.8WXi7s64xO@aivenronan>
Organization: aiven
In-Reply-To: <20210728.152230.2032903346246847782.horikyota.ntt@gmail.com>
References: <18708360.4lzOvYHigE@aivenronan>
 <20210728.152230.2032903346246847782.horikyota.ntt@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"
Archived-At: 
 <https://www.postgresql.org/message-id/15651763.8WXi7s64xO%40aivenronan>
Precedence: bulk

Le mercredi 28 juillet 2021, 08:22:30 CEST Kyotaro Horiguchi a =E9crit :
> At Tue, 27 Jul 2021 07:50:39 +0200, Ronan Dunklau <ronan.dunklau@aiven.io>
> wrote in
> > Hello,
> >=20
> > I've notived that pg_receivewal logic for deciding which LSN to start
> >=20
> > streaming at consists of:
> >   - looking up the latest WAL file in our destination folder, and resume
> >   from
> >=20
> > here
> >=20
> >   - if there isn't, use the current flush location instead.
> >=20
> > This behaviour surprised me when using it with a replication slot: I was
> > expecting it to start streaming at the last flushed location from the
> > replication slot instead. If you consider a backup tool which will take
> > pg_receivewal's output and transfer it somewhere else, using the
> > replication slot position would be the easiest way to ensure we don't
> > miss WAL files.
> >=20
> > Does that make sense ?
> >=20
> > I don't know if it should be the default, toggled by a command line fla=
g,
> > or if we even should let the user provide a LSN.
>=20
> *I* think it is completely reasonable (or at least convenient or less
> astonishing) that pg_receivewal starts from the restart_lsn of the
> replication slot to use.  The tool already decides the clean-start LSN
> a bit unusual way. And it seems to me that proposed behavior can be
> the default when -S is specified.
>=20

As of now we can't get the replication_slot restart_lsn with a replication=
=20
connection AFAIK.

This implies that the patch could require the user to specify a maintenance=
=2Ddb=20
parameter, and we would use that if provided to fetch the replication slot=
=20
info, or fallback to the previous behaviour. I don't really like this appro=
ach=20
as the behaviour changing wether we supply a maintenance-db parameter or no=
t=20
is error-prone for the user.

Another option would be to add a new replication command (for example=20
ACQUIRE_REPLICATION_SLOT <slot_name>) to set the replication slot as the=20
current one, and return some info about it (restart_lsn at least for a=20
physical slot).

I don't see any reason not to make it work for logical replication connecti=
ons=20
/ slots, but it wouldn't be that useful since we can query the database in=
=20
that case.

Acquiring the replication slot instead of just reading it would make sure t=
hat=20
no other process could start the replication between the time we read the=20
restart_lsn and when we issue START_REPLICATION. START_REPLICATION could th=
en=20
check if we already have a replication slot, and ensure it is the same one =
as=20
the one we're trying to use.=20

=46rom pg_receivewal point of view, this would amount to:

 - check if we currently have wal in the target directory.
   - if we do, proceed as currently done, by computing the start lsn and=20
timeline from the last archived wal
  - if we don't, and we have a slot, run ACQUIRE_REPLICATION_SLOT. Use the=
=20
restart_lsn as the start lsn if there is one, and don't provide a timeline
  - if we still don't have a start_lsn, fallback to using the current serve=
r=20
wal position as is done.=20

What do you think ? Which information should we provide about the slot ?


=2D-=20
Ronan Dunklau