MIME-Version: 1.0
References: 
 <CALj2ACUrOB59QaE6=jF2cFAyv1MR7fzD8tr4YM5+OwEYG1SNzA@mail.gmail.com>
 <9290b55b6ae2b04e002ca9dadadd1cca09461482.camel@cybertec.at>
 <CALj2ACWoB3k0kfjA7JxJgskVGGeE3jWzmGRPjP9QTRSCgSjhOg@mail.gmail.com>
 <763B5AF0-1C9E-4796-9639-F969A2E66189@yandex-team.ru>
 <CAFiTN-tzW+pFXGh-NqZCJmBGPsd2EHtvYvSxvsZTH7=cWsdPMw@mail.gmail.com>
 <CALj2ACVuj=Fo55NdWJy=fwweC-+pi7eQPRN3Kij3PQQcWFtYxQ@mail.gmail.com>
 <11FF616C-C78C-41AA-A823-E3D4E745ACE5@yandex-team.ru>
 <CALj2ACU=nzEb_dEfoLqez5CLcwvx1GhkdfYRNX+A4NDRbjYdBg@mail.gmail.com>
 <4F070B19-51EC-4A05-A111-6001A961F991@yandex-team.ru>
 <CALj2ACV1WBqtdwMXV7TCfPiAVdHNF=8n1tiCWpv0yS-mbqeWZg@mail.gmail.com>
In-Reply-To: 
 <CALj2ACV1WBqtdwMXV7TCfPiAVdHNF=8n1tiCWpv0yS-mbqeWZg@mail.gmail.com>
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Thu, 4 Aug 2022 14:01:40 +0530
Message-ID: 
 <CALj2ACUiyE1ui4Daqw1cw-tLTSqkiB1bYZ6rWb5BE70-C65Cog@mail.gmail.com>
Subject: Re: An attempt to avoid
 locally-committed-but-not-replicated-to-standby-transactions
 in synchronous replication
To: Andrey Borodin <x4mmm@yandex-team.ru>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Dilip Kumar <dilipbalaut@gmail.com>,
	Laurenz Albe <laurenz.albe@cybertec.at>,
	SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CALj2ACUiyE1ui4Daqw1cw-tLTSqkiB1bYZ6rWb5BE70-C65Cog%40mail.gmail.com>
Precedence: bulk

On Thu, Aug 4, 2022 at 1:42 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
>
> On Mon, Jul 25, 2022 at 4:20 PM Andrey Borodin <x4mmm@yandex-team.ru> wro=
te:
> >
> > > 25 =D0=B8=D1=8E=D0=BB=D1=8F 2022 =D0=B3., =D0=B2 14:29, Bharath Rupir=
eddy <bharath.rupireddyforpostgres@gmail.com> =D0=BD=D0=B0=D0=BF=D0=B8=D1=
=81=D0=B0=D0=BB(=D0=B0):
> > >
> > > Hm, after thinking for a while, I tend to agree with the above
> > > approach - meaning, query cancel interrupt processing can completely
> > > be disabled in SyncRepWaitForLSN() and process proc die interrupt
> > > immediately, this approach requires no GUC as opposed to the proposed
> > > v1 patch upthread.
> > GUC was proposed here[0] to maintain compatibility with previous behavi=
our. But I think that having no GUC here is fine too. If we do not allow ca=
ncelation of unreplicated backends, of course.
> >
> > >>
> > >> And yes, we need additional complexity - but in some other place. Tr=
ansaction can also be locally committed in presence of a server crash. But =
this another difficult problem. Crashed server must not allow data queries =
until LSN of timeline end is successfully replicated to synchronous_standby=
_names.
> > >
> > > Hm, that needs to be done anyways. How about doing as proposed
> > > initially upthread [1]? Also, quoting the idea here [2].
> > >
> > > Thoughts?
> > >
> > > [1] https://www.postgresql.org/message-id/CALj2ACUrOB59QaE6=3DjF2cFAy=
v1MR7fzD8tr4YM5+OwEYG1SNzA@mail.gmail.com
> > > [2] 2) Wait for sync standbys to catch up upon restart after the cras=
h or
> > > in the next txn after the old locally committed txn was canceled. One
> > > way to achieve this is to let the backend, that's making the first
> > > connection, wait for sync standbys to catch up in ClientAuthenticatio=
n
> > > right after successful authentication. However, I'm not sure this is
> > > the best way to do it at this point.
> >
> >
> > I think ideally startup process should not allow read only connections =
in CheckRecoveryConsistency() until WAL is not replicated to quorum al leas=
t up until new timeline LSN.
>
> We can't do it in CheckRecoveryConsistency() unless I'm missing
> something. Because, the walsenders (required for sending the remaining
> WAL to sync standbys to achieve quorum) can only be started after the
> server reaches a consistent state, after all walsenders are
> specialized backends.

Continuing on the above thought (I inadvertently clicked the send
button previously): A simple approach would be to check for quorum in
PostgresMain() before entering the query loop for (;;) for
non-walsender cases. A disadvantage of this would be that all the
backends will be waiting here in the worst case if it takes time for
achieving the sync quorum after restart -  roughly we can do the
following in PostgresMain(), of course we need locking mechanism so
that all the backends whoever reaches here will wait for the same lsn:

if (sync_replicaion_defined =3D=3D true &&
shmem->wait_for_sync_repl_upon_restart =3D=3D true)
{
      SyncRepWaitForLSN(pg_current_wal_flush_lsn(), false);
      shmem->wait_for_sync_repl_upon_restart =3D false;
}

Thoughts?

--=20
Bharath Rupireddy
RDS Open Source Databases: https://aws.amazon.com/rds/postgresql/