Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n5tIW-0008Sk-PU for pgsql-hackers@arkaria.postgresql.org; Fri, 07 Jan 2022 17:45:04 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1n5tHz-0004rm-Co for pgsql-hackers@arkaria.postgresql.org; Fri, 07 Jan 2022 17:44:31 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n5tHy-0004rd-RA for pgsql-hackers@lists.postgresql.org; Fri, 07 Jan 2022 17:44:31 +0000 Received: from mail-ed1-x533.google.com ([2a00:1450:4864:20::533]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1n5tHw-0007rl-5L for pgsql-hackers@lists.postgresql.org; Fri, 07 Jan 2022 17:44:29 +0000 Received: by mail-ed1-x533.google.com with SMTP id k15so24758321edk.13 for ; Fri, 07 Jan 2022 09:44:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=PxxzoKJ+j+sswQ4BvBkFB+gAopFrpOUTvxhEQrl2ox8=; b=cUcTTwv4+3RB4Wx/gOyTw1hAA2gEVnDfThHhFt0O0Yaa9EbQMdLMwxrM7phQNpIE+T tZ6ECQS05qZ8ykh1ZwzGaZtzeCIcaqFj2Vu81qLPxPiHIo3Yn67MPirBamBK3LhnZofA eCEk+BOU8bejmt6CLLeQ7/JBVoPVIsYnxFfi19qeV8CWfNrlkWYaD4XcQzmeTAg63UiG XQlYYO7GCuIWo/oTMl6SJWvo6ig3vMxckg9DiE/MxGt1sQkr+aCw+eSa+YeTQ0xWd8Mo C1XNfnPLBFvggy7glYKxJ0+SPx7w6sPlJaECXvrFLE02905HBL9KM07Wrce7x7vxIwqA UHfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PxxzoKJ+j+sswQ4BvBkFB+gAopFrpOUTvxhEQrl2ox8=; b=1+AIQCik2LNZncYRVoGj5zzfIKT+wnCC17Tl0U1ZHLjhHshYu5U35bjtV6EOohQaEx 9R1pjuGgX0yCUxThP98VtR8wN1bIuNmzt9x09YYISRztNTeTcFRZ8wOLYR3cxM6EkIB0 z7N+2UB5sCG7PwxOzqjoLtrKeW8c43Cswe44cELWD7jUfUsYE+Fse/0LDjaKblaCzl/5 IQBzPE4zGmEqKeIvFKTMRQoim44JedjJfYEe0L6giLfiWDMfinoDbeo3PuR0RNuwWByB k0M0g3jq/BoDQDplr8epn49s576zqvQ08yyfzcyH4T48KkMgOzY4IUULUqFi0jER3zi0 cqiQ== X-Gm-Message-State: AOAM533vCANu5U9oT1cbBBJuCxs1wHxursJdmAwIgjf0ptoeaneaQbnG hLa/55VA1c7I+X7Lvq7BLVjOBDKn1/prhgXJD8I= X-Google-Smtp-Source: ABdhPJyn4a6XWe71UtXBock/4RQl2/rCLerzBXFHYzALOEStgGB3JAu6pikVn9GSVBDNyhBblht2KNefUX2ldpnZe2U= X-Received: by 2002:a17:907:7b9a:: with SMTP id ne26mr49801777ejc.240.1641577466662; Fri, 07 Jan 2022 09:44:26 -0800 (PST) MIME-Version: 1.0 References: <2df227fddc659d1da61a59a54ffa11c03461a84e.camel@j-davis.com> <20220107.172740.1571453258855138645.horikyota.ntt@gmail.com> In-Reply-To: <20220107.172740.1571453258855138645.horikyota.ntt@gmail.com> From: SATYANARAYANA NARLAPURAM Date: Fri, 7 Jan 2022 09:44:15 -0800 Message-ID: Subject: Re: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers To: Kyotaro Horiguchi Cc: Jeff Davis , Bharath Rupireddy , PostgreSQL Hackers Content-Type: multipart/alternative; boundary="000000000000c5932d05d50189a5" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000c5932d05d50189a5 Content-Type: text/plain; charset="UTF-8" On Fri, Jan 7, 2022 at 12:27 AM Kyotaro Horiguchi wrote: > At Thu, 6 Jan 2022 23:55:01 -0800, SATYANARAYANA NARLAPURAM < > satyanarlapuram@gmail.com> wrote in > > On Thu, Jan 6, 2022 at 11:24 PM Jeff Davis wrote: > > > > > On Wed, 2022-01-05 at 23:59 -0800, SATYANARAYANA NARLAPURAM wrote: > > > > I would like to propose a GUC send_Wal_after_quorum_committed which > > > > when set to ON, walsenders corresponds to async standbys and logical > > > > replication workers wait until the LSN is quorum committed on the > > > > primary before sending it to the standby. This not only simplifies > > > > the post failover steps but avoids unnecessary downtime for the async > > > > replicas. Thoughts? > > > > > > Do we need a GUC? Or should we just always require that sync rep is > > > satisfied before sending to async replicas? > > > > > > > I proposed a GUC to not introduce a behavior change by default. I have no > > strong opinion on having a GUC or making the proposed behavior default, > > would love to get others' perspectives as well. > > > > > > > > > > It feels like the sync quorum should always be ahead of the async > > > replicas. Unless I'm missing a use case, or there is some kind of > > > performance gotcha. > > > > > > > I couldn't think of a case that can cause serious performance issues but > > will run some experiments on this and post the numbers. > > I think Jeff is saying that "quorum commit" already by definition > means that all out-of-quorum standbys are behind of the > quorum-standbys. I agree to that in a dictionary sense. But I can > think of the case where the response from the top-runner standby > vanishes or gets caught somewhere on network for some reason. In that > case the primary happily checks quorum ignoring the top-runner. > > To avoid that misdecision, I can guess two possible "solutions". > > One is to serialize WAL sending (of course it is unacceptable at all) > or aotehr is to send WAL to all standbys at once then make the > decision after making sure receiving replies from all standbys (this > is no longer quorum commit in another sense..) > There is no need to serialize sending the WAL among sync standbys. The only serialization required is first to all the sync replicas and then to sync replicas if any. Once an LSN is quorum committed, no failover subsystem initiates an automatic failover such that the LSN is lost (data loss) > > So I'm afraid that there's no sensible solution to avoid the > hiding-forerunner problem on quorum commit. > Could you elaborate on the problem here? > > regards. > > -- > Kyotaro Horiguchi > NTT Open Source Software Center > --000000000000c5932d05d50189a5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Jan 7, 2022 at 12:27 AM Kyota= ro Horiguchi <horikyota.ntt@g= mail.com> wrote:
At Thu, 6 Jan 2022 23:55:01 -0800, SATYANARAYANA NARLAPURAM <satyanarlapuram= @gmail.com> wrote in
> On Thu, Jan 6, 2022 at 11:24 PM Jeff Davis <pgsql@j-davis.com> wrote:
>
> > On Wed, 2022-01-05 at 23:59 -0800, SATYANARAYANA NARLAPURAM wrote= :
> > > I would like to propose a GUC send_Wal_after_quorum_committe= d which
> > > when set to ON, walsenders corresponds to async standbys and= logical
> > > replication workers wait until the LSN is quorum committed o= n the
> > > primary before sending it to the standby. This not only simp= lifies
> > > the post failover steps but avoids unnecessary downtime for = the async
> > > replicas. Thoughts?
> >
> > Do we need a GUC? Or should we just always require that sync rep = is
> > satisfied before sending to async replicas?
> >
>
> I proposed a GUC to not introduce a behavior change by default. I have= no
> strong opinion on having a GUC or making the proposed behavior default= ,
> would love to get others' perspectives as well.
>
>
> >
> > It feels like the sync quorum should always be ahead of the async=
> > replicas. Unless I'm missing a use case, or there is some kin= d of
> > performance gotcha.
> >
>
> I couldn't think of a case that can cause serious performance issu= es but
> will run some experiments on this and post the numbers.

I think Jeff is saying that "quorum commit" already by definition=
means that all out-of-quorum standbys are behind of the
quorum-standbys.=C2=A0 I agree to that in a dictionary sense. But I can
think of the case where the response from the top-runner standby
vanishes or gets caught somewhere on network for some reason. In that
case the primary happily checks quorum ignoring the top-runner.

To avoid that misdecision, I can guess two possible "solutions".<= br>
One is to serialize WAL sending (of course it is unacceptable at all)
or aotehr is to send WAL to all standbys at once then make the
decision after making sure receiving replies from all standbys (this
is no longer quorum commit in another sense..)

There is no need to serialize sending the WAL among sync standbys. = The only serialization required is first to all the sync replicas and then = to sync replicas if any. Once an LSN is quorum committed, no failover subsy= stem initiates an automatic failover such that the LSN is lost (data loss)<= /div>

So I'm afraid that there's no sensible solution to avoid the
hiding-forerunner problem on quorum commit.
=C2=A0
Could you elaborate on the problem here?
=C2=A0

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center
--000000000000c5932d05d50189a5--