Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sinI5-009TEo-4C for pgsql-hackers@arkaria.postgresql.org; Tue, 27 Aug 2024 03:54:45 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sinI1-001JWH-5b for pgsql-hackers@arkaria.postgresql.org; Tue, 27 Aug 2024 03:54:41 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sinI0-001JW9-Op for pgsql-hackers@lists.postgresql.org; Tue, 27 Aug 2024 03:54:41 +0000 Received: from mail-oa1-x32.google.com ([2001:4860:4864:20::32]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sinHt-001h9m-As for pgsql-hackers@lists.postgresql.org; Tue, 27 Aug 2024 03:54:40 +0000 Received: by mail-oa1-x32.google.com with SMTP id 586e51a60fabf-26ff21d82e4so3521980fac.2 for ; Mon, 26 Aug 2024 20:54:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724730872; x=1725335672; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=qaNsvogac4LirWNYuMyTh2dd6d3lwzdFBypvvG/xEHU=; b=LWxLEa7SsZool+8ILrj1Zu0uQ9jxjvPFmjwE9FGlp0rmP51fXuZo+0MsEFF6p6bQV+ TTPh6t76tetJxG6WJ3rFvl/bNqguD981PY28wG8w9D/wnOgNEihtwIZxgDQdv1BxZlBg 8VGy5a0hhxUYccThh432yKJcjhiODS24W6IN5P/yhIINMqU2ERAXbNVNBP7nYbeOYX5r nJUKve5T1l2fjRoD2Y7ndZ+naYasGzeFu2TPAi0W9x7i0S6uab/CJdRtKELFUPZVRNxZ tPQRsyajeYruQsDXMnXIRrW2TzQ62icl3LebQWrrZuZMM8warmDFSC9mJFkGfZzX8yqI c+4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724730872; x=1725335672; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qaNsvogac4LirWNYuMyTh2dd6d3lwzdFBypvvG/xEHU=; b=KzUQP6KoCc2JwnAZfH/XVXXrJu9buad211+C1OLfe8RVIDuHBRMaA8g0HrKWm6/+P6 7uvc0orIxWgV9vbVT10Pn/GsOSYti32cDbRG0BGXylSw9FzO3+ZGpKI3rFbCw3K0F+nq S8XAxpBw+0aEHdAIKFB4YkVBvmL5tpImps8+HGZhrf4YvRlQaTKFhG4FuEaAnPACzP6+ bkq9Y2q8xhx/2TLJacNzVjl6z2i7kDeRSAde7rZMlUJEqInFa0n2ZaDKtkA9oYK5db0r FSTfxB++IvBJ4cKIGtoJrEHWP269uar4F0zs3aSDMdEsqnEC/58lS2Nhecaz/wCjEDdl oIqA== X-Forwarded-Encrypted: i=1; AJvYcCUHU9sW36ThUVhph5RzIX2ElGbPPps5w2PW32VRGPNbEmhwAIQWWYd8c3d8eRkgYLJhsns4hayCZKw26oaT@lists.postgresql.org X-Gm-Message-State: AOJu0Yy2hrCG705HKtqJ2OFbTRpEf4hLHsXPue4IACdoe/I4X8Z1nsUK lNwQ4FQxMMZYyHvp7BRCk5pWIXnJioM4U8OPHaEsK/a3gfqF3cUsrXBAkvlS18Ybpd/j5bmcXQD ERiDV7tvNL2uvQAGqRGFD8BD8jhpc1HFKuT4= X-Google-Smtp-Source: AGHT+IEOzIjlG9mUAJaKmgnKvV6Dd2QQUmAgCwvzH/iKa0aGhnW13vJ66hhFYEDs0AY3OIrfQRHgqnnz35ZqSbkhZ2A= X-Received: by 2002:a05:6870:c6a1:b0:260:3c34:bc60 with SMTP id 586e51a60fabf-273e64e6b16mr13370028fac.17.1724730871799; Mon, 26 Aug 2024 20:54:31 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac9:5e88:0:b0:557:c384:fb61 with HTTP; Mon, 26 Aug 2024 20:54:31 -0700 (PDT) In-Reply-To: References: From: "David G. Johnston" Date: Mon, 26 Aug 2024 20:54:31 -0700 Message-ID: Subject: Re: Doc: fix the note related to the GUC "synchronized_standby_slots" To: Amit Kapila Cc: "Zhijie Hou (Fujitsu)" , "Masahiro.Ikeda@nttdata.com" , "pgsql-hackers@lists.postgresql.org" , "Masao.Fujii@nttdata.com" , shveta malik Content-Type: multipart/alternative; boundary="000000000000f267400620a2329d" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000f267400620a2329d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Monday, August 26, 2024, Amit Kapila wrote: > On Mon, Aug 26, 2024 at 6:38=E2=80=AFPM Zhijie Hou (Fujitsu) > wrote: > > > > On Monday, August 26, 2024 5:37 PM Amit Kapila > wrote: > > > > > > On Mon, Aug 26, 2024 at 1:30=E2=80=AFPM = wrote: > > > > > > > > When I read the following documentation related to the > > > "synchronized_standby_slots", I misunderstood that data loss would no= t > occur > > > in the case of synchronous physical replication. However, this is > incorrect (see > > > reproduce.txt). > > > > > > > > > Note that in the case of asynchronous replication, there remains = a > risk of > > > data loss for transactions committed on the former primary server but > have yet > > > to be replicated to the new primary server. > > > > https://www.postgresql.org/docs/17/logical-replication-failover.htm= l > > > > > > > > Am I missing something? > > > > > > > > > > It seems part of the paragraph: "Note that in the case of asynchronou= s > > > replication, there remains a risk of data loss for transactions > committed on the > > > former primary server but have yet to be replicated to the new primar= y > server." is > > > a bit confusing. Will it make things clear to me if we remove that > part? > > > > I think the intention is to address a complaint[1] that the date > inserted on > > primary after the primary disconnects with the standby is still lost > after > > failover. But after rethinking, maybe it's doesn't directly belong to > the topic in > > the logical failover section because it's a general fact for async > replication. > > If we think it matters, maybe we can remove this part and slightly modi= fy > > another part: > > > > parameter ensures a seamless transition of those subscriptions after > the > > standby is promoted. They can continue subscribing to publications o= n > the > > - new primary server without losing data. > > + new primary server without losing that has already been replicated > and > > + flushed on the standby server. > > > > Yeah, we can change that way but not sure if that satisfies the OP's > concern. I am waiting for his response. > I=E2=80=99d suggest getting rid of all mention of =E2=80=9Cwithout losing d= ata=E2=80=9D and just emphasize the fact that the subscribers can operate in a hot-standby publishing environment in an automated fashion by connecting using =E2=80=9Cfailover=E2=80=9D enabled slots, assuming the publishing group pre= vents any changes from propagating to any logical subscriber until all standbys in the group have been updated. Whether or not the primary-standby group is resilient in the face of failure during internal group synchronization is out of the hands of logical subscribers - rather they are only guaranteed to see a consistent linear history of activity coming out of the publishing group. Specifically, if the group synchronizes asynchronously there is no guarantee that every committed transaction on the primary makes its way through to the logical subscriber if a slot failover happens. But at the same time its view of the world will be consistent with the newly chosen primary. David J. --000000000000f267400620a2329d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Monday, August 26, 2024, Amit Kapila <amit.kapila16@gmail.com> wrote:
On Mon, Aug 26, 2024 at 6:38=E2=80=AFPM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com>= ; wrote:
>
> On Monday, August 26, 2024 5:37 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Mon, Aug 26, 2024 at 1:30=E2=80=AFPM <Masahiro.Ikeda@nttdata.com> wrote:
> > >
> > > When I read the following documentation related to the
> > "synchronized_standby_slots", I misunderstood that data= loss would not occur
> > in the case of synchronous physical replication. However, this is= incorrect (see
> > reproduce.txt).
> > >
> > > > Note that in the case of asynchronous replication, ther= e remains a risk of
> > data loss for transactions committed on the former primary server= but have yet
> > to be replicated to the new primary server.
> > > https://www.postgresql.org/docs= /17/logical-replication-failover.html
> > >
> > > Am I missing something?
> > >
> >
> > It seems part of the paragraph: "Note that in the case of as= ynchronous
> > replication, there remains a risk of data loss for transactions c= ommitted on the
> > former primary server but have yet to be replicated to the new pr= imary server." is
> > a bit confusing. Will it make things clear to me if we remove tha= t part?
>
> I think the intention is to address a complaint[1] that the date inser= ted on
> primary after the primary disconnects with the standby is still lost a= fter
> failover. But after rethinking, maybe it's doesn't directly be= long to the topic in
> the logical failover section because it's a general fact for async= replication.
> If we think it matters, maybe we can remove this part and slightly mod= ify
> another part:
>
>=C2=A0 =C2=A0 parameter ensures a seamless transition of those subscrip= tions after the
>=C2=A0 =C2=A0 standby is promoted. They can continue subscribing to pub= lications on the
> -=C2=A0 =C2=A0new primary server without losing data.
> +=C2=A0 =C2=A0new primary server without losing that has already been = replicated and
> +=C2=A0 =C2=A0 flushed on the standby server.
>

Yeah, we can change that way but not sure if that satisfies the OP's concern. I am waiting for his response.

I=E2=80=99d suggest getting rid of all men= tion of =E2=80=9Cwithout losing data=E2=80=9D and just emphasize the fact t= hat the subscribers can operate in a hot-standby publishing environment in = an automated fashion by connecting using =E2=80=9Cfailover=E2=80=9D enabled= slots, assuming the publishing group prevents any changes from propagating= to any logical subscriber until all standbys in the group have been update= d.=C2=A0 Whether or not the primary-standby group is resilient in the face = of failure during internal group synchronization is out of the hands of log= ical subscribers - rather they are only guaranteed to see a consistent line= ar history of activity coming out of the publishing group.=C2=A0 Specifical= ly, if the group synchronizes asynchronously there is no guarantee that eve= ry committed transaction on the primary makes its way through to the logica= l subscriber if a slot failover happens.=C2=A0 But at the same time its vie= w of the world will be consistent with the newly chosen primary.
=
David J.

--000000000000f267400620a2329d--