public inbox for [email protected]
help / color / mirror / Atom feedFrom: Hayato Kuroda (Fujitsu) <[email protected]>
To: 'Amit Kapila' <[email protected]>
Cc: Euler Taveira <[email protected]>
Cc: Alexander Lakhin <[email protected]>
Cc: [email protected] <[email protected]>
Cc: Zhijie Hou (Fujitsu) <[email protected]>
Subject: RE: speed up a logical replica setup
Date: Wed, 17 Jul 2024 11:58:47 +0000
Message-ID: <TYCPR01MB5693BC8F357F243196E8DE41F5A32@TYCPR01MB5693.jpnprd01.prod.outlook.com> (raw)
In-Reply-To: <CAA4eK1J5t48z2mfpNdCT8OunECR7qJZmHUiDnrdunU+z83u_oA@mail.gmail.com>
References: <[email protected]>
<CAA4eK1JJq_ER6Kq_H=jKHR75QPRd8y9_D=RtYw=aPYKMfqLi9A@mail.gmail.com>
<CAA4eK1KdCb+5sjYu6qCMXXdCX1y_ihr8kFzMozq0=P=auYxgog@mail.gmail.com>
<CANhcyEV6q1Vhd37i1axUeScLi0UAGVxta1LDa0BV0Eh--TcPMg@mail.gmail.com>
<CAA4eK1JaQcAZ=n513-4VDW4ujE9T9f007fczNCpVDbg1smhT4g@mail.gmail.com>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<OSBPR01MB25528300C71FDD83EA1DCA12F5DD2@OSBPR01MB2552.jpnprd01.prod.outlook.com>
<CAA4eK1+Qmc34cooSNm2=U6YsySSjZTn2_eD_deDFEAZv+aj-AA@mail.gmail.com>
<OSBPR01MB2552440DCE1AE5EDC51BAEC6F5DD2@OSBPR01MB2552.jpnprd01.prod.outlook.com>
<CAA4eK1J9Vn1HB7sXNx-cH4COBd_moGZtrZd82Z160mK6u0HspQ@mail.gmail.com>
<[email protected]>
<CAA4eK1KG7+jUdONhsELY2poyBrmesNnkt-m5QSgPhgyyM6cFHQ@mail.gmail.com>
<[email protected]>
<[email protected]>
<TYAPR01MB5692D87C0E1FD86713525F60F5A32@TYAPR01MB5692.jpnprd01.prod.outlook.com>
<CAA4eK1J5t48z2mfpNdCT8OunECR7qJZmHUiDnrdunU+z83u_oA@mail.gmail.com>
Dear Amit,
> Your analysis sounds correct to me.
Okay, so we could have a same picture...
> > IIUC, the root cause is that pg_create_logical_replication_slot() returns a LSN
> > which is not generated yet. So, I think both mine [1] and Euler's approach [2]
> > can solve the issue. My proposal was to add an extra WAL record after the final
> > slot creation, and Euler's one was to use a restart_lsn as the
> recovery_target_lsn.
> >
>
> I don't think it is correct to set restart_lsn as consistent_lsn point
> because the same is used to set replication origin progress. Later
> when we start the subscriber, the system will use that LSN as a
> start_decoding_at point which is the point after which all the commits
> will be replicated. So, we will end up incorrectly using restart_lsn
> (LSN from where we start reading the WAL) as start_decoding_at point.
> How could that be correct?
I didn't say we could use restart_lsn as consistent point of logical replication,
but I could agree the approach has issues.
> Now, even if we use restart_lsn as recovery_target_lsn and the LSN
> returned by pg_create_logical_replication_slot() as consistent LSN to
> set replication progress, that also could lead to data loss because
> the subscriber may never get data between restart_lsn value and
> consistent LSN value.
You considered the case, e.g., tuples were inserted just after the restart_lsn
but before the RUNNING_XACT record? In this case, yes, the streaming replication
finishes before replicating tuples but logical replication will skip them.
Euler's approach cannot be used as-is.
Best regards,
Hayato Kuroda
FUJITSU LIMITED
view thread (107+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: RE: speed up a logical replica setup
In-Reply-To: <TYCPR01MB5693BC8F357F243196E8DE41F5A32@TYCPR01MB5693.jpnprd01.prod.outlook.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox