public inbox for [email protected]
help / color / mirror / Atom feedFrom: Euler Taveira <[email protected]>
To: [email protected] <[email protected]>
To: '[email protected]' <[email protected]>
Cc: Shlok Kyal <[email protected]>
Cc: vignesh C <[email protected]>
Cc: Michael Paquier <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: Ashutosh Bapat <[email protected]>
Cc: Amit Kapila <[email protected]>
Subject: Re: speed up a logical replica setup
Date: Tue, 23 Jan 2024 20:58:11 -0300
Message-ID: <[email protected]> (raw)
In-Reply-To: <TY3PR01MB9889C5D55206DDD978627D07F5752@TY3PR01MB9889.jpnprd01.prod.outlook.com>
References: <[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<CALDaNm098Jkbh+ye6zMj9Ro9j1bBe6FfPV80BFbs1=pUuTJ07g@mail.gmail.com>
<[email protected]>
<CAA4eK1KtGC-v_CYd8yaCwisrQvoZN=+As90p9hBM29DAxA+Pmg@mail.gmail.com>
<CANhcyEUdYCR9oSt4HsZRqYZZVXsb4x-HoPGqDSxSeA2SYz-YGg@mail.gmail.com>
<CAA4eK1KMQNMhSPmsRdrCQpuc=Tsc08h6=XkLNZ5F35Q2REom6g@mail.gmail.com>
<CANhcyEXjOcnOJKdOfK8Z1+bbymuZzwSjrppbt4FKHbrDD_t7Mg@mail.gmail.com>
<CANhcyEUCt-g4JLQU3Q3ofFk_Vt-Tqh3ZdXoLcpT8fjz9LY_-ww@mail.gmail.com>
<[email protected]>
<CAA4eK1JRgnhv_ySzuFjN7UaX9qxz5Hqcwew7Fk=+AbJbu0Kd9w@mail.gmail.com>
<TY3PR01MB98895BA6C1D72CB8582CACC4F5682@TY3PR01MB9889.jpnprd01.prod.outlook.com>
<[email protected]>
<TY3PR01MB9889C362FF76102C88FA1C29F56F2@TY3PR01MB9889.jpnprd01.prod.outlook.com>
<TY3PR01MB9889C5D55206DDD978627D07F5752@TY3PR01MB9889.jpnprd01.prod.outlook.com>
On Mon, Jan 22, 2024, at 4:06 AM, Hayato Kuroda (Fujitsu) wrote:
> I analyzed and found a reason. This is because publications are invisible for some transactions.
>
> As the first place, below operations were executed in this case.
> Tuples were inserted after getting consistent_lsn, but before starting the standby.
> After doing the workload, I confirmed again that the publication was created.
>
> 1. on primary, logical replication slots were created.
> 2. on primary, another replication slot was created.
> 3. ===on primary, some tuples were inserted. ===
> 4. on standby, a server process was started
> 5. on standby, the process waited until all changes have come.
> 6. on primary, publications were created.
> 7. on standby, subscriptions were created.
> 8. on standby, a replication progress for each subscriptions was set to given LSN (got at step2).
> =====pg_subscriber finished here=====
> 9. on standby, a server process was started again
> 10. on standby, subscriptions were enabled. They referred slots created at step1.
> 11. on primary, decoding was started but ERROR was raised.
Good catch! It is a design flaw.
> In this case, tuples were inserted *before creating publication*.
> So I thought that the decoded transaction could not see the publication because
> it was committed after insertions.
>
> One solution is to create a publication before creating a consistent slot.
> Changes which came before creating the slot were surely replicated to the standby,
> so upcoming transactions can see the object. We are planning to patch set to fix
> the issue in this approach.
I'll include a similar code in the next patch and also explain why we should
create the publication earlier. (I'm renaming
create_all_logical_replication_slots to setup_publisher and calling
create_publication from there and also adding the proposed GUC checks in it.)
--
Euler Taveira
EDB https://www.enterprisedb.com/
view thread (97+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: speed up a logical replica setup
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox