public inbox for [email protected]
help / color / mirror / Atom feedFrom: Michał Kłeczek <[email protected]>
To: PG-General Mailing List <[email protected]>
Subject: Merging logical subscriptions
Date: Thu, 17 Oct 2024 08:45:53 +0200
Message-ID: <[email protected]> (raw)
Hi,
I have a question about an algorithm to “merge” two logical replication subscriptions. Here is the scenario:
We have a server with many tables replicated using logical replication to downstream replicas.
Each table is published with a separate publication.
At the moment each replica uses a single subscription with many publications.
The problem is that a subscription is a unit of concurrency in logical replication and sometimes we need to process replication streams in parallel.
“Splitting” subscriptions so that groups of publications are processed in parallel is (quite) easy:
1. Disable existing subscription
2. Clone replication slot
3. Create a new subscription with a subset of first subscription publications (don’t create a replication slot, use the replication slot created in step 2, don’t copy data)
4. Drop the subset of publications from the first subscription.
5. Enable and refresh the first subscription
But is there a procedure to “merge” subscriptions?
The problem I see is that there is no way to synchronise two subscriptions so that they stop at the same LSN.
We can skip transactions by advancing the LSN. But we cannot process transactions until a specific LSN.
Any ideas?
Thanks,
Michal
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: Merging logical subscriptions
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox