public inbox for [email protected]
help / color / mirror / Atom feedRe: [PATCH] Support automatic sequence replication
58+ messages / 10 participants
[nested] [flat]
* Re: [PATCH] Support automatic sequence replication
@ 2026-02-06 09:15 shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-02-06 09:15 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Thu, Feb 5, 2026 at 10:33 AM Ajin Cherian <[email protected]> wrote:
>
> On Tue, Feb 3, 2026 at 9:22 PM shveta malik <[email protected]> wrote:
> >
> > On Tue, Feb 3, 2026 at 9:18 AM Ajin Cherian <[email protected]> wrote:
> > Thanks for the patch.
> >
> > +1 for the overall idea of patch that once a subscription is created
> > which subscribes to sequences, a sequence sync worker is started which
> > continuously syncs the sequences. This makes usage of REFRESH
> > SEQUENCES redundant and thus it is removed. I am still reviewing the
> > design choice here, and will post my comments soon (if any).
> >
>
> Thanks!
>
> > By quick validation, few issues in current implementation:
> >
> > 1)
> > If the sequence sync worker exits due to some issue (or killed or
> > server restarts), sequence-sync worker is not started again by apply
> > worker unless there is a sequence in INIT state i.e. synchronization
> > of sequences in READY state stops. IIUC, the logic of
> > ProcessSequencesForSync() needs to change to start seq sync worker
> > irrespective of state of sequences.
> >
>
> Yes, I fixed this. I've changed FetchRelationStates to fetch sequences
> in ANY state and not just ones in NON READY state.
>
> > 2)
> > There is some issue in how LOGs (DEBUGs) are getting generated.
> >
> > a) Even if there is no drift, it still keeps on dumping:
> > "logical replication sequence synchronization for subscription "sub1"
> > - total unsynchronized: 3"
> >
>
> Removed this.
>
> > b)
> > When there is a drift in say single sequence, it puts rest (which are
> > in sync) to "missing" section:
> > "logical replication sequence synchronization for subscription "sub1"
> > - batch #1 = 3 attempted, 1 succeeded, 0 mismatched, 0 insufficient
> > permission, 2 missing from publisher, 0 skipped"
> >
>
> Fixed, and added a new section called "no drift". Also now this debug
> message is printed every time the worker attempts to synchronize
> sequences. also mentioning the state of the sequences being synced.
>
> > 3)
> > If a sequence sync worker is taking a nap, and subscription is
> > disabled or the server is stopped just before upgrade, how is the user
> > supposed to know that sequences are synced at the end?
>
> Well, one way is to wait for a debug message that says that all the
> attempted sequences are in the "no drift" state. Also remote
> sequence's LSN is updated in pg_subscription_rel for each sequence.
> Let me know if you have anything more in mind. One option is to leave
> the ALTER SUBSCRIPTION..REFRESH SEQUENCE in place, that will change
> the state of all the sequences to the INIT state, and the user can
> then wait for the sequences to change state to READY.
I think this point needs more analysis. I am analyzing and discussing
it further.
> Attaching patch v3 addressing the above comments.
>
Thank You for the patch. I haven’t reviewed the details yet, but it
would be good to evaluate whether we really need to start the sequence
sync worker so deep inside the apply worker. Currently, the caller of
ProcessSequencesForSync(), namely ProcessSyncingRelations() is invoked
for every apply message to process possible state-changes of relation
and start worker (tablesync/sequence-sync etc). Since monitoring
state-change behavior is no longer required for sequences, should we
consider moving this logic to the main loop of the apply worker,
possibly within LogicalRepApplyLoop()? There, we could check whether
the table has any sequences and, if a worker is not already running,
start one. Thoughts?
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-06 09:17 ` shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-02-06 09:17 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Fri, Feb 6, 2026 at 2:45 PM shveta malik <[email protected]> wrote:
>
> On Thu, Feb 5, 2026 at 10:33 AM Ajin Cherian <[email protected]> wrote:
> >
> > On Tue, Feb 3, 2026 at 9:22 PM shveta malik <[email protected]> wrote:
> > >
> > > On Tue, Feb 3, 2026 at 9:18 AM Ajin Cherian <[email protected]> wrote:
> > > Thanks for the patch.
> > >
> > > +1 for the overall idea of patch that once a subscription is created
> > > which subscribes to sequences, a sequence sync worker is started which
> > > continuously syncs the sequences. This makes usage of REFRESH
> > > SEQUENCES redundant and thus it is removed. I am still reviewing the
> > > design choice here, and will post my comments soon (if any).
> > >
> >
> > Thanks!
> >
> > > By quick validation, few issues in current implementation:
> > >
> > > 1)
> > > If the sequence sync worker exits due to some issue (or killed or
> > > server restarts), sequence-sync worker is not started again by apply
> > > worker unless there is a sequence in INIT state i.e. synchronization
> > > of sequences in READY state stops. IIUC, the logic of
> > > ProcessSequencesForSync() needs to change to start seq sync worker
> > > irrespective of state of sequences.
> > >
> >
> > Yes, I fixed this. I've changed FetchRelationStates to fetch sequences
> > in ANY state and not just ones in NON READY state.
> >
> > > 2)
> > > There is some issue in how LOGs (DEBUGs) are getting generated.
> > >
> > > a) Even if there is no drift, it still keeps on dumping:
> > > "logical replication sequence synchronization for subscription "sub1"
> > > - total unsynchronized: 3"
> > >
> >
> > Removed this.
> >
> > > b)
> > > When there is a drift in say single sequence, it puts rest (which are
> > > in sync) to "missing" section:
> > > "logical replication sequence synchronization for subscription "sub1"
> > > - batch #1 = 3 attempted, 1 succeeded, 0 mismatched, 0 insufficient
> > > permission, 2 missing from publisher, 0 skipped"
> > >
> >
> > Fixed, and added a new section called "no drift". Also now this debug
> > message is printed every time the worker attempts to synchronize
> > sequences. also mentioning the state of the sequences being synced.
> >
> > > 3)
> > > If a sequence sync worker is taking a nap, and subscription is
> > > disabled or the server is stopped just before upgrade, how is the user
> > > supposed to know that sequences are synced at the end?
> >
> > Well, one way is to wait for a debug message that says that all the
> > attempted sequences are in the "no drift" state. Also remote
> > sequence's LSN is updated in pg_subscription_rel for each sequence.
> > Let me know if you have anything more in mind. One option is to leave
> > the ALTER SUBSCRIPTION..REFRESH SEQUENCE in place, that will change
> > the state of all the sequences to the INIT state, and the user can
> > then wait for the sequences to change state to READY.
>
> I think this point needs more analysis. I am analyzing and discussing
> it further.
>
> > Attaching patch v3 addressing the above comments.
> >
>
> Thank You for the patch. I haven’t reviewed the details yet, but it
> would be good to evaluate whether we really need to start the sequence
> sync worker so deep inside the apply worker. Currently, the caller of
> ProcessSequencesForSync(), namely ProcessSyncingRelations() is invoked
> for every apply message to process possible state-changes of relation
> and start worker (tablesync/sequence-sync etc). Since monitoring
> state-change behavior is no longer required for sequences, should we
> consider moving this logic to the main loop of the apply worker,
> possibly within LogicalRepApplyLoop()? There, we could check whether
> the table has any sequences and, if a worker is not already running,
> start one. Thoughts?
Correction to last line:
There, we could check whether *the current susbcription* has any
sequences and, if a worker is not already running,start one.
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-11 03:30 ` shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-02-11 03:30 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Fri, Feb 6, 2026 at 2:47 PM shveta malik <[email protected]> wrote:
>
> On Fri, Feb 6, 2026 at 2:45 PM shveta malik <[email protected]> wrote:
> >
> > On Thu, Feb 5, 2026 at 10:33 AM Ajin Cherian <[email protected]> wrote:
> > >
> > > On Tue, Feb 3, 2026 at 9:22 PM shveta malik <[email protected]> wrote:
> > > >
> > > > On Tue, Feb 3, 2026 at 9:18 AM Ajin Cherian <[email protected]> wrote:
> > > > Thanks for the patch.
> > > >
> > > > +1 for the overall idea of patch that once a subscription is created
> > > > which subscribes to sequences, a sequence sync worker is started which
> > > > continuously syncs the sequences. This makes usage of REFRESH
> > > > SEQUENCES redundant and thus it is removed. I am still reviewing the
> > > > design choice here, and will post my comments soon (if any).
> > > >
> > >
> > > Thanks!
> > >
> > > > By quick validation, few issues in current implementation:
> > > >
> > > > 1)
> > > > If the sequence sync worker exits due to some issue (or killed or
> > > > server restarts), sequence-sync worker is not started again by apply
> > > > worker unless there is a sequence in INIT state i.e. synchronization
> > > > of sequences in READY state stops. IIUC, the logic of
> > > > ProcessSequencesForSync() needs to change to start seq sync worker
> > > > irrespective of state of sequences.
> > > >
> > >
> > > Yes, I fixed this. I've changed FetchRelationStates to fetch sequences
> > > in ANY state and not just ones in NON READY state.
> > >
> > > > 2)
> > > > There is some issue in how LOGs (DEBUGs) are getting generated.
> > > >
> > > > a) Even if there is no drift, it still keeps on dumping:
> > > > "logical replication sequence synchronization for subscription "sub1"
> > > > - total unsynchronized: 3"
> > > >
> > >
> > > Removed this.
> > >
> > > > b)
> > > > When there is a drift in say single sequence, it puts rest (which are
> > > > in sync) to "missing" section:
> > > > "logical replication sequence synchronization for subscription "sub1"
> > > > - batch #1 = 3 attempted, 1 succeeded, 0 mismatched, 0 insufficient
> > > > permission, 2 missing from publisher, 0 skipped"
> > > >
> > >
> > > Fixed, and added a new section called "no drift". Also now this debug
> > > message is printed every time the worker attempts to synchronize
> > > sequences. also mentioning the state of the sequences being synced.
> > >
> > > > 3)
> > > > If a sequence sync worker is taking a nap, and subscription is
> > > > disabled or the server is stopped just before upgrade, how is the user
> > > > supposed to know that sequences are synced at the end?
> > >
> > > Well, one way is to wait for a debug message that says that all the
> > > attempted sequences are in the "no drift" state. Also remote
> > > sequence's LSN is updated in pg_subscription_rel for each sequence.
> > > Let me know if you have anything more in mind. One option is to leave
> > > the ALTER SUBSCRIPTION..REFRESH SEQUENCE in place, that will change
> > > the state of all the sequences to the INIT state, and the user can
> > > then wait for the sequences to change state to READY.
> >
> > I think this point needs more analysis. I am analyzing and discussing
> > it further.
> >
> > > Attaching patch v3 addressing the above comments.
> > >
> >
> > Thank You for the patch. I haven’t reviewed the details yet, but it
> > would be good to evaluate whether we really need to start the sequence
> > sync worker so deep inside the apply worker. Currently, the caller of
> > ProcessSequencesForSync(), namely ProcessSyncingRelations() is invoked
> > for every apply message to process possible state-changes of relation
> > and start worker (tablesync/sequence-sync etc). Since monitoring
> > state-change behavior is no longer required for sequences, should we
> > consider moving this logic to the main loop of the apply worker,
> > possibly within LogicalRepApplyLoop()? There, we could check whether
> > the table has any sequences and, if a worker is not already running,
> > start one. Thoughts?
>
> Correction to last line:
> There, we could check whether *the current susbcription* has any
> sequences and, if a worker is not already running,start one.
>
Few more comments:
1)
+ /* Run sync for sequences in READY state */
+ sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
SUBREL_STATE_READY);
+
+ /* Call initial sync for sequences in INIT state */
+ sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
SUBREL_STATE_INIT);
Above logic means we ping primary twice for one seq-sync cycle? Is it
possible to do it in below way:
Get all sequences from the publisher in one go.
If local-seq's state is INIT, copy those sequences, move the state to READY.
If local-seq's state is READY, check the drift and proceed accordingly.
i.e, there should be a single call to LogicalRepSyncSequences() and
relstate shouldn't even be an argument.
2)
GetSequence:
+ /* Open and lock the sequence relation */
+ seqrel = table_open(relid, AccessShareLock);
GetSequence is called from check_sequence_drift and
check_sequence_drift() from copy_sequences(). In copy_sequences(), we
already open the seq's (localrelid) table in
get_and_validate_seq_info() and give it as output (see sequence_rel).
Same can be passed to this instead of trying opening the table again.
3)
+ /* Verify it's actually a sequence */
+ if (seqrel->rd_rel->relkind != RELKIND_SEQUENCE)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not a sequence",
+ RelationGetRelationName(seqrel))));
Do we need this check? IIUC, sequence_rel is opened using
RowExclusiveLock in get_and_validate_seq_info() and thus should not
hit this case so that it becomes non-sequence later on. If required,
we can have Assert.
I can review further once these and previous comments are addressed.
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-12 09:23 ` Ajin Cherian <[email protected]>
2026-02-12 10:02 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-13 05:42 ` Re: [PATCH] Support automatic sequence replication Peter Smith <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
0 siblings, 3 replies; 58+ messages in thread
From: Ajin Cherian @ 2026-02-12 09:23 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>
On Fri, Feb 6, 2026 at 8:15 PM shveta malik <[email protected]> wrote:
>
> On Thu, Feb 5, 2026 at 10:33 AM Ajin Cherian <[email protected]> wrote:
> Thank You for the patch. I haven’t reviewed the details yet, but it
> would be good to evaluate whether we really need to start the sequence
> sync worker so deep inside the apply worker. Currently, the caller of
> ProcessSequencesForSync(), namely ProcessSyncingRelations() is invoked
> for every apply message to process possible state-changes of relation
> and start worker (tablesync/sequence-sync etc). Since monitoring
> state-change behavior is no longer required for sequences, should we
> consider moving this logic to the main loop of the apply worker,
> possibly within LogicalRepApplyLoop()? There, we could check whether
> the table has any sequences and, if a worker is not already running,
> start one. Thoughts?
>
I've moved this to LogicalRepApplyLoop()
On Mon, Feb 9, 2026 at 10:55 AM Peter Smith <[email protected]> wrote:
>
> Some review comments for v3-0001.
>
> ======
> .../replication/logical/sequencesync.c
>
> copy_sequences:
>
> 1.
> - if (sync_status == COPYSEQ_SUCCESS)
> +
> + /*
> + * For sequences in READY state, only sync if there's drift
> + */
> + if (relstate == SUBREL_STATE_READY && sync_status == COPYSEQ_SUCCESS)
> + {
> + should_sync = check_sequence_drift(seqinfo);
> + if (should_sync)
> + drift_detected = true;
> + }
> +
> + if (sync_status == COPYSEQ_SUCCESS && should_sync)
> sync_status = copy_sequence(seqinfo,
> - sequence_rel->rd_rel->relowner);
> + sequence_rel->rd_rel->relowner,
> + relstate);
> + else if (sync_status == COPYSEQ_SUCCESS && !should_sync)
> + sync_status = COPYSEQ_NOWORK;
>
> I'm struggling somewhat to follow this logic.
>
> For example, every condition refers to COPYSEQ_SUCCESS -- is that
> really necessary; can't this all be boiled down to something like
> below?
>
> SUGGESTION
>
> /*
> * For sequences in INIT state, always sync.
> * Otherwise, for sequences in READY state, only sync if there's drift.
> */
> if (sync_status == COPYSEQ_SUCCESS)
> {
> if ((relstate == SUBREL_STATE_INIT) || check_sequence_drift(seqinfo))
> sync_status = copy_sequence(...);
> else
> sync_status = COPYSEQ_NOWORK;
> }
Changed as suggested.
On Wed, Feb 11, 2026 at 2:30 PM shveta malik <[email protected]> wrote:
>
> On Fri, Feb 6, 2026 at 2:47 PM shveta malik <[email protected]> wrote:
> >
> Few more comments:
>
> 1)
> + /* Run sync for sequences in READY state */
> + sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
> SUBREL_STATE_READY);
> +
> + /* Call initial sync for sequences in INIT state */
> + sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
> SUBREL_STATE_INIT);
>
> Above logic means we ping primary twice for one seq-sync cycle? Is it
> possible to do it in below way:
>
> Get all sequences from the publisher in one go.
> If local-seq's state is INIT, copy those sequences, move the state to READY.
> If local-seq's state is READY, check the drift and proceed accordingly.
>
> i.e, there should be a single call to LogicalRepSyncSequences() and
> relstate shouldn't even be an argument.
>
Changed as suggested.
> 2)
> GetSequence:
> + /* Open and lock the sequence relation */
> + seqrel = table_open(relid, AccessShareLock);
>
> GetSequence is called from check_sequence_drift and
> check_sequence_drift() from copy_sequences(). In copy_sequences(), we
> already open the seq's (localrelid) table in
> get_and_validate_seq_info() and give it as output (see sequence_rel).
> Same can be passed to this instead of trying opening the table again.
>
> 3)
> + /* Verify it's actually a sequence */
> + if (seqrel->rd_rel->relkind != RELKIND_SEQUENCE)
> + ereport(ERROR,
> + (errcode(ERRCODE_WRONG_OBJECT_TYPE),
> + errmsg("\"%s\" is not a sequence",
> + RelationGetRelationName(seqrel))));
>
Changed these.
Other than these, I've changed seqinfos from being a global static
list to a list that gets passed around and destroyed in each
iteration.
I haven't addressed the upgrade issue raised by Vignesh and Shveta in
this patch. I will address that in the next patch.
Here's patch v4 addressing the above comments.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v4-0001-Support-automatic-sequence-replication.patch (42.9K, 2-v4-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From db053c5e3e6a9738f32d0f3223282dd3d128bc8a Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Thu, 12 Feb 2026 20:16:16 +1100
Subject: [PATCH v4] Support automatic sequence replication.
Logical replication sequences can drift between publisher and
subscriber as values are consumed independently on each node.
Previously, the sequence sync worker exited after the initial
synchronization, allowing sequences to diverge over time.
This change keeps the sequence sync worker running continuously
so it can monitor sequences and resynchronize them when drift
is detected. The worker uses an adaptive sleep interval:
it starts at 2 seconds, doubles up to a maximum of 30 seconds
when no drift is observed, and resets to the minimum interval
once drift is found.
Since synchronization is now continuous,
ALTER SUBSCRIPTION ... REFRESH SEQUENCES is no longer needed
and has been removed.
Sequences remain in the READY state during continuous
synchronization.
Author: Ajin Cherian [email protected]
---
doc/src/sgml/catalogs.sgml | 6 +-
doc/src/sgml/logical-replication.sgml | 56 +---
doc/src/sgml/ref/alter_subscription.sgml | 30 --
src/backend/commands/sequence.c | 27 ++
src/backend/commands/subscriptioncmds.c | 65 ----
src/backend/parser/gram.y | 9 -
.../replication/logical/sequencesync.c | 294 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 48 ++-
src/backend/replication/logical/worker.c | 3 +
src/bin/psql/tab-complete.in.c | 4 +-
src/include/commands/sequence.h | 1 +
src/include/nodes/parsenodes.h | 1 -
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 71 +----
14 files changed, 294 insertions(+), 323 deletions(-)
mode change 100644 => 100755 src/backend/replication/logical/sequencesync.c
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 332193565e2..ec51a87c253 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8261,11 +8261,9 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<para>
This catalog contains tables and sequences known to the subscription
after running:
- <link linkend="sql-createsubscription"><command>CREATE SUBSCRIPTION</command></link>,
+ <link linkend="sql-createsubscription"><command>CREATE SUBSCRIPTION</command></link>, or
<link linkend="sql-altersubscription-params-refresh-publication">
- <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command></link>, or
- <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
+ <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command></link>.
</para>
<table>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..cfe9c2788c4 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1775,20 +1775,14 @@ Publications:
to synchronize only newly added sequences.
</para>
</listitem>
- <listitem>
- <para>
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
- to re-synchronize all sequences currently known to the subscription.
- </para>
- </listitem>
</itemizedlist>
</para>
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1815,21 +1809,7 @@ Publications:
</sect2>
<sect2 id="sequences-out-of-sync">
- <title>Refreshing Out-of-Sync Sequences</title>
- <para>
- Subscriber sequence values will become out of sync as the publisher
- advances them.
- </para>
- <para>
- To detect this, compare the
- <link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
- on the subscriber with the <structfield>page_lsn</structfield> obtained
- from the <link linkend="func-pg-get-sequence-data"><function>pg_get_sequence_data</function></link>
- function for the sequence on the publisher. Then run
- <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> to
- re-synchronize if necessary.
- </para>
+ <title>Out-of-Sync Sequences</title>
<warning>
<para>
Each sequence caches a block of values (typically 32) in memory before
@@ -1961,16 +1941,6 @@ Publications:
------------+-----------+------------
610 | t | 0/017CEDF8
(1 row)
-</programlisting></para>
-
- <para>
- The difference between the sequence page LSNs on the publisher and the
- sequence page LSNs on the subscriber indicates that the sequences are out
- of sync. Re-synchronize all sequences known to the subscriber using
- <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
-<programlisting>
-/* sub # */ ALTER SUBSCRIPTION sub1 REFRESH SEQUENCES;
</programlisting></para>
<para>
@@ -2333,24 +2303,6 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
</para>
</listitem>
- <listitem>
- <para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
- or by copying the current data from the publisher (perhaps using
- <command>pg_dump</command>) or by determining a sufficiently high value
- from the tables themselves.
- </para>
- </listitem>
-
<listitem>
<para>
Replication of <command>TRUNCATE</command> commands is supported, but
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 27c06439f4f..c0fd426d712 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -26,7 +26,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> SET PUBLICA
ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> ADD PUBLICATION <replaceable class="parameter">publication_name</replaceable> [, ...] [ WITH ( <replaceable class="parameter">publication_option</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) ]
ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> DROP PUBLICATION <replaceable class="parameter">publication_name</replaceable> [, ...] [ WITH ( <replaceable class="parameter">publication_option</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) ]
ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> REFRESH PUBLICATION [ WITH ( <replaceable class="parameter">refresh_option</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) ]
-ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> REFRESH SEQUENCES
ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> ENABLE
ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> DISABLE
ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">subscription_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
@@ -190,11 +189,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
@@ -219,30 +213,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
</listitem>
</varlistentry>
- <varlistentry id="sql-altersubscription-params-refresh-sequences">
- <term><literal>REFRESH SEQUENCES</literal></term>
- <listitem>
- <para>
- Re-synchronize sequence data with the publisher. Unlike
- <link linkend="sql-altersubscription-params-refresh-publication">
- <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command></link> which
- only has the ability to synchronize newly added sequences,
- <literal>REFRESH SEQUENCES</literal> will re-synchronize the sequence
- data for all currently subscribed sequences. It does not add or remove
- sequences from the subscription to match the publication.
- </para>
- <para>
- See <xref linkend="sequence-definition-mismatches"/> for
- recommendations on how to handle any warnings about sequence definition
- differences between the publisher and the subscriber.
- </para>
- <para>
- See <xref linkend="sequences-out-of-sync"/> for recommendations on how to
- identify and handle out-of-sync sequences.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="sql-altersubscription-params-enable">
<term><literal>ENABLE</literal></term>
<listitem>
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..3ca382e89b9 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,33 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation that acquires AccessShareLock on the sequence.
+ * Used by logical replication sequence synchronization to detect drift.
+ */
+void
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+
+ /* Confirm that the relation is a sequence */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 0b3c8499b49..c1b903c5908 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1183,58 +1183,6 @@ AlterSubscription_refresh(Subscription *sub, bool copy_data,
table_close(rel, NoLock);
}
-/*
- * Marks all sequences with INIT state.
- */
-static void
-AlterSubscription_refresh_seq(Subscription *sub)
-{
- char *err = NULL;
- WalReceiverConn *wrconn;
- bool must_use_password;
-
- /* Load the library providing us libpq calls. */
- load_file("libpqwalreceiver", false);
-
- /* Try to connect to the publisher. */
- must_use_password = sub->passwordrequired && !sub->ownersuperuser;
- wrconn = walrcv_connect(sub->conninfo, true, true, must_use_password,
- sub->name, &err);
- if (!wrconn)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("subscription \"%s\" could not connect to the publisher: %s",
- sub->name, err));
-
- PG_TRY();
- {
- List *subrel_states;
-
- check_publications_origin_sequences(wrconn, sub->publications, true,
- sub->origin, NULL, 0, sub->name);
-
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
- }
- PG_FINALLY();
- {
- walrcv_disconnect(wrconn);
- }
- PG_END_TRY();
-}
-
/*
* Common checks for altering failover, two_phase, and retain_dead_tuples
* options.
@@ -1871,19 +1819,6 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
break;
}
- case ALTER_SUBSCRIPTION_REFRESH_SEQUENCES:
- {
- if (!sub->enabled)
- ereport(ERROR,
- errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("%s is not allowed for disabled subscriptions",
- "ALTER SUBSCRIPTION ... REFRESH SEQUENCES"));
-
- AlterSubscription_refresh_seq(sub);
-
- break;
- }
-
case ALTER_SUBSCRIPTION_SKIP:
{
parse_subscription_options(pstate, stmt->options, SUBOPT_LSN, &opts);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 713ee5c10a2..9d362127632 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -11069,15 +11069,6 @@ AlterSubscriptionStmt:
n->options = $6;
$$ = (Node *) n;
}
- | ALTER SUBSCRIPTION name REFRESH SEQUENCES
- {
- AlterSubscriptionStmt *n =
- makeNode(AlterSubscriptionStmt);
-
- n->kind = ALTER_SUBSCRIPTION_REFRESH_SEQUENCES;
- n->subname = $3;
- $$ = (Node *) n;
- }
| ALTER SUBSCRIPTION name ADD_P PUBLICATION name_list opt_definition
{
AlterSubscriptionStmt *n =
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
old mode 100644
new mode 100755
index 165f909b3ba..e4c08ac1063
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronzize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -78,10 +90,15 @@ typedef enum CopySeqResult
COPYSEQ_SUCCESS,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NOWORK,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
/*
* Apply worker determines if sequence synchronization is needed.
@@ -144,7 +161,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +188,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +200,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +211,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +222,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +246,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -325,11 +343,12 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Apply remote sequence state to local sequence. If we are doing this
+ * for sequences in the INIT state, move them to the READY state once
+ * synchronized.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner, char relstate)
{
UserContext ucxt;
AclResult aclresult;
@@ -368,19 +387,46 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (relstate == SUBREL_STATE_INIT)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
+/*
+ * check_sequence_drift
+ *
+ * Check if the remote sequence values differ from the local sequence.
+ * Returns true/false if any sequences drifted.
+ */
+static bool
+check_sequence_drift(Relation sequence_rel, LogicalRepSequenceInfo *seqinfo)
+{
+ int64 local_last_value;
+ bool local_is_called;
+
+ /* Get current local sequence state */
+ GetSequence(sequence_rel, &local_last_value, &local_is_called);
+
+ /* Check if values have drifted and return accordingly */
+ return (local_last_value != seqinfo->last_value ||
+ local_is_called != seqinfo->is_called);
+}
+
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If relstate is SUBREL_STATE_READY, only synchronize sequences that
+ * have drifted from their publisher values. Otherwise, synchronize
+ * all sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos, char relstate)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +436,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +449,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,20 +545,33 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
+ &seqinfo, &seqidx, seqinfos);
+
+ /*
+ * For sequences in INIT state, always sync.
+ * Otherwise, for sequences in READY state, only sync if there's drift.
+ */
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ {
+ if ((relstate == SUBREL_STATE_INIT) || check_sequence_drift(sequence_rel, seqinfo))
+ sync_status = copy_sequence(seqinfo,
+ sequence_rel->rd_rel->relowner,
+ relstate);
+ else
+ sync_status = COPYSEQ_NOWORK;
+ }
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
+ "logical replication sync for subscription \"%s\", sequence \"%s.%s\" has been updated",
MySubscription->name, seqinfo->nspname,
seqinfo->seqname);
batch_succeeded_count++;
+ sequence_copied = true;
break;
+
case COPYSEQ_MISMATCH:
/*
@@ -528,6 +585,7 @@ copy_sequences(WalReceiverConn *conn)
MemoryContextSwitchTo(oldctx);
batch_mismatched_count++;
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
/*
@@ -541,6 +599,7 @@ copy_sequences(WalReceiverConn *conn)
MemoryContextSwitchTo(oldctx);
batch_insuffperm_count++;
break;
+
case COPYSEQ_SKIPPED:
/*
@@ -558,6 +617,15 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+
+ case COPYSEQ_NOWORK:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -572,14 +640,16 @@ copy_sequences(WalReceiverConn *conn)
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
- MySubscription->name,
- (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
- batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ "logical replication sequence synchronization for subscription \"%s\" - for sequences in %s state batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
+ MySubscription->name,
+ (relstate == 'r') ? "READY" : (relstate == 'i') ? "INIT" : "UNKNOWN",
+ (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
+ batch_size, batch_succeeded_count, batch_mismatched_count,
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,51 +677,55 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ List *init_seqs = NIL;
+ List *ready_seqs = NIL;
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
subrel = (Form_pg_subscription_rel) GETSTRUCT(tup);
+ relstate = subrel->srsubstate;
sequence_rel = try_table_open(subrel->srrelid, RowExclusiveLock);
@@ -666,6 +740,19 @@ LogicalRepSyncSequences(void)
continue;
}
+ /* Error on unexpected states */
+ if (relstate != SUBREL_STATE_INIT && relstate != SUBREL_STATE_READY)
+ {
+ table_close(sequence_rel, NoLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("unexpected relstate '%c' for sequence \"%s.%s\" in subscription \"%s\"",
+ relstate,
+ get_namespace_name(RelationGetNamespace(sequence_rel)),
+ RelationGetRelationName(sequence_rel),
+ MySubscription->name)));
+ }
+
/*
* Worker needs to process sequences across transaction boundary, so
* allocate them under long-lived context.
@@ -678,6 +765,12 @@ LogicalRepSyncSequences(void)
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seqinfos = lappend(seqinfos, seq);
+ /* Separate sequences by state for processing */
+ if (relstate == SUBREL_STATE_INIT)
+ init_seqs = lappend(init_seqs, seq);
+ else if (relstate == SUBREL_STATE_READY)
+ ready_seqs = lappend(ready_seqs, seq);
+
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -693,36 +786,30 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
+ return false;
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ /* Process INIT sequences first */
+ if (init_seqs)
+ {
+ sequence_copied |= copy_sequences(conn, init_seqs, SUBREL_STATE_INIT);
+ }
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Then process READY sequences */
+ if (ready_seqs)
+ {
+ sequence_copied |= copy_sequences(conn, ready_seqs, SUBREL_STATE_READY);
+ }
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ /* Clean up */
+ list_free(init_seqs);
+ list_free(ready_seqs);
+ list_free(seqinfos);
+
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +822,67 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ * The function will process INIT sequences first, then READY sequences.
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ /* Adjust sleep interval based on whether sequences were copied over */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+
+ /*
+ * Double the sleep time, but not beyond
+ * the maximum allowable value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index 535ffb6f09e..ac7669bc273 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -171,7 +171,6 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -190,13 +189,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -204,53 +203,47 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
+ * has_subtables and has_subsequences are declared as static,
* since the same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
-
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
-
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
-
if (!IsTransactionState())
{
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
-
+ list_free_deep(seq_states);
/*
* Does the subscription have tables?
*
@@ -260,7 +253,6 @@ FetchRelationStates(bool *has_pending_subtables,
*/
has_subtables = (table_states_not_ready != NIL) ||
HasSubscriptionTables(MySubscription->oid);
-
/*
* If the subscription relation cache has been invalidated since we
* entered this routine, we still use and return the relations we just
@@ -271,10 +263,8 @@ FetchRelationStates(bool *has_pending_subtables,
if (relation_states_validity == SYNC_RELATIONS_STATE_REBUILD_STARTED)
relation_states_validity = SYNC_RELATIONS_STATE_VALID;
}
-
if (has_pending_subtables)
*has_pending_subtables = has_subtables;
-
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 32725c48623..5a9139ace63 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -4219,6 +4219,9 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
ProcessConfigFile(PGC_SIGHUP);
}
+ /* Check if any new sequences need syncing */
+ ProcessSequencesForSync();
+
if (rc & WL_TIMEOUT)
{
/*
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 8b91bc00062..b8bdfebb15a 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2331,8 +2331,8 @@ match_previous_words(int pattern_id,
/* ALTER SUBSCRIPTION <name> */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
- "RENAME TO", "REFRESH PUBLICATION", "REFRESH SEQUENCES",
- "SET", "SKIP (", "ADD PUBLICATION", "DROP PUBLICATION");
+ "RENAME TO", "REFRESH PUBLICATION", "SET", "SKIP (",
+ "ADD PUBLICATION", "DROP PUBLICATION");
/* ALTER SUBSCRIPTION <name> REFRESH */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny, MatchAnyN, "REFRESH"))
COMPLETE_WITH("PUBLICATION", "SEQUENCES");
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..f00eea9fbd1 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern void GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 646d6ced763..43b3292fd2c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -4391,7 +4391,6 @@ typedef enum AlterSubscriptionType
ALTER_SUBSCRIPTION_ADD_PUBLICATION,
ALTER_SUBSCRIPTION_DROP_PUBLICATION,
ALTER_SUBSCRIPTION_REFRESH_PUBLICATION,
- ALTER_SUBSCRIPTION_REFRESH_SEQUENCES,
ALTER_SUBSCRIPTION_ENABLED,
ALTER_SUBSCRIPTION_SKIP,
} AlterSubscriptionType;
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..9c308cf4f2a 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,8 +75,7 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
@@ -97,19 +96,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,12 +105,9 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
# Create a new sequence 'regress_s3', and update the existing sequence
@@ -138,51 +121,25 @@ $node_publisher->safe_psql(
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.47.3
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-12 10:02 ` shveta malik <[email protected]>
2 siblings, 0 replies; 58+ messages in thread
From: shveta malik @ 2026-02-12 10:02 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Thu, Feb 12, 2026 at 2:54 PM Ajin Cherian <[email protected]> wrote:
>
> On Fri, Feb 6, 2026 at 8:15 PM shveta malik <[email protected]> wrote:
> >
> > On Thu, Feb 5, 2026 at 10:33 AM Ajin Cherian <[email protected]> wrote:
>
> > Thank You for the patch. I haven’t reviewed the details yet, but it
> > would be good to evaluate whether we really need to start the sequence
> > sync worker so deep inside the apply worker. Currently, the caller of
> > ProcessSequencesForSync(), namely ProcessSyncingRelations() is invoked
> > for every apply message to process possible state-changes of relation
> > and start worker (tablesync/sequence-sync etc). Since monitoring
> > state-change behavior is no longer required for sequences, should we
> > consider moving this logic to the main loop of the apply worker,
> > possibly within LogicalRepApplyLoop()? There, we could check whether
> > the table has any sequences and, if a worker is not already running,
> > start one. Thoughts?
> >
>
> I've moved this to LogicalRepApplyLoop()
>
>
>
> On Mon, Feb 9, 2026 at 10:55 AM Peter Smith <[email protected]> wrote:
> >
> > Some review comments for v3-0001.
> >
> > ======
> > .../replication/logical/sequencesync.c
> >
> > copy_sequences:
> >
> > 1.
> > - if (sync_status == COPYSEQ_SUCCESS)
> > +
> > + /*
> > + * For sequences in READY state, only sync if there's drift
> > + */
> > + if (relstate == SUBREL_STATE_READY && sync_status == COPYSEQ_SUCCESS)
> > + {
> > + should_sync = check_sequence_drift(seqinfo);
> > + if (should_sync)
> > + drift_detected = true;
> > + }
> > +
> > + if (sync_status == COPYSEQ_SUCCESS && should_sync)
> > sync_status = copy_sequence(seqinfo,
> > - sequence_rel->rd_rel->relowner);
> > + sequence_rel->rd_rel->relowner,
> > + relstate);
> > + else if (sync_status == COPYSEQ_SUCCESS && !should_sync)
> > + sync_status = COPYSEQ_NOWORK;
> >
> > I'm struggling somewhat to follow this logic.
> >
> > For example, every condition refers to COPYSEQ_SUCCESS -- is that
> > really necessary; can't this all be boiled down to something like
> > below?
> >
> > SUGGESTION
> >
> > /*
> > * For sequences in INIT state, always sync.
> > * Otherwise, for sequences in READY state, only sync if there's drift.
> > */
> > if (sync_status == COPYSEQ_SUCCESS)
> > {
> > if ((relstate == SUBREL_STATE_INIT) || check_sequence_drift(seqinfo))
> > sync_status = copy_sequence(...);
> > else
> > sync_status = COPYSEQ_NOWORK;
> > }
>
> Changed as suggested.
>
>
>
> On Wed, Feb 11, 2026 at 2:30 PM shveta malik <[email protected]> wrote:
> >
> > On Fri, Feb 6, 2026 at 2:47 PM shveta malik <[email protected]> wrote:
> > >
> > Few more comments:
> >
> > 1)
> > + /* Run sync for sequences in READY state */
> > + sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
> > SUBREL_STATE_READY);
> > +
> > + /* Call initial sync for sequences in INIT state */
> > + sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
> > SUBREL_STATE_INIT);
> >
> > Above logic means we ping primary twice for one seq-sync cycle? Is it
> > possible to do it in below way:
> >
> > Get all sequences from the publisher in one go.
> > If local-seq's state is INIT, copy those sequences, move the state to READY.
> > If local-seq's state is READY, check the drift and proceed accordingly.
> >
> > i.e, there should be a single call to LogicalRepSyncSequences() and
> > relstate shouldn't even be an argument.
> >
>
> Changed as suggested.
>
> > 2)
> > GetSequence:
> > + /* Open and lock the sequence relation */
> > + seqrel = table_open(relid, AccessShareLock);
> >
> > GetSequence is called from check_sequence_drift and
> > check_sequence_drift() from copy_sequences(). In copy_sequences(), we
> > already open the seq's (localrelid) table in
> > get_and_validate_seq_info() and give it as output (see sequence_rel).
> > Same can be passed to this instead of trying opening the table again.
> >
> > 3)
> > + /* Verify it's actually a sequence */
> > + if (seqrel->rd_rel->relkind != RELKIND_SEQUENCE)
> > + ereport(ERROR,
> > + (errcode(ERRCODE_WRONG_OBJECT_TYPE),
> > + errmsg("\"%s\" is not a sequence",
> > + RelationGetRelationName(seqrel))));
> >
>
> Changed these.
>
> Other than these, I've changed seqinfos from being a global static
> list to a list that gets passed around and destroyed in each
> iteration.
> I haven't addressed the upgrade issue raised by Vignesh and Shveta in
> this patch. I will address that in the next patch.
Thanks Ajin, I will review. I will suggest waiting for others'
feedback before doing anything about the upgrade issue. Meanwhile, we
can try to improve the current patch itself.
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-13 05:42 ` Peter Smith <[email protected]>
2 siblings, 0 replies; 58+ messages in thread
From: Peter Smith @ 2026-02-13 05:42 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: shveta malik <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi Ajin.
Some review comments for patch v4-0001
======
src/backend/commands/sequence.c
GetSequence:
1.
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation that acquires AccessShareLock on the sequence.
+ * Used by logical replication sequence synchronization to detect drift.
+ */
The comment seems stale. e.g. the function is not acquiring a lock
anymore, contrary to what that comment says.
======
.../replication/logical/sequencesync.c
2.
-static List *seqinfos = NIL;
The removal of this global was not strictly part of this patch; it is
more like a prerequisite to make everything tidier, so your new code
does not go further down that track of side-affecting a global. From
that POV, I thought this global removal should be implemented as a
first/separate (0001) patch so that it might be quickly reviewed and
committed independently of the new seq-sync logic.
~~~
LogicalRepSyncSequences:
3.
+ /* Error on unexpected states */
+ if (relstate != SUBREL_STATE_INIT && relstate != SUBREL_STATE_READY)
+ {
+ table_close(sequence_rel, NoLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("unexpected relstate '%c' for sequence \"%s.%s\" in
subscription \"%s\"",
+ relstate,
+ get_namespace_name(RelationGetNamespace(sequence_rel)),
+ RelationGetRelationName(sequence_rel),
+ MySubscription->name)));
+ }
+
How is this possible? Should it just be Assert?
~~~
start_sequence_sync:
4.
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ * The function will process INIT sequences first, then READY sequences.
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
Why is talking about the processing order relevant?
======
src/backend/replication/logical/syncutils.c
5.
+ /* Check if any new sequences need syncing */
+ ProcessSequencesForSync();
+
Maybe don't say "new" because IIUC it also handles older sequences
where the values have drifted.
======
src/test/subscription/t/036_sequences.pl
6.
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
Since you are no longer testing "existing sequences" in this test
part, should you also remove the earlier INSERT for 'regress_s1'?
======
Kind Regards,
Peter Smith.
Fujitsu Australia.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-17 11:21 ` Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2 siblings, 1 reply; 58+ messages in thread
From: Ashutosh Sharma @ 2026-02-17 11:21 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: shveta malik <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On Thu, Feb 12, 2026 at 2:54 PM Ajin Cherian <[email protected]> wrote:
>
> On Fri, Feb 6, 2026 at 8:15 PM shveta malik <[email protected]> wrote:
> >
> > On Thu, Feb 5, 2026 at 10:33 AM Ajin Cherian <[email protected]> wrote:
>
> > Thank You for the patch. I haven’t reviewed the details yet, but it
> > would be good to evaluate whether we really need to start the sequence
> > sync worker so deep inside the apply worker. Currently, the caller of
> > ProcessSequencesForSync(), namely ProcessSyncingRelations() is invoked
> > for every apply message to process possible state-changes of relation
> > and start worker (tablesync/sequence-sync etc). Since monitoring
> > state-change behavior is no longer required for sequences, should we
> > consider moving this logic to the main loop of the apply worker,
> > possibly within LogicalRepApplyLoop()? There, we could check whether
> > the table has any sequences and, if a worker is not already running,
> > start one. Thoughts?
> >
>
> I've moved this to LogicalRepApplyLoop()
>
>
>
> On Mon, Feb 9, 2026 at 10:55 AM Peter Smith <[email protected]> wrote:
> >
> > Some review comments for v3-0001.
> >
> > ======
> > .../replication/logical/sequencesync.c
> >
> > copy_sequences:
> >
> > 1.
> > - if (sync_status == COPYSEQ_SUCCESS)
> > +
> > + /*
> > + * For sequences in READY state, only sync if there's drift
> > + */
> > + if (relstate == SUBREL_STATE_READY && sync_status == COPYSEQ_SUCCESS)
> > + {
> > + should_sync = check_sequence_drift(seqinfo);
> > + if (should_sync)
> > + drift_detected = true;
> > + }
> > +
> > + if (sync_status == COPYSEQ_SUCCESS && should_sync)
> > sync_status = copy_sequence(seqinfo,
> > - sequence_rel->rd_rel->relowner);
> > + sequence_rel->rd_rel->relowner,
> > + relstate);
> > + else if (sync_status == COPYSEQ_SUCCESS && !should_sync)
> > + sync_status = COPYSEQ_NOWORK;
> >
> > I'm struggling somewhat to follow this logic.
> >
> > For example, every condition refers to COPYSEQ_SUCCESS -- is that
> > really necessary; can't this all be boiled down to something like
> > below?
> >
> > SUGGESTION
> >
> > /*
> > * For sequences in INIT state, always sync.
> > * Otherwise, for sequences in READY state, only sync if there's drift.
> > */
> > if (sync_status == COPYSEQ_SUCCESS)
> > {
> > if ((relstate == SUBREL_STATE_INIT) || check_sequence_drift(seqinfo))
> > sync_status = copy_sequence(...);
> > else
> > sync_status = COPYSEQ_NOWORK;
> > }
>
> Changed as suggested.
>
>
>
> On Wed, Feb 11, 2026 at 2:30 PM shveta malik <[email protected]> wrote:
> >
> > On Fri, Feb 6, 2026 at 2:47 PM shveta malik <[email protected]> wrote:
> > >
> > Few more comments:
> >
> > 1)
> > + /* Run sync for sequences in READY state */
> > + sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
> > SUBREL_STATE_READY);
> > +
> > + /* Call initial sync for sequences in INIT state */
> > + sequence_copied |= LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
> > SUBREL_STATE_INIT);
> >
> > Above logic means we ping primary twice for one seq-sync cycle? Is it
> > possible to do it in below way:
> >
> > Get all sequences from the publisher in one go.
> > If local-seq's state is INIT, copy those sequences, move the state to READY.
> > If local-seq's state is READY, check the drift and proceed accordingly.
> >
> > i.e, there should be a single call to LogicalRepSyncSequences() and
> > relstate shouldn't even be an argument.
> >
>
> Changed as suggested.
>
> > 2)
> > GetSequence:
> > + /* Open and lock the sequence relation */
> > + seqrel = table_open(relid, AccessShareLock);
> >
> > GetSequence is called from check_sequence_drift and
> > check_sequence_drift() from copy_sequences(). In copy_sequences(), we
> > already open the seq's (localrelid) table in
> > get_and_validate_seq_info() and give it as output (see sequence_rel).
> > Same can be passed to this instead of trying opening the table again.
> >
> > 3)
> > + /* Verify it's actually a sequence */
> > + if (seqrel->rd_rel->relkind != RELKIND_SEQUENCE)
> > + ereport(ERROR,
> > + (errcode(ERRCODE_WRONG_OBJECT_TYPE),
> > + errmsg("\"%s\" is not a sequence",
> > + RelationGetRelationName(seqrel))));
> >
>
> Changed these.
>
> Other than these, I've changed seqinfos from being a global static
> list to a list that gets passed around and destroyed in each
> iteration.
> I haven't addressed the upgrade issue raised by Vignesh and Shveta in
> this patch. I will address that in the next patch.
> Here's patch v4 addressing the above comments.
>
How about retaining ALTER SUBSCRIPTION ... REFRESH SEQUENCES command?
It can be useful in scenarios where automatic sequence replication
cannot be enabled, for example, due to the max_worker_processes limit
on the server preventing a new worker from being registered. Since
increasing max_worker_processes requires a server restart, which the
user may not want to perform immediately, this command would provide a
way to manually synchronize the sequences.
Thoughts?
--
One minor comment:
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronzize)
+ * | |
+ * +--<---+
"synchronzize" → "synchronize"
--
With Regards,
Ashutosh Sharma.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
@ 2026-02-18 04:48 ` Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-02-18 04:48 UTC (permalink / raw)
To: Ashutosh Sharma <[email protected]>; +Cc: Ajin Cherian <[email protected]>; shveta malik <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 17, 2026 at 4:51 PM Ashutosh Sharma <[email protected]> wrote:
>
>
> How about retaining ALTER SUBSCRIPTION ... REFRESH SEQUENCES command?
>
Yes, Shveta and myself also advocated the same for the upgrade case.
> It can be useful in scenarios where automatic sequence replication
> cannot be enabled, for example, due to the max_worker_processes limit
> on the server preventing a new worker from being registered. Since
> increasing max_worker_processes requires a server restart, which the
> user may not want to perform immediately, this command would provide a
> way to manually synchronize the sequences.
>
Good point. I think we won't be able to launch sync workers even when
we reach max_logical_replication_workers limit. So, this is one more
reason to retain the REFRESH SEQUENCES command.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-02-18 07:06 ` shveta malik <[email protected]>
2026-02-18 07:41 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 2 replies; 58+ messages in thread
From: shveta malik @ 2026-02-18 07:06 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>; shveta malik <[email protected]>
I tested a few scenarios on the latest patch. Sequence sync worker did
not stop in below scenarios:
1) When the subscription was disabled.
2) When the only publication for sequences was dropped from
subscription ( ALTER SUBSCRIPTION sub1 DROP PUBLICATION pub_seq;)
3) When all sequences were dropped on sub.
Application worker stops in scenario 1, seq-sync worker should also
stop. See maybe_reread_subscription().
We need to decide the behavior of the seq-sync worker for 2 and 3.
IMO, for scenario 2, we should stop the sequence sync worker. It is of
no use to keep holding one worker slot when there are no publisher
publishing sequences for that subscription.
For scenario 3, it might be acceptable(or may be even expected?) to
keep the seq-sync worker running. But the challenge is how we would
distinguish scenario 2 from scenario 3. I believe scenario 2 will rely
on the absence of sequence entries in pg_subscription_rel to sotp
seq-sync worker. But in both scenario 2 and scenario 3, there would be
no sequence entries in pg_subscription_rel. Given that, to keep the
logic simpler, we can stop the seq-sync worker in scenario 3 as well.
This seems like a corner case, and it should not cause much harm to
stop the worker and restart it later when needed.
Thoughts?
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-18 07:41 ` Amit Kapila <[email protected]>
2026-02-18 11:28 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-02-18 07:41 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, Feb 18, 2026 at 12:36 PM shveta malik <[email protected]> wrote:
>
> I tested a few scenarios on the latest patch. Sequence sync worker did
> not stop in below scenarios:
>
> 1) When the subscription was disabled.
> 2) When the only publication for sequences was dropped from
> subscription ( ALTER SUBSCRIPTION sub1 DROP PUBLICATION pub_seq;)
> 3) When all sequences were dropped on sub.
>
> Application worker stops in scenario 1, seq-sync worker should also
> stop. See maybe_reread_subscription().
>
> We need to decide the behavior of the seq-sync worker for 2 and 3.
>
Shouldn't we try to map (2) and (3) to what we do for table publication?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 07:41 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-02-18 11:28 ` shveta malik <[email protected]>
2026-02-19 02:15 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-02-18 11:28 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Wed, Feb 18, 2026 at 1:12 PM Amit Kapila <[email protected]> wrote:
>
> On Wed, Feb 18, 2026 at 12:36 PM shveta malik <[email protected]> wrote:
> >
> > I tested a few scenarios on the latest patch. Sequence sync worker did
> > not stop in below scenarios:
> >
> > 1) When the subscription was disabled.
> > 2) When the only publication for sequences was dropped from
> > subscription ( ALTER SUBSCRIPTION sub1 DROP PUBLICATION pub_seq;)
> > 3) When all sequences were dropped on sub.
> >
> > Application worker stops in scenario 1, seq-sync worker should also
> > stop. See maybe_reread_subscription().
> >
> > We need to decide the behavior of the seq-sync worker for 2 and 3.
> >
>
> Shouldn't we try to map (2) and (3) to what we do for table publication?
>
I thought about it, this is what I have in mind:
1) When there is no sequences and tables to be synced, we will be
blocking 2 workers slot, one for apply worker and one for seq-sync
worker. While only apply-worker is enough, as it may restart seq-sync
worker as soon as it notices a sequence.
2) In case of apply-worker, when no tables are being published, it
hardly does any work. IIUC, it just sends responses to keep-alive
messages. OTOH, seq-sync worker will begin a transaction and query
pg_subscription_rel every few seconds. I feel, we should avoid this
unnecessary activity if possible.
Overall, I feel the sequence sync worker is logically different from
the table apply worker. It behaves more like an auxiliary worker
managed by the apply worker and derives all of its state from the
system catalogs. In contrast, the apply worker actively consumes the
logical replication stream and is responsible for advancing the slot
and origin. If the apply worker is stopped for an extended period, the
slot cannot advance, which may lead to WAL accumulation on the
publisher. There is no such constraint associated with the seq-sync
worker. And thus stopping the seq-sync worker aggressively (in
scenarios 2 and 3) is both safer and simpler as compared to apply
worker and will also avoid some unnecessary transactions and activity
as I stated above. Thoughts?
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 07:41 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 11:28 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-19 02:15 ` Amit Kapila <[email protected]>
2026-02-19 04:19 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-02-19 02:15 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, Feb 18, 2026 at 4:58 PM shveta malik <[email protected]> wrote:
>
> On Wed, Feb 18, 2026 at 1:12 PM Amit Kapila <[email protected]> wrote:
> >
> > On Wed, Feb 18, 2026 at 12:36 PM shveta malik <[email protected]> wrote:
> > >
> > > I tested a few scenarios on the latest patch. Sequence sync worker did
> > > not stop in below scenarios:
> > >
> > > 1) When the subscription was disabled.
> > > 2) When the only publication for sequences was dropped from
> > > subscription ( ALTER SUBSCRIPTION sub1 DROP PUBLICATION pub_seq;)
> > > 3) When all sequences were dropped on sub.
> > >
> > > Application worker stops in scenario 1, seq-sync worker should also
> > > stop. See maybe_reread_subscription().
> > >
> > > We need to decide the behavior of the seq-sync worker for 2 and 3.
> > >
> >
> > Shouldn't we try to map (2) and (3) to what we do for table publication?
> >
>
> I thought about it, this is what I have in mind:
>
> 1) When there is no sequences and tables to be synced, we will be
> blocking 2 workers slot, one for apply worker and one for seq-sync
> worker. While only apply-worker is enough, as it may restart seq-sync
> worker as soon as it notices a sequence.
>
> 2) In case of apply-worker, when no tables are being published, it
> hardly does any work. IIUC, it just sends responses to keep-alive
> messages. OTOH, seq-sync worker will begin a transaction and query
> pg_subscription_rel every few seconds. I feel, we should avoid this
> unnecessary activity if possible.
>
> Overall, I feel the sequence sync worker is logically different from
> the table apply worker. It behaves more like an auxiliary worker
> managed by the apply worker and derives all of its state from the
> system catalogs.
>
I understand that sequence-sync worker won't be doing anything useful
in such corner cases but the chances of such situations are rare and
the consequences are not that bad. Similarly, one can say that we can
exit the launcher process when no subscription is present and the
system should figure out and restart when required. In this case also,
the apply-worker needs to check from time-to-time or needs to be
informed to launch the new sequence-sync worker. I think we can
evaluate these cases separately and can decide to write a top-up patch
if found useful to make sequence-sync worker detect and exit.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 07:41 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 11:28 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-19 02:15 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-02-19 04:19 ` shveta malik <[email protected]>
0 siblings, 0 replies; 58+ messages in thread
From: shveta malik @ 2026-02-19 04:19 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Thu, Feb 19, 2026 at 7:45 AM Amit Kapila <[email protected]> wrote:
>
> On Wed, Feb 18, 2026 at 4:58 PM shveta malik <[email protected]> wrote:
> >
> > On Wed, Feb 18, 2026 at 1:12 PM Amit Kapila <[email protected]> wrote:
> > >
> > > On Wed, Feb 18, 2026 at 12:36 PM shveta malik <[email protected]> wrote:
> > > >
> > > > I tested a few scenarios on the latest patch. Sequence sync worker did
> > > > not stop in below scenarios:
> > > >
> > > > 1) When the subscription was disabled.
> > > > 2) When the only publication for sequences was dropped from
> > > > subscription ( ALTER SUBSCRIPTION sub1 DROP PUBLICATION pub_seq;)
> > > > 3) When all sequences were dropped on sub.
> > > >
> > > > Application worker stops in scenario 1, seq-sync worker should also
> > > > stop. See maybe_reread_subscription().
> > > >
> > > > We need to decide the behavior of the seq-sync worker for 2 and 3.
> > > >
> > >
> > > Shouldn't we try to map (2) and (3) to what we do for table publication?
> > >
> >
> > I thought about it, this is what I have in mind:
> >
> > 1) When there is no sequences and tables to be synced, we will be
> > blocking 2 workers slot, one for apply worker and one for seq-sync
> > worker. While only apply-worker is enough, as it may restart seq-sync
> > worker as soon as it notices a sequence.
> >
> > 2) In case of apply-worker, when no tables are being published, it
> > hardly does any work. IIUC, it just sends responses to keep-alive
> > messages. OTOH, seq-sync worker will begin a transaction and query
> > pg_subscription_rel every few seconds. I feel, we should avoid this
> > unnecessary activity if possible.
> >
> > Overall, I feel the sequence sync worker is logically different from
> > the table apply worker. It behaves more like an auxiliary worker
> > managed by the apply worker and derives all of its state from the
> > system catalogs.
> >
>
> I understand that sequence-sync worker won't be doing anything useful
> in such corner cases but the chances of such situations are rare and
> the consequences are not that bad. Similarly, one can say that we can
> exit the launcher process when no subscription is present and the
> system should figure out and restart when required.
No, I don’t think this is comparable to our case. For that scenario to
work, we would need to expose subscription-related awareness and state
to the postmaster, which is not recommended. In our case, the apply
worker is already responsible for monitoring the sequence-sync worker.
It is already checking two things:
--whether the subscription includes sequences, and
--whether the sequence-sync worker is running and if not, it starts it.
So there is no additional logic required. None. We don’t need any
extra awareness in the apply worker. Given that, I don’t think the two
cases are comparable at all.
But I also agree that it is a corner case, and running additional
seq-sync worker per subscription may not harm that much.
> In this case also,
> the apply-worker needs to check from time-to-time or needs to be
> informed to launch the new sequence-sync worker.
It is already doing that. There is always a case where seq-sync worker
errors out or exits unexpectedly. In such a case, the apply worker is
the one to start it again. And for that, apply worker has to keep
checking it.
> I think we can
> evaluate these cases separately and can decide to write a top-up patch
> if found useful to make sequence-sync worker detect and exit.
>
Sure, we can do that.
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-18 10:04 ` shveta malik <[email protected]>
2026-02-18 21:56 ` Re: [PATCH] Support automatic sequence replication Peter Smith <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
1 sibling, 2 replies; 58+ messages in thread
From: shveta malik @ 2026-02-18 10:04 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>; shveta malik <[email protected]>
Few comments:
1)
+ /* Check if any new sequences need syncing */
+ ProcessSequencesForSync();
+
Shall we name it 'maybe_start_seqsync_worker()' (or something better?)
and move it immediately after 'maybe_advance_nonremovable_xid()'.
Thoughts? This is because we do not process sequences here, we just
check if the subscription includes sequences and start worker.
2)
We can also change the comment atop ProcessSequencesForSync to mention
that. Currently it says:
/*
* Apply worker determines if sequence synchronization is needed.
*
* Start a sequencesync worker if one is not already running.
Now, we shall change it to:
Apply worker determines whether a sequence sync worker is needed.
Check if the subscription includes sequences and start a sequencesync
worker if one is not already running.
3)
I noticed that copy_sequences is invoked twice per sync cycle—once for
sequences in the INIT state, and once for sequences in the READY state
to detect drift. This means we ping the primary twice during each sync
cycle. We should consider combining this logic into a single
copy_sequences call. Since we already have the state information
(INIT, READY, etc.) for each local sequence, copy_sequences can
internally check the current state and decide whether to transition a
sequence to READY based on its previous state. This approach would
allow us to fetch all required information from the primary in a
single call.
I also think that we do not even need to mention the states here and
we can give a single message instead of 2:
DEBUG: logical replication sequence synchronization for subscription
"sub1" - for sequences in INIT state batch #1 = 5 attempted, 5
succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
publisher, 0 skipped, 0 no drift
DEBUG: logical replication sequence synchronization for subscription
"sub1" - for sequences in READY state batch #1 = 5 attempted, 0
succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
publisher, 0 skipped, 5 no drift
4)
FetchRelationStates():
+ List *seq_states;
It would be better to initialize it with NIL.
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-18 21:56 ` Peter Smith <[email protected]>
2026-02-19 02:22 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-19 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
1 sibling, 2 replies; 58+ messages in thread
From: Peter Smith @ 2026-02-18 21:56 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>
On Wed, Feb 18, 2026 at 9:05 PM shveta malik <[email protected]> wrote:
>
...
> 3)
> I noticed that copy_sequences is invoked twice per sync cycle—once for
> sequences in the INIT state, and once for sequences in the READY state
> to detect drift. This means we ping the primary twice during each sync
> cycle. We should consider combining this logic into a single
> copy_sequences call. Since we already have the state information
> (INIT, READY, etc.) for each local sequence, copy_sequences can
> internally check the current state and decide whether to transition a
> sequence to READY based on its previous state. This approach would
> allow us to fetch all required information from the primary in a
> single call.
>
> I also think that we do not even need to mention the states here and
> we can give a single message instead of 2:
> DEBUG: logical replication sequence synchronization for subscription
> "sub1" - for sequences in INIT state batch #1 = 5 attempted, 5
> succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> publisher, 0 skipped, 0 no drift
> DEBUG: logical replication sequence synchronization for subscription
> "sub1" - for sequences in READY state batch #1 = 5 attempted, 0
> succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> publisher, 0 skipped, 5 no drift
>
Those are DEBUG1 messages, not LOG messages, so I think the primary
goal is to ensure that they are full of useful information to help
debugging. Knowing the prior state information means we know that the
"drift" logic was needed when deciding to sync or not.
If message volume can be reduced without any loss of debugging
usefulness, then great, but we need to take care not to throw the baby
out with the bath water.
If message volume is a problem, then maybe that should be addressed by
using more log levels DEBUG1,DEBUG2,DEBUG3,... etc.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 21:56 ` Re: [PATCH] Support automatic sequence replication Peter Smith <[email protected]>
@ 2026-02-19 02:22 ` Amit Kapila <[email protected]>
1 sibling, 0 replies; 58+ messages in thread
From: Amit Kapila @ 2026-02-19 02:22 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: shveta malik <[email protected]>; Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Feb 19, 2026 at 3:26 AM Peter Smith <[email protected]> wrote:
>
> On Wed, Feb 18, 2026 at 9:05 PM shveta malik <[email protected]> wrote:
> >
> ...
> > 3)
> > I noticed that copy_sequences is invoked twice per sync cycle—once for
> > sequences in the INIT state, and once for sequences in the READY state
> > to detect drift. This means we ping the primary twice during each sync
> > cycle. We should consider combining this logic into a single
> > copy_sequences call. Since we already have the state information
> > (INIT, READY, etc.) for each local sequence, copy_sequences can
> > internally check the current state and decide whether to transition a
> > sequence to READY based on its previous state. This approach would
> > allow us to fetch all required information from the primary in a
> > single call.
> >
> > I also think that we do not even need to mention the states here and
> > we can give a single message instead of 2:
> > DEBUG: logical replication sequence synchronization for subscription
> > "sub1" - for sequences in INIT state batch #1 = 5 attempted, 5
> > succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> > publisher, 0 skipped, 0 no drift
> > DEBUG: logical replication sequence synchronization for subscription
> > "sub1" - for sequences in READY state batch #1 = 5 attempted, 0
> > succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> > publisher, 0 skipped, 5 no drift
> >
>
> Those are DEBUG1 messages, not LOG messages, so I think the primary
> goal is to ensure that they are full of useful information to help
> debugging. Knowing the prior state information means we know that the
> "drift" logic was needed when deciding to sync or not.
>
> If message volume can be reduced without any loss of debugging
> usefulness, then great, but we need to take care not to throw the baby
> out with the bath water.
>
I think it is useful to discuss how much DEBUG information is
required. However, I would like to know if this is related to the
patch being discussed or a case in HEAD? If later, then it is better
to discuss it separately and address it as a separate patch if
required.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 21:56 ` Re: [PATCH] Support automatic sequence replication Peter Smith <[email protected]>
@ 2026-02-19 04:05 ` shveta malik <[email protected]>
1 sibling, 0 replies; 58+ messages in thread
From: shveta malik @ 2026-02-19 04:05 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>; shveta malik <[email protected]>
On Thu, Feb 19, 2026 at 3:26 AM Peter Smith <[email protected]> wrote:
>
> On Wed, Feb 18, 2026 at 9:05 PM shveta malik <[email protected]> wrote:
> >
> ...
> > 3)
> > I noticed that copy_sequences is invoked twice per sync cycle—once for
> > sequences in the INIT state, and once for sequences in the READY state
> > to detect drift. This means we ping the primary twice during each sync
> > cycle. We should consider combining this logic into a single
> > copy_sequences call. Since we already have the state information
> > (INIT, READY, etc.) for each local sequence, copy_sequences can
> > internally check the current state and decide whether to transition a
> > sequence to READY based on its previous state. This approach would
> > allow us to fetch all required information from the primary in a
> > single call.
> >
> > I also think that we do not even need to mention the states here and
> > we can give a single message instead of 2:
> > DEBUG: logical replication sequence synchronization for subscription
> > "sub1" - for sequences in INIT state batch #1 = 5 attempted, 5
> > succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> > publisher, 0 skipped, 0 no drift
> > DEBUG: logical replication sequence synchronization for subscription
> > "sub1" - for sequences in READY state batch #1 = 5 attempted, 0
> > succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> > publisher, 0 skipped, 5 no drift
> >
>
> Those are DEBUG1 messages, not LOG messages, so I think the primary
> goal is to ensure that they are full of useful information to help
> debugging. Knowing the prior state information means we know that the
> "drift" logic was needed when deciding to sync or not.
>
> If message volume can be reduced without any loss of debugging
> usefulness, then great, but we need to take care not to throw the baby
> out with the bath water.
>
> If message volume is a problem, then maybe that should be addressed by
> using more log levels DEBUG1,DEBUG2,DEBUG3,... etc.
>
I am not totally against it, but I don’t find it particularly useful
to dump the state (INIT, READY), since we already log exactly which
sequences were updated individually (see below). The output below
covers both cases—sequences with drift and those in the INIT state. We
also include the “no drift” count. So all of these effectively
summarizes the overall status.
logical replication sync for subscription "sub2", sequence
"public.myseq1" has been updated
logical replication sync for subscription "sub2", sequence
"public.myseq2" has been updated
Even if we plan to retain multiple messages for INIT vs READY, I
guess, with copy_sequences() now suggested to be called only once
irrespective of 'state' will make it further tricky to separate out
these log messages for INIT and READY. But let's see.
> I think it is useful to discuss how much DEBUG information is
> required. However, I would like to know if this is related to the
> patch being discussed or a case in HEAD? If later, then it is better
> to discuss it separately and address it as a separate patch if
> required.
Amit, this is related to the new patch only.
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-02-20 10:57 ` Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: Ajin Cherian @ 2026-02-20 10:57 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>
On Fri, Feb 13, 2026 at 4:43 PM Peter Smith <[email protected]> wrote:
>
> Hi Ajin.
>
> Some review comments for patch v4-0001
>
> ======
> src/backend/commands/sequence.c
>
> GetSequence:
>
> 1.
> +/*
> + * Read the current sequence values (last_value and is_called)
> + *
> + * This is a read-only operation that acquires AccessShareLock on the sequence.
> + * Used by logical replication sequence synchronization to detect drift.
> + */
>
> The comment seems stale. e.g. the function is not acquiring a lock
> anymore, contrary to what that comment says.
>
Fixed.
>
> ======
> .../replication/logical/sequencesync.c
>
> 2.
> -static List *seqinfos = NIL;
>
> The removal of this global was not strictly part of this patch; it is
> more like a prerequisite to make everything tidier, so your new code
> does not go further down that track of side-affecting a global. From
> that POV, I thought this global removal should be implemented as a
> first/separate (0001) patch so that it might be quickly reviewed and
> committed independently of the new seq-sync logic.
>
Keeping it as is for the time being. Let's see what everybody else thinks.
> ~~~
>
> LogicalRepSyncSequences:
>
> 3.
> + /* Error on unexpected states */
> + if (relstate != SUBREL_STATE_INIT && relstate != SUBREL_STATE_READY)
> + {
> + table_close(sequence_rel, NoLock);
> + ereport(ERROR,
> + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> + errmsg("unexpected relstate '%c' for sequence \"%s.%s\" in
> subscription \"%s\"",
> + relstate,
> + get_namespace_name(RelationGetNamespace(sequence_rel)),
> + RelationGetRelationName(sequence_rel),
> + MySubscription->name)));
> + }
> +
>
> How is this possible? Should it just be Assert?
>
Fixed.
> ~~~
>
> start_sequence_sync:
>
> 4.
> + /*
> + * Synchronize all sequences (both READY and INIT states).
> + * The function will process INIT sequences first, then READY sequences.
> + */
> + sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
>
> Why is talking about the processing order relevant?
>
>
Removed.
>
> ======
> src/backend/replication/logical/syncutils.c
>
> 5.
> + /* Check if any new sequences need syncing */
> + ProcessSequencesForSync();
> +
>
> Maybe don't say "new" because IIUC it also handles older sequences
> where the values have drifted.
>
> ======
> src/test/subscription/t/036_sequences.pl
>
> 6.
> -$result = $node_publisher->safe_psql(
> - 'postgres', qq(
> - SELECT last_value, is_called FROM regress_s1;
> -));
> -is($result, '200|t', 'Check sequence value in the publisher');
> -
> -# Check - existing sequence ('regress_s1') is not synced
> -$result = $node_subscriber->safe_psql(
> - 'postgres', qq(
> - SELECT last_value, is_called FROM regress_s1;
> -));
> -is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
> -
>
> Since you are no longer testing "existing sequences" in this test
> part, should you also remove the earlier INSERT for 'regress_s1'?
>
Fixed both the above comments.
On Tue, Feb 17, 2026 at 10:21 PM Ashutosh Sharma <[email protected]> wrote:
>
>
> How about retaining ALTER SUBSCRIPTION ... REFRESH SEQUENCES command?
>
> It can be useful in scenarios where automatic sequence replication
> cannot be enabled, for example, due to the max_worker_processes limit
> on the server preventing a new worker from being registered. Since
> increasing max_worker_processes requires a server restart, which the
> user may not want to perform immediately, this command would provide a
> way to manually synchronize the sequences.
>
Retained ALTER SUBSCRIPTION ... REFRESH SEQUENCES similar to the
behaviour on HEAD, it only changes the state of the sequence to INIT.
The sequence worker will update these sequences unconditionally.
>
> One minor comment:
>
> * Sequence state transitions follow this pattern:
> - * INIT -> READY
> + * INIT --> READY ->-+
> + * ^ | (check/synchronzize)
> + * | |
> + * +--<---+
>
>
> "synchronzize" → "synchronize"
>
Fixed.
On Wed, Feb 18, 2026 at 9:04 PM shveta malik <[email protected]> wrote:
>
> Few comments:
>
> 1)
> + /* Check if any new sequences need syncing */
> + ProcessSequencesForSync();
> +
>
> Shall we name it 'maybe_start_seqsync_worker()' (or something better?)
> and move it immediately after 'maybe_advance_nonremovable_xid()'.
> Thoughts? This is because we do not process sequences here, we just
> check if the subscription includes sequences and start worker.
>
Since snak_case are used for static functions, I've renamed it to
MaybeLaunchSequenceSyncWorker and called it immediately after
maybe_advance_nonremovable_xid()
> 2)
> We can also change the comment atop ProcessSequencesForSync to mention
> that. Currently it says:
> /*
> * Apply worker determines if sequence synchronization is needed.
> *
> * Start a sequencesync worker if one is not already running.
>
> Now, we shall change it to:
> Apply worker determines whether a sequence sync worker is needed.
>
> Check if the subscription includes sequences and start a sequencesync
> worker if one is not already running.
>
Fixed.
> 3)
> I noticed that copy_sequences is invoked twice per sync cycle—once for
> sequences in the INIT state, and once for sequences in the READY state
> to detect drift. This means we ping the primary twice during each sync
> cycle. We should consider combining this logic into a single
> copy_sequences call. Since we already have the state information
> (INIT, READY, etc.) for each local sequence, copy_sequences can
> internally check the current state and decide whether to transition a
> sequence to READY based on its previous state. This approach would
> allow us to fetch all required information from the primary in a
> single call.
>
Changed it.
> I also think that we do not even need to mention the states here and
> we can give a single message instead of 2:
> DEBUG: logical replication sequence synchronization for subscription
> "sub1" - for sequences in INIT state batch #1 = 5 attempted, 5
> succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> publisher, 0 skipped, 0 no drift
> DEBUG: logical replication sequence synchronization for subscription
> "sub1" - for sequences in READY state batch #1 = 5 attempted, 0
> succeeded, 0 mismatched, 0 insufficient permission, 0 missing from
> publisher, 0 skipped, 5 no drift
>
Yes, since states are now specific to each sequence and not to
batches, I've removed it.
I've also found that the sequence worker was not being stopped when
SUBSCRIPTION was disabled. I've added checks to do this as well.
All the above changes are part of the attached v5 patch .
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v5-0001-Support-automatic-sequence-replication.patch (38.8K, 2-v5-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From e947256178a88887db2f9c046b8a675c0ae87d24 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Fri, 20 Feb 2026 21:10:42 +1100
Subject: [PATCH v5] Support automatic sequence replication.
Logical replication sequences can drift between publisher and
subscriber as values are consumed independently on each node.
Previously, the sequence sync worker exited after the initial
synchronization, allowing sequences to diverge over time.
This change keeps the sequence sync worker running continuously
so it can monitor sequences and resynchronize them when drift
is detected. The worker uses an adaptive sleep interval:
it starts at 2 seconds, doubles up to a maximum of 30 seconds
when no drift is observed, and resets to the minimum interval
once drift is found.
Sequences remain in the READY state during continuous
synchronization.
Author: Ajin Cherian <[email protected]>
Reviewed-by: Shveta Malik <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Reviewed-by: Ashutosh Sharma <[email protected]>
---
doc/src/sgml/logical-replication.sgml | 56 +---
doc/src/sgml/ref/alter_subscription.sgml | 9 -
src/backend/commands/sequence.c | 27 ++
.../replication/logical/sequencesync.c | 302 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 48 ++-
src/backend/replication/logical/worker.c | 4 +
src/bin/psql/tab-complete.in.c | 4 +-
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 79 +----
12 files changed, 300 insertions(+), 235 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..cfe9c2788c4 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1775,20 +1775,14 @@ Publications:
to synchronize only newly added sequences.
</para>
</listitem>
- <listitem>
- <para>
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
- to re-synchronize all sequences currently known to the subscription.
- </para>
- </listitem>
</itemizedlist>
</para>
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1815,21 +1809,7 @@ Publications:
</sect2>
<sect2 id="sequences-out-of-sync">
- <title>Refreshing Out-of-Sync Sequences</title>
- <para>
- Subscriber sequence values will become out of sync as the publisher
- advances them.
- </para>
- <para>
- To detect this, compare the
- <link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
- on the subscriber with the <structfield>page_lsn</structfield> obtained
- from the <link linkend="func-pg-get-sequence-data"><function>pg_get_sequence_data</function></link>
- function for the sequence on the publisher. Then run
- <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> to
- re-synchronize if necessary.
- </para>
+ <title>Out-of-Sync Sequences</title>
<warning>
<para>
Each sequence caches a block of values (typically 32) in memory before
@@ -1961,16 +1941,6 @@ Publications:
------------+-----------+------------
610 | t | 0/017CEDF8
(1 row)
-</programlisting></para>
-
- <para>
- The difference between the sequence page LSNs on the publisher and the
- sequence page LSNs on the subscriber indicates that the sequences are out
- of sync. Re-synchronize all sequences known to the subscriber using
- <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
-<programlisting>
-/* sub # */ ALTER SUBSCRIPTION sub1 REFRESH SEQUENCES;
</programlisting></para>
<para>
@@ -2333,24 +2303,6 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
</para>
</listitem>
- <listitem>
- <para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
- or by copying the current data from the publisher (perhaps using
- <command>pg_dump</command>) or by determining a sufficiently high value
- from the tables themselves.
- </para>
- </listitem>
-
<listitem>
<para>
Replication of <command>TRUNCATE</command> commands is supported, but
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 27c06439f4f..385f685456b 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -190,11 +190,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
@@ -236,10 +231,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
recommendations on how to handle any warnings about sequence definition
differences between the publisher and the subscriber.
</para>
- <para>
- See <xref linkend="sequences-out-of-sync"/> for recommendations on how to
- identify and handle out-of-sync sequences.
- </para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..aa815dd19af 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,33 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift.
+ */
+void
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+
+ /* Confirm that the relation is a sequence */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 165f909b3ba..9b502a89dd6 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -60,6 +72,7 @@
#include "postmaster/interrupt.h"
#include "replication/logicalworker.h"
#include "replication/worker_internal.h"
+#include "storage/ipc.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
@@ -78,21 +91,27 @@ typedef enum CopySeqResult
COPYSEQ_SUCCESS,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NOWORK,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
*
- * Start a sequencesync worker if one is not already running. The active
- * sequencesync worker will handle all pending sequence synchronization. If any
- * sequences remain unsynchronized after it exits, a new worker can be started
- * in the next iteration.
+ * Check if the subscription includes sequences and start a sequencesync
+ * worker if one is not already running. The active sequencesync worker will
+ * handle all pending sequence synchronization. If any sequences remain
+ * unsynchronized after it exits, a new worker can be started in the next
+ * iteration.
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
LogicalRepWorker *sequencesync_worker;
int nsyncworkers;
@@ -144,7 +163,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +190,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +202,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +213,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +224,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +248,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -325,11 +345,12 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Apply remote sequence state to local sequence. If we are doing this
+ * for sequences in the INIT state, move them to the READY state once
+ * synchronized.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner, char relstate)
{
UserContext ucxt;
AclResult aclresult;
@@ -368,19 +389,46 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (relstate == SUBREL_STATE_INIT)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
+/*
+ * check_sequence_drift
+ *
+ * Check if the remote sequence values differ from the local sequence.
+ * Returns true/false if any sequences drifted.
+ */
+static bool
+check_sequence_drift(Relation sequence_rel, LogicalRepSequenceInfo *seqinfo)
+{
+ int64 local_last_value;
+ bool local_is_called;
+
+ /* Get current local sequence state */
+ GetSequence(sequence_rel, &local_last_value, &local_is_called);
+
+ /* Check if values have drifted and return accordingly */
+ return (local_last_value != seqinfo->last_value ||
+ local_is_called != seqinfo->is_called);
+}
+
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If relstate is SUBREL_STATE_READY, only synchronize sequences that
+ * have drifted from their publisher values. Otherwise, synchronize
+ * all sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +438,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +451,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,20 +547,34 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
+ &seqinfo, &seqidx, seqinfos);
+
+ /*
+ * For sequences in INIT state, always sync.
+ * Otherwise, for sequences in READY state, only sync if there's drift.
+ */
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ {
+ if ((seqinfo->relstate == SUBREL_STATE_INIT) ||
+ check_sequence_drift(sequence_rel, seqinfo))
+ sync_status = copy_sequence(seqinfo,
+ sequence_rel->rd_rel->relowner,
+ seqinfo->relstate);
+ else
+ sync_status = COPYSEQ_NOWORK;
+ }
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
+ "logical replication sync for subscription \"%s\", sequence \"%s.%s\" has been updated",
MySubscription->name, seqinfo->nspname,
seqinfo->seqname);
batch_succeeded_count++;
+ sequence_copied = true;
break;
+
case COPYSEQ_MISMATCH:
/*
@@ -528,6 +588,7 @@ copy_sequences(WalReceiverConn *conn)
MemoryContextSwitchTo(oldctx);
batch_mismatched_count++;
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
/*
@@ -541,6 +602,7 @@ copy_sequences(WalReceiverConn *conn)
MemoryContextSwitchTo(oldctx);
batch_insuffperm_count++;
break;
+
case COPYSEQ_SKIPPED:
/*
@@ -558,6 +620,15 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+
+ case COPYSEQ_NOWORK:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -572,14 +643,15 @@ copy_sequences(WalReceiverConn *conn)
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
- MySubscription->name,
- (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
- batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
+ MySubscription->name,
+ (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
+ batch_size, batch_succeeded_count, batch_mismatched_count,
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,51 +679,53 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
subrel = (Form_pg_subscription_rel) GETSTRUCT(tup);
+ relstate = subrel->srsubstate;
sequence_rel = try_table_open(subrel->srrelid, RowExclusiveLock);
@@ -666,6 +740,8 @@ LogicalRepSyncSequences(void)
continue;
}
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
* allocate them under long-lived context.
@@ -676,6 +752,7 @@ LogicalRepSyncSequences(void)
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
MemoryContextSwitchTo(oldctx);
@@ -693,36 +770,19 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
+ return false;
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Clean up */
+ list_free(seqinfos);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +795,90 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+
+ CHECK_FOR_INTERRUPTS();
+
+
+ /* Need to start transaction for cache lookup */
+ StartTransactionCommand();
+
+ /*
+ * Check if subscription has been disabled.
+ * Switch to ApplyContext that outlives the transaction so that
+ * MySubscription remains valid even after CommitTransactionCommand().
+ */
+ oldctx = MemoryContextSwitchTo(ApplyContext);
+ MySubscription = GetSubscription(MyLogicalRepWorker->subid, false);
+ MemoryContextSwitchTo(oldctx);
+
+ if (!MySubscription->enabled)
+ {
+ ereport(LOG,
+ (errmsg("logical replication sequencesync worker for subscription \"%s\" will stop because the subscription was disabled",
+ MySubscription->name)));
+ proc_exit(0);
+ }
+
+ CommitTransactionCommand();
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ /* Adjust sleep interval based on whether sequences were copied over */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+
+ /*
+ * Double the sleep time, but not beyond
+ * the maximum allowable value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index 535ffb6f09e..ac7669bc273 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -171,7 +171,6 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -190,13 +189,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -204,53 +203,47 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
+ * has_subtables and has_subsequences are declared as static,
* since the same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
-
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
-
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
-
if (!IsTransactionState())
{
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
-
+ list_free_deep(seq_states);
/*
* Does the subscription have tables?
*
@@ -260,7 +253,6 @@ FetchRelationStates(bool *has_pending_subtables,
*/
has_subtables = (table_states_not_ready != NIL) ||
HasSubscriptionTables(MySubscription->oid);
-
/*
* If the subscription relation cache has been invalidated since we
* entered this routine, we still use and return the relations we just
@@ -271,10 +263,8 @@ FetchRelationStates(bool *has_pending_subtables,
if (relation_states_validity == SYNC_RELATIONS_STATE_REBUILD_STARTED)
relation_states_validity = SYNC_RELATIONS_STATE_VALID;
}
-
if (has_pending_subtables)
*has_pending_subtables = has_subtables;
-
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 32725c48623..5b479be7ec7 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -4219,6 +4219,7 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
ProcessConfigFile(PGC_SIGHUP);
}
+
if (rc & WL_TIMEOUT)
{
/*
@@ -4266,6 +4267,9 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
maybe_advance_nonremovable_xid(&rdt_data, false);
+ /* Check if any new sequences need syncing */
+ MaybeLaunchSequenceSyncWorker();
+
/*
* Force reporting to ensure long idle periods don't lead to
* arbitrarily delayed stats. Stats can only be reported outside
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 8b91bc00062..b8bdfebb15a 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -2331,8 +2331,8 @@ match_previous_words(int pattern_id,
/* ALTER SUBSCRIPTION <name> */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
- "RENAME TO", "REFRESH PUBLICATION", "REFRESH SEQUENCES",
- "SET", "SKIP (", "ADD PUBLICATION", "DROP PUBLICATION");
+ "RENAME TO", "REFRESH PUBLICATION", "SET", "SKIP (",
+ "ADD PUBLICATION", "DROP PUBLICATION");
/* ALTER SUBSCRIPTION <name> REFRESH */
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny, MatchAnyN, "REFRESH"))
COMPLETE_WITH("PUBLICATION", "SEQUENCES");
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index f810b34c78d..a7c808acf23 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -92,6 +92,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..f00eea9fbd1 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern void GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index c1285fdd1bc..537e13fe9b8 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, int cacheid, uint32 hashvalue);
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..1d81518fe22 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,8 +75,7 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
@@ -84,9 +83,6 @@ $node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.47.3
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-23 01:26 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-23 05:44 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 05:36 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
0 siblings, 3 replies; 58+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-02-23 01:26 UTC (permalink / raw)
To: 'Ajin Cherian' <[email protected]>; shveta malik <[email protected]>; +Cc: Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>
Dear Ajin,
Thanks for updating the patch. Here are my comments.
01. start_sequence_sync()
```
/* Need to start transaction for cache lookup */
StartTransactionCommand();
```
Here, we must check additional parameter changes, such as conninfo and passwordrequired.
Also, it's inefficient because transactions are started each time.
Can we re-use maybe_reread_subscription() here? Some parameters do not take
effect for the sequence sync worker, but it is OK to exit even if they
are changed. If we use the function, no need to include "storage/ipc.h".
02. match_previous_words
No need to remove "REFRESH SEQUENCES" anymore.
03. CopySeqResult
```
COPYSEQ_NOWORK,
```
It describes why the copying is skipped. How about "COPYSEQ_NO_DRIFT"?
04. LogicalRepSyncSequences()
```
oldctx = MemoryContextSwitchTo(ApplyContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
MemoryContextSwitchTo(oldctx);
```
ISTM they are palloc'd but not pfree'd.
Since the sequencesync worker now has a long lifetime, we must take care of the
memory allocation/freeing more carefully. How about introducing per-interaction
memory context like ApplyMessageContext?
05. LogicalRepApplyLoop()
MaybeLaunchSequenceSyncWorker() should be called more; otherwise, the sequencesync
worker won't be launched if the worker always receives messages and WL_TIMEOUT does
not happen. Can you add most of the places under maybe_advance_nonremovable_xid()?
Personally considered, no need to add within `else if (c == PqReplMsg_PrimaryStatusUpdate)`
because it just consumes status updates from the primary.
06.
Not sure if the issue should be discussed here, but I found that sequences more
likely to go backward if users use sequences on the subscriber side.
Previously, the sync could happen based on the request, and users could understand
the risk. But now everything would be done automatically, thus they may be
surprised more.
Should we consider some ratchet mechanisms, or retain it now because it's not
expected usage?
E.g., `nextval()` is called three times, and synchronization occurs between them.
```
subscriber=# SELECT nextval('seq');
nextval
---------
2
(1 row)
subscriber=# SELECT nextval('seq');
nextval
---------
3
(1 row)
subscriber=# -- synchronization happened
subscriber=# SELECT nextval('seq');
nextval
---------
1
(1 row)
```
07.
Question: Can we introduce an intermediate state, such as SYNC, to clarify
whether synchronization is proceeding?
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-02-23 05:44 ` Amit Kapila <[email protected]>
2 siblings, 0 replies; 58+ messages in thread
From: Amit Kapila @ 2026-02-23 05:44 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Ajin Cherian <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Feb 23, 2026 at 6:56 AM Hayato Kuroda (Fujitsu)
<[email protected]> wrote:
>
> 06.
> Not sure if the issue should be discussed here, but I found that sequences more
> likely to go backward if users use sequences on the subscriber side.
> Previously, the sync could happen based on the request, and users could understand
> the risk. But now everything would be done automatically, thus they may be
> surprised more.
>
> Should we consider some ratchet mechanisms, or retain it now because it's not
> expected usage?
>
We discussed this case upthread. We ideally can handle it via
conflict/resolution strategy or simply avoid updating the sequences
that are synced from the publisher. If we do later it would be tricky
because we need to maintain a persistent state and then after
failover, that state should be cleared. We discussed to have it
documented that users should use such sequences only during upgrade or
for failover cases, allowing to update it actively on multiple nodes
can lead to inconsistency.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-02-24 05:36 ` Amit Kapila <[email protected]>
2 siblings, 0 replies; 58+ messages in thread
From: Amit Kapila @ 2026-02-24 05:36 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Ajin Cherian <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Feb 23, 2026 at 6:56 AM Hayato Kuroda (Fujitsu)
<[email protected]> wrote:
>
> Thanks for updating the patch. Here are my comments.
>
> 01. start_sequence_sync()
> ```
> /* Need to start transaction for cache lookup */
> StartTransactionCommand();
> ```
>
> Here, we must check additional parameter changes, such as conninfo and passwordrequired.
> Also, it's inefficient because transactions are started each time.
>
> Can we re-use maybe_reread_subscription() here? Some parameters do not take
> effect for the sequence sync worker, but it is OK to exit even if they
> are changed. If we use the function, no need to include "storage/ipc.h".
>
I also think it is better to restart the sequencesync worker on
subscription parameter change and probably it is okay to use
maybe_reread_subscription() but need to be careful to avoid anything
apply_worker specific in that function. BTW, what happens now without
this change when apply_worker stops due to parameter change, does
sequencesync worker continue? If so, we should make it restart as we
are doing for apply worker.
Also, shouldn't we need to invoke AcceptInvalidationMessages() as we
are doing in apply worker when not in a remote transaction? I think it
will be required to get local_sequence definition changes , if any.
> 02. match_previous_words
>
> No need to remove "REFRESH SEQUENCES" anymore.
>
> 03. CopySeqResult
> ```
> COPYSEQ_NOWORK,
> ```
>
> It describes why the copying is skipped. How about "COPYSEQ_NO_DRIFT"?
>
+1.
> 04. LogicalRepSyncSequences()
>
> ```
> oldctx = MemoryContextSwitchTo(ApplyContext);
>
> seq = palloc0_object(LogicalRepSequenceInfo);
> seq->localrelid = subrel->srrelid;
> seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
> seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
> seq->relstate = relstate;
> seqinfos = lappend(seqinfos, seq);
>
> MemoryContextSwitchTo(oldctx);
> ```
>
> ISTM they are palloc'd but not pfree'd.
>
They are freed, see following code. But it seems nspname and seqname
should be freed separately for each sequence element and each sequence
element also needs to be freed independently.
+ /* Clean up */
+ list_free(seqinfos);
> Since the sequencesync worker now has a long lifetime, we must take care of the
> memory allocation/freeing more carefully. How about introducing per-interaction
> memory context like ApplyMessageContext?
>
Yeah, I feel that would be a better approach than retail pfree
especially because it is also required at other places in sequencesync
worker (see places where currently ApplyContext is used in
sequencesync worker). I think it would be better if we name this new
context as SequenceSyncContext.
Few other miscellaneous comments
=================================
1.
<sect2 id="sequences-out-of-sync">
- <title>Refreshing Out-of-Sync Sequences</title>
- <para>
- Subscriber sequence values will become out of sync as the publisher
- advances them.
- </para>
- <para>
- To detect this, compare the
- <link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
- on the subscriber with the <structfield>page_lsn</structfield> obtained
- from the <link
linkend="func-pg-get-sequence-data"><function>pg_get_sequence_data</function></link>
- function for the sequence on the publisher. Then run
- <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> to
- re-synchronize if necessary.
- </para>
+ <title>Out-of-Sync Sequences</title>
<warning>
<para>
Each sequence caches a block of values (typically 32) in memory before
@@ -1961,16 +1941,6 @@ Publications:
------------+-----------+------------
610 | t | 0/017CEDF8
(1 row)
-</programlisting></para>
-
- <para>
- The difference between the sequence page LSNs on the publisher and the
- sequence page LSNs on the subscriber indicates that the sequences are out
- of sync. Re-synchronize all sequences known to the subscriber using
- <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
-<programlisting>
-/* sub # */ ALTER SUBSCRIPTION sub1 REFRESH SEQUENCES;
...
...
- <listitem>
- <para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
- or by copying the current data from the publisher (perhaps using
- <command>pg_dump</command>) or by determining a sufficiently high value
- from the tables themselves.
- </para>
- </listitem>
-
I think this can still happen after this patch but chances are much
lower, so we need some of this before upgrade. We should reword it
accordingly. Similarly check other parts of the doc you removed.
2.
- "logical replication synchronization for subscription \"%s\",
sequence \"%s.%s\" has finished",
+ "logical replication sync for subscription \"%s\", sequence
\"%s.%s\" has been updated",
Is there a need to shorten synchronization to sync in above message?
3.
elog(DEBUG1,
- "logical replication sequence synchronization for subscription
\"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d
insufficient permission, %d missing from publisher, %d skipped",
- MySubscription->name,
- (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
- batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ "logical replication sequence synchronization for subscription
\"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d
insufficient permission, %d missing from publisher, %d skipped, %d no
drift",
+ MySubscription->name,
+ (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
+ batch_size, batch_succeeded_count, batch_mismatched_count,
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count,
batch_no_drift);
Here, the formatting of message is incorrect.
4.
table_states_not_ready = NIL;
-
if (!IsTransactionState())
Spurious line removal.
5.
MemoryContextSwitchTo(oldctx);
-
+ list_free_deep(seq_states);
/*
* Does the subscription have tables?
*
@@ -260,7 +253,6 @@ FetchRelationStates(bool *has_pending_subtables,
*/
has_subtables = (table_states_not_ready != NIL) ||
HasSubscriptionTables(MySubscription->oid);
-
/*
* If the subscription relation cache has been invalidated since we
* entered this routine, we still use and return the relations we just
@@ -271,10 +263,8 @@ FetchRelationStates(bool *has_pending_subtables,
if (relation_states_validity == SYNC_RELATIONS_STATE_REBUILD_STARTED)
relation_states_validity = SYNC_RELATIONS_STATE_VALID;
}
-
if (has_pending_subtables)
*has_pending_subtables = has_subtables;
-
if (has_pending_subsequences)
...
...
}
+
if (rc & WL_TIMEOUT)
All above places have spurious line removals and additions which made
code harder to understand.
6.
else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
- "RENAME TO", "REFRESH PUBLICATION", "REFRESH SEQUENCES",
- "SET", "SKIP (", "ADD PUBLICATION", "DROP PUBLICATION");
+ "RENAME TO", "REFRESH PUBLICATION", "SET", "SKIP (",
+ "ADD PUBLICATION", "DROP PUBLICATION");
unrelated change.
7.
+ else
+ {
+
+ /*
+ * Double the sleep time, but not beyond
+ * the maximum allowable value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
Comments are not properly aligned.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-02-24 06:17 ` Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2 siblings, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-02-24 06:17 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Ajin Cherian <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Feb 23, 2026 at 6:56 AM Hayato Kuroda (Fujitsu)
<[email protected]> wrote:
>
> 05. LogicalRepApplyLoop()
>
> MaybeLaunchSequenceSyncWorker() should be called more; otherwise, the sequencesync
> worker won't be launched if the worker always receives messages and WL_TIMEOUT does
> not happen. Can you add most of the places under maybe_advance_nonremovable_xid()?
> Personally considered, no need to add within `else if (c == PqReplMsg_PrimaryStatusUpdate)`
> because it just consumes status updates from the primary.
>
I don't think we need to be as aggressive as
maybe_advance_nonremovable_xid because not doing that can lead to
bload if slot is not advanced. The only minor downside with checking
too frequently is that we need to traverse the all logical replication
workers to find if sequencesync worker is available. I feel doing in
ProcessSyncingRelations() where earlier we were doing
ProcessSequencesForSync() should be sufficient. Can we find some cheap
way to detect if sequencesync worker is present or not? Can you think
some other way to not incur the cost of traversing the worker array
and also detect sequence worker exit without much delay?
...
>
> 07.
> Question: Can we introduce an intermediate state, such as SYNC, to clarify
> whether synchronization is proceeding?
>
What is the advantage of this? For external purposes, the presence of
sequencesync worker, which can be checked via pg_stat_subscription
should be sufficient.
BTW, what is the behavior of REFRESH SEQUENCES command if the sequence
worker is active? Does it still try to refresh sequences, if so, is
that required/good idea?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-02-24 11:04 ` Ajin Cherian <[email protected]>
2026-02-24 11:25 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
0 siblings, 2 replies; 58+ messages in thread
From: Ajin Cherian @ 2026-02-24 11:04 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 24, 2026 at 4:36 PM Amit Kapila <[email protected]> wrote:
>
> On Mon, Feb 23, 2026 at 6:56 AM Hayato Kuroda (Fujitsu)
> <[email protected]> wrote:
> >
> > Thanks for updating the patch. Here are my comments.
> >
> > 01. start_sequence_sync()
> > ```
> > /* Need to start transaction for cache lookup */
> > StartTransactionCommand();
> > ```
> >
> > Here, we must check additional parameter changes, such as conninfo and passwordrequired.
> > Also, it's inefficient because transactions are started each time.
> >
> > Can we re-use maybe_reread_subscription() here? Some parameters do not take
> > effect for the sequence sync worker, but it is OK to exit even if they
> > are changed. If we use the function, no need to include "storage/ipc.h".
> >
>
> I also think it is better to restart the sequencesync worker on
> subscription parameter change and probably it is okay to use
> maybe_reread_subscription() but need to be careful to avoid anything
> apply_worker specific in that function. BTW, what happens now without
> this change when apply_worker stops due to parameter change, does
> sequencesync worker continue? If so, we should make it restart as we
> are doing for apply worker.
>
I have changed it to use maybe_reread_subscription() and modified
maybe_reread_subscription() to have have some special handling
sequence sync workers as well.
Yes, the apply worker will again restart the sequence worker when it
finds that it isn't running and there are sequences in the
subscription_rel.
> Also, shouldn't we need to invoke AcceptInvalidationMessages() as we
> are doing in apply worker when not in a remote transaction? I think it
> will be required to get local_sequence definition changes , if any.
I will need to investigate this further.
>
> > 02. match_previous_words
> >
> > No need to remove "REFRESH SEQUENCES" anymore.
> >
> > 03. CopySeqResult
> > ```
> > COPYSEQ_NOWORK,
> > ```
> >
> > It describes why the copying is skipped. How about "COPYSEQ_NO_DRIFT"?
> >
>
> +1.
Fixed.
>
> > 04. LogicalRepSyncSequences()
> >
> > ```
> > oldctx = MemoryContextSwitchTo(ApplyContext);
> >
> > seq = palloc0_object(LogicalRepSequenceInfo);
> > seq->localrelid = subrel->srrelid;
> > seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
> > seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
> > seq->relstate = relstate;
> > seqinfos = lappend(seqinfos, seq);
> >
> > MemoryContextSwitchTo(oldctx);
> > ```
> >
> > ISTM they are palloc'd but not pfree'd.
> >
>
> They are freed, see following code. But it seems nspname and seqname
> should be freed separately for each sequence element and each sequence
> element also needs to be freed independently.
>
> + /* Clean up */
> + list_free(seqinfos);
>
> > Since the sequencesync worker now has a long lifetime, we must take care of the
> > memory allocation/freeing more carefully. How about introducing per-interaction
> > memory context like ApplyMessageContext?
> >
>
> Yeah, I feel that would be a better approach than retail pfree
> especially because it is also required at other places in sequencesync
> worker (see places where currently ApplyContext is used in
> sequencesync worker). I think it would be better if we name this new
> context as SequenceSyncContext.
>
Done. Changed as requested.
> Few other miscellaneous comments
> =================================
> 1.
> <sect2 id="sequences-out-of-sync">
> - <title>Refreshing Out-of-Sync Sequences</title>
> - <para>
> - Subscriber sequence values will become out of sync as the publisher
> - advances them.
> - </para>
> - <para>
> - To detect this, compare the
> - <link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
> - on the subscriber with the <structfield>page_lsn</structfield> obtained
> - from the <link
> linkend="func-pg-get-sequence-data"><function>pg_get_sequence_data</function></link>
> - function for the sequence on the publisher. Then run
> - <link linkend="sql-altersubscription-params-refresh-sequences">
> - <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> to
> - re-synchronize if necessary.
> - </para>
> + <title>Out-of-Sync Sequences</title>
> <warning>
> <para>
> Each sequence caches a block of values (typically 32) in memory before
> @@ -1961,16 +1941,6 @@ Publications:
> ------------+-----------+------------
> 610 | t | 0/017CEDF8
> (1 row)
> -</programlisting></para>
> -
> - <para>
> - The difference between the sequence page LSNs on the publisher and the
> - sequence page LSNs on the subscriber indicates that the sequences are out
> - of sync. Re-synchronize all sequences known to the subscriber using
> - <link linkend="sql-altersubscription-params-refresh-sequences">
> - <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
> -<programlisting>
> -/* sub # */ ALTER SUBSCRIPTION sub1 REFRESH SEQUENCES;
>
> ...
> ...
> - <listitem>
> - <para>
> - Incremental sequence changes are not replicated. Although the data in
> - serial or identity columns backed by sequences will be replicated as part
> - of the table, the sequences themselves do not replicate ongoing changes.
> - On the subscriber, a sequence will retain the last value it synchronized
> - from the publisher. If the subscriber is used as a read-only database,
> - then this should typically not be a problem. If, however, some kind of
> - switchover or failover to the subscriber database is intended, then the
> - sequences would need to be updated to the latest values, either by
> - executing <link linkend="sql-altersubscription-params-refresh-sequences">
> - <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
> - or by copying the current data from the publisher (perhaps using
> - <command>pg_dump</command>) or by determining a sufficiently high value
> - from the tables themselves.
> - </para>
> - </listitem>
> -
>
> I think this can still happen after this patch but chances are much
> lower, so we need some of this before upgrade. We should reword it
> accordingly. Similarly check other parts of the doc you removed.
>
I have re-added them and modified them.
> 2.
> - "logical replication synchronization for subscription \"%s\",
> sequence \"%s.%s\" has finished",
> + "logical replication sync for subscription \"%s\", sequence
> \"%s.%s\" has been updated",
>
> Is there a need to shorten synchronization to sync in above message?
>
Removed.
> 3.
> elog(DEBUG1,
> - "logical replication sequence synchronization for subscription
> \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d
> insufficient permission, %d missing from publisher, %d skipped",
> - MySubscription->name,
> - (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
> - batch_size, batch_succeeded_count, batch_mismatched_count,
> - batch_insuffperm_count, batch_missing_count, batch_skipped_count);
> + "logical replication sequence synchronization for subscription
> \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d
> insufficient permission, %d missing from publisher, %d skipped, %d no
> drift",
> + MySubscription->name,
> + (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
> + batch_size, batch_succeeded_count, batch_mismatched_count,
> + batch_insuffperm_count, batch_missing_count, batch_skipped_count,
> batch_no_drift);
>
> Here, the formatting of message is incorrect.
>
Fixed.
> 4.
> table_states_not_ready = NIL;
> -
> if (!IsTransactionState())
>
> Spurious line removal.
>
> 5.
> MemoryContextSwitchTo(oldctx);
> -
> + list_free_deep(seq_states);
> /*
> * Does the subscription have tables?
> *
> @@ -260,7 +253,6 @@ FetchRelationStates(bool *has_pending_subtables,
> */
> has_subtables = (table_states_not_ready != NIL) ||
> HasSubscriptionTables(MySubscription->oid);
> -
> /*
> * If the subscription relation cache has been invalidated since we
> * entered this routine, we still use and return the relations we just
> @@ -271,10 +263,8 @@ FetchRelationStates(bool *has_pending_subtables,
> if (relation_states_validity == SYNC_RELATIONS_STATE_REBUILD_STARTED)
> relation_states_validity = SYNC_RELATIONS_STATE_VALID;
> }
> -
> if (has_pending_subtables)
> *has_pending_subtables = has_subtables;
> -
> if (has_pending_subsequences)
> ...
> ...
>
> }
>
> +
> if (rc & WL_TIMEOUT)
>
> All above places have spurious line removals and additions which made
> code harder to understand.
>
Fixed.
> 6.
> else if (Matches("ALTER", "SUBSCRIPTION", MatchAny))
> COMPLETE_WITH("CONNECTION", "ENABLE", "DISABLE", "OWNER TO",
> - "RENAME TO", "REFRESH PUBLICATION", "REFRESH SEQUENCES",
> - "SET", "SKIP (", "ADD PUBLICATION", "DROP PUBLICATION");
> + "RENAME TO", "REFRESH PUBLICATION", "SET", "SKIP (",
> + "ADD PUBLICATION", "DROP PUBLICATION");
>
> unrelated change.
>
Removed.
> 7.
> + else
> + {
> +
> + /*
> + * Double the sleep time, but not beyond
> + * the maximum allowable value.
> + */
> + sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
>
> Comments are not properly aligned.
>
Fixed.
On Tue, Feb 24, 2026 at 5:17 PM Amit Kapila <[email protected]> wrote:
>
> On Mon, Feb 23, 2026 at 6:56 AM Hayato Kuroda (Fujitsu)
> <[email protected]> wrote:
> >
> > 05. LogicalRepApplyLoop()
> >
> > MaybeLaunchSequenceSyncWorker() should be called more; otherwise, the sequencesync
> > worker won't be launched if the worker always receives messages and WL_TIMEOUT does
> > not happen. Can you add most of the places under maybe_advance_nonremovable_xid()?
> > Personally considered, no need to add within `else if (c == PqReplMsg_PrimaryStatusUpdate)`
> > because it just consumes status updates from the primary.
> >
>
> I don't think we need to be as aggressive as
> maybe_advance_nonremovable_xid because not doing that can lead to
> bload if slot is not advanced. The only minor downside with checking
> too frequently is that we need to traverse the all logical replication
> workers to find if sequencesync worker is available. I feel doing in
> ProcessSyncingRelations() where earlier we were doing
> ProcessSequencesForSync() should be sufficient.
Added it back as previous.
Can we find some cheap
> way to detect if sequencesync worker is present or not? Can you think
> some other way to not incur the cost of traversing the worker array
> and also detect sequence worker exit without much delay?
>
I will need to investigate this further.
> ...
> >
> > 07.
> > Question: Can we introduce an intermediate state, such as SYNC, to clarify
> > whether synchronization is proceeding?
> >
>
> What is the advantage of this? For external purposes, the presence of
> sequencesync worker, which can be checked via pg_stat_subscription
> should be sufficient.
>
> BTW, what is the behavior of REFRESH SEQUENCES command if the sequence
> worker is active? Does it still try to refresh sequences, if so, is
> that required/good idea?
>
Currently REFRESH SEQUENCES can only be called if the subscription is
enabled. All it does is change the states of all the sequences in
subscription_rel to INIT, this will prompt the sequence worker to wake
up and unconditionally sync all the sequences. For sequences in the
INIT state, it doesn't check if there is drift or not, it updates all
sequences unconditionally. From your discussions with Dilip, I
understand we want to reduce the time it takes to REFRESH SEQUENCES at
the time of an upgrade. If so, then this might not be a good approach.
As not only does the sequence worker have to update all sequences even
if they have not drifted, it also has to update the relstate of all
these sequences to READY. I propose, we change the logic such that
REFRESH SEQUENCES only wakes up the sequence worker, that will reduce
the time as most sequences will already be in READY state (unless
newly added) and only sequences that have not drifted need to be
updated and no catalog update is required for sequence states already
in READY state. One downside to this is that, earlier users could
issue a REFRESH SEQUENCES and wait to see if all the sequences have
returned to the READY state to confirm that the sync has completed,
with this approach that advantage is not there. Users might have to
use a query to find the sequence values of all the sequences and
compare that with the values in the publisher.
I have addressed the above comments in the attached patch v6.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v6-0001-Support-automatic-sequence-replication.patch (38.4K, 2-v6-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From e26287ea1341127cd71381f782827aab98c337be Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Tue, 24 Feb 2026 21:37:01 +1100
Subject: [PATCH v6] Support automatic sequence replication.
Logical replication sequences can drift between publisher and
subscriber as values are consumed independently on each node.
Previously, the sequence sync worker exited after the initial
synchronization, allowing sequences to diverge over time.
This change keeps the sequence sync worker running continuously
so it can monitor sequences and resynchronize them when drift
is detected. The worker uses an adaptive sleep interval:
it starts at 2 seconds, doubles up to a maximum of 30 seconds
when no drift is observed, and resets to the minimum interval
once drift is found.
Sequences remain in the READY state during continuous
synchronization.
Author: Ajin Cherian <[email protected]>
Reviewed-by: Shveta Malik <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Reviewed-by: Ashutosh Sharma <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
---
doc/src/sgml/logical-replication.sgml | 23 +-
doc/src/sgml/ref/alter_subscription.sgml | 9 -
src/backend/commands/sequence.c | 27 ++
.../replication/logical/sequencesync.c | 319 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 46 ++-
src/backend/replication/logical/worker.c | 12 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 79 +----
11 files changed, 320 insertions(+), 201 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..bb523af5d37 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,8 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1817,7 +1818,7 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
+ Subscriber sequence values can become out of sync as the publisher
advances them.
</para>
<para>
@@ -2335,15 +2336,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 5318998e80c..a8dd7a6b42d 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -190,11 +190,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
@@ -236,10 +231,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
recommendations on how to handle any warnings about sequence definition
differences between the publisher and the subscriber.
</para>
- <para>
- See <xref linkend="sequences-out-of-sync"/> for recommendations on how to
- identify and handle out-of-sync sequences.
- </para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..aa815dd19af 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,33 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift.
+ */
+void
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+
+ /* Confirm that the relation is a sequence */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 9c92fddd624..db015e9b5de 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -78,21 +90,29 @@ typedef enum CopySeqResult
COPYSEQ_SUCCESS,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+
+MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
*
- * Start a sequencesync worker if one is not already running. The active
- * sequencesync worker will handle all pending sequence synchronization. If any
- * sequences remain unsynchronized after it exits, a new worker can be started
- * in the next iteration.
+ * Check if the subscription includes sequences and start a sequencesync
+ * worker if one is not already running. The active sequencesync worker will
+ * handle all pending sequence synchronization. If any sequences remain
+ * unsynchronized after it exits, a new worker can be started in the next
+ * iteration.
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
LogicalRepWorker *sequencesync_worker;
int nsyncworkers;
@@ -144,7 +164,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +191,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +203,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +214,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +225,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +249,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -325,11 +346,12 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Apply remote sequence state to local sequence. If we are doing this
+ * for sequences in the INIT state, move them to the READY state once
+ * synchronized.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner, char relstate)
{
UserContext ucxt;
AclResult aclresult;
@@ -368,19 +390,46 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (relstate == SUBREL_STATE_INIT)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
+/*
+ * check_sequence_drift
+ *
+ * Check if the remote sequence values differ from the local sequence.
+ * Returns true/false if any sequences drifted.
+ */
+static bool
+check_sequence_drift(Relation sequence_rel, LogicalRepSequenceInfo *seqinfo)
+{
+ int64 local_last_value;
+ bool local_is_called;
+
+ /* Get current local sequence state */
+ GetSequence(sequence_rel, &local_last_value, &local_is_called);
+
+ /* Check if values have drifted and return accordingly */
+ return (local_last_value != seqinfo->last_value ||
+ local_is_called != seqinfo->is_called);
+}
+
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If relstate is SUBREL_STATE_READY, only synchronize sequences that
+ * have drifted from their publisher values. Otherwise, synchronize
+ * all sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +439,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +452,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,46 +548,62 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
+ &seqinfo, &seqidx, seqinfos);
+
+ /*
+ * For sequences in INIT state, always sync.
+ * Otherwise, for sequences in READY state, only sync if there's drift.
+ */
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ {
+ if ((seqinfo->relstate == SUBREL_STATE_INIT) ||
+ check_sequence_drift(sequence_rel, seqinfo))
+ sync_status = copy_sequence(seqinfo,
+ sequence_rel->rd_rel->relowner,
+ seqinfo->relstate);
+ else
+ sync_status = COPYSEQ_NO_DRIFT;
+ }
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
+ "logical replication synchronizatio for subscription \"%s\", sequence \"%s.%s\" has been updated",
MySubscription->name, seqinfo->nspname,
seqinfo->seqname);
batch_succeeded_count++;
+ sequence_copied = true;
break;
+
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_mismatched_count++;
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_insuffperm_count++;
break;
+
case COPYSEQ_SKIPPED:
/*
@@ -558,6 +621,15 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -566,20 +638,19 @@ copy_sequences(WalReceiverConn *conn)
ExecDropSingleTupleTableSlot(slot);
walrcv_clear_result(res);
- resetStringInfo(seqstr);
- resetStringInfo(cmd);
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
- MySubscription->name,
- (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
- batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
+ MySubscription->name,
+ (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
+ batch_size, batch_succeeded_count, batch_mismatched_count,
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,51 +678,55 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
subrel = (Form_pg_subscription_rel) GETSTRUCT(tup);
+ relstate = subrel->srsubstate;
sequence_rel = try_table_open(subrel->srrelid, RowExclusiveLock);
@@ -666,16 +741,19 @@ LogicalRepSyncSequences(void)
continue;
}
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
MemoryContextSwitchTo(oldctx);
@@ -693,36 +771,23 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
+ return false;
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ MemoryContextSwitchTo(oldctx);
+
+ /* Cleanup the memory. */
+ MemoryContextReset(SequenceSyncContext);
+
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +800,76 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+
+ CHECK_FOR_INTERRUPTS();
+
+ maybe_reread_subscription();
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ /* Adjust sleep interval based on whether sequences were copied over */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+
+ /*
+ * Double the sleep time, but not beyond
+ * the maximum allowable value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..e8d21f55af0 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,9 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+
+ /* Check if sequence worker needs to be started */
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +193,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +207,20 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
+ * has_subtables and has_subsequences are declared as static,
* since the same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
-
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
-
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +230,26 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
/*
* Does the subscription have tables?
*
@@ -277,5 +275,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index bae8c011390..6dce3ced90b 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -4221,6 +4221,7 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
ProcessConfigFile(PGC_SIGHUP);
}
+
if (rc & WL_TIMEOUT)
{
/*
@@ -5098,6 +5099,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5113,6 +5117,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5131,6 +5139,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..f00eea9fbd1 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern void GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 4ecbdcfadac..a41cb045f19 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..1d81518fe22 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,8 +75,7 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
@@ -84,9 +83,6 @@ $node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.47.3
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-24 11:25 ` Amit Kapila <[email protected]>
2026-02-24 13:18 ` Re: [PATCH] Support automatic sequence replication Dilip Kumar <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-02-24 11:25 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 24, 2026 at 4:34 PM Ajin Cherian <[email protected]> wrote:
>
> Currently REFRESH SEQUENCES can only be called if the subscription is
> enabled. All it does is change the states of all the sequences in
> subscription_rel to INIT, this will prompt the sequence worker to wake
> up and unconditionally sync all the sequences. For sequences in the
> INIT state, it doesn't check if there is drift or not, it updates all
> sequences unconditionally. From your discussions with Dilip, I
> understand we want to reduce the time it takes to REFRESH SEQUENCES at
> the time of an upgrade. If so, then this might not be a good approach.
>
The point I was trying to make is that with automatic sequence sync,
we won't need to execute REFRESH SEQUENCES before upgrade or failover
as the sequences will be in sync.
> As not only does the sequence worker have to update all sequences even
> if they have not drifted, it also has to update the relstate of all
> these sequences to READY. I propose, we change the logic such that
> REFRESH SEQUENCES only wakes up the sequence worker, that will reduce
> the time as most sequences will already be in READY state (unless
> newly added) and only sequences that have not drifted need to be
> updated and no catalog update is required for sequence states already
> in READY state. One downside to this is that, earlier users could
> issue a REFRESH SEQUENCES and wait to see if all the sequences have
> returned to the READY state to confirm that the sync has completed,
> with this approach that advantage is not there. Users might have to
> use a query to find the sequence values of all the sequences and
> compare that with the values in the publisher.
>
Yeah, this point needs more thoughts as the existing method for
REFRESH SEQUENCES also allows users to easily verify if the sequences
are in sync.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-24 11:25 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-02-24 13:18 ` Dilip Kumar <[email protected]>
2026-02-25 03:02 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Dilip Kumar @ 2026-02-24 13:18 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 24, 2026 at 4:55 PM Amit Kapila <[email protected]> wrote:
>
> On Tue, Feb 24, 2026 at 4:34 PM Ajin Cherian <[email protected]> wrote:
> >
> > Currently REFRESH SEQUENCES can only be called if the subscription is
> > enabled. All it does is change the states of all the sequences in
> > subscription_rel to INIT, this will prompt the sequence worker to wake
> > up and unconditionally sync all the sequences. For sequences in the
> > INIT state, it doesn't check if there is drift or not, it updates all
> > sequences unconditionally. From your discussions with Dilip, I
> > understand we want to reduce the time it takes to REFRESH SEQUENCES at
> > the time of an upgrade. If so, then this might not be a good approach.
> >
>
> The point I was trying to make is that with automatic sequence sync,
> we won't need to execute REFRESH SEQUENCES before upgrade or failover
> as the sequences will be in sync.
>
That is a valid goal, however, the sequence sync worker is an
asynchronous process triggered at specific intervals. For a switchover
to guarantee consistency, we must disconnect all connections from the
current publisher and wait for all pending data to sync. Relying on an
automated, interval-based worker at this critical time is risky, as it
directly extends our downtime. To minimize the cutover window, we
should either treat sequence data the same as relational data
(synchronous streaming) or manually trigger for the sequence sync
during the switchover phase, i.e. REFRESH SEQUENCES. Anyway this is
my understanding, do you have anything else in mind that how you are
trying to completely avoid REFRESH SEQUENCES. Having said that I
agree that in many cases where sequences are not frequently modified
it is very much possible that when we are in switchover phase all
sequences are already in sync and if we can identify that then we can
avoid REFRESH SEQUENCES, but my point is we can not completely avoid
that in all caes. And the cases where we are frequently inserting
into relation which has a sequence type field the sequences will be
freqnectly modified and we have to call REFRESH SEQUENCES.
--
Regards,
Dilip Kumar
Google
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-24 11:25 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 13:18 ` Re: [PATCH] Support automatic sequence replication Dilip Kumar <[email protected]>
@ 2026-02-25 03:02 ` Amit Kapila <[email protected]>
2026-02-25 03:57 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-02-25 03:02 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 24, 2026 at 6:49 PM Dilip Kumar <[email protected]> wrote:
>
> On Tue, Feb 24, 2026 at 4:55 PM Amit Kapila <[email protected]> wrote:
> >
> > On Tue, Feb 24, 2026 at 4:34 PM Ajin Cherian <[email protected]> wrote:
> > >
> > > Currently REFRESH SEQUENCES can only be called if the subscription is
> > > enabled. All it does is change the states of all the sequences in
> > > subscription_rel to INIT, this will prompt the sequence worker to wake
> > > up and unconditionally sync all the sequences. For sequences in the
> > > INIT state, it doesn't check if there is drift or not, it updates all
> > > sequences unconditionally. From your discussions with Dilip, I
> > > understand we want to reduce the time it takes to REFRESH SEQUENCES at
> > > the time of an upgrade. If so, then this might not be a good approach.
> > >
> >
> > The point I was trying to make is that with automatic sequence sync,
> > we won't need to execute REFRESH SEQUENCES before upgrade or failover
> > as the sequences will be in sync.
> >
>
> That is a valid goal, however, the sequence sync worker is an
> asynchronous process triggered at specific intervals. For a switchover
> to guarantee consistency, we must disconnect all connections from the
> current publisher and wait for all pending data to sync. Relying on an
> automated, interval-based worker at this critical time is risky, as it
> directly extends our downtime. To minimize the cutover window, we
> should either treat sequence data the same as relational data
> (synchronous streaming) or manually trigger for the sequence sync
> during the switchover phase, i.e. REFRESH SEQUENCES. Anyway this is
> my understanding, do you have anything else in mind that how you are
> trying to completely avoid REFRESH SEQUENCES. Having said that I
> agree that in many cases where sequences are not frequently modified
> it is very much possible that when we are in switchover phase all
> sequences are already in sync and if we can identify that then we can
> avoid REFRESH SEQUENCES, but my point is we can not completely avoid
> that in all caes.
>
Right, I am also of the point that we can't completely avoid REFRESH
SEQUENCES in all cases. Having sequence sync worker syncing
periodically reduces the chances that one need to perform REFRESH
SEQUENCES, so we need both. Do we agree with that?
Additionally, I am thinking that after sync-worker we change REFRESH
SEQUENCES functionality such that it will refresh all sequences during
command. And, even we fail to copy one sequence, the command should
fail. This should use almost the same functionality (like we sync only
the required ones) as syncworker but online. This is because the case
where it would be used will be less likely, say when due to some
reason like max_logical_workers limit, we are not able to invoke
sequencesync worker or just before upgrade (when sequences are not
already synced), so doing sync during refresh command sounds
reasonable. What do you think?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-24 11:25 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 13:18 ` Re: [PATCH] Support automatic sequence replication Dilip Kumar <[email protected]>
2026-02-25 03:02 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-02-25 03:57 ` shveta malik <[email protected]>
0 siblings, 0 replies; 58+ messages in thread
From: shveta malik @ 2026-02-25 03:57 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Dilip Kumar <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Wed, Feb 25, 2026 at 8:32 AM Amit Kapila <[email protected]> wrote:
>
> On Tue, Feb 24, 2026 at 6:49 PM Dilip Kumar <[email protected]> wrote:
> >
> > On Tue, Feb 24, 2026 at 4:55 PM Amit Kapila <[email protected]> wrote:
> > >
> > > On Tue, Feb 24, 2026 at 4:34 PM Ajin Cherian <[email protected]> wrote:
> > > >
> > > > Currently REFRESH SEQUENCES can only be called if the subscription is
> > > > enabled. All it does is change the states of all the sequences in
> > > > subscription_rel to INIT, this will prompt the sequence worker to wake
> > > > up and unconditionally sync all the sequences. For sequences in the
> > > > INIT state, it doesn't check if there is drift or not, it updates all
> > > > sequences unconditionally. From your discussions with Dilip, I
> > > > understand we want to reduce the time it takes to REFRESH SEQUENCES at
> > > > the time of an upgrade. If so, then this might not be a good approach.
> > > >
> > >
> > > The point I was trying to make is that with automatic sequence sync,
> > > we won't need to execute REFRESH SEQUENCES before upgrade or failover
> > > as the sequences will be in sync.
> > >
> >
> > That is a valid goal, however, the sequence sync worker is an
> > asynchronous process triggered at specific intervals. For a switchover
> > to guarantee consistency, we must disconnect all connections from the
> > current publisher and wait for all pending data to sync. Relying on an
> > automated, interval-based worker at this critical time is risky, as it
> > directly extends our downtime. To minimize the cutover window, we
> > should either treat sequence data the same as relational data
> > (synchronous streaming) or manually trigger for the sequence sync
> > during the switchover phase, i.e. REFRESH SEQUENCES. Anyway this is
> > my understanding, do you have anything else in mind that how you are
> > trying to completely avoid REFRESH SEQUENCES. Having said that I
> > agree that in many cases where sequences are not frequently modified
> > it is very much possible that when we are in switchover phase all
> > sequences are already in sync and if we can identify that then we can
> > avoid REFRESH SEQUENCES, but my point is we can not completely avoid
> > that in all caes.
> >
>
> Right, I am also of the point that we can't completely avoid REFRESH
> SEQUENCES in all cases. Having sequence sync worker syncing
> periodically reduces the chances that one need to perform REFRESH
> SEQUENCES, so we need both. Do we agree with that?
>
> Additionally, I am thinking that after sync-worker we change REFRESH
> SEQUENCES functionality such that it will refresh all sequences during
> command. And, even we fail to copy one sequence, the command should
> fail. This should use almost the same functionality (like we sync only
> the required ones) as syncworker but online. This is because the case
> where it would be used will be less likely, say when due to some
> reason like max_logical_workers limit, we are not able to invoke
> sequencesync worker or just before upgrade (when sequences are not
> already synced), so doing sync during refresh command sounds
> reasonable. What do you think?
>
I agree with the overall direction of the discussion. But this is how
I look at it:
Given that sequences can be large, it’s not always practical and may
place an unnecessary burden on users to manually verify whether all
sequences are in sync before an upgrade. Requiring users to inspect
sequence values and manually determine drift before deciding whether
to run REFRESH introduces unnecessary operational complexity. So
instead of trying to find ways to avoid REFRESH, we can position it as
the final safety step before upgrade, optimized to refresh only
drifted sequences and to fail if synchronization of any required
sequence does not succeed. This approach gives us several advantages:
--Users don’t need to manually check for drift.
--REFRESH becomes a safe, final synchronization step before upgrade.
--In most real-world scenarios, sequences are likely already in sync
due to the periodic sync worker. When everything is already
synchronized, REFRESH effectively becomes a lightweight confirmation
check.
--If some sequences are out of sync (due to high activity, worker
limits such as max_logical_replication_workers, or other edge cases),
REFRESH will correct only those.
This creates a practical win-win situation: fast execution when
sequences are already in sync, guaranteed correctness when they are
not, and no additional manual validation burden on the user. Thoughts?
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-26 07:37 ` Ajin Cherian <[email protected]>
2026-02-26 14:13 ` Re: [PATCH] Support automatic sequence replication Nisha Moond <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
1 sibling, 2 replies; 58+ messages in thread
From: Ajin Cherian @ 2026-02-26 07:37 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 24, 2026 at 5:17 PM Amit Kapila <[email protected]> wrote:
>
> Can we find some cheap
> way to detect if sequencesync worker is present or not? Can you think
> some other way to not incur the cost of traversing the worker array
> and also detect sequence worker exit without much delay?
>
Added this.
> Also, shouldn't we need to invoke AcceptInvalidationMessages() as we
> are doing in apply worker when not in a remote transaction? I think it
> will be required to get local_sequence definition changes , if any.
Changed.
Thanks Hou-san for helping me with these changes.
I also did some performance testing on HEAD to see how long REFRESH
SEQUENCES takes for a large number of sequences.
I ran these on a 2× Intel Xeon E5-2699 v4 (22 cores each, 44 cores
total / 88 threads) 512 GB RAM. I didn't see much value in
differentiating between cases where half the sequences were different
or all the sequences were different as REFRESH SEQUENCES updates all
sequences after changing the state of all of them to INIT, it doesn't
matter if they drifted or not.
On HEAD:
time to sync 10000 sequences: 1.080s (1080ms)
time to sync 100000 sequences: 12.069s (12069ms)
time to sync 1000000 sequences: 139.414s (139414ms)
testing script attached (pass in the number of sequences as a run time
parameter).
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v7-0001-Support-automatic-sequence-replication.patch (39.4K, 2-v7-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From 5d2e390460d67b8fececb7f051e898b153f5717b Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Thu, 26 Feb 2026 17:38:33 +1100
Subject: [PATCH v7] Support automatic sequence replication.
Logical replication sequences can drift between publisher and
subscriber as values are consumed independently on each node.
Previously, the sequence sync worker exited after the initial
synchronization, allowing sequences to diverge over time.
This change keeps the sequence sync worker running continuously
so it can monitor sequences and resynchronize them when drift
is detected. The worker uses an adaptive sleep interval:
it starts at 2 seconds, doubles up to a maximum of 30 seconds
when no drift is observed, and resets to the minimum interval
once drift is found.
Sequences remain in the READY state during continuous
synchronization.
Author: Ajin Cherian <[email protected]>
Reviewed-by: Shveta Malik <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Reviewed-by: Ashutosh Sharma <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
---
doc/src/sgml/logical-replication.sgml | 23 +-
doc/src/sgml/ref/alter_subscription.sgml | 9 -
src/backend/commands/sequence.c | 27 ++
.../replication/logical/sequencesync.c | 341 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 46 ++-
src/backend/replication/logical/worker.c | 12 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 79 +---
11 files changed, 341 insertions(+), 202 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..bb523af5d37 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,8 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1817,7 +1818,7 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
+ Subscriber sequence values can become out of sync as the publisher
advances them.
</para>
<para>
@@ -2335,15 +2336,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 5318998e80c..a8dd7a6b42d 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -190,11 +190,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
@@ -236,10 +231,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
recommendations on how to handle any warnings about sequence definition
differences between the publisher and the subscriber.
</para>
- <para>
- See <xref linkend="sequences-out-of-sync"/> for recommendations on how to
- identify and handle out-of-sync sequences.
- </para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..aa815dd19af 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,33 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift.
+ */
+void
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+
+ /* Confirm that the relation is a sequence */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 9c92fddd624..ffbbd1257d0 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -78,23 +90,35 @@ typedef enum CopySeqResult
COPYSEQ_SUCCESS,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
*
- * Start a sequencesync worker if one is not already running. The active
- * sequencesync worker will handle all pending sequence synchronization. If any
- * sequences remain unsynchronized after it exits, a new worker can be started
- * in the next iteration.
+ * Check if the subscription includes sequences and start a sequencesync
+ * worker if one is not already running. The active sequencesync worker will
+ * handle all pending sequence synchronization. If any sequences remain
+ * unsynchronized after it exits, a new worker can be started in the next
+ * iteration.
+ *
+ * The pointer to the sequencesync worker is cached to avoid acquiring an
+ * LWLock and scanning the workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -112,6 +136,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Return if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -144,7 +181,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +208,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +220,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +231,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +242,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +266,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -325,11 +363,12 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Apply remote sequence state to local sequence. If we are doing this
+ * for sequences in the INIT state, move them to the READY state once
+ * synchronized.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner, char relstate)
{
UserContext ucxt;
AclResult aclresult;
@@ -368,19 +407,46 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (relstate == SUBREL_STATE_INIT)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
+/*
+ * check_sequence_drift
+ *
+ * Check if the remote sequence values differ from the local sequence.
+ * Returns true/false if any sequences drifted.
+ */
+static bool
+check_sequence_drift(Relation sequence_rel, LogicalRepSequenceInfo *seqinfo)
+{
+ int64 local_last_value;
+ bool local_is_called;
+
+ /* Get current local sequence state */
+ GetSequence(sequence_rel, &local_last_value, &local_is_called);
+
+ /* Check if values have drifted and return accordingly */
+ return (local_last_value != seqinfo->last_value ||
+ local_is_called != seqinfo->is_called);
+}
+
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If relstate is SUBREL_STATE_READY, only synchronize sequences that
+ * have drifted from their publisher values. Otherwise, synchronize
+ * all sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +456,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +469,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,46 +565,62 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
+ &seqinfo, &seqidx, seqinfos);
+
+ /*
+ * For sequences in INIT state, always sync.
+ * Otherwise, for sequences in READY state, only sync if there's drift.
+ */
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ {
+ if ((seqinfo->relstate == SUBREL_STATE_INIT) ||
+ check_sequence_drift(sequence_rel, seqinfo))
+ sync_status = copy_sequence(seqinfo,
+ sequence_rel->rd_rel->relowner,
+ seqinfo->relstate);
+ else
+ sync_status = COPYSEQ_NO_DRIFT;
+ }
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
+ "logical replication synchronizatio for subscription \"%s\", sequence \"%s.%s\" has been updated",
MySubscription->name, seqinfo->nspname,
seqinfo->seqname);
batch_succeeded_count++;
+ sequence_copied = true;
break;
+
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_mismatched_count++;
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_insuffperm_count++;
break;
+
case COPYSEQ_SKIPPED:
/*
@@ -558,6 +638,15 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -566,20 +655,19 @@ copy_sequences(WalReceiverConn *conn)
ExecDropSingleTupleTableSlot(slot);
walrcv_clear_result(res);
- resetStringInfo(seqstr);
- resetStringInfo(cmd);
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
- MySubscription->name,
- (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
- batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
+ MySubscription->name,
+ (cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
+ batch_size, batch_succeeded_count, batch_mismatched_count,
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,51 +695,55 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
subrel = (Form_pg_subscription_rel) GETSTRUCT(tup);
+ relstate = subrel->srsubstate;
sequence_rel = try_table_open(subrel->srrelid, RowExclusiveLock);
@@ -666,16 +758,19 @@ LogicalRepSyncSequences(void)
continue;
}
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
MemoryContextSwitchTo(oldctx);
@@ -693,36 +788,23 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
+ return false;
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ MemoryContextSwitchTo(oldctx);
+
+ /* Cleanup the memory. */
+ MemoryContextReset(SequenceSyncContext);
+
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +817,79 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+
+ maybe_reread_subscription();
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ /* Adjust sleep interval based on whether sequences were copied over */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+
+ /*
+ * Double the sleep time, but not beyond
+ * the maximum allowable value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..e8d21f55af0 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,9 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+
+ /* Check if sequence worker needs to be started */
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +193,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +207,20 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
+ * has_subtables and has_subsequences are declared as static,
* since the same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
-
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
-
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +230,26 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
/*
* Does the subscription have tables?
*
@@ -277,5 +275,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index bae8c011390..6dce3ced90b 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -4221,6 +4221,7 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
ProcessConfigFile(PGC_SIGHUP);
}
+
if (rc & WL_TIMEOUT)
{
/*
@@ -5098,6 +5099,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5113,6 +5117,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5131,6 +5139,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..f00eea9fbd1 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern void GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 4ecbdcfadac..a41cb045f19 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..1d81518fe22 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,8 +75,7 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
@@ -84,9 +83,6 @@ $node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.47.3
[application/x-zip-compressed] test_seq_sync_performance.zip (2.2K, 3-test_seq_sync_performance.zip)
download
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-26 14:13 ` Nisha Moond <[email protected]>
2026-02-27 06:01 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: Nisha Moond @ 2026-02-26 14:13 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: Amit Kapila <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Feb 26, 2026 at 1:07 PM Ajin Cherian <[email protected]> wrote:
>
> On Tue, Feb 24, 2026 at 5:17 PM Amit Kapila <[email protected]> wrote:
> >
> > Can we find some cheap
> > way to detect if sequencesync worker is present or not? Can you think
> > some other way to not incur the cost of traversing the worker array
> > and also detect sequence worker exit without much delay?
> >
>
> Added this.
>
> > Also, shouldn't we need to invoke AcceptInvalidationMessages() as we
> > are doing in apply worker when not in a remote transaction? I think it
> > will be required to get local_sequence definition changes , if any.
>
> Changed.
>
> Thanks Hou-san for helping me with these changes.
> I also did some performance testing on HEAD to see how long REFRESH
> SEQUENCES takes for a large number of sequences.
> I ran these on a 2× Intel Xeon E5-2699 v4 (22 cores each, 44 cores
> total / 88 threads) 512 GB RAM. I didn't see much value in
> differentiating between cases where half the sequences were different
> or all the sequences were different as REFRESH SEQUENCES updates all
> sequences after changing the state of all of them to INIT, it doesn't
> matter if they drifted or not.
>
> On HEAD:
> time to sync 10000 sequences: 1.080s (1080ms)
> time to sync 100000 sequences: 12.069s (12069ms)
> time to sync 1000000 sequences: 139.414s (139414ms)
>
> testing script attached (pass in the number of sequences as a run time
> parameter).
Hi Ajin,
Thanks for sharing the performance results. I ran the same tests using
your scripts on a different machine with the configuration:
- Chip: Apple M4 Pro, 14 CPU cores
- RAM: 24 GB
- Postgres installation on pg_Head - commit 77c7a17a6e5
For these tests, I used shared_buffers = 4GB. The time taken for 1M
sequences is increased significantly:
time to sync 10000 sequences: .994s (994ms)
time to sync 100000 sequences: 11.032s (11032ms)
time to sync 1000000 sequences: 426.850s (426850ms)
I also tested with shared_buffers = 8GB, and the time for 1M sequences
was 441.794s (441794 ms)
--
Thanks,
Nisha
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 14:13 ` Re: [PATCH] Support automatic sequence replication Nisha Moond <[email protected]>
@ 2026-02-27 06:01 ` Amit Kapila <[email protected]>
0 siblings, 0 replies; 58+ messages in thread
From: Amit Kapila @ 2026-02-27 06:01 UTC (permalink / raw)
To: Nisha Moond <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Feb 26, 2026 at 7:43 PM Nisha Moond <[email protected]> wrote:
>
> On Thu, Feb 26, 2026 at 1:07 PM Ajin Cherian <[email protected]> wrote:
> >
> > I also did some performance testing on HEAD to see how long REFRESH
> > SEQUENCES takes for a large number of sequences.
> > I ran these on a 2× Intel Xeon E5-2699 v4 (22 cores each, 44 cores
> > total / 88 threads) 512 GB RAM. I didn't see much value in
> > differentiating between cases where half the sequences were different
> > or all the sequences were different as REFRESH SEQUENCES updates all
> > sequences after changing the state of all of them to INIT, it doesn't
> > matter if they drifted or not.
> >
> > On HEAD:
> > time to sync 10000 sequences: 1.080s (1080ms)
> > time to sync 100000 sequences: 12.069s (12069ms)
> > time to sync 1000000 sequences: 139.414s (139414ms)
> >
> > testing script attached (pass in the number of sequences as a run time
> > parameter).
>
> Hi Ajin,
> Thanks for sharing the performance results. I ran the same tests using
> your scripts on a different machine with the configuration:
> - Chip: Apple M4 Pro, 14 CPU cores
> - RAM: 24 GB
> - Postgres installation on pg_Head - commit 77c7a17a6e5
>
> For these tests, I used shared_buffers = 4GB. The time taken for 1M
> sequences is increased significantly:
> time to sync 10000 sequences: .994s (994ms)
> time to sync 100000 sequences: 11.032s (11032ms)
> time to sync 1000000 sequences: 426.850s (426850ms)
>
> I also tested with shared_buffers = 8GB, and the time for 1M sequences
> was 441.794s (441794 ms)
>
IIUC, the tests by Ajin and Nisha show the time to sync sequences
varies from a few seconds to minutes depending on the number of
sequences and machine configuration. This indicates that having an
autosync worker can avoid downtime before upgrade, otherwise, users
always need to use REFRESH SEQUENCES before upgrade.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-02-27 11:07 ` Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-02-27 11:07 UTC (permalink / raw)
To: Ajin Cherian <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Feb 26, 2026 at 1:07 PM Ajin Cherian <[email protected]> wrote:
>
Few comments:
=============
1.
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not
connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ MemoryContextSwitchTo(oldctx);
It is better to switch to SequenceSyncContext at the caller of
LogicalRepSyncSequences similar to what we are doing for
ApplyMessageContext.
2.
@@ -4221,6 +4221,7 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
ProcessConfigFile(PGC_SIGHUP);
}
+
if (rc & WL_TIMEOUT)
Spurious line addition.
3. Apart from above, the attached patch contains comments and cosmetic changes.
--
With Regards,
Amit Kapila.
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index ffbbd1257d0..3898b315bb8 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -111,8 +111,8 @@ static MemoryContext SequenceSyncContext = NULL;
* unsynchronized after it exits, a new worker can be started in the next
* iteration.
*
- * The pointer to the sequencesync worker is cached to avoid acquiring an
- * LWLock and scanning the workers array each time via logicalrep_worker_find().
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
MaybeLaunchSequenceSyncWorker(void)
@@ -137,7 +137,7 @@ MaybeLaunchSequenceSyncWorker(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
/*
- * Return if the sequence sync worker for the current subscription is
+ * Quick exit if the sequence sync worker for the current subscription is
* already alive.
*/
if (sequencesync_worker &&
@@ -586,9 +586,8 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronizatio for subscription \"%s\", sequence \"%s.%s\" has been updated",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -765,14 +764,12 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
* allocate them under SequenceSyncContext.
*/
oldctx = MemoryContextSwitchTo(SequenceSyncContext);
-
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -860,7 +857,6 @@ start_sequence_sync(void)
/* Process any invalidation messages that might have accumulated */
AcceptInvalidationMessages();
-
maybe_reread_subscription();
/*
@@ -875,7 +871,6 @@ start_sequence_sync(void)
}
else
{
-
/*
* Double the sleep time, but not beyond
* the maximum allowable value.
Attachments:
[text/plain] v7_amit_1.patch.txt (2.5K, 2-v7_amit_1.patch.txt)
download | inline diff:
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index ffbbd1257d0..3898b315bb8 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -111,8 +111,8 @@ static MemoryContext SequenceSyncContext = NULL;
* unsynchronized after it exits, a new worker can be started in the next
* iteration.
*
- * The pointer to the sequencesync worker is cached to avoid acquiring an
- * LWLock and scanning the workers array each time via logicalrep_worker_find().
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
MaybeLaunchSequenceSyncWorker(void)
@@ -137,7 +137,7 @@ MaybeLaunchSequenceSyncWorker(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
/*
- * Return if the sequence sync worker for the current subscription is
+ * Quick exit if the sequence sync worker for the current subscription is
* already alive.
*/
if (sequencesync_worker &&
@@ -586,9 +586,8 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronizatio for subscription \"%s\", sequence \"%s.%s\" has been updated",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -765,14 +764,12 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
* allocate them under SequenceSyncContext.
*/
oldctx = MemoryContextSwitchTo(SequenceSyncContext);
-
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -860,7 +857,6 @@ start_sequence_sync(void)
/* Process any invalidation messages that might have accumulated */
AcceptInvalidationMessages();
-
maybe_reread_subscription();
/*
@@ -875,7 +871,6 @@ start_sequence_sync(void)
}
else
{
-
/*
* Double the sleep time, but not beyond
* the maximum allowable value.
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-02-28 08:40 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-02-28 08:40 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Friday, February 27, 2026 7:07 PM Amit Kapila <[email protected]> wrote:
> Few comments:
> =============
> 1.
> + oldctx = MemoryContextSwitchTo(SequenceSyncContext);
>
> - initStringInfo(&app_name);
> - appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
> - MySubscription->oid, GetSystemIdentifier());
> + /* Process sequences */
> + sequence_copied = copy_sequences(conn, seqinfos);
>
> - /*
> - * Establish the connection to the publisher for sequence synchronization.
> - */
> - LogRepWorkerWalRcvConn =
> - walrcv_connect(MySubscription->conninfo, true, true,
> - must_use_password,
> - app_name.data, &err);
> - if (LogRepWorkerWalRcvConn == NULL)
> - ereport(ERROR,
> - errcode(ERRCODE_CONNECTION_FAILURE),
> - errmsg("sequencesync worker for subscription \"%s\" could not connect to
> the publisher: %s",
> - MySubscription->name, err));
> -
> - pfree(app_name.data);
> -
> - copy_sequences(LogRepWorkerWalRcvConn);
> + MemoryContextSwitchTo(oldctx);
>
> It is better to switch to SequenceSyncContext at the caller of
> LogicalRepSyncSequences similar to what we are doing for
> ApplyMessageContext.
Agreed. I changed as suggested.
>
> 2.
> @@ -4221,6 +4221,7 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
> ProcessConfigFile(PGC_SIGHUP);
> }
>
> +
> if (rc & WL_TIMEOUT)
>
> Spurious line addition.
>
> 3. Apart from above, the attached patch contains comments and cosmetic
> changes.
Thanks for the patch, I have merged them into 0001.
Here is the V8 patch set addressing the previous review comments:
- For 0001, I noticed that the GetSequence() function added in the patch
fetches the local sequence value without any privilege check. This
allows the worker to read sequence data even without proper SELECT
privilege, which seems unsafe. I've added a SELECT privilege check
before fetching the sequence value. Additionally, I've updated several
comments, made cosmetic changes, commit message update, and run
pgindent on all patches.
- 0002 includes the changes to synchronize sequences directly in the
REFRESH SEQUENCES command
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v8-0001-Support-automatic-sequence-replication.patch (41.3K, 2-v8-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From a90dbc2d9abc70d066a4a180c5334980d8d2c880 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Tue, 24 Feb 2026 21:37:01 +1100
Subject: [PATCH v8 1/2] Support automatic sequence replication.
Currently, sequence values are synchronized from publisher to subscriber only
when the user manually runs ALTER SUBSCRIPTION ... REFRESH PUBLICATION (which
affects only newly subscribed sequences) or REFRESH SEQUENCES. The sequence sync
worker exits immediately after completing each synchronization round.
The primary use case for sequence replication is during upgrades, where it's
recommended that users ensure sequences are in sync by running REFRESH SEQUENCES
before finishing the upgrade. However, this command can be slow when
synchronizing a large number of sequences, potentially increasing downtime.
To address this, this commit makes the sequence sync worker long-lived,
continuously monitoring sequences and resynchronizing them when drift is
detected. The worker uses an adaptive sleep interval: it starts at 2 seconds,
doubles up to a maximum of 30 seconds when no drift is observed, and resets to
the minimum interval once drift is found.
With this change, most sequences are silently synchronized in the background,
eliminating the need to run REFRESH SEQUENCES for the majority of cases.
However, frequently updated sequences may still lag behind, requiring a final
REFRESH SEQUENCES before upgrade completion. Users can monitor progress by
checking whether sequence states transition from INIT to READY in
pg_subscription_rel.
The REFRESH SEQUENCES command is retained for this final synchronization step,
though it currently updates all sequence states to INIT, which has room for
improvement. A future patch will enhance this command to synchronize sequences
directly without launching a worker, reducing catalog overhead.
Author: Ajin Cherian <[email protected]>
Author: Zhijie Hou <[email protected]>
Reviewed-by: Shveta Malik <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Reviewed-by: Ashutosh Sharma <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
---
doc/src/sgml/logical-replication.sgml | 23 +-
doc/src/sgml/ref/alter_subscription.sgml | 5 -
src/backend/commands/sequence.c | 29 ++
.../replication/logical/sequencesync.c | 373 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 47 +--
src/backend/replication/logical/worker.c | 11 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 79 +---
11 files changed, 369 insertions(+), 204 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..bb523af5d37 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,8 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1817,7 +1818,7 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
+ Subscriber sequence values can become out of sync as the publisher
advances them.
</para>
<para>
@@ -2335,15 +2336,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 5318998e80c..5a7b4f3c2c2 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -190,11 +190,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..ac5a0197e1c 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,35 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift. The caller must hold a lock on the sequence
+ * and have sufficient privileges to access it.
+ */
+void
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+
+ /* Confirm that the relation is a sequence and is locked */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+ Assert(CheckRelationLockedByMe(seqrel, AccessShareLock, true));
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 9c92fddd624..7df871ae7bc 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -78,23 +90,35 @@ typedef enum CopySeqResult
COPYSEQ_SUCCESS,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
+ *
+ * Check if the subscription includes sequences and start a sequencesync
+ * worker if one is not already running. The active sequencesync worker will
+ * handle all pending sequence synchronization. If any sequences remain
+ * unsynchronized after it exits, a new worker can be started in the next
+ * iteration.
*
- * Start a sequencesync worker if one is not already running. The active
- * sequencesync worker will handle all pending sequence synchronization. If any
- * sequences remain unsynchronized after it exits, a new worker can be started
- * in the next iteration.
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -112,6 +136,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Quick exit if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -144,7 +181,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +208,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +220,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +231,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +242,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +266,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -325,32 +363,77 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Preliminary check to determine if copying the sequence is allowed.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
{
- UserContext ucxt;
AclResult aclresult;
+ Oid seqoid = seqinfo->localrelid;
+ int64 local_last_value;
+ bool local_is_called;
+
+ /* Perform drift check if it's not the initial sync */
+ if (seqinfo->relstate == SUBREL_STATE_READY)
+ {
+ /*
+ * Verify that the current user has SELECT privilege on the sequence.
+ * This is required to read the sequence state below.
+ */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_SELECT);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ /* Get current local sequence state */
+ GetSequence(sequence_rel, &local_last_value, &local_is_called);
+
+ /*
+ * Skip synchronization if the sequence is already in READY state and
+ * has not drifted from the publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Verify that the current user can update the sequence */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ return COPYSEQ_SUCCESS;
+}
+
+/*
+ * Apply remote sequence state to local sequence. If we are doing this
+ * for sequences in the INIT state, move them to the READY state once
+ * synchronized.
+ */
+static CopySeqResult
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+{
+ UserContext ucxt;
bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
+ CopySeqResult result;
/*
* If the user did not opt to run as the owner of the subscription
* ('run_as_owner'), then copy the sequence as the owner of the sequence.
*/
if (!run_as_owner)
- SwitchToUntrustedUser(seqowner, &ucxt);
+ SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+ result = validate_seqsync_state(seqinfo, sequence_rel);
- if (aclresult != ACLCHECK_OK)
+ if (result != COPYSEQ_SUCCESS)
{
if (!run_as_owner)
RestoreUserContext(&ucxt);
- return COPYSEQ_INSUFFICIENT_PERM;
+ return result;
}
/*
@@ -368,19 +451,26 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (seqinfo->relstate == SUBREL_STATE_INIT)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If relstate is SUBREL_STATE_READY, only synchronize sequences that
+ * have drifted from their publisher values. Otherwise, synchronize
+ * all sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +480,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +493,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,46 +589,53 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
+ &seqinfo, &seqidx, seqinfos);
+
+ /*
+ * For sequences in INIT state, always sync. Otherwise, for
+ * sequences in READY state, only sync if there's drift.
+ */
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ sync_status = copy_sequence(seqinfo, sequence_rel);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
+ sequence_copied = true;
break;
+
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_mismatched_count++;
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_insuffperm_count++;
break;
+
case COPYSEQ_SKIPPED:
/*
@@ -558,6 +653,15 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -566,20 +670,19 @@ copy_sequences(WalReceiverConn *conn)
ExecDropSingleTupleTableSlot(slot);
walrcv_clear_result(res);
- resetStringInfo(seqstr);
- resetStringInfo(cmd);
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
MySubscription->name,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,51 +710,55 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
subrel = (Form_pg_subscription_rel) GETSTRUCT(tup);
+ relstate = subrel->srsubstate;
sequence_rel = try_table_open(subrel->srrelid, RowExclusiveLock);
@@ -666,18 +773,19 @@ LogicalRepSyncSequences(void)
continue;
}
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
-
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -693,36 +801,16 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
-
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ return false;
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +823,91 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence
+ * synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+ maybe_reread_subscription();
+
+ /*
+ * Perform sequence synchronization under SequenceSyncContext and
+ * reset it each cycle to avoid manual memory management.
+ */
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ MemoryContextReset(SequenceSyncContext);
+ MemoryContextSwitchTo(oldctx);
+
+ /*
+ * Adjust sleep interval based on whether sequences were copied
+ * over
+ */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+ /*
+ * Double the sleep time, but not beyond the maximum allowable
+ * value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..233ac7ae873 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,9 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+
+ /* Check if sequence worker needs to be started */
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +193,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +207,22 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
- * since the same value can be used until the system table is invalidated.
+ * has_subtables and has_subsequences are declared as static, since the
+ * same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +232,27 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
+
/*
* Does the subscription have tables?
*
@@ -277,5 +278,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index bae8c011390..93b7e1b9993 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -5098,6 +5098,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5113,6 +5116,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5131,6 +5138,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..f00eea9fbd1 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern void GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 4ecbdcfadac..a41cb045f19 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..1d81518fe22 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,8 +75,7 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
@@ -84,9 +83,6 @@ $node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.51.1.windows.1
[application/octet-stream] v8-0002-Synchronize-sequences-directly-in-REFRESH-SEQUENC.patch (17.2K, 3-v8-0002-Synchronize-sequences-directly-in-REFRESH-SEQUENC.patch)
download | inline diff:
From 53d4a199cb4703dea8eaa568e7aa624277496793 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Sat, 28 Feb 2026 16:14:14 +0800
Subject: [PATCH v8 2/2] Synchronize sequences directly in REFRESH SEQUENCES
command
The ALTER SUBSCRIPTION ... REFRESH SEQUENCES command currently sets all
sequence states in pg_subscription_rel to INIT and relies on the sequence sync
worker to perform the actual synchronization and update states to READY.
With the recent change making the sequence sync worker long-lived, most
sequences are now synchronized in the background, reducing the need for
REFRESH SEQUENCES. However, the command remains necessary for sequences that
haven't been synchronized.
This commit enhances REFRESH SEQUENCES to synchronize sequences directly within
the command itself, eliminating the overhead of launching a worker and updating
catalog entries unnecessarily.
---
src/backend/commands/subscriptioncmds.c | 17 +-
.../replication/logical/sequencesync.c | 165 ++++++++++++++----
src/include/replication/logicalworker.h | 5 +
src/test/subscription/t/036_sequences.pl | 49 ++++++
4 files changed, 190 insertions(+), 46 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 5e3c0964d38..0a5acfda0ff 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1245,25 +1245,10 @@ AlterSubscription_refresh_seq(Subscription *sub)
PG_TRY();
{
- List *subrel_states;
-
check_publications_origin_sequences(wrconn, sub->publications, true,
sub->origin, NULL, 0, sub->name);
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
+ AlterSubSyncSequences(wrconn, sub->oid, sub->name, sub->runasowner);
}
PG_FINALLY();
{
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 7df871ae7bc..71452261dd2 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -208,7 +208,7 @@ get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx, List *seqinfos)
+ List *missing_seqs_idx, List *seqinfos, char *subname)
{
StringInfo seqstr;
@@ -254,7 +254,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
ereport(ERROR,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("logical replication sequence synchronization failed for subscription \"%s\"",
- MySubscription->name));
+ subname));
}
/*
@@ -282,6 +282,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
+ LOCKMODE lockmode;
*seqidx = DatumGetInt32(slot_getattr(slot, ++col, &isnull));
Assert(!isnull);
@@ -328,7 +329,20 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
seqinfo_local->found_on_pub = true;
- *sequence_rel = try_table_open(seqinfo_local->localrelid, RowExclusiveLock);
+ /*
+ * We take a stronger lock during DDL commands (currently only ALTER
+ * SUBSCRIPTION ... REFRESH SEQUENCES) to prevent concurrent sequencesync
+ * workers from updating the page_lsn while the DDL is also updating the
+ * same sequence. This ensures we can always fetch the latest page_lsn to
+ * determine whether the remote sequence value should be synchronized (see
+ * should_sync_sequence).
+ */
+ if (IsLogicalWorker())
+ lockmode = RowExclusiveLock;
+ else
+ lockmode = ShareRowExclusiveLock;
+
+ *sequence_rel = try_table_open(seqinfo_local->localrelid, lockmode);
/* Sequence was concurrently dropped? */
if (!*sequence_rel)
@@ -366,7 +380,8 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
* Preliminary check to determine if copying the sequence is allowed.
*/
static CopySeqResult
-validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ XLogRecPtr local_page_lsn)
{
AclResult aclresult;
Oid seqoid = seqinfo->localrelid;
@@ -376,6 +391,16 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
/* Perform drift check if it's not the initial sync */
if (seqinfo->relstate == SUBREL_STATE_READY)
{
+ /*
+ * Skip synchronization if we are processing outdated sequence info
+ * based on the LSN. This occurs when the sequence has been updated to
+ * more recent data concurrently (via either ALTER SUBSCRIPTION ...
+ * REFRESH SEQUENCES or the sequencesync worker).
+ */
+ if (XLogRecPtrIsValid(local_page_lsn) &&
+ local_page_lsn > seqinfo->page_lsn)
+ return COPYSEQ_NO_DRIFT;
+
/*
* Verify that the current user has SELECT privilege on the sequence.
* This is required to read the sequence state below.
@@ -389,9 +414,32 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
GetSequence(sequence_rel, &local_last_value, &local_is_called);
/*
- * Skip synchronization if the sequence is already in READY state and
- * has not drifted from the publisher's value.
+ * Skip synchronization if the local sequence value is already ahead of
+ * the publisher's value.
+ *
+ * XXX This occurs not only when the local sequence has been
+ * synchronized to a newer value from the publisher (where skipping is
+ * necessary to avoid backward movement), but also when the local
+ * sequence has been manually updated by the user on the subscriber. The
+ * latter could be considered a replication conflict, and overwriting
+ * the user's update might be acceptable. However, since we cannot
+ * easily distinguish between these two scenarios, we choose to skip
+ * synchronization in all cases and emit a WARNING to notify the user to
+ * manually resolve the conflict.
*/
+ if (local_last_value > seqinfo->last_value)
+ {
+ ereport(WARNING,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("skipped synchronizing the sequence \"%s.%s\"",
+ seqinfo->nspname, seqinfo->seqname),
+ errdetail("The local last_value %lld is ahead of the one on publisher",
+ (long long int) local_last_value));
+
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Skip synchronization if the sequence hasn't drifted */
if (local_last_value == seqinfo->last_value &&
local_is_called == seqinfo->is_called)
return COPYSEQ_NO_DRIFT;
@@ -412,12 +460,16 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* synchronized.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ Oid subid, bool run_as_owner)
{
UserContext ucxt;
- bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
CopySeqResult result;
+ XLogRecPtr local_page_lsn;
+
+ (void) GetSubscriptionRelState(subid, RelationGetRelid(sequence_rel),
+ &local_page_lsn);
/*
* If the user did not opt to run as the owner of the subscription
@@ -426,7 +478,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
if (!run_as_owner)
SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- result = validate_seqsync_state(seqinfo, sequence_rel);
+ result = validate_seqsync_state(seqinfo, sequence_rel, local_page_lsn);
if (result != COPYSEQ_SUCCESS)
{
@@ -453,8 +505,9 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* Record the remote sequence's LSN in pg_subscription_rel and mark the
* sequence as READY if updating a sequence that is in INIT state.
*/
- if (seqinfo->relstate == SUBREL_STATE_INIT)
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ if (seqinfo->relstate == SUBREL_STATE_INIT ||
+ seqinfo->page_lsn != local_page_lsn)
+ UpdateSubscriptionRelState(subid, seqoid, SUBREL_STATE_READY,
seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
@@ -470,7 +523,8 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* Returns true/false if any sequences were actually copied.
*/
static bool
-copy_sequences(WalReceiverConn *conn, List *seqinfos)
+copy_sequences(WalReceiverConn *conn, List *seqinfos, Oid subid, char *subname,
+ bool runasowner)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -496,11 +550,16 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
+ bool started_tx = false;
WalRcvExecResult *res;
TupleTableSlot *slot;
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
for (int idx = cur_batch_base_index; idx < n_seqinfos; idx++)
{
@@ -596,14 +655,15 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
* sequences in READY state, only sync if there's drift.
*/
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo, sequence_rel);
+ sync_status = copy_sequence(seqinfo, sequence_rel,
+ subid, runasowner);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
"logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
- seqinfo->nspname, seqinfo->seqname, MySubscription->name);
+ seqinfo->nspname, seqinfo->seqname, subname);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -611,9 +671,8 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in SequenceSyncContext
- * since these will be used after the transaction is
- * committed.
+ * Remember mismatched sequences in SequenceSyncContext since
+ * these will be used after the transaction is committed.
*/
oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
@@ -679,13 +738,17 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
elog(DEBUG1,
"logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
- MySubscription->name,
+ subname,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
- /* Commit this batch, and prepare for next batch */
- CommitTransactionCommand();
+ /*
+ * Commit this batch if started a transaction, and prepare for next
+ * batch.
+ */
+ if (started_tx)
+ CommitTransactionCommand();
if (batch_missing_count)
{
@@ -710,7 +773,7 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx, seqinfos);
+ missing_seqs_idx, seqinfos, subname);
return sequence_copied;
}
@@ -722,20 +785,23 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
* Returns true if sequences have been updated.
*/
static bool
-LogicalRepSyncSequences(WalReceiverConn *conn)
+LogicalRepSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
- Oid subid = MyLogicalRepWorker->subid;
bool sequence_copied = false;
List *seqinfos = NIL;
MemoryContext oldctx;
+ bool started_tx = false;
- Assert(SequenceSyncContext);
-
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
@@ -795,7 +861,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- CommitTransactionCommand();
+ if (started_tx)
+ CommitTransactionCommand();
/*
* Exit early if no catalog entries found, likely due to concurrent drops.
@@ -804,7 +871,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
return false;
/* Process sequences */
- sequence_copied = copy_sequences(conn, seqinfos);
+ sequence_copied = copy_sequences(conn, seqinfos, subid, subname,
+ runasowner);
return sequence_copied;
}
@@ -879,7 +947,10 @@ start_sequence_sync(void)
/*
* Synchronize all sequences (both READY and INIT states).
*/
- sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
+ MySubscription->oid,
+ MySubscription->name,
+ MySubscription->runasowner);
MemoryContextReset(SequenceSyncContext);
MemoryContextSwitchTo(oldctx);
@@ -941,3 +1012,37 @@ SequenceSyncWorkerMain(Datum main_arg)
FinishSyncWorker();
}
+
+/*
+ * Wrapper for LogicalRepSyncSequences to synchronize all sequences of a
+ * subscription from outside the sequencesync worker
+ */
+void
+AlterSubSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
+{
+ /*
+ * Init the SequenceSyncContext which we clean up after the sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(CurrentMemoryContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ MemoryContext oldctx;
+
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ LogicalRepSyncSequences(conn, subid, subname, runasowner);
+
+ MemoryContextSwitchTo(oldctx);
+ }
+ PG_FINALLY();
+ {
+ MemoryContextDelete(SequenceSyncContext);
+ SequenceSyncContext = NULL;
+ }
+ PG_END_TRY();
+}
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..73afd7853d0 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,6 +14,8 @@
#include <signal.h>
+#include "replication/walreceiver.h"
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
extern void ApplyWorkerMain(Datum main_arg);
@@ -31,4 +33,7 @@ extern void LogicalRepWorkersWakeupAtCommit(Oid subid);
extern void AtEOXact_LogicalRepWorkers(bool isCommit);
+extern void AlterSubSyncSequences(WalReceiverConn *conn, Oid subid,
+ char *subname, bool runasowner);
+
#endif /* LOGICALWORKER_H */
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 1d81518fe22..9a61b7bd0c8 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -176,4 +176,53 @@ $node_subscriber->wait_for_log(
qr/WARNING: ( [A-Z0-9]+:)? missing sequence on publisher \("public.regress_s4"\)/,
$log_offset);
+##########
+# ALTER SUBSCRIPTION ... REFRESH SEQUENCES synchronizes sequences online,
+# eliminating the need to launch a sequencesync worker.
+##########
+
+# Reduce max_logical_replication_workers to disallow sequence worker from running
+$node_subscriber->append_conf('postgresql.conf',
+ qq(max_logical_replication_workers = 0));
+$node_subscriber->restart;
+
+# Verify there is no logical replication apply worker running
+$result = $node_subscriber->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'logical replication apply worker'");
+
+is($result, '0', 'no logical replication worker is running');
+
+# Increment sequence on publisher
+$node_publisher->safe_psql('postgres',
+ qq(SELECT nextval('regress_s1');));
+
+# The command should fail due to missing sequence ('regress_s4')
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+like(
+ $stderr,
+ qr/WARNING: missing sequence on publisher \("public.regress_s4"\)/,
+ "output the wanring for the missing sequence regress_s4");
+
+like(
+ $stderr,
+ qr/ERROR: logical replication sequence synchronization failed for subscription \"regress_seq_sub\"/,
+ "the command failed due to the missing sequence regress_s4");
+
+# Refresh the publication to remove the missing sequence
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION;");
+
+# Sync the sequence regress_s1
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+# Get the current sequence value on subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ qq(SELECT last_value FROM regress_s1;));
+
+is($result, '201', 'sequence regress_s1 is synced now');
+
done_testing();
--
2.51.1.windows.1
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-02 07:58 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 04:20 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 2 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-02 07:58 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Saturday, February 28, 2026 4:41 PM Zhijie Hou (Fujitsu) <[email protected]> wrote:
> Here is the V8 patch set addressing the previous review comments:
>
> - For 0001, I noticed that the GetSequence() function added in the patch
> fetches the local sequence value without any privilege check. This
> allows the worker to read sequence data even without proper SELECT
> privilege, which seems unsafe. I've added a SELECT privilege check
> before fetching the sequence value. Additionally, I've updated several
> comments, made cosmetic changes, commit message update, and run
> pgindent on all patches.
>
> - 0002 includes the changes to synchronize sequences directly in the
> REFRESH SEQUENCES command
Rebased the patch to silence compile warning due to a recent commit
a2c89835.
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v9-0002-Synchronize-sequences-directly-in-REFRESH-SEQUENC.patch (17.2K, 2-v9-0002-Synchronize-sequences-directly-in-REFRESH-SEQUENC.patch)
download | inline diff:
From 25d17db1e61f9078dab37417ebb8e06d2a4ba766 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Sat, 28 Feb 2026 16:14:14 +0800
Subject: [PATCH v9 2/2] Synchronize sequences directly in REFRESH SEQUENCES
command
The ALTER SUBSCRIPTION ... REFRESH SEQUENCES command currently sets all
sequence states in pg_subscription_rel to INIT and relies on the sequence sync
worker to perform the actual synchronization and update states to READY.
With the recent change making the sequence sync worker long-lived, most
sequences are now synchronized in the background, reducing the need for
REFRESH SEQUENCES. However, the command remains necessary for sequences that
haven't been synchronized.
This commit enhances REFRESH SEQUENCES to synchronize sequences directly within
the command itself, eliminating the overhead of launching a worker and updating
catalog entries unnecessarily.
---
src/backend/commands/subscriptioncmds.c | 17 +-
.../replication/logical/sequencesync.c | 165 ++++++++++++++----
src/include/replication/logicalworker.h | 5 +
src/test/subscription/t/036_sequences.pl | 49 ++++++
4 files changed, 190 insertions(+), 46 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 5e3c0964d38..0a5acfda0ff 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1245,25 +1245,10 @@ AlterSubscription_refresh_seq(Subscription *sub)
PG_TRY();
{
- List *subrel_states;
-
check_publications_origin_sequences(wrconn, sub->publications, true,
sub->origin, NULL, 0, sub->name);
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
+ AlterSubSyncSequences(wrconn, sub->oid, sub->name, sub->runasowner);
}
PG_FINALLY();
{
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index fad1bb548b2..b4081216477 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -209,7 +209,7 @@ get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx, List *seqinfos)
+ List *missing_seqs_idx, List *seqinfos, char *subname)
{
StringInfo seqstr;
@@ -255,7 +255,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
ereport(ERROR,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("logical replication sequence synchronization failed for subscription \"%s\"",
- MySubscription->name));
+ subname));
}
/*
@@ -283,6 +283,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
+ LOCKMODE lockmode;
*seqidx = DatumGetInt32(slot_getattr(slot, ++col, &isnull));
Assert(!isnull);
@@ -329,7 +330,20 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
seqinfo_local->found_on_pub = true;
- *sequence_rel = try_table_open(seqinfo_local->localrelid, RowExclusiveLock);
+ /*
+ * We take a stronger lock during DDL commands (currently only ALTER
+ * SUBSCRIPTION ... REFRESH SEQUENCES) to prevent concurrent sequencesync
+ * workers from updating the page_lsn while the DDL is also updating the
+ * same sequence. This ensures we can always fetch the latest page_lsn to
+ * determine whether the remote sequence value should be synchronized (see
+ * validate_seqsync_state).
+ */
+ if (IsLogicalWorker())
+ lockmode = RowExclusiveLock;
+ else
+ lockmode = ShareRowExclusiveLock;
+
+ *sequence_rel = try_table_open(seqinfo_local->localrelid, lockmode);
/* Sequence was concurrently dropped? */
if (!*sequence_rel)
@@ -367,7 +381,8 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
* Preliminary check to determine if copying the sequence is allowed.
*/
static CopySeqResult
-validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ XLogRecPtr local_page_lsn)
{
AclResult aclresult;
Oid seqoid = seqinfo->localrelid;
@@ -377,6 +392,16 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
/* Perform drift check if it's not the initial sync */
if (seqinfo->relstate == SUBREL_STATE_READY)
{
+ /*
+ * Skip synchronization if we are processing outdated sequence info
+ * based on the LSN. This occurs when the sequence has been updated to
+ * more recent data concurrently (via either ALTER SUBSCRIPTION ...
+ * REFRESH SEQUENCES or the sequencesync worker).
+ */
+ if (XLogRecPtrIsValid(local_page_lsn) &&
+ local_page_lsn > seqinfo->page_lsn)
+ return COPYSEQ_NO_DRIFT;
+
/*
* Verify that the current user has SELECT privilege on the sequence.
* This is required to read the sequence state below.
@@ -390,9 +415,32 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
GetSequence(sequence_rel, &local_last_value, &local_is_called);
/*
- * Skip synchronization if the sequence is already in READY state and
- * has not drifted from the publisher's value.
+ * Skip synchronization if the local sequence value is already ahead of
+ * the publisher's value.
+ *
+ * XXX This occurs not only when the local sequence has been
+ * synchronized to a newer value from the publisher (where skipping is
+ * necessary to avoid backward movement), but also when the local
+ * sequence has been manually updated by the user on the subscriber. The
+ * latter could be considered a replication conflict, and overwriting
+ * the user's update might be acceptable. However, since we cannot
+ * easily distinguish between these two scenarios, we choose to skip
+ * synchronization in all cases and emit a WARNING to notify the user to
+ * manually resolve the conflict.
*/
+ if (local_last_value > seqinfo->last_value)
+ {
+ ereport(WARNING,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("skipped synchronizing the sequence \"%s.%s\"",
+ seqinfo->nspname, seqinfo->seqname),
+ errdetail("The local last_value %lld is ahead of the one on publisher",
+ (long long int) local_last_value));
+
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Skip synchronization if the sequence hasn't drifted */
if (local_last_value == seqinfo->last_value &&
local_is_called == seqinfo->is_called)
return COPYSEQ_NO_DRIFT;
@@ -413,12 +461,16 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* synchronized.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ Oid subid, bool run_as_owner)
{
UserContext ucxt;
- bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
CopySeqResult result;
+ XLogRecPtr local_page_lsn;
+
+ (void) GetSubscriptionRelState(subid, RelationGetRelid(sequence_rel),
+ &local_page_lsn);
/*
* If the user did not opt to run as the owner of the subscription
@@ -427,7 +479,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
if (!run_as_owner)
SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- result = validate_seqsync_state(seqinfo, sequence_rel);
+ result = validate_seqsync_state(seqinfo, sequence_rel, local_page_lsn);
if (result != COPYSEQ_SUCCESS)
{
@@ -454,8 +506,9 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* Record the remote sequence's LSN in pg_subscription_rel and mark the
* sequence as READY if updating a sequence that is in INIT state.
*/
- if (seqinfo->relstate == SUBREL_STATE_INIT)
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ if (seqinfo->relstate == SUBREL_STATE_INIT ||
+ seqinfo->page_lsn != local_page_lsn)
+ UpdateSubscriptionRelState(subid, seqoid, SUBREL_STATE_READY,
seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
@@ -471,7 +524,8 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* Returns true/false if any sequences were actually copied.
*/
static bool
-copy_sequences(WalReceiverConn *conn, List *seqinfos)
+copy_sequences(WalReceiverConn *conn, List *seqinfos, Oid subid, char *subname,
+ bool runasowner)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -497,11 +551,16 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
+ bool started_tx = false;
WalRcvExecResult *res;
TupleTableSlot *slot;
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
for (int idx = cur_batch_base_index; idx < n_seqinfos; idx++)
{
@@ -597,14 +656,15 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
* sequences in READY state, only sync if there's drift.
*/
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo, sequence_rel);
+ sync_status = copy_sequence(seqinfo, sequence_rel,
+ subid, runasowner);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
"logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
- seqinfo->nspname, seqinfo->seqname, MySubscription->name);
+ seqinfo->nspname, seqinfo->seqname, subname);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -612,9 +672,8 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in SequenceSyncContext
- * since these will be used after the transaction is
- * committed.
+ * Remember mismatched sequences in SequenceSyncContext since
+ * these will be used after the transaction is committed.
*/
oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
@@ -680,13 +739,17 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
elog(DEBUG1,
"logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
- MySubscription->name,
+ subname,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
- /* Commit this batch, and prepare for next batch */
- CommitTransactionCommand();
+ /*
+ * Commit this batch if started a transaction, and prepare for next
+ * batch.
+ */
+ if (started_tx)
+ CommitTransactionCommand();
if (batch_missing_count)
{
@@ -711,7 +774,7 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx, seqinfos);
+ missing_seqs_idx, seqinfos, subname);
return sequence_copied;
}
@@ -723,20 +786,23 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
* Returns true if sequences have been updated.
*/
static bool
-LogicalRepSyncSequences(WalReceiverConn *conn)
+LogicalRepSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
- Oid subid = MyLogicalRepWorker->subid;
bool sequence_copied = false;
List *seqinfos = NIL;
MemoryContext oldctx;
+ bool started_tx = false;
- Assert(SequenceSyncContext);
-
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
@@ -796,7 +862,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- CommitTransactionCommand();
+ if (started_tx)
+ CommitTransactionCommand();
/*
* Exit early if no catalog entries found, likely due to concurrent drops.
@@ -805,7 +872,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
return false;
/* Process sequences */
- sequence_copied = copy_sequences(conn, seqinfos);
+ sequence_copied = copy_sequences(conn, seqinfos, subid, subname,
+ runasowner);
return sequence_copied;
}
@@ -880,7 +948,10 @@ start_sequence_sync(void)
/*
* Synchronize all sequences (both READY and INIT states).
*/
- sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
+ MySubscription->oid,
+ MySubscription->name,
+ MySubscription->runasowner);
MemoryContextReset(SequenceSyncContext);
MemoryContextSwitchTo(oldctx);
@@ -942,3 +1013,37 @@ SequenceSyncWorkerMain(Datum main_arg)
FinishSyncWorker();
}
+
+/*
+ * Wrapper for LogicalRepSyncSequences to synchronize all sequences of a
+ * subscription from outside the sequencesync worker
+ */
+void
+AlterSubSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
+{
+ /*
+ * Init the SequenceSyncContext which we clean up after the sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(CurrentMemoryContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ MemoryContext oldctx;
+
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ LogicalRepSyncSequences(conn, subid, subname, runasowner);
+
+ MemoryContextSwitchTo(oldctx);
+ }
+ PG_FINALLY();
+ {
+ MemoryContextDelete(SequenceSyncContext);
+ SequenceSyncContext = NULL;
+ }
+ PG_END_TRY();
+}
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..73afd7853d0 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,6 +14,8 @@
#include <signal.h>
+#include "replication/walreceiver.h"
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
extern void ApplyWorkerMain(Datum main_arg);
@@ -31,4 +33,7 @@ extern void LogicalRepWorkersWakeupAtCommit(Oid subid);
extern void AtEOXact_LogicalRepWorkers(bool isCommit);
+extern void AlterSubSyncSequences(WalReceiverConn *conn, Oid subid,
+ char *subname, bool runasowner);
+
#endif /* LOGICALWORKER_H */
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 1d81518fe22..9a61b7bd0c8 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -176,4 +176,53 @@ $node_subscriber->wait_for_log(
qr/WARNING: ( [A-Z0-9]+:)? missing sequence on publisher \("public.regress_s4"\)/,
$log_offset);
+##########
+# ALTER SUBSCRIPTION ... REFRESH SEQUENCES synchronizes sequences online,
+# eliminating the need to launch a sequencesync worker.
+##########
+
+# Reduce max_logical_replication_workers to disallow sequence worker from running
+$node_subscriber->append_conf('postgresql.conf',
+ qq(max_logical_replication_workers = 0));
+$node_subscriber->restart;
+
+# Verify there is no logical replication apply worker running
+$result = $node_subscriber->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'logical replication apply worker'");
+
+is($result, '0', 'no logical replication worker is running');
+
+# Increment sequence on publisher
+$node_publisher->safe_psql('postgres',
+ qq(SELECT nextval('regress_s1');));
+
+# The command should fail due to missing sequence ('regress_s4')
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+like(
+ $stderr,
+ qr/WARNING: missing sequence on publisher \("public.regress_s4"\)/,
+ "output the wanring for the missing sequence regress_s4");
+
+like(
+ $stderr,
+ qr/ERROR: logical replication sequence synchronization failed for subscription \"regress_seq_sub\"/,
+ "the command failed due to the missing sequence regress_s4");
+
+# Refresh the publication to remove the missing sequence
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION;");
+
+# Sync the sequence regress_s1
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+# Get the current sequence value on subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ qq(SELECT last_value FROM regress_s1;));
+
+is($result, '201', 'sequence regress_s1 is synced now');
+
done_testing();
--
2.51.1.windows.1
[application/octet-stream] v9-0001-Support-automatic-sequence-replication.patch (41.6K, 3-v9-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From 641532c6b3034c345968d74215576f7020ee537f Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Tue, 24 Feb 2026 21:37:01 +1100
Subject: [PATCH v9 1/2] Support automatic sequence replication.
Currently, sequence values are synchronized from publisher to subscriber only
when the user manually runs ALTER SUBSCRIPTION ... REFRESH PUBLICATION (which
affects only newly subscribed sequences) or REFRESH SEQUENCES. The sequence sync
worker exits immediately after completing each synchronization round.
The primary use case for sequence replication is during upgrades, where it's
recommended that users ensure sequences are in sync by running REFRESH SEQUENCES
before finishing the upgrade. However, this command can be slow when
synchronizing a large number of sequences, potentially increasing downtime.
To address this, this commit makes the sequence sync worker long-lived,
continuously monitoring sequences and resynchronizing them when drift is
detected. The worker uses an adaptive sleep interval: it starts at 2 seconds,
doubles up to a maximum of 30 seconds when no drift is observed, and resets to
the minimum interval once drift is found.
With this change, most sequences are silently synchronized in the background,
eliminating the need to run REFRESH SEQUENCES for the majority of cases.
However, frequently updated sequences may still lag behind, requiring a final
REFRESH SEQUENCES before upgrade completion. Users can monitor progress by
checking whether sequence states transition from INIT to READY in
pg_subscription_rel.
The REFRESH SEQUENCES command is retained for this final synchronization step,
though it currently updates all sequence states to INIT, which has room for
improvement. A future patch will enhance this command to synchronize sequences
directly without launching a worker, reducing catalog overhead.
Author: Ajin Cherian <[email protected]>
Author: Zhijie Hou <[email protected]>
Reviewed-by: Shveta Malik <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Reviewed-by: Ashutosh Sharma <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
---
doc/src/sgml/logical-replication.sgml | 23 +-
doc/src/sgml/ref/alter_subscription.sgml | 5 -
src/backend/commands/sequence.c | 29 ++
.../replication/logical/sequencesync.c | 374 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 47 +--
src/backend/replication/logical/worker.c | 11 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 79 +---
11 files changed, 370 insertions(+), 204 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..bb523af5d37 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,8 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1817,7 +1818,7 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
+ Subscriber sequence values can become out of sync as the publisher
advances them.
</para>
<para>
@@ -2335,15 +2336,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 5318998e80c..5a7b4f3c2c2 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -190,11 +190,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..ac5a0197e1c 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,35 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift. The caller must hold a lock on the sequence
+ * and have sufficient privileges to access it.
+ */
+void
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+
+ /* Confirm that the relation is a sequence and is locked */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+ Assert(CheckRelationLockedByMe(seqrel, AccessShareLock, true));
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 9c92fddd624..fad1bb548b2 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -60,6 +72,7 @@
#include "postmaster/interrupt.h"
#include "replication/logicalworker.h"
#include "replication/worker_internal.h"
+#include "storage/latch.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
@@ -78,23 +91,35 @@ typedef enum CopySeqResult
COPYSEQ_SUCCESS,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
+ *
+ * Check if the subscription includes sequences and start a sequencesync
+ * worker if one is not already running. The active sequencesync worker will
+ * handle all pending sequence synchronization. If any sequences remain
+ * unsynchronized after it exits, a new worker can be started in the next
+ * iteration.
*
- * Start a sequencesync worker if one is not already running. The active
- * sequencesync worker will handle all pending sequence synchronization. If any
- * sequences remain unsynchronized after it exits, a new worker can be started
- * in the next iteration.
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -112,6 +137,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Quick exit if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -144,7 +182,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +209,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +221,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +232,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +243,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +267,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -325,32 +364,77 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Preliminary check to determine if copying the sequence is allowed.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
{
- UserContext ucxt;
AclResult aclresult;
+ Oid seqoid = seqinfo->localrelid;
+ int64 local_last_value;
+ bool local_is_called;
+
+ /* Perform drift check if it's not the initial sync */
+ if (seqinfo->relstate == SUBREL_STATE_READY)
+ {
+ /*
+ * Verify that the current user has SELECT privilege on the sequence.
+ * This is required to read the sequence state below.
+ */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_SELECT);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ /* Get current local sequence state */
+ GetSequence(sequence_rel, &local_last_value, &local_is_called);
+
+ /*
+ * Skip synchronization if the sequence is already in READY state and
+ * has not drifted from the publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Verify that the current user can update the sequence */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ return COPYSEQ_SUCCESS;
+}
+
+/*
+ * Apply remote sequence state to local sequence. If we are doing this
+ * for sequences in the INIT state, move them to the READY state once
+ * synchronized.
+ */
+static CopySeqResult
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+{
+ UserContext ucxt;
bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
+ CopySeqResult result;
/*
* If the user did not opt to run as the owner of the subscription
* ('run_as_owner'), then copy the sequence as the owner of the sequence.
*/
if (!run_as_owner)
- SwitchToUntrustedUser(seqowner, &ucxt);
+ SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+ result = validate_seqsync_state(seqinfo, sequence_rel);
- if (aclresult != ACLCHECK_OK)
+ if (result != COPYSEQ_SUCCESS)
{
if (!run_as_owner)
RestoreUserContext(&ucxt);
- return COPYSEQ_INSUFFICIENT_PERM;
+ return result;
}
/*
@@ -368,19 +452,26 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (seqinfo->relstate == SUBREL_STATE_INIT)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If relstate is SUBREL_STATE_READY, only synchronize sequences that
+ * have drifted from their publisher values. Otherwise, synchronize
+ * all sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +481,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +494,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,46 +590,53 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
+ &seqinfo, &seqidx, seqinfos);
+
+ /*
+ * For sequences in INIT state, always sync. Otherwise, for
+ * sequences in READY state, only sync if there's drift.
+ */
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ sync_status = copy_sequence(seqinfo, sequence_rel);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
+ sequence_copied = true;
break;
+
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_mismatched_count++;
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_insuffperm_count++;
break;
+
case COPYSEQ_SKIPPED:
/*
@@ -558,6 +654,15 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -566,20 +671,19 @@ copy_sequences(WalReceiverConn *conn)
ExecDropSingleTupleTableSlot(slot);
walrcv_clear_result(res);
- resetStringInfo(seqstr);
- resetStringInfo(cmd);
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
MySubscription->name,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,51 +711,55 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
subrel = (Form_pg_subscription_rel) GETSTRUCT(tup);
+ relstate = subrel->srsubstate;
sequence_rel = try_table_open(subrel->srrelid, RowExclusiveLock);
@@ -666,18 +774,19 @@ LogicalRepSyncSequences(void)
continue;
}
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
-
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -693,36 +802,16 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
-
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ return false;
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +824,91 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence
+ * synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+ maybe_reread_subscription();
+
+ /*
+ * Perform sequence synchronization under SequenceSyncContext and
+ * reset it each cycle to avoid manual memory management.
+ */
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ MemoryContextReset(SequenceSyncContext);
+ MemoryContextSwitchTo(oldctx);
+
+ /*
+ * Adjust sleep interval based on whether sequences were copied
+ * over
+ */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+ /*
+ * Double the sleep time, but not beyond the maximum allowable
+ * value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..233ac7ae873 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,9 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+
+ /* Check if sequence worker needs to be started */
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +193,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +207,22 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
- * since the same value can be used until the system table is invalidated.
+ * has_subtables and has_subsequences are declared as static, since the
+ * same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +232,27 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
+
/*
* Does the subscription have tables?
*
@@ -277,5 +278,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index f9c4b484754..f91c8f9ecde 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -5099,6 +5099,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5114,6 +5117,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5132,6 +5139,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..f00eea9fbd1 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern void GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 4ecbdcfadac..a41cb045f19 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..1d81518fe22 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,8 +75,7 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
@@ -84,9 +83,6 @@ $node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.51.1.windows.1
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-04 04:20 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-05 02:45 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 12:57 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 2 replies; 58+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-03-04 04:20 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; +Cc: shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
Dear Hou,
Thanks for updating the patch. Here are my comments.
01.
```
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
```
I think it's not accurate, because REFRESH SEQUENCE command does not need the
sequencesync worker anymore.
02.
```
void
GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
```
I think GetSequence() itself should conitan the permission check like SetSequence().
My idea is to set NULL for last_value and is_called in this case.
03.
```
/*
* Verify that the current user has SELECT privilege on the sequence.
* This is required to read the sequence state below.
*/
aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_SELECT);
if (aclresult != ACLCHECK_OK)
return COPYSEQ_INSUFFICIENT_PERM;
/* Get current local sequence state */
GetSequence(sequence_rel, &local_last_value, &local_is_called);
```
If you accept above comment, this part can be simplified.
04.
```
/*
* get_and_validate_seq_info
*
* Extracts remote sequence information from the tuple slot received from the
* publisher, and validates it against the corresponding local sequence
* definition.
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
LogicalRepSequenceInfo **seqinfo, int *seqidx,
List *seqinfos)
```
It can return COPYSEQ_SUCCESS, but it might be misleading; copying is not
happened yet here. How about returning boolean and add another argument to
indicate the reason if the validation is failed?
05.
LogicalRepSyncSequences() starts the transaction and read sequences every time.
Can we cache the seqinfos to reuse in the next iteration? My idea is to introduce
a syscache callback for the pg_subscription_relto invalidate the cached list.
How about measuring performance once and considering it's a good improvement?
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 04:20 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-03-05 02:45 ` Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 0 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-05 02:45 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; +Cc: shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wednesday, March 4, 2026 12:21 PM Kuroda, Hayato/黒田 隼人 <[email protected]> wrote:
>
> Thanks for updating the patch. Here are my comments.
Thanks for the comments.
>
> 01.
> ```
> <para>
> A <firstterm>sequence synchronization worker</firstterm> will be started
> - after executing any of the above subscriber commands, and will exit once
> the
> - sequences are synchronized.
> + after executing any of the above subscriber commands. The worker will
> + remain running for the life of the subscription, periodically
> + synchronizing all published sequences.
> </para>
> ```
>
> I think it's not accurate, because REFRESH SEQUENCE command does not
> need the sequencesync worker anymore.
Since the command is changed in 0002, I updated the doc there.
>
> 02.
> ```
> void
> GetSequence(Relation seqrel, int64 *last_value, bool *is_called) ```
>
> I think GetSequence() itself should conitan the permission check like
> SetSequence().
> My idea is to set NULL for last_value and is_called in this case.
Changed.
>
> 03.
> ```
> /*
> * Verify that the current user has SELECT privilege on the
> sequence.
> * This is required to read the sequence state below.
> */
> aclresult = pg_class_aclcheck(seqoid, GetUserId(),
> ACL_SELECT);
>
> if (aclresult != ACLCHECK_OK)
> return COPYSEQ_INSUFFICIENT_PERM;
>
> /* Get current local sequence state */
> GetSequence(sequence_rel, &local_last_value,
> &local_is_called); ```
>
> If you accept above comment, this part can be simplified.
Changed.
>
> 04.
> ```
> /*
> * get_and_validate_seq_info
> *
> * Extracts remote sequence information from the tuple slot received from the
> * publisher, and validates it against the corresponding local sequence
> * definition.
> */
> static CopySeqResult
> get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
> LogicalRepSequenceInfo
> **seqinfo, int *seqidx,
> List *seqinfos)
> ```
>
> It can return COPYSEQ_SUCCESS, but it might be misleading; copying is not
> happened yet here. How about returning boolean and add another argument
> to indicate the reason if the validation is failed?
I have added one more enum value COPYSEQ_ALLOWED for cases where
the validation passes to avoid changing the function signature.
>
> 05.
>
> LogicalRepSyncSequences() starts the transaction and read sequences every
> time.
> Can we cache the seqinfos to reuse in the next iteration? My idea is to
> introduce a syscache callback for the pg_subscription_relto invalidate the
> cached list.
>
> How about measuring performance once and considering it's a good
> improvement?
I think the time of scanning sequence might be negligible when most of sequence
needs to be updated, since the most cost would be on sequence update. There
might be some value to save the CPU cycle when the most of sequence
has not drifted in which case the cost of scanning might be noticeable. But we
will do some more tests and share later.
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 04:20 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-03-05 12:57 ` Hayato Kuroda (Fujitsu) <[email protected]>
1 sibling, 0 replies; 58+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-03-05 12:57 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; Zhijie Hou (Fujitsu) <[email protected]>; Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; +Cc: shveta malik <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
Dear Hackers,
> 05.
>
> LogicalRepSyncSequences() starts the transaction and read sequences every
> time.
> Can we cache the seqinfos to reuse in the next iteration? My idea is to introduce
> a syscache callback for the pg_subscription_relto invalidate the cached list.
>
> How about measuring performance once and considering it's a good
> improvement?
I profiled the sequencesync worker when sequences were less actively updated on
the publisher side. In the actively updated system, copying sequences used most
of the CPU time; thus, we could not observe the effect.
Abstract
--------------
Sequencesync worker spent 20-25% of the working time scanning pg_subscription_rel
in the workload. It's not so large compared with the total CPU time; the worker
can work once per 2 seconds or longer. We may able to consider the optimization
if there are easy ways.
Source
-----------
ea47447 + v8 patch set + attached fix patch.
To simplify the analysis, I extracted the scan part into the function
scan_subscription_relations. No configure options are set at build.
Workload
---------------
Two workloads were tested.
A - profile with no sequence updates
0. Defined 100 sequences on both nodes
1. Built a pub-sub replication system.
2. Attached the sequencesync worker as early as creating the subscription.
3. Waited 10 minutes.
B - profile with 10% sequences updates
0. Defined 100 sequences on both nodes
1. Built a pub-sub replication system.
2. Waited till the initial sync was done. On my env 100s was enough
3. Attached the sequencesync worker
4. Updated 10 sequences per second.
5. Repeat step 4 for 10 minutes.
Result
----------
The attached profiles show the detailed results: noupdate.out corresponds to
workload A, while 10percent_update.out is for workload B.
In both cases, scan_subscription_relations spends more than 20% of their working
time. Notable points are to open sequence relations with the AccessShareLock,
committing the transaction, starting the catalog scan, etc.
workload A:
```
| --20.83%--scan_subscription_relations
| |
| |--10.83%--try_table_open
| | try_relation_open
```
workload B:
```
| --24.01%--scan_subscription_relations
| |
| |--12.52%--try_table_open
| | try_relation_open
```
Consideration
--------------
Based on that, we may be able to cache seqinfos to avoid starting the
transaction and opening the sequence. But we need to introduce a relcache
callback to invalidate the specific entry of the list, not sure it's beneficial
more than the complexity.
Configuration
----------------------
Each node had shared_buffer=1GB, and others had the default GUC values.
Environment
--------------------
CPU: Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz, 4 cores, 1 thread per core
Memory: 15GiB
OS: AlmaLinux 9.7
Best regards,
Hayato Kuroda
FUJITSU LIMITED
Attachments:
[application/octet-stream] 10percent_update.out (817.2K, 2-10percent_update.out)
download
[application/octet-stream] noupdate.out (717.0K, 3-noupdate.out)
download
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-04 10:24 ` shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-03-04 10:24 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Mon, Mar 2, 2026 at 1:28 PM Zhijie Hou (Fujitsu)
<[email protected]> wrote:
>
> Rebased the patch to silence compile warning due to a recent commit
> a2c89835.
>
Thanks for the patch. Please find a few comments for 001:
1)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (seqinfo->relstate == SUBREL_STATE_INIT)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
What if page-lsn has changed and we are in READY state, don't we need
to update that in pg_subscription_rel? Or is that done somewhere else?
2)
+ *
+ * If relstate is SUBREL_STATE_READY, only synchronize sequences that
+ * have drifted from their publisher values. Otherwise, synchronize
+ * all sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
There is no relstate, comments need correction.
3)
Currently we use same state 'COPYSEQ_SUCCESS' for 2 cases: allowed to
copy (as returned by validate_seqsync_state and
get_and_validate_seq_info) and copy-done (by copy_sequences,
copy_sequence). It is slightly confusing. Shall we add one more state
for 'allowed' case, could be COPYSEQ_ELIGIBLE or COPYSEQ_PROCEED or
COPYSEQ_ALLOWED? COPYSEQ_SUCCESS was used for such a case in previous
seq-sync commands too (on HEAD), but now its usage is more in
'allowed' case as compared to HEAD, so perhaps we can change in this
patch. But I would like to know what others think here.
4)
+ * Preliminary check to determine if copying the sequence is allowed.
How about this comment:
Check whether the user has required privileges on the sequence and
whether the sequence has drifted.
5)
validate_seqsync_state():
+ /*
+ * Skip synchronization if the sequence is already in READY state and
+ * has not drifted from the publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
Since we already have a comment where we check READY state in outer
if-block and since we are not checking READY state here, perhaps we
can change the above comment to simply:
"Skip synchronization if it has not drifted from the publisher's value."
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-03-05 02:46 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-05 02:46 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wednesday, March 4, 2026 6:25 PM shveta malik <[email protected]> wrote:
>
> On Mon, Mar 2, 2026 at 1:28 PM Zhijie Hou (Fujitsu) <[email protected]>
> wrote:
> >
> > Rebased the patch to silence compile warning due to a recent commit
> > a2c89835.
> >
>
> Thanks for the patch. Please find a few comments for 001:
Thanks for the comments.
>
> 1)
> /*
> * Record the remote sequence's LSN in pg_subscription_rel and mark the
> - * sequence as READY.
> + * sequence as READY if updating a sequence that is in INIT state.
> */
> - UpdateSubscriptionRelState(MySubscription->oid, seqoid,
> SUBREL_STATE_READY,
> - seqinfo->page_lsn, false);
> + if (seqinfo->relstate == SUBREL_STATE_INIT)
> + UpdateSubscriptionRelState(MySubscription->oid, seqoid,
> SUBREL_STATE_READY,
> + seqinfo->page_lsn, false);
>
> What if page-lsn has changed and we are in READY state, don't we need to
> update that in pg_subscription_rel? Or is that done somewhere else?
The check was done in 0002, I moved it to 0001 now.
>
> 2)
> + *
> + * If relstate is SUBREL_STATE_READY, only synchronize sequences that
> + * have drifted from their publisher values. Otherwise, synchronize
> + * all sequences.
> + *
> + * Returns true/false if any sequences were actually copied.
> */
> +static bool
> +copy_sequences(WalReceiverConn *conn, List *seqinfos)
>
> There is no relstate, comments need correction.
Changed.
>
> 3)
> Currently we use same state 'COPYSEQ_SUCCESS' for 2 cases: allowed to
> copy (as returned by validate_seqsync_state and
> get_and_validate_seq_info) and copy-done (by copy_sequences,
> copy_sequence). It is slightly confusing. Shall we add one more state for
> 'allowed' case, could be COPYSEQ_ELIGIBLE or COPYSEQ_PROCEED or
> COPYSEQ_ALLOWED? COPYSEQ_SUCCESS was used for such a case in
> previous seq-sync commands too (on HEAD), but now its usage is more in
> 'allowed' case as compared to HEAD, so perhaps we can change in this patch.
> But I would like to know what others think here.
I agree that adding a new value can make the code easier to read.
>
>
> 4)
> + * Preliminary check to determine if copying the sequence is allowed.
>
> How about this comment:
> Check whether the user has required privileges on the sequence and whether
> the sequence has drifted.
Changed as suggested.
>
> 5)
> validate_seqsync_state():
> + /*
> + * Skip synchronization if the sequence is already in READY state and
> + * has not drifted from the publisher's value.
> + */
> + if (local_last_value == seqinfo->last_value && local_is_called ==
> + seqinfo->is_called) return COPYSEQ_NO_DRIFT;
>
> Since we already have a comment where we check READY state in outer if-
> block and since we are not checking READY state here, perhaps we can
> change the above comment to simply:
> "Skip synchronization if it has not drifted from the publisher's value."
Changed.
Here is V10 patch set which addressed all comments.
I also fixed a bug that Kuroda-San reported off-list, where we were not
resetting the StringInfo used to build the query in the copy_sequences loop,
which caused the built query to be incorrect.
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v10-0002-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch (17.2K, 2-v10-0002-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch)
download | inline diff:
From 2c714f0ec91cc491af9851f77f63d1f3d8100a5d Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Thu, 5 Mar 2026 10:44:17 +0800
Subject: [PATCH v10 2/2] Synchronize sequences directly in REFRESH SEQUENCES
command
The ALTER SUBSCRIPTION ... REFRESH SEQUENCES command currently sets all
sequence states in pg_subscription_rel to INIT and relies on the sequence sync
worker to perform the actual synchronization and update states to READY.
With the recent change making the sequence sync worker long-lived, most
sequences are now synchronized in the background, reducing the need for
REFRESH SEQUENCES. However, the command remains necessary for sequences that
haven't been synchronized.
This commit enhances REFRESH SEQUENCES to synchronize sequences directly within
the command itself, eliminating the overhead of launching a worker and updating
catalog entries unnecessarily.
---
doc/src/sgml/logical-replication.sgml | 6 +-
src/backend/commands/subscriptioncmds.c | 17 +-
.../replication/logical/sequencesync.c | 158 ++++++++++++++----
src/include/replication/logicalworker.h | 5 +
src/test/subscription/t/036_sequences.pl | 49 ++++++
5 files changed, 188 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index bb523af5d37..323d5b7c7a0 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,9 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands. The worker will
- remain running for the life of the subscription, periodically
- synchronizing all published sequences.
+ after executing <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command>
+ command. The worker will remain running for the life of the subscription,
+ periodically synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 5e3c0964d38..0a5acfda0ff 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1245,25 +1245,10 @@ AlterSubscription_refresh_seq(Subscription *sub)
PG_TRY();
{
- List *subrel_states;
-
check_publications_origin_sequences(wrconn, sub->publications, true,
sub->origin, NULL, 0, sub->name);
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
+ AlterSubSyncSequences(wrconn, sub->oid, sub->name, sub->runasowner);
}
PG_FINALLY();
{
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index fb293487657..8b910aa1bc0 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -210,7 +210,7 @@ get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx, List *seqinfos)
+ List *missing_seqs_idx, List *seqinfos, char *subname)
{
StringInfo seqstr;
@@ -256,7 +256,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
ereport(ERROR,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("logical replication sequence synchronization failed for subscription \"%s\"",
- MySubscription->name));
+ subname));
}
/*
@@ -284,6 +284,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
+ LOCKMODE lockmode;
*seqidx = DatumGetInt32(slot_getattr(slot, ++col, &isnull));
Assert(!isnull);
@@ -330,7 +331,20 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
seqinfo_local->found_on_pub = true;
- *sequence_rel = try_table_open(seqinfo_local->localrelid, RowExclusiveLock);
+ /*
+ * We take a stronger lock during DDL commands (currently only ALTER
+ * SUBSCRIPTION ... REFRESH SEQUENCES) to prevent concurrent sequencesync
+ * workers from updating the page_lsn while the DDL is also updating the
+ * same sequence. This ensures we can always fetch the latest page_lsn to
+ * determine whether the remote sequence value should be synchronized (see
+ * validate_seqsync_state).
+ */
+ if (IsLogicalWorker())
+ lockmode = RowExclusiveLock;
+ else
+ lockmode = ShareRowExclusiveLock;
+
+ *sequence_rel = try_table_open(seqinfo_local->localrelid, lockmode);
/* Sequence was concurrently dropped? */
if (!*sequence_rel)
@@ -369,7 +383,8 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
* whether the sequence has drifted.
*/
static CopySeqResult
-validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ XLogRecPtr local_page_lsn)
{
AclResult aclresult;
Oid seqoid = seqinfo->localrelid;
@@ -380,6 +395,16 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
int64 local_last_value;
bool local_is_called;
+ /*
+ * Skip synchronization if we are processing outdated sequence info
+ * based on the LSN. This occurs when the sequence has been updated to
+ * more recent data concurrently (via either ALTER SUBSCRIPTION ...
+ * REFRESH SEQUENCES or the sequencesync worker).
+ */
+ if (XLogRecPtrIsValid(local_page_lsn) &&
+ local_page_lsn > seqinfo->page_lsn)
+ return COPYSEQ_NO_DRIFT;
+
/* Get current local sequence state */
GetSequence(sequence_rel, &local_last_value, &local_is_called);
@@ -390,6 +415,32 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
if (local_last_value == 0)
return COPYSEQ_INSUFFICIENT_PERM;
+ /*
+ * Skip synchronization if the local sequence value is already ahead of
+ * the publisher's value.
+ *
+ * XXX This occurs not only when the local sequence has been
+ * synchronized to a newer value from the publisher (where skipping is
+ * necessary to avoid backward movement), but also when the local
+ * sequence has been manually updated by the user on the subscriber. The
+ * latter could be considered a replication conflict, and overwriting
+ * the user's update might be acceptable. However, since we cannot
+ * easily distinguish between these two scenarios, we choose to skip
+ * synchronization in all cases and emit a WARNING to notify the user to
+ * manually resolve the conflict.
+ */
+ if (local_last_value > seqinfo->last_value)
+ {
+ ereport(WARNING,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("skipped synchronizing the sequence \"%s.%s\"",
+ seqinfo->nspname, seqinfo->seqname),
+ errdetail("The local last_value %lld is ahead of the one on publisher",
+ (long long int) local_last_value));
+
+ return COPYSEQ_NO_DRIFT;
+ }
+
/*
* Skip synchronization if the sequence has not drifted from the
* publisher's value.
@@ -414,16 +465,15 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* synchronized.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ Oid subid, bool run_as_owner)
{
UserContext ucxt;
- bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
CopySeqResult result;
XLogRecPtr local_page_lsn;
- (void) GetSubscriptionRelState(MySubscription->oid,
- RelationGetRelid(sequence_rel),
+ (void) GetSubscriptionRelState(subid, RelationGetRelid(sequence_rel),
&local_page_lsn);
/*
@@ -433,7 +483,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
if (!run_as_owner)
SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- result = validate_seqsync_state(seqinfo, sequence_rel);
+ result = validate_seqsync_state(seqinfo, sequence_rel, local_page_lsn);
if (result != COPYSEQ_ALLOWED)
{
@@ -478,7 +528,8 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* Returns true/false if any sequences were actually copied.
*/
static bool
-copy_sequences(WalReceiverConn *conn, List *seqinfos)
+copy_sequences(WalReceiverConn *conn, List *seqinfos, Oid subid, char *subname,
+ bool runasowner)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -504,11 +555,16 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
+ bool started_tx = false;
WalRcvExecResult *res;
TupleTableSlot *slot;
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
for (int idx = cur_batch_base_index; idx < n_seqinfos; idx++)
{
@@ -604,14 +660,15 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
* sequences in READY state, only sync if there's drift.
*/
if (sync_status == COPYSEQ_ALLOWED)
- sync_status = copy_sequence(seqinfo, sequence_rel);
+ sync_status = copy_sequence(seqinfo, sequence_rel,
+ subid, runasowner);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
"logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
- seqinfo->nspname, seqinfo->seqname, MySubscription->name);
+ seqinfo->nspname, seqinfo->seqname, subname);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -619,9 +676,8 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in SequenceSyncContext
- * since these will be used after the transaction is
- * committed.
+ * Remember mismatched sequences in SequenceSyncContext since
+ * these will be used after the transaction is committed.
*/
oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
@@ -689,13 +745,17 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
elog(DEBUG1,
"logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
- MySubscription->name,
+ subname,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
- /* Commit this batch, and prepare for next batch */
- CommitTransactionCommand();
+ /*
+ * Commit this batch if started a transaction, and prepare for next
+ * batch.
+ */
+ if (started_tx)
+ CommitTransactionCommand();
if (batch_missing_count)
{
@@ -720,7 +780,7 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx, seqinfos);
+ missing_seqs_idx, seqinfos, subname);
return sequence_copied;
}
@@ -732,20 +792,23 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
* Returns true if sequences have been updated.
*/
static bool
-LogicalRepSyncSequences(WalReceiverConn *conn)
+LogicalRepSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
- Oid subid = MyLogicalRepWorker->subid;
bool sequence_copied = false;
List *seqinfos = NIL;
MemoryContext oldctx;
+ bool started_tx = false;
- Assert(SequenceSyncContext);
-
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
@@ -805,7 +868,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- CommitTransactionCommand();
+ if (started_tx)
+ CommitTransactionCommand();
/*
* Exit early if no catalog entries found, likely due to concurrent drops.
@@ -814,7 +878,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
return false;
/* Process sequences */
- sequence_copied = copy_sequences(conn, seqinfos);
+ sequence_copied = copy_sequences(conn, seqinfos, subid, subname,
+ runasowner);
return sequence_copied;
}
@@ -889,7 +954,10 @@ start_sequence_sync(void)
/*
* Synchronize all sequences (both READY and INIT states).
*/
- sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
+ MySubscription->oid,
+ MySubscription->name,
+ MySubscription->runasowner);
MemoryContextReset(SequenceSyncContext);
MemoryContextSwitchTo(oldctx);
@@ -951,3 +1019,37 @@ SequenceSyncWorkerMain(Datum main_arg)
FinishSyncWorker();
}
+
+/*
+ * Wrapper for LogicalRepSyncSequences to synchronize all sequences of a
+ * subscription from outside the sequencesync worker
+ */
+void
+AlterSubSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
+{
+ /*
+ * Init the SequenceSyncContext which we clean up after the sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(CurrentMemoryContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ MemoryContext oldctx;
+
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ LogicalRepSyncSequences(conn, subid, subname, runasowner);
+
+ MemoryContextSwitchTo(oldctx);
+ }
+ PG_FINALLY();
+ {
+ MemoryContextDelete(SequenceSyncContext);
+ SequenceSyncContext = NULL;
+ }
+ PG_END_TRY();
+}
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..73afd7853d0 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,6 +14,8 @@
#include <signal.h>
+#include "replication/walreceiver.h"
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
extern void ApplyWorkerMain(Datum main_arg);
@@ -31,4 +33,7 @@ extern void LogicalRepWorkersWakeupAtCommit(Oid subid);
extern void AtEOXact_LogicalRepWorkers(bool isCommit);
+extern void AlterSubSyncSequences(WalReceiverConn *conn, Oid subid,
+ char *subname, bool runasowner);
+
#endif /* LOGICALWORKER_H */
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 1d81518fe22..9a61b7bd0c8 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -176,4 +176,53 @@ $node_subscriber->wait_for_log(
qr/WARNING: ( [A-Z0-9]+:)? missing sequence on publisher \("public.regress_s4"\)/,
$log_offset);
+##########
+# ALTER SUBSCRIPTION ... REFRESH SEQUENCES synchronizes sequences online,
+# eliminating the need to launch a sequencesync worker.
+##########
+
+# Reduce max_logical_replication_workers to disallow sequence worker from running
+$node_subscriber->append_conf('postgresql.conf',
+ qq(max_logical_replication_workers = 0));
+$node_subscriber->restart;
+
+# Verify there is no logical replication apply worker running
+$result = $node_subscriber->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'logical replication apply worker'");
+
+is($result, '0', 'no logical replication worker is running');
+
+# Increment sequence on publisher
+$node_publisher->safe_psql('postgres',
+ qq(SELECT nextval('regress_s1');));
+
+# The command should fail due to missing sequence ('regress_s4')
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+like(
+ $stderr,
+ qr/WARNING: missing sequence on publisher \("public.regress_s4"\)/,
+ "output the wanring for the missing sequence regress_s4");
+
+like(
+ $stderr,
+ qr/ERROR: logical replication sequence synchronization failed for subscription \"regress_seq_sub\"/,
+ "the command failed due to the missing sequence regress_s4");
+
+# Refresh the publication to remove the missing sequence
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION;");
+
+# Sync the sequence regress_s1
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+# Get the current sequence value on subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ qq(SELECT last_value FROM regress_s1;));
+
+is($result, '201', 'sequence regress_s1 is synced now');
+
done_testing();
--
2.51.1.windows.1
[application/octet-stream] v10-0001-Support-automatic-sequence-replication.patch (42.2K, 3-v10-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From 53e3fcbae4e53f3b53cf9defea2e920418fa68ec Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Tue, 24 Feb 2026 21:37:01 +1100
Subject: [PATCH v10 1/2] Support automatic sequence replication.
Currently, sequence values are synchronized from publisher to subscriber only
when the user manually runs ALTER SUBSCRIPTION ... REFRESH PUBLICATION (which
affects only newly subscribed sequences) or REFRESH SEQUENCES. The sequence sync
worker exits immediately after completing each synchronization round.
The primary use case for sequence replication is during upgrades, where it's
recommended that users ensure sequences are in sync by running REFRESH SEQUENCES
before finishing the upgrade. However, this command can be slow when
synchronizing a large number of sequences, potentially increasing downtime.
To address this, this commit makes the sequence sync worker long-lived,
continuously monitoring sequences and resynchronizing them when drift is
detected. The worker uses an adaptive sleep interval: it starts at 2 seconds,
doubles up to a maximum of 30 seconds when no drift is observed, and resets to
the minimum interval once drift is found.
With this change, most sequences are silently synchronized in the background,
eliminating the need to run REFRESH SEQUENCES for the majority of cases.
However, frequently updated sequences may still lag behind, requiring a final
REFRESH SEQUENCES before upgrade completion. Users can monitor progress by
checking whether sequence states transition from INIT to READY in
pg_subscription_rel.
The REFRESH SEQUENCES command is retained for this final synchronization step,
though it currently updates all sequence states to INIT, which has room for
improvement. A future patch will enhance this command to synchronize sequences
directly without launching a worker, reducing catalog overhead.
Author: Ajin Cherian <[email protected]>
Author: Zhijie Hou <[email protected]>
Reviewed-by: Shveta Malik <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Reviewed-by: Ashutosh Sharma <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
---
doc/src/sgml/logical-replication.sgml | 23 +-
doc/src/sgml/ref/alter_subscription.sgml | 5 -
src/backend/commands/sequence.c | 39 ++
.../replication/logical/sequencesync.c | 383 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 47 +--
src/backend/replication/logical/worker.c | 11 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 79 +---
11 files changed, 389 insertions(+), 204 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..bb523af5d37 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,8 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1817,7 +1818,7 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
+ Subscriber sequence values can become out of sync as the publisher
advances them.
</para>
<para>
@@ -2335,15 +2336,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 5318998e80c..5a7b4f3c2c2 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -190,11 +190,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..692b6b6857c 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,45 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift. The caller must hold a lock on the sequence.
+ *
+ * If the caller does not have sufficient privileges to access the sequence,
+ * *last_value will be set to 0.
+ */
+void
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Oid relid = RelationGetRelid(seqrel);
+
+ /* Confirm that the relation is a sequence and is locked */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+ Assert(CheckRelationLockedByMe(seqrel, AccessShareLock, true));
+
+ if (pg_class_aclcheck(relid, GetUserId(), ACL_SELECT) != ACLCHECK_OK)
+ {
+ *last_value = 0;
+ *is_called = false;
+ return;
+ }
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 9c92fddd624..fb293487657 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -60,6 +72,7 @@
#include "postmaster/interrupt.h"
#include "replication/logicalworker.h"
#include "replication/worker_internal.h"
+#include "storage/latch.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
@@ -76,25 +89,38 @@
typedef enum CopySeqResult
{
COPYSEQ_SUCCESS,
+ COPYSEQ_ALLOWED,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
+ *
+ * Check if the subscription includes sequences and start a sequencesync
+ * worker if one is not already running. The active sequencesync worker will
+ * handle all pending sequence synchronization. If any sequences remain
+ * unsynchronized after it exits, a new worker can be started in the next
+ * iteration.
*
- * Start a sequencesync worker if one is not already running. The active
- * sequencesync worker will handle all pending sequence synchronization. If any
- * sequences remain unsynchronized after it exits, a new worker can be started
- * in the next iteration.
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -112,6 +138,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Quick exit if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -144,7 +183,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +210,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +222,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +233,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +244,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +268,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -240,7 +280,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
int64 remote_min;
int64 remote_max;
bool remote_cycle;
- CopySeqResult result = COPYSEQ_SUCCESS;
+ CopySeqResult result = COPYSEQ_ALLOWED;
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
@@ -325,32 +365,82 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Check whether the user has required privileges on the sequence and
+ * whether the sequence has drifted.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
{
- UserContext ucxt;
AclResult aclresult;
+ Oid seqoid = seqinfo->localrelid;
+
+ /* Perform drift check if it's not the initial sync */
+ if (seqinfo->relstate == SUBREL_STATE_READY)
+ {
+ int64 local_last_value;
+ bool local_is_called;
+
+ /* Get current local sequence state */
+ GetSequence(sequence_rel, &local_last_value, &local_is_called);
+
+ /*
+ * Skip synchronization if the current user does not have sufficient
+ * privileges to read the sequence data.
+ */
+ if (local_last_value == 0)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ /*
+ * Skip synchronization if the sequence has not drifted from the
+ * publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Verify that the current user can update the sequence */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ return COPYSEQ_ALLOWED;
+}
+
+/*
+ * Apply remote sequence state to local sequence. If we are doing this
+ * for sequences in the INIT state, move them to the READY state once
+ * synchronized.
+ */
+static CopySeqResult
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+{
+ UserContext ucxt;
bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
+ CopySeqResult result;
+ XLogRecPtr local_page_lsn;
+
+ (void) GetSubscriptionRelState(MySubscription->oid,
+ RelationGetRelid(sequence_rel),
+ &local_page_lsn);
/*
* If the user did not opt to run as the owner of the subscription
* ('run_as_owner'), then copy the sequence as the owner of the sequence.
*/
if (!run_as_owner)
- SwitchToUntrustedUser(seqowner, &ucxt);
+ SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+ result = validate_seqsync_state(seqinfo, sequence_rel);
- if (aclresult != ACLCHECK_OK)
+ if (result != COPYSEQ_ALLOWED)
{
if (!run_as_owner)
RestoreUserContext(&ucxt);
- return COPYSEQ_INSUFFICIENT_PERM;
+ return result;
}
/*
@@ -368,19 +458,27 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (seqinfo->relstate == SUBREL_STATE_INIT ||
+ seqinfo->page_lsn != local_page_lsn)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If the state of sequence is SUBREL_STATE_READY, only synchronize sequences
+ * that have drifted from their publisher values. Otherwise, synchronize all
+ * sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +488,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +501,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,46 +597,53 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
- if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ &seqinfo, &seqidx, seqinfos);
+
+ /*
+ * For sequences in INIT state, always sync. Otherwise, for
+ * sequences in READY state, only sync if there's drift.
+ */
+ if (sync_status == COPYSEQ_ALLOWED)
+ sync_status = copy_sequence(seqinfo, sequence_rel);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
+ sequence_copied = true;
break;
+
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_mismatched_count++;
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
batch_insuffperm_count++;
break;
+
case COPYSEQ_SKIPPED:
/*
@@ -558,6 +661,15 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -572,14 +684,15 @@ copy_sequences(WalReceiverConn *conn)
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
MySubscription->name,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,51 +720,55 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
subrel = (Form_pg_subscription_rel) GETSTRUCT(tup);
+ relstate = subrel->srsubstate;
sequence_rel = try_table_open(subrel->srrelid, RowExclusiveLock);
@@ -666,18 +783,19 @@ LogicalRepSyncSequences(void)
continue;
}
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
-
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -693,36 +811,16 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
-
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ return false;
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +833,91 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence
+ * synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+ maybe_reread_subscription();
+
+ /*
+ * Perform sequence synchronization under SequenceSyncContext and
+ * reset it each cycle to avoid manual memory management.
+ */
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ MemoryContextReset(SequenceSyncContext);
+ MemoryContextSwitchTo(oldctx);
+
+ /*
+ * Adjust sleep interval based on whether sequences were copied
+ * over
+ */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+ /*
+ * Double the sleep time, but not beyond the maximum allowable
+ * value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..233ac7ae873 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,9 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+
+ /* Check if sequence worker needs to be started */
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +193,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +207,22 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
- * since the same value can be used until the system table is invalidated.
+ * has_subtables and has_subsequences are declared as static, since the
+ * same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +232,27 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
+
/*
* Does the subscription have tables?
*
@@ -277,5 +278,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index f9c4b484754..f91c8f9ecde 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -5099,6 +5099,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5114,6 +5117,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5132,6 +5139,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..f00eea9fbd1 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern void GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 4ecbdcfadac..a41cb045f19 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..1d81518fe22 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,8 +75,7 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
@@ -84,9 +83,6 @@ $node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.51.1.windows.1
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-05 04:05 ` shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 10:46 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
0 siblings, 2 replies; 58+ messages in thread
From: shveta malik @ 2026-03-05 04:05 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Thu, Mar 5, 2026 at 8:16 AM Zhijie Hou (Fujitsu)
<[email protected]> wrote:
>
>
> Here is V10 patch set which addressed all comments.
>
Thank You. Please find a few comments on 001:
1)
+ /*
+ * Skip synchronization if the current user does not have sufficient
+ * privileges to read the sequence data.
+ */
+ if (local_last_value == 0)
+ return COPYSEQ_INSUFFICIENT_PERM;
I don't think it is the right way to handle this. The local_last_value
can be genuinely 0 for some cases and we may end up giving the wrong
ERROR.
Try this:
CREATE SEQUENCE my_seq START WITH 0 INCREMENT BY 1 MINVALUE 0;
And then set-up pub-sub.
We get:
2026-03-05 08:57:39.591 IST [92281] WARNING: insufficient privileges
on sequence ("public.my_seq")
2026-03-05 08:57:39.591 IST [92281] ERROR: logical replication
sequence synchronization failed for subscription "subi1"
Either we shall move back the acl check to the caller of GetSequence
or pass the info of acl-check failure in a new argument or return
value.
2)
+ /*
+ * For sequences in INIT state, always sync. Otherwise, for
+ * sequences in READY state, only sync if there's drift.
+ */
if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ sync_status = copy_sequence(seqinfo, sequence_rel);
We shall add such a comment atop copy_sequence as well. I am unsure if
we need it here or not.
3)
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
I don't think this wait-event is appropriate. Unlike tablesync, we are
not waiting for any state change here. Shall we add a new one for our
case? How about WAIT_EVENT_LOGICAL_SEQSYNC_MAIN? Thoughts?
4)
+ relstate = subrel->srsubstate;
it will be good to move it just after below part:
/* Skip if the relation is not a sequence */
5)
}
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
One blank line before new change will improve readability.
6)
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
Last comment needs to be changed. Remove this please: 'and update
existing sequence 'regress_s1''
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-03-05 05:48 ` shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-03-05 05:48 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Thu, Mar 5, 2026 at 9:35 AM shveta malik <[email protected]> wrote:
>
> On Thu, Mar 5, 2026 at 8:16 AM Zhijie Hou (Fujitsu)
> <[email protected]> wrote:
> >
> >
> > Here is V10 patch set which addressed all comments.
> >
>
> Thank You. Please find a few comments on 001:
>
A concern in 002:
I realized that below might not be the correct logic to avoid
overwriting sequences at sub which are already at latest values.
+ /*
+ * Skip synchronization if the local sequence value is already ahead of
+ * the publisher's value.
...
+ */
+ if (local_last_value > seqinfo->last_value)
+ {
+ ereport(WARNING,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("skipped synchronizing the sequence \"%s.%s\"",
+ seqinfo->nspname, seqinfo->seqname),
+ errdetail("The local last_value %lld is ahead of the one on publisher",
+ (long long int) local_last_value));
+
+ return COPYSEQ_NO_DRIFT;
+ }
A sequence could be descending one too and thus we may wrongly end up
avoiding synchronization. We should first check if it is descending or
ascending (perhaps by checking if increment_by < 0 or >0), then decide
to manage conflict.
Example:
postgres=# CREATE SEQUENCE desc_seq START WITH 1000 INCREMENT BY -1
MINVALUE 1 MAXVALUE 1000;
CREATE SEQUENCE
postgres=# select nextval('desc_seq');
nextval
---------
1000
postgres=# select nextval('desc_seq');
nextval
---------
999
Doc also mentions descending sequences. See [1] (search for descending).
[1]: https://www.postgresql.org/docs/18/sql-createsequence.html
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-03-05 11:34 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-06 08:52 ` Re: [PATCH] Support automatic sequence replication Chao Li <[email protected]>
0 siblings, 2 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-05 11:34 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thursday, March 5, 2026 1:48 PM shveta malik <[email protected]> wrote:
>
> On Thu, Mar 5, 2026 at 9:35 AM shveta malik <[email protected]>
> wrote:
> >
> > On Thu, Mar 5, 2026 at 8:16 AM Zhijie Hou (Fujitsu)
> > <[email protected]> wrote:
> > >
> > >
> > > Here is V10 patch set which addressed all comments.
> > >
> >
> > Thank You. Please find a few comments on 001:
> >
>
> A concern in 002:
>
> I realized that below might not be the correct logic to avoid overwriting
> sequences at sub which are already at latest values.
>
> + /*
> + * Skip synchronization if the local sequence value is already ahead of
> + * the publisher's value.
> ...
> + */
> + if (local_last_value > seqinfo->last_value) { ereport(WARNING,
> + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> + errmsg("skipped synchronizing the sequence \"%s.%s\"",
> + seqinfo->nspname, seqinfo->seqname), errdetail("The local
> + last_value %lld is ahead of the one on publisher",
> + (long long int) local_last_value));
> +
> + return COPYSEQ_NO_DRIFT;
> + }
>
>
> A sequence could be descending one too and thus we may wrongly end up
> avoiding synchronization. We should first check if it is descending or ascending
> (perhaps by checking if increment_by < 0 or >0), then decide to manage
> conflict.
Thanks for catching this, I changed it to check the increment before reporting warning.
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-06 05:25 ` shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-03-06 05:25 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Thu, Mar 5, 2026 at 5:04 PM Zhijie Hou (Fujitsu)
<[email protected]> wrote:
>
>
> Thanks for catching this, I changed it to check the increment before reporting warning.
>
Thanks! I have changed comments atop sequencesync.c to get rid of old
implementation details (worker dependency upon INIT state) and
rephrased a little bit. Please take it if you find it okay. Attaching
patch as txt file. It is a top-up for 0001.
thanks
Shveta
From 22a575bc2ab43795c72f2240485c189262672035 Mon Sep 17 00:00:00 2001
From: Shveta Malik <[email protected]>
Date: Fri, 6 Mar 2026 10:45:01 +0530
Subject: [PATCH] header comment improvement
---
.../replication/logical/sequencesync.c | 49 +++++++++----------
1 file changed, 22 insertions(+), 27 deletions(-)
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index cde9eaba474..a80d5e5b335 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,37 +19,32 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * The apply worker periodically scans pg_subscription_rel for sequences in
- * INIT state. When such sequences are found, it spawns a sequencesync worker
- * to handle synchronization.
- *
- * A single sequencesync worker is responsible for synchronizing all sequences.
- * It begins by retrieving the list of sequences that are flagged for
- * synchronization, i.e., those in the INIT state. These sequences are then
- * processed in batches, allowing multiple entries to be synchronized within a
- * single transaction. The worker fetches the current sequence values and page
- * LSNs from the remote publisher, updates the corresponding sequences on the
- * local subscriber, and finally marks each sequence as READY upon successful
+ * The apply worker periodically scans pg_subscription_rel for sequences
+ * When sequences are found, it spawns a sequencesync worker to handle
* synchronization.
*
- * The sequencesync worker then fetches all sequences that are
- * in the READY state, queries the publisher for current sequence values, and
- * updates any sequences that have drifted and then goes to sleep. The sleep
- * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
- * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
- * the minimum to ensure timely updates.
- *
- * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
- * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
- * state. All INIT sequences are synchronized unconditionally, then transition
- * to the READY state. Once in the READY state, sequences are checked for drift
- * from the publisher and synchronized only when drift is detected.
+ * A single sequencesync worker is responsible for synchronizing all sequences
+ * for a subscription. It begins by retrieving the list of sequences. These
+ * sequences are then processed in batches, allowing multiple entries to be
+ * synchronized within a single transaction. The worker fetches the current
+ * sequence values and page LSNs from the remote publisher and updates the
+ * corresponding sequences on the local subscriber. Sequences in the INIT
+ * state are unconditionally updated to the latest values from the publisher
+ * and then moved to the READY state. For sequences already in the READY
+ * state, the worker checks for drift and updates them only when needed.
*
* Sequence state transitions follow this pattern:
- * INIT --> READY ->-+
- * ^ | (check/synchronize)
- * | |
- * +--<---+
+
+ * (synchronize)
+ * INIT --------------> READY ->-+
+ * ^ | (check-drift/synchronize)
+ * | |
+ * +--<---+
+ *
+ * Between cycles, the worker sleeps for SEQSYNC_MIN_SLEEP_MS. If no drift is
+ * observed in any sequence, the sleep interval doubles after each wake cycle
+ * up to SEQSYNC_MAX_SLEEP_MS. When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
--
2.34.1
Attachments:
[text/plain] 0001-header-comment-improvement.patch.txt (3.8K, 2-0001-header-comment-improvement.patch.txt)
download | inline diff:
From 22a575bc2ab43795c72f2240485c189262672035 Mon Sep 17 00:00:00 2001
From: Shveta Malik <[email protected]>
Date: Fri, 6 Mar 2026 10:45:01 +0530
Subject: [PATCH] header comment improvement
---
.../replication/logical/sequencesync.c | 49 +++++++++----------
1 file changed, 22 insertions(+), 27 deletions(-)
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index cde9eaba474..a80d5e5b335 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,37 +19,32 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * The apply worker periodically scans pg_subscription_rel for sequences in
- * INIT state. When such sequences are found, it spawns a sequencesync worker
- * to handle synchronization.
- *
- * A single sequencesync worker is responsible for synchronizing all sequences.
- * It begins by retrieving the list of sequences that are flagged for
- * synchronization, i.e., those in the INIT state. These sequences are then
- * processed in batches, allowing multiple entries to be synchronized within a
- * single transaction. The worker fetches the current sequence values and page
- * LSNs from the remote publisher, updates the corresponding sequences on the
- * local subscriber, and finally marks each sequence as READY upon successful
+ * The apply worker periodically scans pg_subscription_rel for sequences
+ * When sequences are found, it spawns a sequencesync worker to handle
* synchronization.
*
- * The sequencesync worker then fetches all sequences that are
- * in the READY state, queries the publisher for current sequence values, and
- * updates any sequences that have drifted and then goes to sleep. The sleep
- * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
- * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
- * the minimum to ensure timely updates.
- *
- * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
- * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
- * state. All INIT sequences are synchronized unconditionally, then transition
- * to the READY state. Once in the READY state, sequences are checked for drift
- * from the publisher and synchronized only when drift is detected.
+ * A single sequencesync worker is responsible for synchronizing all sequences
+ * for a subscription. It begins by retrieving the list of sequences. These
+ * sequences are then processed in batches, allowing multiple entries to be
+ * synchronized within a single transaction. The worker fetches the current
+ * sequence values and page LSNs from the remote publisher and updates the
+ * corresponding sequences on the local subscriber. Sequences in the INIT
+ * state are unconditionally updated to the latest values from the publisher
+ * and then moved to the READY state. For sequences already in the READY
+ * state, the worker checks for drift and updates them only when needed.
*
* Sequence state transitions follow this pattern:
- * INIT --> READY ->-+
- * ^ | (check/synchronize)
- * | |
- * +--<---+
+
+ * (synchronize)
+ * INIT --------------> READY ->-+
+ * ^ | (check-drift/synchronize)
+ * | |
+ * +--<---+
+ *
+ * Between cycles, the worker sleeps for SEQSYNC_MIN_SLEEP_MS. If no drift is
+ * observed in any sequence, the sleep interval doubles after each wake cycle
+ * up to SEQSYNC_MAX_SLEEP_MS. When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
--
2.34.1
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-03-09 03:13 ` shveta malik <[email protected]>
2026-03-13 07:13 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: shveta malik @ 2026-03-09 03:13 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Fri, Mar 6, 2026 at 10:55 AM shveta malik <[email protected]> wrote:
>
> On Thu, Mar 5, 2026 at 5:04 PM Zhijie Hou (Fujitsu)
> <[email protected]> wrote:
> >
> >
> > Thanks for catching this, I changed it to check the increment before reporting warning.
> >
>
> Thanks! I have changed comments atop sequencesync.c to get rid of old
> implementation details (worker dependency upon INIT state) and
> rephrased a little bit. Please take it if you find it okay. Attaching
> patch as txt file. It is a top-up for 0001.
>
No major concerns on 001, just a few trivial things. Do these only if
you feel okay about these.
1)
copy_sequence():
+ (void) GetSubscriptionRelState(MySubscription->oid,
+ RelationGetRelid(sequence_rel),
+ &local_page_lsn);
IIUC, we only need it to get local_page_lsn which is used at the end
of copy_sequence(). Shall we move fetching it towards the end. That
way if validate_seqsync_state() returns copy-not-allowed, then there
is no need to even do 'GetSubscriptionRelState'.
2)
validate_seqsync_state() and get_and_validate_seq_info() are a bit
confusing (at least to me). They have similar names but serve
different purposes. Would it make sense to rename the new function
validate_seqsync_state() to check_seq_privileges_and_drift()?
3)
I feel that the section below in the doc needs some change to briefly
mention the role of the SeqSync worker, at least at a high level, to
indicate that it is running to perform incremental synchronization
already. Thoughts?
---------
29.7.2. Refreshing Out-of-Sync Sequences
Subscriber sequence values can become out of sync as the publisher
advances them. To detect this, compare the
pg_subscription_rel.srsublsn on the subscriber with the page_lsn
obtained from the pg_get_sequence_data function for the sequence on
the publisher. Then run ALTER SUBSCRIPTION ... REFRESH SEQUENCES to
re-synchronize if necessary.
---------
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-03-13 07:13 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-13 08:37 ` Re: [PATCH] Support automatic sequence replication Chao Li <[email protected]>
2026-03-13 10:41 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 11:35 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
0 siblings, 3 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-13 07:13 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Monday, March 9, 2026 11:13 AM shveta malik <[email protected]> wrote:
>
> No major concerns on 001, just a few trivial things. Do these only if you feel
> okay about these.
>
Thanks for the reviews. I've updated the patch set addressing all comments.
In 0001, aside from addressing comments in [1][2][3][4], I've changed the
logic to delay updating the page LSN for each sequence in
pg_subscription_rel. Frequent catalog updates would generate many
invalidations, degrading the performance of the apply worker which
relies on cached data from pg_subscription_rel.
0002 adds caching of sequence information for the current subscription in the
sequence sync worker. The cache is invalidated immediately when
pg_subscription_rel is modified. Concurrent changes to sequence names or
namespaces are detected before synchronization, as the worker verifies the
sequence data at sync time.
0003 (formerly 0002) modifies REFRESH SEQUENCES to synchronize sequence values
directly without launching a worker.
[1] https://www.postgresql.org/message-id/02EDB3D2-4E5A-4EDE-BADF-3DF62D707831%40gmail.com
[2] https://www.postgresql.org/message-id/OS9PR01MB12149E4614DA95963670772EEF579A%40OS9PR01MB12149.jpnpr...
[3] https://www.postgresql.org/message-id/CAJpy0uAmEkjsBS6RxPv9iDcK2kfJ5%3Dbq4Mq1zMCQtaYFoDfbbQ%40mail.g...
[4] https://www.postgresql.org/message-id/CAJpy0uC0T_tp62zxJN_2d_A%3DYpvf14ebjGFepckeJugW5OHOyA%40mail.g...
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v12-0003-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch (15.7K, 2-v12-0003-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch)
download | inline diff:
From 6614db9174c440e0ed855bf7304c77e4bb400d6c Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Fri, 13 Mar 2026 13:10:30 +0800
Subject: [PATCH v12 3/3] Synchronize sequences directly in REFRESH SEQUENCES
command
The ALTER SUBSCRIPTION ... REFRESH SEQUENCES command currently sets all
sequence states in pg_subscription_rel to INIT and relies on the sequence sync
worker to perform the actual synchronization and update states to READY.
With the recent change making the sequence sync worker long-lived, most
sequences are now synchronized in the background, reducing the need for
REFRESH SEQUENCES. However, the command remains necessary for sequences that
haven't been synchronized.
This commit enhances REFRESH SEQUENCES to synchronize sequences directly within
the command itself, eliminating the overhead of launching a worker and updating
catalog entries unnecessarily.
---
doc/src/sgml/logical-replication.sgml | 5 +-
src/backend/commands/subscriptioncmds.c | 27 ++--
.../replication/logical/sequencesync.c | 148 +++++++++++++-----
src/include/replication/logicalworker.h | 5 +
src/test/subscription/t/036_sequences.pl | 49 ++++++
5 files changed, 176 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 3865f617816..e3472cc952e 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1791,8 +1791,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands. The worker will
- remain running for the life of the subscription, periodically
+ after executing <command>CREATE SUBSCRIPTION</command> or
+ <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command> command. The
+ worker will remain running for the life of the subscription, periodically
synchronizing all published sequences.
</para>
<para>
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 724637cff5b..d7fb634df12 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1288,25 +1288,20 @@ AlterSubscription_refresh_seq(Subscription *sub)
PG_TRY();
{
- List *subrel_states;
-
check_publications_origin_sequences(wrconn, sub->publications, true,
sub->origin, NULL, 0, sub->name);
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
+ /*
+ * Stop the sequencesync worker to prevent concurrent updates. This
+ * avoids a race condition where the sequence value could be updated
+ * by this command and then immediately moved backward by a
+ * concurrently running worker. Stopping the worker is safe even if it
+ * attempts to restart, as it will wait on the subscription lock
+ * already held by this ALTER SUBSCRIPTION command.
+ */
+ logicalrep_worker_stop(WORKERTYPE_SEQUENCESYNC, sub->oid, InvalidOid);
+
+ AlterSubSyncSequences(wrconn, sub->oid, sub->name, sub->runasowner);
}
PG_FINALLY();
{
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index a4807ee8017..9ea35c388f4 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -216,7 +216,7 @@ get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx, List *seqinfos)
+ List *missing_seqs_idx, List *seqinfos, char *subname)
{
StringInfo seqstr;
@@ -262,7 +262,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
ereport(ERROR,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("logical replication sequence synchronization failed for subscription \"%s\"",
- MySubscription->name));
+ subname));
}
/*
@@ -419,10 +419,9 @@ check_seq_privileges_and_drift(LogicalRepSequenceInfo *seqinfo,
*/
static CopySeqResult
copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
- bool update_lsn)
+ Oid subid, bool run_as_owner, bool update_lsn)
{
UserContext ucxt;
- bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
CopySeqResult result;
bool need_lsn_update = false;
@@ -466,8 +465,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
XLogRecPtr local_page_lsn;
/* Get the local page LSN for comparison with the remote value */
- (void) GetSubscriptionRelState(MySubscription->oid,
- RelationGetRelid(sequence_rel),
+ (void) GetSubscriptionRelState(subid, RelationGetRelid(sequence_rel),
&local_page_lsn);
need_lsn_update = (local_page_lsn != seqinfo->page_lsn);
@@ -479,7 +477,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
* cycle (update_lsn is true).
*/
if (seqinfo->relstate == SUBREL_STATE_INIT || need_lsn_update)
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ UpdateSubscriptionRelState(subid, seqoid, SUBREL_STATE_READY,
seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
@@ -498,7 +496,8 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
* Returns true/false if any sequences were actually copied.
*/
static bool
-copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
+copy_sequences(WalReceiverConn *conn, List *seqinfos, Oid subid, char *subname,
+ bool runasowner, bool update_lsn)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -527,11 +526,16 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
+ bool started_tx = false;
WalRcvExecResult *res;
TupleTableSlot *slot;
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
for (int idx = cur_batch_base_index; idx < n_seqinfos; idx++)
{
@@ -623,14 +627,15 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
&seqinfo, &seqidx, seqinfos);
if (sync_status == COPYSEQ_ALLOWED)
- sync_status = copy_sequence(seqinfo, sequence_rel, update_lsn);
+ sync_status = copy_sequence(seqinfo, sequence_rel, subid,
+ runasowner, update_lsn);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
"logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
- seqinfo->nspname, seqinfo->seqname, MySubscription->name);
+ seqinfo->nspname, seqinfo->seqname, subname);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -703,13 +708,17 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
elog(DEBUG1,
"logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
- MySubscription->name,
+ subname,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
- /* Commit this batch, and prepare for next batch */
- CommitTransactionCommand();
+ /*
+ * Commit this batch if started a transaction, and prepare for next
+ * batch.
+ */
+ if (started_tx)
+ CommitTransactionCommand();
if (batch_missing_count)
{
@@ -734,7 +743,7 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx, seqinfos);
+ missing_seqs_idx, seqinfos, subname);
return sequence_copied;
}
@@ -750,37 +759,28 @@ invalidate_syncing_sequence_infos(Datum arg, SysCacheIdentifier cacheid,
}
/*
- * Get the list of sequence information for the current subscription.
+ * Get the list of sequence information for the given subscription from
+ * catalog.
*
- * Return cached sequence states if valid; otherwise fetches them from the
- * catalog, caches the result, and return it.
+ * All entries in the returned list are allocated in the specified memory
+ * context.
*/
static List *
-fetch_sequence_infos(void)
+fetch_sequences_from_catalog(Oid subid, MemoryContext ctx)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
- Oid subid = MyLogicalRepWorker->subid;
- List *tmp_seqinfos = NIL;
+ List *seqinfos = NIL;
+ bool started_tx = false;
- if (sequence_infos_valid)
- return sequence_infos;
-
- /* Free the existing invalid cache entries */
- foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ if (!IsTransactionState())
{
- pfree(seqinfo->nspname);
- pfree(seqinfo->seqname);
- pfree(seqinfo);
+ StartTransactionCommand();
+ started_tx = true;
}
- list_free(sequence_infos);
- sequence_infos = NIL;
-
- StartTransactionCommand();
-
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
/* Scan for all sequences belonging to this subscription */
@@ -821,14 +821,14 @@ fetch_sequence_infos(void)
Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
- /* Cache the information in a permanent memory context */
- oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ /* Cache the information in the given memory context */
+ oldctx = MemoryContextSwitchTo(ctx);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
- tmp_seqinfos = lappend(tmp_seqinfos, seq);
+ seqinfos = lappend(seqinfos, seq);
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -838,10 +838,38 @@ fetch_sequence_infos(void)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- sequence_infos = tmp_seqinfos;
- sequence_infos_valid = true;
+ if (started_tx)
+ CommitTransactionCommand();
- CommitTransactionCommand();
+ return seqinfos;
+}
+
+/*
+ * Get the list of sequence information for the current subscription.
+ *
+ * Return cached sequence states if valid; otherwise fetches them from the
+ * catalog, caches the result, and return it.
+ */
+static List *
+fetch_sequence_infos(void)
+{
+ if (sequence_infos_valid)
+ return sequence_infos;
+
+ /* Free the existing invalid cache entries */
+ foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ {
+ pfree(seqinfo->nspname);
+ pfree(seqinfo->seqname);
+ pfree(seqinfo);
+ }
+
+ list_free(sequence_infos);
+ sequence_infos = NIL;
+
+ sequence_infos = fetch_sequences_from_catalog(MySubscription->oid,
+ CacheMemoryContext);
+ sequence_infos_valid = true;
return sequence_infos;
}
@@ -946,6 +974,9 @@ start_sequence_sync(void)
seqinfos = fetch_sequence_infos();
sequence_copied = copy_sequences(LogRepWorkerWalRcvConn, seqinfos,
+ MySubscription->oid,
+ MySubscription->name,
+ MySubscription->runasowner,
update_lsn);
MemoryContextReset(SequenceSyncContext);
@@ -1015,3 +1046,40 @@ SequenceSyncWorkerMain(Datum main_arg)
FinishSyncWorker();
}
+
+/*
+ * Wrapper for LogicalRepSyncSequences to synchronize all sequences of a
+ * subscription from outside the sequencesync worker.
+ */
+void
+AlterSubSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
+{
+ List *seqinfos;
+
+ Assert(!SequenceSyncContext);
+
+ /*
+ * Fetch sequences directly from the catalog rather than using the
+ * sequence cache, which is maintained only for the sequence sync
+ * worker.
+ */
+ seqinfos = fetch_sequences_from_catalog(subid, CurrentMemoryContext);
+
+ PG_TRY();
+ {
+ /*
+ * Use the current memory context for synchronization. Since this should
+ * be short-lived command context that will be cleaned up automatically,
+ * we can simply assign it as the synchronization context.
+ */
+ SequenceSyncContext = CurrentMemoryContext;
+
+ (void) copy_sequences(conn, seqinfos, subid, subname, runasowner, true);
+ }
+ PG_FINALLY();
+ {
+ SequenceSyncContext = NULL;
+ }
+ PG_END_TRY();
+}
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..73afd7853d0 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,6 +14,8 @@
#include <signal.h>
+#include "replication/walreceiver.h"
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
extern void ApplyWorkerMain(Datum main_arg);
@@ -31,4 +33,7 @@ extern void LogicalRepWorkersWakeupAtCommit(Oid subid);
extern void AtEOXact_LogicalRepWorkers(bool isCommit);
+extern void AlterSubSyncSequences(WalReceiverConn *conn, Oid subid,
+ char *subname, bool runasowner);
+
#endif /* LOGICALWORKER_H */
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index af190713b2b..8d25ac40ce0 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -176,4 +176,53 @@ $node_subscriber->wait_for_log(
qr/WARNING: ( [A-Z0-9]+:)? missing sequence on publisher \("public.regress_s4"\)/,
$log_offset);
+##########
+# ALTER SUBSCRIPTION ... REFRESH SEQUENCES synchronizes sequences online,
+# eliminating the need to launch a sequencesync worker.
+##########
+
+# Reduce max_logical_replication_workers to disallow sequence worker from running
+$node_subscriber->append_conf('postgresql.conf',
+ qq(max_logical_replication_workers = 0));
+$node_subscriber->restart;
+
+# Verify there is no logical replication apply worker running
+$result = $node_subscriber->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'logical replication apply worker'");
+
+is($result, '0', 'no logical replication worker is running');
+
+# Increment sequence on publisher
+$node_publisher->safe_psql('postgres',
+ qq(SELECT nextval('regress_s1');));
+
+# The command should fail due to missing sequence ('regress_s4')
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+like(
+ $stderr,
+ qr/WARNING: missing sequence on publisher \("public.regress_s4"\)/,
+ "output the wanring for the missing sequence regress_s4");
+
+like(
+ $stderr,
+ qr/ERROR: logical replication sequence synchronization failed for subscription \"regress_seq_sub\"/,
+ "the command failed due to the missing sequence regress_s4");
+
+# Refresh the publication to remove the missing sequence
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION;");
+
+# Sync the sequence regress_s1
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+# Get the current sequence value on subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ qq(SELECT last_value FROM regress_s1;));
+
+is($result, '201', 'sequence regress_s1 is synced now');
+
done_testing();
--
2.53.0.windows.2
[application/octet-stream] v12-0001-Support-automatic-sequence-replication.patch (46.4K, 3-v12-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From ce4733ab4f3ed960812906a345807650cb8485b1 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Fri, 13 Mar 2026 13:28:27 +0800
Subject: [PATCH v12 1/3] Support automatic sequence replication.
Currently, sequence values are synchronized from publisher to subscriber only
when the user manually runs ALTER SUBSCRIPTION ... REFRESH PUBLICATION (which
affects only newly subscribed sequences) or REFRESH SEQUENCES. The sequence sync
worker exits immediately after completing each synchronization round.
The primary use case for sequence replication is during upgrades, where it's
recommended that users ensure sequences are in sync by running REFRESH SEQUENCES
before finishing the upgrade. However, this command can be slow when
synchronizing a large number of sequences, potentially increasing downtime.
To address this, this commit makes the sequence sync worker long-lived,
continuously monitoring sequences and resynchronizing them when drift is
detected. The worker uses an adaptive sleep interval: it starts at 2 seconds,
doubles up to a maximum of 30 seconds when no drift is observed, and resets to
the minimum interval once drift is found.
With this change, most sequences are silently synchronized in the background,
eliminating the need to run REFRESH SEQUENCES for the majority of cases.
However, frequently updated sequences may still lag behind, requiring a final
REFRESH SEQUENCES before upgrade completion. Users can monitor progress by
checking whether sequence states transition from INIT to READY in
pg_subscription_rel.
The REFRESH SEQUENCES command is retained for this final synchronization step,
though it currently updates all sequence states to INIT, which has room for
improvement. A future patch will enhance this command to synchronize sequences
directly without launching a worker, reducing catalog overhead.
---
doc/src/sgml/logical-replication.sgml | 39 +-
doc/src/sgml/ref/alter_subscription.sgml | 5 -
src/backend/commands/sequence.c | 41 ++
.../replication/logical/sequencesync.c | 424 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 46 +-
src/backend/replication/logical/worker.c | 11 +
.../utils/activity/wait_event_names.txt | 1 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 81 +---
12 files changed, 437 insertions(+), 217 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 72c8d3d59bd..3865f617816 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1791,8 +1791,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1821,18 +1822,26 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
- advances them.
+ Subscriber sequence values can become out of sync as the publisher advances
+ them and the sequence synchronization worker has not yet caught up.
</para>
<para>
To detect this, compare the
<link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
on the subscriber with the <structfield>page_lsn</structfield> obtained
from the <link linkend="func-pg-get-sequence-data"><function>pg_get_sequence_data</function></link>
- function for the sequence on the publisher. Then run
+ function for the sequence on the publisher. If drift is detected, the user
+ can wait for the sequence synchronization worker to catch up (by
+ periodically checking whether the LSNs match).
+ </para>
+ <para>
+ Note that the sequence synchronization worker updates
+ <link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
+ periodically. To reduce the delay and force an immediate update, execute the
<link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> to
- re-synchronize if necessary.
+ <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> command,
+ which updates the page LSN immediately after synchronizing the sequence
+ value.
</para>
<warning>
<para>
@@ -2339,15 +2348,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index f215fb0e5a2..ee96e4823a3 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -202,11 +202,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..92ae5aab488 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,47 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift. The caller must hold a lock on the sequence.
+ *
+ * Return false if the caller does not have sufficient privileges to access the
+ * sequence, true otherwise.
+ */
+bool
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Oid relid = RelationGetRelid(seqrel);
+
+ /* Confirm that the relation is a sequence and is locked */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+ Assert(CheckRelationLockedByMe(seqrel, AccessShareLock, true));
+
+ if (pg_class_aclcheck(relid, GetUserId(), ACL_SELECT) != ACLCHECK_OK)
+ {
+ *last_value = 0;
+ *is_called = false;
+ return false;
+ }
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+
+ return true;
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 9c92fddd624..f04c50f6cae 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,25 +19,32 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
- * The apply worker periodically scans pg_subscription_rel for sequences in
- * INIT state. When such sequences are found, it spawns a sequencesync worker
- * to handle synchronization.
- *
- * A single sequencesync worker is responsible for synchronizing all sequences.
- * It begins by retrieving the list of sequences that are flagged for
- * synchronization, i.e., those in the INIT state. These sequences are then
- * processed in batches, allowing multiple entries to be synchronized within a
- * single transaction. The worker fetches the current sequence values and page
- * LSNs from the remote publisher, updates the corresponding sequences on the
- * local subscriber, and finally marks each sequence as READY upon successful
+ * The apply worker periodically scans pg_subscription_rel for sequences.
+ * When sequences are found, it spawns a sequencesync worker to handle
* synchronization.
*
+ * A single sequencesync worker is responsible for synchronizing all sequences
+ * for a subscription. It begins by retrieving the list of sequences. These
+ * sequences are then processed in batches, allowing multiple entries to be
+ * synchronized within a single transaction. The worker fetches the current
+ * sequence values and page LSNs from the remote publisher and updates the
+ * corresponding sequences on the local subscriber. Sequences in the INIT
+ * state are unconditionally updated to the latest values from the publisher
+ * and then moved to the READY state. For sequences already in the READY
+ * state, the worker checks for drift and updates them only when needed.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+
+ * (synchronize)
+ * INIT --------------> READY ->-+
+ * ^ | (check-drift/synchronize)
+ * | |
+ * +--<---+
+ *
+ * Between cycles, the worker sleeps for SEQSYNC_MIN_SLEEP_MS. If no drift is
+ * observed in any sequence, the sleep interval doubles after each wake cycle
+ * up to SEQSYNC_MAX_SLEEP_MS. When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -60,6 +67,7 @@
#include "postmaster/interrupt.h"
#include "replication/logicalworker.h"
#include "replication/worker_internal.h"
+#include "storage/latch.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
@@ -70,31 +78,42 @@
#include "utils/pg_lsn.h"
#include "utils/syscache.h"
#include "utils/usercontext.h"
+#include "utils/wait_event.h"
#define REMOTE_SEQ_COL_COUNT 10
typedef enum CopySeqResult
{
COPYSEQ_SUCCESS,
+ COPYSEQ_ALLOWED,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
*
* Start a sequencesync worker if one is not already running. The active
* sequencesync worker will handle all pending sequence synchronization. If any
* sequences remain unsynchronized after it exits, a new worker can be started
* in the next iteration.
+ *
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -112,6 +131,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Quick exit if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -144,7 +176,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +203,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +215,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +226,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +237,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +261,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -240,7 +273,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
int64 remote_min;
int64 remote_max;
bool remote_cycle;
- CopySeqResult result = COPYSEQ_SUCCESS;
+ CopySeqResult result = COPYSEQ_ALLOWED;
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
@@ -325,32 +358,77 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Check whether the user has required privileges on the sequence and
+ * whether the sequence has drifted.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+check_seq_privileges_and_drift(LogicalRepSequenceInfo *seqinfo,
+ Relation sequence_rel)
{
- UserContext ucxt;
AclResult aclresult;
+ Oid seqoid = seqinfo->localrelid;
+
+ /* Perform drift check if it's not the initial sync */
+ if (seqinfo->relstate == SUBREL_STATE_READY)
+ {
+ int64 local_last_value;
+ bool local_is_called;
+
+ /*
+ * Skip synchronization if the current user does not have sufficient
+ * privileges to read the sequence data.
+ */
+ if (!GetSequence(sequence_rel, &local_last_value, &local_is_called))
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ /*
+ * Skip synchronization if the sequence has not drifted from the
+ * publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Verify that the current user can update the sequence */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ return COPYSEQ_ALLOWED;
+}
+
+/*
+ * Apply remote sequence state to local sequence. For sequences in INIT state,
+ * always synchronize and then move them to READY state upon completion. For
+ * sequences already in READY state, synchronize only if drift is detected.
+ */
+static CopySeqResult
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ bool update_lsn)
+{
+ UserContext ucxt;
bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
+ CopySeqResult result;
+ bool need_lsn_update = false;
/*
* If the user did not opt to run as the owner of the subscription
* ('run_as_owner'), then copy the sequence as the owner of the sequence.
*/
if (!run_as_owner)
- SwitchToUntrustedUser(seqowner, &ucxt);
+ SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+ result = check_seq_privileges_and_drift(seqinfo, sequence_rel);
- if (aclresult != ACLCHECK_OK)
+ if (result != COPYSEQ_ALLOWED)
{
if (!run_as_owner)
RestoreUserContext(&ucxt);
- return COPYSEQ_INSUFFICIENT_PERM;
+ return result;
}
/*
@@ -367,20 +445,47 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
RestoreUserContext(&ucxt);
/*
- * Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * If LSN updates are requested, check whether the remote LSN differs from
+ * the locally stored value.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (update_lsn)
+ {
+ XLogRecPtr local_page_lsn;
+
+ /* Get the local page LSN for comparison with the remote value */
+ (void) GetSubscriptionRelState(MySubscription->oid,
+ RelationGetRelid(sequence_rel),
+ &local_page_lsn);
+
+ need_lsn_update = (local_page_lsn != seqinfo->page_lsn);
+ }
+
+ /*
+ * Update the catalog if either the sequence is in INIT state and needs to
+ * transition to READY, or the LSN has changed and this is an LSN update
+ * cycle (update_lsn is true).
+ */
+ if (seqinfo->relstate == SUBREL_STATE_INIT || need_lsn_update)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
/*
* Copy existing data of sequences from the publisher.
+ *
+ * Sequences in INIT state are always synchronized. Sequences in READY state are
+ * synchronized only when drift is detected.
+ *
+ * When 'update_lsn' is true, update the page LSN in pg_subscription_rel for any
+ * synchronized sequences whose LSN has changed. When false, the LSN is updated
+ * only for sequences transitioning from INIT to READY state.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +495,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +508,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,28 +604,28 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
- if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ &seqinfo, &seqidx, seqinfos);
+
+ if (sync_status == COPYSEQ_ALLOWED)
+ sync_status = copy_sequence(seqinfo, sequence_rel, update_lsn);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
+ sequence_copied = true;
break;
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -531,11 +634,11 @@ copy_sequences(WalReceiverConn *conn)
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -558,6 +661,13 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+ default:
+ elog(ERROR, "unrecognized sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -572,14 +682,15 @@ copy_sequences(WalReceiverConn *conn)
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
MySubscription->name,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,47 +718,50 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
@@ -666,18 +780,21 @@ LogicalRepSyncSequences(void)
continue;
}
+ relstate = subrel->srsubstate;
+
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
-
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -693,36 +810,16 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
-
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ return false;
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos, update_lsn);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +832,117 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+ long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ TimestampTz next_lsn_update = GetCurrentTimestamp();
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence
+ * synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+ bool update_lsn;
+ TimestampTz now = GetCurrentTimestamp();
+
+ CHECK_FOR_INTERRUPTS();
+
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+ maybe_reread_subscription();
+
+ /*
+ * We avoid updating pg_subscription_rel's page LSN in every cycle
+ * to prevent excessive catalog invalidations, which would slow
+ * the apply worker that relies on this cache (see
+ * FetchRelationStates). Instead, updates occur every 30 seconds.
+ */
+ update_lsn = (now >= next_lsn_update);
+
+ /*
+ * Perform sequence synchronization under SequenceSyncContext and
+ * reset it each cycle to avoid manual memory management.
+ */
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
+ update_lsn);
+
+ MemoryContextReset(SequenceSyncContext);
+ MemoryContextSwitchTo(oldctx);
+
+ /*
+ * Adjust sleep interval based on sync activity. If sequences were
+ * copied, reset to the minimum interval to poll more frequently.
+ * Otherwise, exponentially back off up to the maximum interval.
+ */
+ if (sequence_copied)
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ else
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+
+ /* Refresh timestamp after potentially time-consuming sync work */
+ now = GetCurrentTimestamp();
+
+ /*
+ * Schedule the next LSN update. If we just performed an update,
+ * set the next update time to 30 seconds from now. Otherwise,
+ * ensure the sleep interval doesn't exceed the time remaining
+ * until the next update.
+ */
+ if (update_lsn)
+ next_lsn_update = TimestampTzPlusMilliseconds(now, SEQSYNC_MAX_SLEEP_MS);
+ else
+ sleep_ms = Min(sleep_ms, TimestampDifferenceMilliseconds(now, next_lsn_update));
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SEQSYNC_MAIN);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..3a0dc8669f9 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,7 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +191,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +205,22 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
- * since the same value can be used until the system table is invalidated.
+ * has_subtables and has_subsequences are declared as static, since the
+ * same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +230,28 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
+
/*
* Does the subscription have tables?
*
@@ -277,5 +277,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 033858752d9..b1cfa7f55bb 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -5100,6 +5100,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5115,6 +5118,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5133,6 +5140,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index 4aa864fe3c3..a91085e7723 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -61,6 +61,7 @@ IO_WORKER_MAIN "Waiting in main loop of IO Worker process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_SEQSYNC_MAIN "Waiting in main loop of logical replication sequence sync process."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot synchronization."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..fd4f69bdd1c 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern bool GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 4ecbdcfadac..a41cb045f19 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..af190713b2b 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,18 +75,14 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
-# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
+# Create a new sequence 'regress_s2'
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.53.0.windows.2
[application/octet-stream] v12-0002-Cache-sequence-information-in-the-sequence-sync-.patch (6.2K, 4-v12-0002-Cache-sequence-information-in-the-sequence-sync-.patch)
download | inline diff:
From 52b649b39343a179f4857189b209c6fb862bfe12 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Fri, 13 Mar 2026 12:37:25 +0800
Subject: [PATCH v12 2/3] Cache sequence information in the sequence sync
worker
Previously, the sequence sync worker would fetch sequence metadata from
the catalog each time it needed to synchronize sequences. This could be
inefficient when many sequences are involved, as the worker would need
to repeatedly open and scan pg_subscription_rel.
To improve this, introduce a cache for sequence information in the sequence sync
worker. The cache is populated on first use and kept across synchronization
cycles. It is invalidated when pg_subscription_rel is modified, ensuring that
changes to subscription relations are reflected promptly.
---
.../replication/logical/sequencesync.c | 93 +++++++++++++------
1 file changed, 66 insertions(+), 27 deletions(-)
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index f04c50f6cae..a4807ee8017 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -98,6 +98,19 @@ typedef enum CopySeqResult
static MemoryContext SequenceSyncContext = NULL;
+/*
+ * Cached list of sequence information (LogicalRepSequenceInfo) for the current
+ * subscription. The cache is invalidated when pg_subscription_rel is modified.
+ *
+ * Note: To avoid the cost of searching for a specific sequence on relcache
+ * invalidation, we do not invalidate the cache immediately when a sequence is
+ * altered (e.g., renamed or moved to another namespace). Instead, we validate
+ * the sequence name and namespace when next attempting to sync it, at which
+ * point we verify the local sequence state.
+ */
+static List *sequence_infos = NIL;
+static bool sequence_infos_valid = false;
+
/*
* Apply worker determines whether a sequence sync worker is needed.
*
@@ -499,6 +512,9 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
+ if (seqinfos == NIL)
+ return false;
+
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -724,24 +740,44 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
}
/*
- * Identifies sequences that require synchronization and initiates the
- * synchronization process.
+ * Callback from syscache invalidation.
+ */
+static void
+invalidate_syncing_sequence_infos(Datum arg, SysCacheIdentifier cacheid,
+ uint32 hashvalue)
+{
+ sequence_infos_valid = false;
+}
+
+/*
+ * Get the list of sequence information for the current subscription.
*
- * Returns true if sequences have been updated.
+ * Return cached sequence states if valid; otherwise fetches them from the
+ * catalog, caches the result, and return it.
*/
-static bool
-LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
+static List *
+fetch_sequence_infos(void)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- bool sequence_copied = false;
- List *seqinfos = NIL;
- MemoryContext oldctx;
+ List *tmp_seqinfos = NIL;
+
+ if (sequence_infos_valid)
+ return sequence_infos;
- Assert(SequenceSyncContext);
+ /* Free the existing invalid cache entries */
+ foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ {
+ pfree(seqinfo->nspname);
+ pfree(seqinfo->seqname);
+ pfree(seqinfo);
+ }
+
+ list_free(sequence_infos);
+ sequence_infos = NIL;
StartTransactionCommand();
@@ -762,6 +798,7 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
char relstate;
+ MemoryContext oldctx;
CHECK_FOR_INTERRUPTS();
@@ -784,17 +821,14 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
- /*
- * Worker needs to process sequences across transaction boundary, so
- * allocate them under SequenceSyncContext.
- */
- oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+ /* Cache the information in a permanent memory context */
+ oldctx = MemoryContextSwitchTo(CacheMemoryContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
- seqinfos = lappend(seqinfos, seq);
+ tmp_seqinfos = lappend(tmp_seqinfos, seq);
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -804,18 +838,12 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- CommitTransactionCommand();
-
- /*
- * Exit early if no catalog entries found, likely due to concurrent drops.
- */
- if (!seqinfos)
- return false;
+ sequence_infos = tmp_seqinfos;
+ sequence_infos_valid = true;
- /* Process sequences */
- sequence_copied = copy_sequences(conn, seqinfos, update_lsn);
+ CommitTransactionCommand();
- return sequence_copied;
+ return sequence_infos;
}
/*
@@ -830,6 +858,14 @@ start_sequence_sync(void)
{
Assert(am_sequencesync_worker());
+ /*
+ * Setup callback for syscache so that we know when something changes in
+ * the subscription relation state.
+ */
+ CacheRegisterSyscacheCallback(SUBSCRIPTIONRELMAP,
+ invalidate_syncing_sequence_infos,
+ (Datum) 0);
+
PG_TRY();
{
char *err;
@@ -875,6 +911,7 @@ start_sequence_sync(void)
bool sequence_copied = false;
MemoryContext oldctx;
bool update_lsn;
+ List *seqinfos;
TimestampTz now = GetCurrentTimestamp();
CHECK_FOR_INTERRUPTS();
@@ -906,8 +943,10 @@ start_sequence_sync(void)
/*
* Synchronize all sequences (both READY and INIT states).
*/
- sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
- update_lsn);
+ seqinfos = fetch_sequence_infos();
+
+ sequence_copied = copy_sequences(LogRepWorkerWalRcvConn, seqinfos,
+ update_lsn);
MemoryContextReset(SequenceSyncContext);
MemoryContextSwitchTo(oldctx);
--
2.53.0.windows.2
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 07:13 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-13 08:37 ` Chao Li <[email protected]>
2 siblings, 0 replies; 58+ messages in thread
From: Chao Li @ 2026-03-13 08:37 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: shveta malik <[email protected]>; Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
> On Mar 13, 2026, at 15:13, Zhijie Hou (Fujitsu) <[email protected]> wrote:
>
> On Monday, March 9, 2026 11:13 AM shveta malik <[email protected]> wrote:
>>
>> No major concerns on 001, just a few trivial things. Do these only if you feel
>> okay about these.
>>
>
> Thanks for the reviews. I've updated the patch set addressing all comments.
>
> In 0001, aside from addressing comments in [1][2][3][4], I've changed the
> logic to delay updating the page LSN for each sequence in
> pg_subscription_rel. Frequent catalog updates would generate many
> invalidations, degrading the performance of the apply worker which
> relies on cached data from pg_subscription_rel.
>
> 0002 adds caching of sequence information for the current subscription in the
> sequence sync worker. The cache is invalidated immediately when
> pg_subscription_rel is modified. Concurrent changes to sequence names or
> namespaces are detected before synchronization, as the worker verifies the
> sequence data at sync time.
>
> 0003 (formerly 0002) modifies REFRESH SEQUENCES to synchronize sequence values
> directly without launching a worker.
>
> [1] https://www.postgresql.org/message-id/02EDB3D2-4E5A-4EDE-BADF-3DF62D707831%40gmail.com
> [2] https://www.postgresql.org/message-id/OS9PR01MB12149E4614DA95963670772EEF579A%40OS9PR01MB12149.jpnpr...
> [3] https://www.postgresql.org/message-id/CAJpy0uAmEkjsBS6RxPv9iDcK2kfJ5%3Dbq4Mq1zMCQtaYFoDfbbQ%40mail.g...
> [4] https://www.postgresql.org/message-id/CAJpy0uC0T_tp62zxJN_2d_A%3DYpvf14ebjGFepckeJugW5OHOyA%40mail.g...
>
> Best Regards,
> Hou zj
> <v12-0003-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch><v12-0001-Support-automatic-sequence-replication.patch><v12-0002-Cache-sequence-information-in-the-sequence-sync-.patch>
I reviewed v12 again. 0001 looks good. A few comments on 0002 and 0003.
1 - 0002
```
+ /*
+ * Setup callback for syscache so that we know when something changes in
+ * the subscription relation state.
+ */
+ CacheRegisterSyscacheCallback(SUBSCRIPTIONRELMAP,
+ invalidate_syncing_sequence_infos,
+ (Datum) 0);
```
I wonder if SUBSCRIPTIONRELMAP should be SUBSCRIPTIONREL?
2 - 0003
```
+ /*
+ * Use the current memory context for synchronization. Since this should
+ * be short-lived command context that will be cleaned up automatically,
+ * we can simply assign it as the synchronization context.
+ */
+ SequenceSyncContext = CurrentMemoryContext;
```
I think it’s still better to create a memory context from CurrentMemoryContext for SequenceSyncContext, and destroy it after copy_sequence.
Today, this is only on the SQL command path, CurrentMemoryContext is supposed to be short-lived. But AlterSubSyncSequences() might be called somewhere else in future, then we could not predict what would be CurrentMemoryContext.
3 - 0003
```
"output the wanring for the missing sequence regress_s4”);
```
Typo: wanring -> warning
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 07:13 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-13 10:41 ` shveta malik <[email protected]>
2 siblings, 0 replies; 58+ messages in thread
From: shveta malik @ 2026-03-13 10:41 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; shveta malik <[email protected]>
On Fri, Mar 13, 2026 at 12:43 PM Zhijie Hou (Fujitsu)
<[email protected]> wrote:
>
> On Monday, March 9, 2026 11:13 AM shveta malik <[email protected]> wrote:
> >
> > No major concerns on 001, just a few trivial things. Do these only if you feel
> > okay about these.
> >
>
> Thanks for the reviews. I've updated the patch set addressing all comments.
>
Thanks Hou-San. Please find my concerns on 001:
1)
Consider a case where the page LSN has changed and the sequence has
drifted, but the page LSN was not updated because the update interval
had not yet elapsed. Later, if there is no further drift for a couple
of minutes, we may continue invoking copy_sequence with update_lsn =
true. However, since check_seq_privileges_and_drift() keeps returning
no drift, the LSN might never get updated.
2)
Also, IIUC, we will end up advancing 'next_lsn_update' based on
'update_lsn' even though no actual lsn-update has occurred. As a
result, the next page LSN update may never happen if the update_lsn =
true cases always coincide with the no-drift case.
Shall copy_sequence() call UpdateSubscriptionRelState() even if there
is no drift but need_lsn_update is true?
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 07:13 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-13 11:35 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-16 06:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2 siblings, 1 reply; 58+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-03-13 11:35 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; shveta malik <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
Dear Hou,
Thanks for updating the patch. Comments for v12-0002.
01.
```
+ /* Free the existing invalid cache entries */
+ foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ {
+ pfree(seqinfo->nspname);
+ pfree(seqinfo->seqname);
+ pfree(seqinfo);
+ }
```
According to the comment atop foreach_delete_current, we should not directly pfree()
the iterator.
02.
```
+ /* Cache the information in a permanent memory context */
+ oldctx = MemoryContextSwitchTo(CacheMemoryContext);
```
Do you have a reason to use CacheMemoryContext instead of ApplyContext?
According to the readme, the context can be used for the limited purpose, like catcache
and relcache. Not sure we can easily use it.
03.
I think seqinfo->found_on_pub must be set to false before doing copy_sequences() again.
Otherwise, sequences dropped on the publisher cannot be detected as the missing ones.
Or we may have to have another array to indicate it.
I found a below scenario.
There are 10 sequences on both instances, and sequencesync worker synchronizes once.
Now two of them are dropped on the publisher. In the next iteration by the worker,
it can find that only 8 sequeces are found on the publisher. Then it scans the
cache to check each found_on_pub in sequence_infos, but they were cached as true.
Thus sequencesync worker cannot report anything for missing ones.
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 07:13 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-13 11:35 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-03-16 06:58 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-26 04:25 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-16 06:58 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; +Cc: Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Friday, March 13, 2026 7:36 PM Kuroda, Hayato/黒田 隼人 <[email protected]> wrote:
> Thanks for updating the patch. Comments for v12-0002.
Thanks for the comments.
After some off-list discussion with Amit, we agreed that further analysis is
needed, which means rescheduling this feature for the next release.
The main issue requiring analysis is how to reduce the impact of invalidations
that can occur once the sequence synchronization worker begins modifying
pg_subscription_rel regularly. The current patch updates pg_subscription_rel
only every 30 seconds, which seems acceptable for an initial version. However,
there are other potential approaches worth exploring, such as: 1) adding a
subscription option to let users control the update frequency, or 2) updating
the catalog only after modifying a specific number of sequences.
It would also be worthwhile to examine other modules for similar issues and
their solutions. For example, autoanalyze and autovacuum also modify the catalog
regularly. We should investigate whether they face the same invalidation
challenges and how those challenges are addressed.
Overall, we will continue working on this to improve the patch set, but will
schedule it for PG20.
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 07:13 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-13 11:35 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-16 06:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-26 04:25 ` Ajin Cherian <[email protected]>
2026-04-15 10:41 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
0 siblings, 1 reply; 58+ messages in thread
From: Ajin Cherian @ 2026-03-26 04:25 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Mar 16, 2026 at 5:59 PM Zhijie Hou (Fujitsu)
<[email protected]> wrote:
> Overall, we will continue working on this to improve the patch set, but will
> schedule it for PG20.
>
Just rebasing the patch so that it doesn't break cfbot.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v12-0003-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch (15.7K, 2-v12-0003-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch)
download | inline diff:
From 832496b6175514eb8a00203f376809f3688a26b5 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Thu, 26 Mar 2026 14:41:41 +1100
Subject: [PATCH v12 3/3] Synchronize sequences directly in REFRESH SEQUENCES
command.
The ALTER SUBSCRIPTION ... REFRESH SEQUENCES command currently sets all
sequence states in pg_subscription_rel to INIT and relies on the sequence sync
worker to perform the actual synchronization and update states to READY.
With the recent change making the sequence sync worker long-lived, most
sequences are now synchronized in the background, reducing the need for
REFRESH SEQUENCES. However, the command remains necessary for sequences that
haven't been synchronized.
This commit enhances REFRESH SEQUENCES to synchronize sequences directly within
the command itself, eliminating the overhead of launching a worker and updating
catalog entries unnecessarily.
---
doc/src/sgml/logical-replication.sgml | 5 +-
src/backend/commands/subscriptioncmds.c | 27 ++--
.../replication/logical/sequencesync.c | 148 +++++++++++++-----
src/include/replication/logicalworker.h | 5 +
src/test/subscription/t/036_sequences.pl | 49 ++++++
5 files changed, 176 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 3865f617816..e3472cc952e 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1791,8 +1791,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands. The worker will
- remain running for the life of the subscription, periodically
+ after executing <command>CREATE SUBSCRIPTION</command> or
+ <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command> command. The
+ worker will remain running for the life of the subscription, periodically
synchronizing all published sequences.
</para>
<para>
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 7375e214cb4..bf4fef91f56 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1288,25 +1288,20 @@ AlterSubscription_refresh_seq(Subscription *sub)
PG_TRY();
{
- List *subrel_states;
-
check_publications_origin_sequences(wrconn, sub->publications, true,
sub->origin, NULL, 0, sub->name);
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
+ /*
+ * Stop the sequencesync worker to prevent concurrent updates. This
+ * avoids a race condition where the sequence value could be updated
+ * by this command and then immediately moved backward by a
+ * concurrently running worker. Stopping the worker is safe even if it
+ * attempts to restart, as it will wait on the subscription lock
+ * already held by this ALTER SUBSCRIPTION command.
+ */
+ logicalrep_worker_stop(WORKERTYPE_SEQUENCESYNC, sub->oid, InvalidOid);
+
+ AlterSubSyncSequences(wrconn, sub->oid, sub->name, sub->runasowner);
}
PG_FINALLY();
{
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index d12bd31f09d..bead305156d 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -217,7 +217,7 @@ get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx, List *seqinfos)
+ List *missing_seqs_idx, List *seqinfos, char *subname)
{
StringInfo seqstr;
@@ -263,7 +263,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
ereport(ERROR,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("logical replication sequence synchronization failed for subscription \"%s\"",
- MySubscription->name));
+ subname));
}
/*
@@ -420,10 +420,9 @@ check_seq_privileges_and_drift(LogicalRepSequenceInfo *seqinfo,
*/
static CopySeqResult
copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
- bool update_lsn)
+ Oid subid, bool run_as_owner, bool update_lsn)
{
UserContext ucxt;
- bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
CopySeqResult result;
bool need_lsn_update = false;
@@ -467,8 +466,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
XLogRecPtr local_page_lsn;
/* Get the local page LSN for comparison with the remote value */
- (void) GetSubscriptionRelState(MySubscription->oid,
- RelationGetRelid(sequence_rel),
+ (void) GetSubscriptionRelState(subid, RelationGetRelid(sequence_rel),
&local_page_lsn);
need_lsn_update = (local_page_lsn != seqinfo->page_lsn);
@@ -480,7 +478,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
* cycle (update_lsn is true).
*/
if (seqinfo->relstate == SUBREL_STATE_INIT || need_lsn_update)
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ UpdateSubscriptionRelState(subid, seqoid, SUBREL_STATE_READY,
seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
@@ -499,7 +497,8 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
* Returns true/false if any sequences were actually copied.
*/
static bool
-copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
+copy_sequences(WalReceiverConn *conn, List *seqinfos, Oid subid, char *subname,
+ bool runasowner, bool update_lsn)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -528,11 +527,16 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
+ bool started_tx = false;
WalRcvExecResult *res;
TupleTableSlot *slot;
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
for (int idx = cur_batch_base_index; idx < n_seqinfos; idx++)
{
@@ -624,14 +628,15 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
&seqinfo, &seqidx, seqinfos);
if (sync_status == COPYSEQ_ALLOWED)
- sync_status = copy_sequence(seqinfo, sequence_rel, update_lsn);
+ sync_status = copy_sequence(seqinfo, sequence_rel, subid,
+ runasowner, update_lsn);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
"logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
- seqinfo->nspname, seqinfo->seqname, MySubscription->name);
+ seqinfo->nspname, seqinfo->seqname, subname);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -704,13 +709,17 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
elog(DEBUG1,
"logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
- MySubscription->name,
+ subname,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
- /* Commit this batch, and prepare for next batch */
- CommitTransactionCommand();
+ /*
+ * Commit this batch if started a transaction, and prepare for next
+ * batch.
+ */
+ if (started_tx)
+ CommitTransactionCommand();
if (batch_missing_count)
{
@@ -735,7 +744,7 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx, seqinfos);
+ missing_seqs_idx, seqinfos, subname);
return sequence_copied;
}
@@ -751,37 +760,28 @@ invalidate_syncing_sequence_infos(Datum arg, SysCacheIdentifier cacheid,
}
/*
- * Get the list of sequence information for the current subscription.
+ * Get the list of sequence information for the given subscription from
+ * catalog.
*
- * Return cached sequence states if valid; otherwise fetches them from the
- * catalog, caches the result, and return it.
+ * All entries in the returned list are allocated in the specified memory
+ * context.
*/
static List *
-fetch_sequence_infos(void)
+fetch_sequences_from_catalog(Oid subid, MemoryContext ctx)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
- Oid subid = MyLogicalRepWorker->subid;
- List *tmp_seqinfos = NIL;
+ List *seqinfos = NIL;
+ bool started_tx = false;
- if (sequence_infos_valid)
- return sequence_infos;
-
- /* Free the existing invalid cache entries */
- foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ if (!IsTransactionState())
{
- pfree(seqinfo->nspname);
- pfree(seqinfo->seqname);
- pfree(seqinfo);
+ StartTransactionCommand();
+ started_tx = true;
}
- list_free(sequence_infos);
- sequence_infos = NIL;
-
- StartTransactionCommand();
-
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
/* Scan for all sequences belonging to this subscription */
@@ -822,14 +822,14 @@ fetch_sequence_infos(void)
Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
- /* Cache the information in a permanent memory context */
- oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ /* Cache the information in the given memory context */
+ oldctx = MemoryContextSwitchTo(ctx);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
- tmp_seqinfos = lappend(tmp_seqinfos, seq);
+ seqinfos = lappend(seqinfos, seq);
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -839,10 +839,38 @@ fetch_sequence_infos(void)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- sequence_infos = tmp_seqinfos;
- sequence_infos_valid = true;
+ if (started_tx)
+ CommitTransactionCommand();
- CommitTransactionCommand();
+ return seqinfos;
+}
+
+/*
+ * Get the list of sequence information for the current subscription.
+ *
+ * Return cached sequence states if valid; otherwise fetches them from the
+ * catalog, caches the result, and return it.
+ */
+static List *
+fetch_sequence_infos(void)
+{
+ if (sequence_infos_valid)
+ return sequence_infos;
+
+ /* Free the existing invalid cache entries */
+ foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ {
+ pfree(seqinfo->nspname);
+ pfree(seqinfo->seqname);
+ pfree(seqinfo);
+ }
+
+ list_free(sequence_infos);
+ sequence_infos = NIL;
+
+ sequence_infos = fetch_sequences_from_catalog(MySubscription->oid,
+ CacheMemoryContext);
+ sequence_infos_valid = true;
return sequence_infos;
}
@@ -947,6 +975,9 @@ start_sequence_sync(void)
seqinfos = fetch_sequence_infos();
sequence_copied = copy_sequences(LogRepWorkerWalRcvConn, seqinfos,
+ MySubscription->oid,
+ MySubscription->name,
+ MySubscription->runasowner,
update_lsn);
MemoryContextReset(SequenceSyncContext);
@@ -1016,3 +1047,40 @@ SequenceSyncWorkerMain(Datum main_arg)
FinishSyncWorker();
}
+
+/*
+ * Wrapper for LogicalRepSyncSequences to synchronize all sequences of a
+ * subscription from outside the sequencesync worker.
+ */
+void
+AlterSubSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
+{
+ List *seqinfos;
+
+ Assert(!SequenceSyncContext);
+
+ /*
+ * Fetch sequences directly from the catalog rather than using the
+ * sequence cache, which is maintained only for the sequence sync
+ * worker.
+ */
+ seqinfos = fetch_sequences_from_catalog(subid, CurrentMemoryContext);
+
+ PG_TRY();
+ {
+ /*
+ * Use the current memory context for synchronization. Since this should
+ * be short-lived command context that will be cleaned up automatically,
+ * we can simply assign it as the synchronization context.
+ */
+ SequenceSyncContext = CurrentMemoryContext;
+
+ (void) copy_sequences(conn, seqinfos, subid, subname, runasowner, true);
+ }
+ PG_FINALLY();
+ {
+ SequenceSyncContext = NULL;
+ }
+ PG_END_TRY();
+}
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..73afd7853d0 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,6 +14,8 @@
#include <signal.h>
+#include "replication/walreceiver.h"
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
extern void ApplyWorkerMain(Datum main_arg);
@@ -31,4 +33,7 @@ extern void LogicalRepWorkersWakeupAtCommit(Oid subid);
extern void AtEOXact_LogicalRepWorkers(bool isCommit);
+extern void AlterSubSyncSequences(WalReceiverConn *conn, Oid subid,
+ char *subname, bool runasowner);
+
#endif /* LOGICALWORKER_H */
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index af190713b2b..8d25ac40ce0 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -176,4 +176,53 @@ $node_subscriber->wait_for_log(
qr/WARNING: ( [A-Z0-9]+:)? missing sequence on publisher \("public.regress_s4"\)/,
$log_offset);
+##########
+# ALTER SUBSCRIPTION ... REFRESH SEQUENCES synchronizes sequences online,
+# eliminating the need to launch a sequencesync worker.
+##########
+
+# Reduce max_logical_replication_workers to disallow sequence worker from running
+$node_subscriber->append_conf('postgresql.conf',
+ qq(max_logical_replication_workers = 0));
+$node_subscriber->restart;
+
+# Verify there is no logical replication apply worker running
+$result = $node_subscriber->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'logical replication apply worker'");
+
+is($result, '0', 'no logical replication worker is running');
+
+# Increment sequence on publisher
+$node_publisher->safe_psql('postgres',
+ qq(SELECT nextval('regress_s1');));
+
+# The command should fail due to missing sequence ('regress_s4')
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+like(
+ $stderr,
+ qr/WARNING: missing sequence on publisher \("public.regress_s4"\)/,
+ "output the wanring for the missing sequence regress_s4");
+
+like(
+ $stderr,
+ qr/ERROR: logical replication sequence synchronization failed for subscription \"regress_seq_sub\"/,
+ "the command failed due to the missing sequence regress_s4");
+
+# Refresh the publication to remove the missing sequence
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION;");
+
+# Sync the sequence regress_s1
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+# Get the current sequence value on subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ qq(SELECT last_value FROM regress_s1;));
+
+is($result, '201', 'sequence regress_s1 is synced now');
+
done_testing();
--
2.47.3
[application/octet-stream] v12-0001-Support-automatic-sequence-replication.patch (46.3K, 3-v12-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From 39164ea8103efa803d93af06ac2ae9be4c6037fe Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Thu, 26 Mar 2026 14:15:21 +1100
Subject: [PATCH v12 1/3] Support automatic sequence replication.
Currently, sequence values are synchronized from publisher to subscriber only
when the user manually runs ALTER SUBSCRIPTION ... REFRESH PUBLICATION (which
affects only newly subscribed sequences) or REFRESH SEQUENCES. The sequence sync
worker exits immediately after completing each synchronization round.
The primary use case for sequence replication is during upgrades, where it's
recommended that users ensure sequences are in sync by running REFRESH SEQUENCES
before finishing the upgrade. However, this command can be slow when
synchronizing a large number of sequences, potentially increasing downtime.
To address this, this commit makes the sequence sync worker long-lived,
continuously monitoring sequences and resynchronizing them when drift is
detected. The worker uses an adaptive sleep interval: it starts at 2 seconds,
doubles up to a maximum of 30 seconds when no drift is observed, and resets to
the minimum interval once drift is found.
With this change, most sequences are silently synchronized in the background,
eliminating the need to run REFRESH SEQUENCES for the majority of cases.
However, frequently updated sequences may still lag behind, requiring a final
REFRESH SEQUENCES before upgrade completion. Users can monitor progress by
checking whether sequence states transition from INIT to READY in
pg_subscription_rel.
The REFRESH SEQUENCES command is retained for this final synchronization step,
though it currently updates all sequence states to INIT, which has room for
improvement. A future patch will enhance this command to synchronize sequences
directly without launching a worker, reducing catalog overhead.
---
doc/src/sgml/logical-replication.sgml | 39 +-
doc/src/sgml/ref/alter_subscription.sgml | 5 -
src/backend/commands/sequence.c | 41 ++
.../replication/logical/sequencesync.c | 424 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 46 +-
src/backend/replication/logical/worker.c | 11 +
.../utils/activity/wait_event_names.txt | 1 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 81 +---
12 files changed, 437 insertions(+), 217 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 72c8d3d59bd..3865f617816 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1791,8 +1791,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1821,18 +1822,26 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
- advances them.
+ Subscriber sequence values can become out of sync as the publisher advances
+ them and the sequence synchronization worker has not yet caught up.
</para>
<para>
To detect this, compare the
<link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
on the subscriber with the <structfield>page_lsn</structfield> obtained
from the <link linkend="func-pg-get-sequence-data"><function>pg_get_sequence_data</function></link>
- function for the sequence on the publisher. Then run
+ function for the sequence on the publisher. If drift is detected, the user
+ can wait for the sequence synchronization worker to catch up (by
+ periodically checking whether the LSNs match).
+ </para>
+ <para>
+ Note that the sequence synchronization worker updates
+ <link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
+ periodically. To reduce the delay and force an immediate update, execute the
<link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> to
- re-synchronize if necessary.
+ <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> command,
+ which updates the page LSN immediately after synchronizing the sequence
+ value.
</para>
<warning>
<para>
@@ -2339,15 +2348,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index f215fb0e5a2..ee96e4823a3 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -202,11 +202,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 551667650ba..9a9fa6e25e6 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,47 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift. The caller must hold a lock on the sequence.
+ *
+ * Return false if the caller does not have sufficient privileges to access the
+ * sequence, true otherwise.
+ */
+bool
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Oid relid = RelationGetRelid(seqrel);
+
+ /* Confirm that the relation is a sequence and is locked */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+ Assert(CheckRelationLockedByMe(seqrel, AccessShareLock, true));
+
+ if (pg_class_aclcheck(relid, GetUserId(), ACL_SELECT) != ACLCHECK_OK)
+ {
+ *last_value = 0;
+ *is_called = false;
+ return false;
+ }
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+
+ return true;
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index a4fb6783ba9..18321d3b316 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,25 +19,32 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
- * The apply worker periodically scans pg_subscription_rel for sequences in
- * INIT state. When such sequences are found, it spawns a sequencesync worker
- * to handle synchronization.
- *
- * A single sequencesync worker is responsible for synchronizing all sequences.
- * It begins by retrieving the list of sequences that are flagged for
- * synchronization, i.e., those in the INIT state. These sequences are then
- * processed in batches, allowing multiple entries to be synchronized within a
- * single transaction. The worker fetches the current sequence values and page
- * LSNs from the remote publisher, updates the corresponding sequences on the
- * local subscriber, and finally marks each sequence as READY upon successful
+ * The apply worker periodically scans pg_subscription_rel for sequences.
+ * When sequences are found, it spawns a sequencesync worker to handle
* synchronization.
*
+ * A single sequencesync worker is responsible for synchronizing all sequences
+ * for a subscription. It begins by retrieving the list of sequences. These
+ * sequences are then processed in batches, allowing multiple entries to be
+ * synchronized within a single transaction. The worker fetches the current
+ * sequence values and page LSNs from the remote publisher and updates the
+ * corresponding sequences on the local subscriber. Sequences in the INIT
+ * state are unconditionally updated to the latest values from the publisher
+ * and then moved to the READY state. For sequences already in the READY
+ * state, the worker checks for drift and updates them only when needed.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+
+ * (synchronize)
+ * INIT --------------> READY ->-+
+ * ^ | (check-drift/synchronize)
+ * | |
+ * +--<---+
+ *
+ * Between cycles, the worker sleeps for SEQSYNC_MIN_SLEEP_MS. If no drift is
+ * observed in any sequence, the sleep interval doubles after each wake cycle
+ * up to SEQSYNC_MAX_SLEEP_MS. When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -60,6 +67,7 @@
#include "postmaster/interrupt.h"
#include "replication/logicalworker.h"
#include "replication/worker_internal.h"
+#include "storage/latch.h"
#include "storage/lwlock.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -71,31 +79,42 @@
#include "utils/pg_lsn.h"
#include "utils/syscache.h"
#include "utils/usercontext.h"
+#include "utils/wait_event.h"
#define REMOTE_SEQ_COL_COUNT 10
typedef enum CopySeqResult
{
COPYSEQ_SUCCESS,
+ COPYSEQ_ALLOWED,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
*
* Start a sequencesync worker if one is not already running. The active
* sequencesync worker will handle all pending sequence synchronization. If any
* sequences remain unsynchronized after it exits, a new worker can be started
* in the next iteration.
+ *
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -113,6 +132,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Quick exit if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -145,7 +177,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -172,7 +204,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -184,7 +216,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -195,7 +227,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -206,7 +238,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -230,7 +262,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -241,7 +274,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
int64 remote_min;
int64 remote_max;
bool remote_cycle;
- CopySeqResult result = COPYSEQ_SUCCESS;
+ CopySeqResult result = COPYSEQ_ALLOWED;
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
@@ -326,32 +359,77 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Check whether the user has required privileges on the sequence and
+ * whether the sequence has drifted.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+check_seq_privileges_and_drift(LogicalRepSequenceInfo *seqinfo,
+ Relation sequence_rel)
{
- UserContext ucxt;
AclResult aclresult;
+ Oid seqoid = seqinfo->localrelid;
+
+ /* Perform drift check if it's not the initial sync */
+ if (seqinfo->relstate == SUBREL_STATE_READY)
+ {
+ int64 local_last_value;
+ bool local_is_called;
+
+ /*
+ * Skip synchronization if the current user does not have sufficient
+ * privileges to read the sequence data.
+ */
+ if (!GetSequence(sequence_rel, &local_last_value, &local_is_called))
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ /*
+ * Skip synchronization if the sequence has not drifted from the
+ * publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Verify that the current user can update the sequence */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ return COPYSEQ_ALLOWED;
+}
+
+/*
+ * Apply remote sequence state to local sequence. For sequences in INIT state,
+ * always synchronize and then move them to READY state upon completion. For
+ * sequences already in READY state, synchronize only if drift is detected.
+ */
+static CopySeqResult
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ bool update_lsn)
+{
+ UserContext ucxt;
bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
+ CopySeqResult result;
+ bool need_lsn_update = false;
/*
* If the user did not opt to run as the owner of the subscription
* ('run_as_owner'), then copy the sequence as the owner of the sequence.
*/
if (!run_as_owner)
- SwitchToUntrustedUser(seqowner, &ucxt);
+ SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+ result = check_seq_privileges_and_drift(seqinfo, sequence_rel);
- if (aclresult != ACLCHECK_OK)
+ if (result != COPYSEQ_ALLOWED)
{
if (!run_as_owner)
RestoreUserContext(&ucxt);
- return COPYSEQ_INSUFFICIENT_PERM;
+ return result;
}
/*
@@ -368,20 +446,47 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
RestoreUserContext(&ucxt);
/*
- * Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * If LSN updates are requested, check whether the remote LSN differs from
+ * the locally stored value.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (update_lsn)
+ {
+ XLogRecPtr local_page_lsn;
+
+ /* Get the local page LSN for comparison with the remote value */
+ (void) GetSubscriptionRelState(MySubscription->oid,
+ RelationGetRelid(sequence_rel),
+ &local_page_lsn);
+
+ need_lsn_update = (local_page_lsn != seqinfo->page_lsn);
+ }
+
+ /*
+ * Update the catalog if either the sequence is in INIT state and needs to
+ * transition to READY, or the LSN has changed and this is an LSN update
+ * cycle (update_lsn is true).
+ */
+ if (seqinfo->relstate == SUBREL_STATE_INIT || need_lsn_update)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
/*
* Copy existing data of sequences from the publisher.
+ *
+ * Sequences in INIT state are always synchronized. Sequences in READY state are
+ * synchronized only when drift is detected.
+ *
+ * When 'update_lsn' is true, update the page LSN in pg_subscription_rel for any
+ * synchronized sequences whose LSN has changed. When false, the LSN is updated
+ * only for sequences transitioning from INIT to READY state.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -391,13 +496,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -407,6 +509,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -502,28 +605,28 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
- if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ &seqinfo, &seqidx, seqinfos);
+
+ if (sync_status == COPYSEQ_ALLOWED)
+ sync_status = copy_sequence(seqinfo, sequence_rel, update_lsn);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
+ sequence_copied = true;
break;
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -532,11 +635,11 @@ copy_sequences(WalReceiverConn *conn)
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -559,6 +662,13 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+ default:
+ elog(ERROR, "unrecognized sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -573,14 +683,15 @@ copy_sequences(WalReceiverConn *conn)
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
MySubscription->name,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -608,47 +719,50 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
@@ -667,18 +781,21 @@ LogicalRepSyncSequences(void)
continue;
}
+ relstate = subrel->srsubstate;
+
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
-
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -694,36 +811,16 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
-
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ return false;
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos, update_lsn);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -736,8 +833,117 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+ long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ TimestampTz next_lsn_update = GetCurrentTimestamp();
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence
+ * synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+ bool update_lsn;
+ TimestampTz now = GetCurrentTimestamp();
+
+ CHECK_FOR_INTERRUPTS();
+
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+ maybe_reread_subscription();
+
+ /*
+ * We avoid updating pg_subscription_rel's page LSN in every cycle
+ * to prevent excessive catalog invalidations, which would slow
+ * the apply worker that relies on this cache (see
+ * FetchRelationStates). Instead, updates occur every 30 seconds.
+ */
+ update_lsn = (now >= next_lsn_update);
+
+ /*
+ * Perform sequence synchronization under SequenceSyncContext and
+ * reset it each cycle to avoid manual memory management.
+ */
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
+ update_lsn);
+
+ MemoryContextReset(SequenceSyncContext);
+ MemoryContextSwitchTo(oldctx);
+
+ /*
+ * Adjust sleep interval based on sync activity. If sequences were
+ * copied, reset to the minimum interval to poll more frequently.
+ * Otherwise, exponentially back off up to the maximum interval.
+ */
+ if (sequence_copied)
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ else
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+
+ /* Refresh timestamp after potentially time-consuming sync work */
+ now = GetCurrentTimestamp();
+
+ /*
+ * Schedule the next LSN update. If we just performed an update,
+ * set the next update time to 30 seconds from now. Otherwise,
+ * ensure the sleep interval doesn't exceed the time remaining
+ * until the next update.
+ */
+ if (update_lsn)
+ next_lsn_update = TimestampTzPlusMilliseconds(now, SEQSYNC_MAX_SLEEP_MS);
+ else
+ sleep_ms = Min(sleep_ms, TimestampDifferenceMilliseconds(now, next_lsn_update));
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SEQSYNC_MAIN);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..3a0dc8669f9 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,7 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +191,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +205,22 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
- * since the same value can be used until the system table is invalidated.
+ * has_subtables and has_subsequences are declared as static, since the
+ * same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +230,28 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
+
/*
* Does the subscription have tables?
*
@@ -277,5 +277,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 27d398d576d..c8841831530 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -5102,6 +5102,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5117,6 +5120,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5135,6 +5142,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index 4aa864fe3c3..a91085e7723 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -61,6 +61,7 @@ IO_WORKER_MAIN "Waiting in main loop of IO Worker process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_SEQSYNC_MAIN "Waiting in main loop of logical replication sequence sync process."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot synchronization."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..fd4f69bdd1c 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern bool GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 745b7d9e969..4c7b6f4634b 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -285,7 +285,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..af190713b2b 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,18 +75,14 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
-# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
+# Create a new sequence 'regress_s2'
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.47.3
[application/octet-stream] v12-0002-Cache-sequence-information-in-the-sequence-sync-.patch (6.2K, 4-v12-0002-Cache-sequence-information-in-the-sequence-sync-.patch)
download | inline diff:
From 4c9c11ee6e07c799ef5cd2845a248d8a70e8d09e Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Thu, 26 Mar 2026 14:26:52 +1100
Subject: [PATCH v12 2/3] Cache sequence information in the sequence sync
worker.
Previously, the sequence sync worker would fetch sequence metadata from
the catalog each time it needed to synchronize sequences. This could be
inefficient when many sequences are involved, as the worker would need
to repeatedly open and scan pg_subscription_rel.
To improve this, introduce a cache for sequence information in the sequence sync
worker. The cache is populated on first use and kept across synchronization
cycles. It is invalidated when pg_subscription_rel is modified, ensuring that
changes to subscription relations are reflected promptly.
---
.../replication/logical/sequencesync.c | 93 +++++++++++++------
1 file changed, 66 insertions(+), 27 deletions(-)
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 18321d3b316..d12bd31f09d 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -99,6 +99,19 @@ typedef enum CopySeqResult
static MemoryContext SequenceSyncContext = NULL;
+/*
+ * Cached list of sequence information (LogicalRepSequenceInfo) for the current
+ * subscription. The cache is invalidated when pg_subscription_rel is modified.
+ *
+ * Note: To avoid the cost of searching for a specific sequence on relcache
+ * invalidation, we do not invalidate the cache immediately when a sequence is
+ * altered (e.g., renamed or moved to another namespace). Instead, we validate
+ * the sequence name and namespace when next attempting to sync it, at which
+ * point we verify the local sequence state.
+ */
+static List *sequence_infos = NIL;
+static bool sequence_infos_valid = false;
+
/*
* Apply worker determines whether a sequence sync worker is needed.
*
@@ -500,6 +513,9 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
+ if (seqinfos == NIL)
+ return false;
+
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -725,24 +741,44 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
}
/*
- * Identifies sequences that require synchronization and initiates the
- * synchronization process.
+ * Callback from syscache invalidation.
+ */
+static void
+invalidate_syncing_sequence_infos(Datum arg, SysCacheIdentifier cacheid,
+ uint32 hashvalue)
+{
+ sequence_infos_valid = false;
+}
+
+/*
+ * Get the list of sequence information for the current subscription.
*
- * Returns true if sequences have been updated.
+ * Return cached sequence states if valid; otherwise fetches them from the
+ * catalog, caches the result, and return it.
*/
-static bool
-LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
+static List *
+fetch_sequence_infos(void)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- bool sequence_copied = false;
- List *seqinfos = NIL;
- MemoryContext oldctx;
+ List *tmp_seqinfos = NIL;
+
+ if (sequence_infos_valid)
+ return sequence_infos;
- Assert(SequenceSyncContext);
+ /* Free the existing invalid cache entries */
+ foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ {
+ pfree(seqinfo->nspname);
+ pfree(seqinfo->seqname);
+ pfree(seqinfo);
+ }
+
+ list_free(sequence_infos);
+ sequence_infos = NIL;
StartTransactionCommand();
@@ -763,6 +799,7 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
char relstate;
+ MemoryContext oldctx;
CHECK_FOR_INTERRUPTS();
@@ -785,17 +822,14 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
- /*
- * Worker needs to process sequences across transaction boundary, so
- * allocate them under SequenceSyncContext.
- */
- oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+ /* Cache the information in a permanent memory context */
+ oldctx = MemoryContextSwitchTo(CacheMemoryContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
- seqinfos = lappend(seqinfos, seq);
+ tmp_seqinfos = lappend(tmp_seqinfos, seq);
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -805,18 +839,12 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- CommitTransactionCommand();
-
- /*
- * Exit early if no catalog entries found, likely due to concurrent drops.
- */
- if (!seqinfos)
- return false;
+ sequence_infos = tmp_seqinfos;
+ sequence_infos_valid = true;
- /* Process sequences */
- sequence_copied = copy_sequences(conn, seqinfos, update_lsn);
+ CommitTransactionCommand();
- return sequence_copied;
+ return sequence_infos;
}
/*
@@ -831,6 +859,14 @@ start_sequence_sync(void)
{
Assert(am_sequencesync_worker());
+ /*
+ * Setup callback for syscache so that we know when something changes in
+ * the subscription relation state.
+ */
+ CacheRegisterSyscacheCallback(SUBSCRIPTIONRELMAP,
+ invalidate_syncing_sequence_infos,
+ (Datum) 0);
+
PG_TRY();
{
char *err;
@@ -876,6 +912,7 @@ start_sequence_sync(void)
bool sequence_copied = false;
MemoryContext oldctx;
bool update_lsn;
+ List *seqinfos;
TimestampTz now = GetCurrentTimestamp();
CHECK_FOR_INTERRUPTS();
@@ -907,8 +944,10 @@ start_sequence_sync(void)
/*
* Synchronize all sequences (both READY and INIT states).
*/
- sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
- update_lsn);
+ seqinfos = fetch_sequence_infos();
+
+ sequence_copied = copy_sequences(LogRepWorkerWalRcvConn, seqinfos,
+ update_lsn);
MemoryContextReset(SequenceSyncContext);
MemoryContextSwitchTo(oldctx);
--
2.47.3
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-09 03:13 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 07:13 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-13 11:35 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-16 06:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-26 04:25 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
@ 2026-04-15 10:41 ` Ajin Cherian <[email protected]>
0 siblings, 0 replies; 58+ messages in thread
From: Ajin Cherian @ 2026-04-15 10:41 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Hayato Kuroda (Fujitsu) <[email protected]>; shveta malik <[email protected]>; Amit Kapila <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
Rebasing patch to keep cfbot happy.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v12-0001-Support-automatic-sequence-replication.patch (46.4K, 2-v12-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From 1c832054b4c8bb2286784667eedeb25c3d18123b Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 15 Apr 2026 20:22:43 +1000
Subject: [PATCH v12 1/3] Support automatic sequence replication.
Currently, sequence values are synchronized from publisher to subscriber only
when the user manually runs ALTER SUBSCRIPTION ... REFRESH PUBLICATION (which
affects only newly subscribed sequences) or REFRESH SEQUENCES. The sequence sync
worker exits immediately after completing each synchronization round.
The primary use case for sequence replication is during upgrades, where it's
recommended that users ensure sequences are in sync by running REFRESH SEQUENCES
before finishing the upgrade. However, this command can be slow when
synchronizing a large number of sequences, potentially increasing downtime.
To address this, this commit makes the sequence sync worker long-lived,
continuously monitoring sequences and resynchronizing them when drift is
detected. The worker uses an adaptive sleep interval: it starts at 2 seconds,
doubles up to a maximum of 30 seconds when no drift is observed, and resets to
the minimum interval once drift is found.
With this change, most sequences are silently synchronized in the background,
eliminating the need to run REFRESH SEQUENCES for the majority of cases.
However, frequently updated sequences may still lag behind, requiring a final
REFRESH SEQUENCES before upgrade completion. Users can monitor progress by
checking whether sequence states transition from INIT to READY in
pg_subscription_rel.
The REFRESH SEQUENCES command is retained for this final synchronization step,
though it currently updates all sequence states to INIT, which has room for
improvement. A future patch will enhance this command to synchronize sequences
directly without launching a worker, reducing catalog overhead.
---
doc/src/sgml/logical-replication.sgml | 39 +-
doc/src/sgml/ref/alter_subscription.sgml | 5 -
src/backend/commands/sequence.c | 41 ++
.../replication/logical/sequencesync.c | 424 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 46 +-
src/backend/replication/logical/worker.c | 11 +
.../utils/activity/wait_event_names.txt | 1 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 81 +---
12 files changed, 437 insertions(+), 217 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 23b268273b9..430d2581699 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1791,8 +1791,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1821,18 +1822,26 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
- advances them.
+ Subscriber sequence values can become out of sync as the publisher advances
+ them and the sequence synchronization worker has not yet caught up.
</para>
<para>
To detect this, compare the
<link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
on the subscriber with the <structfield>page_lsn</structfield> obtained
from the <link linkend="func-pg-get-sequence-data"><function>pg_get_sequence_data</function></link>
- function for the sequence on the publisher. Then run
+ function for the sequence on the publisher. If drift is detected, the user
+ can wait for the sequence synchronization worker to catch up (by
+ periodically checking whether the LSNs match).
+ </para>
+ <para>
+ Note that the sequence synchronization worker updates
+ <link linkend="catalog-pg-subscription-rel">pg_subscription_rel</link>.<structfield>srsublsn</structfield>
+ periodically. To reduce the delay and force an immediate update, execute the
<link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> to
- re-synchronize if necessary.
+ <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link> command,
+ which updates the page LSN immediately after synchronizing the sequence
+ value.
</para>
<warning>
<para>
@@ -2339,15 +2348,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index f215fb0e5a2..ee96e4823a3 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -202,11 +202,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 551667650ba..9a9fa6e25e6 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,47 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift. The caller must hold a lock on the sequence.
+ *
+ * Return false if the caller does not have sufficient privileges to access the
+ * sequence, true otherwise.
+ */
+bool
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Oid relid = RelationGetRelid(seqrel);
+
+ /* Confirm that the relation is a sequence and is locked */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+ Assert(CheckRelationLockedByMe(seqrel, AccessShareLock, true));
+
+ if (pg_class_aclcheck(relid, GetUserId(), ACL_SELECT) != ACLCHECK_OK)
+ {
+ *last_value = 0;
+ *is_called = false;
+ return false;
+ }
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+
+ return true;
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index ec7e76abf93..22e32dbba57 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,25 +19,32 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
- * The apply worker periodically scans pg_subscription_rel for sequences in
- * INIT state. When such sequences are found, it spawns a sequencesync worker
- * to handle synchronization.
- *
- * A single sequencesync worker is responsible for synchronizing all sequences.
- * It begins by retrieving the list of sequences that are flagged for
- * synchronization, i.e., those in the INIT state. These sequences are then
- * processed in batches, allowing multiple entries to be synchronized within a
- * single transaction. The worker fetches the current sequence values and page
- * LSNs from the remote publisher, updates the corresponding sequences on the
- * local subscriber, and finally marks each sequence as READY upon successful
+ * The apply worker periodically scans pg_subscription_rel for sequences.
+ * When sequences are found, it spawns a sequencesync worker to handle
* synchronization.
*
+ * A single sequencesync worker is responsible for synchronizing all sequences
+ * for a subscription. It begins by retrieving the list of sequences. These
+ * sequences are then processed in batches, allowing multiple entries to be
+ * synchronized within a single transaction. The worker fetches the current
+ * sequence values and page LSNs from the remote publisher and updates the
+ * corresponding sequences on the local subscriber. Sequences in the INIT
+ * state are unconditionally updated to the latest values from the publisher
+ * and then moved to the READY state. For sequences already in the READY
+ * state, the worker checks for drift and updates them only when needed.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+
+ * (synchronize)
+ * INIT --------------> READY ->-+
+ * ^ | (check-drift/synchronize)
+ * | |
+ * +--<---+
+ *
+ * Between cycles, the worker sleeps for SEQSYNC_MIN_SLEEP_MS. If no drift is
+ * observed in any sequence, the sleep interval doubles after each wake cycle
+ * up to SEQSYNC_MAX_SLEEP_MS. When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -60,6 +67,7 @@
#include "postmaster/interrupt.h"
#include "replication/logicalworker.h"
#include "replication/worker_internal.h"
+#include "storage/latch.h"
#include "storage/lwlock.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -71,31 +79,42 @@
#include "utils/pg_lsn.h"
#include "utils/syscache.h"
#include "utils/usercontext.h"
+#include "utils/wait_event.h"
#define REMOTE_SEQ_COL_COUNT 10
typedef enum CopySeqResult
{
COPYSEQ_SUCCESS,
+ COPYSEQ_ALLOWED,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
*
* Start a sequencesync worker if one is not already running. The active
* sequencesync worker will handle all pending sequence synchronization. If any
* sequences remain unsynchronized after it exits, a new worker can be started
* in the next iteration.
+ *
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -113,6 +132,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Quick exit if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -145,7 +177,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -172,7 +204,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfoData seqstr;
@@ -184,7 +216,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, &seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, &seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -195,7 +227,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, &seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, &seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -206,7 +238,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, &seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, &seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -230,7 +262,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -241,7 +274,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
int64 remote_min;
int64 remote_max;
bool remote_cycle;
- CopySeqResult result = COPYSEQ_SUCCESS;
+ CopySeqResult result = COPYSEQ_ALLOWED;
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
@@ -326,32 +359,77 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Check whether the user has required privileges on the sequence and
+ * whether the sequence has drifted.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+check_seq_privileges_and_drift(LogicalRepSequenceInfo *seqinfo,
+ Relation sequence_rel)
{
- UserContext ucxt;
AclResult aclresult;
+ Oid seqoid = seqinfo->localrelid;
+
+ /* Perform drift check if it's not the initial sync */
+ if (seqinfo->relstate == SUBREL_STATE_READY)
+ {
+ int64 local_last_value;
+ bool local_is_called;
+
+ /*
+ * Skip synchronization if the current user does not have sufficient
+ * privileges to read the sequence data.
+ */
+ if (!GetSequence(sequence_rel, &local_last_value, &local_is_called))
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ /*
+ * Skip synchronization if the sequence has not drifted from the
+ * publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Verify that the current user can update the sequence */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ return COPYSEQ_ALLOWED;
+}
+
+/*
+ * Apply remote sequence state to local sequence. For sequences in INIT state,
+ * always synchronize and then move them to READY state upon completion. For
+ * sequences already in READY state, synchronize only if drift is detected.
+ */
+static CopySeqResult
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ bool update_lsn)
+{
+ UserContext ucxt;
bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
+ CopySeqResult result;
+ bool need_lsn_update = false;
/*
* If the user did not opt to run as the owner of the subscription
* ('run_as_owner'), then copy the sequence as the owner of the sequence.
*/
if (!run_as_owner)
- SwitchToUntrustedUser(seqowner, &ucxt);
+ SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+ result = check_seq_privileges_and_drift(seqinfo, sequence_rel);
- if (aclresult != ACLCHECK_OK)
+ if (result != COPYSEQ_ALLOWED)
{
if (!run_as_owner)
RestoreUserContext(&ucxt);
- return COPYSEQ_INSUFFICIENT_PERM;
+ return result;
}
/*
@@ -368,26 +446,54 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
RestoreUserContext(&ucxt);
/*
- * Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * If LSN updates are requested, check whether the remote LSN differs from
+ * the locally stored value.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (update_lsn)
+ {
+ XLogRecPtr local_page_lsn;
+
+ /* Get the local page LSN for comparison with the remote value */
+ (void) GetSubscriptionRelState(MySubscription->oid,
+ RelationGetRelid(sequence_rel),
+ &local_page_lsn);
+
+ need_lsn_update = (local_page_lsn != seqinfo->page_lsn);
+ }
+
+ /*
+ * Update the catalog if either the sequence is in INIT state and needs to
+ * transition to READY, or the LSN has changed and this is an LSN update
+ * cycle (update_lsn is true).
+ */
+ if (seqinfo->relstate == SUBREL_STATE_INIT || need_lsn_update)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
/*
* Copy existing data of sequences from the publisher.
+ *
+ * Sequences in INIT state are always synchronized. Sequences in READY state are
+ * synchronized only when drift is detected.
+ *
+ * When 'update_lsn' is true, update the page LSN in pg_subscription_rel for any
+ * synchronized sequences whose LSN has changed. When false, the LSN is updated
+ * only for sequences transitioning from INIT to READY state.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
List *mismatched_seqs_idx = NIL;
List *missing_seqs_idx = NIL;
List *insuffperm_seqs_idx = NIL;
+ bool sequence_copied = false;
StringInfoData seqstr;
StringInfoData cmd;
MemoryContext oldctx;
@@ -397,10 +503,6 @@ copy_sequences(WalReceiverConn *conn)
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -410,6 +512,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -505,28 +608,28 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
- if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ &seqinfo, &seqidx, seqinfos);
+
+ if (sync_status == COPYSEQ_ALLOWED)
+ sync_status = copy_sequence(seqinfo, sequence_rel, update_lsn);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
+ sequence_copied = true;
break;
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -535,11 +638,11 @@ copy_sequences(WalReceiverConn *conn)
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -562,6 +665,13 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+ default:
+ elog(ERROR, "unrecognized sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -576,14 +686,15 @@ copy_sequences(WalReceiverConn *conn)
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
MySubscription->name,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -611,47 +722,50 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
@@ -670,18 +784,21 @@ LogicalRepSyncSequences(void)
continue;
}
+ relstate = subrel->srsubstate;
+
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
-
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -697,36 +814,16 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
-
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ return false;
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos, update_lsn);
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -739,8 +836,117 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+ long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ TimestampTz next_lsn_update = GetCurrentTimestamp();
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence
+ * synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+ bool update_lsn;
+ TimestampTz now = GetCurrentTimestamp();
+
+ CHECK_FOR_INTERRUPTS();
+
+ if (ConfigReloadPending)
+ {
+ ConfigReloadPending = false;
+ ProcessConfigFile(PGC_SIGHUP);
+ }
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+ maybe_reread_subscription();
+
+ /*
+ * We avoid updating pg_subscription_rel's page LSN in every cycle
+ * to prevent excessive catalog invalidations, which would slow
+ * the apply worker that relies on this cache (see
+ * FetchRelationStates). Instead, updates occur every 30 seconds.
+ */
+ update_lsn = (now >= next_lsn_update);
+
+ /*
+ * Perform sequence synchronization under SequenceSyncContext and
+ * reset it each cycle to avoid manual memory management.
+ */
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
+ update_lsn);
+
+ MemoryContextReset(SequenceSyncContext);
+ MemoryContextSwitchTo(oldctx);
+
+ /*
+ * Adjust sleep interval based on sync activity. If sequences were
+ * copied, reset to the minimum interval to poll more frequently.
+ * Otherwise, exponentially back off up to the maximum interval.
+ */
+ if (sequence_copied)
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ else
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+
+ /* Refresh timestamp after potentially time-consuming sync work */
+ now = GetCurrentTimestamp();
+
+ /*
+ * Schedule the next LSN update. If we just performed an update,
+ * set the next update time to 30 seconds from now. Otherwise,
+ * ensure the sleep interval doesn't exceed the time remaining
+ * until the next update.
+ */
+ if (update_lsn)
+ next_lsn_update = TimestampTzPlusMilliseconds(now, SEQSYNC_MAX_SLEEP_MS);
+ else
+ sleep_ms = Min(sleep_ms, TimestampDifferenceMilliseconds(now, next_lsn_update));
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SEQSYNC_MAIN);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..3a0dc8669f9 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,7 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +191,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +205,22 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
- * since the same value can be used until the system table is invalidated.
+ * has_subtables and has_subsequences are declared as static, since the
+ * same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +230,28 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
+
/*
* Does the subscription have tables?
*
@@ -277,5 +277,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index b38170f0fbe..510f7c2485b 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -5103,6 +5103,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5118,6 +5121,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5136,6 +5143,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index 560659f9568..c43db0c6198 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -61,6 +61,7 @@ IO_WORKER_MAIN "Waiting in main loop of IO Worker process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_SEQSYNC_MAIN "Waiting in main loop of logical replication sequence sync process."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot synchronization."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..fd4f69bdd1c 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern bool GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 745b7d9e969..4c7b6f4634b 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -285,7 +285,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..af190713b2b 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,18 +75,14 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
-# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
+# Create a new sequence 'regress_s2'
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.47.3
[application/octet-stream] v12-0002-Cache-sequence-information-in-the-sequence-sync-.patch (6.2K, 3-v12-0002-Cache-sequence-information-in-the-sequence-sync-.patch)
download | inline diff:
From 914b734175f65ad21b44d620627f5ae47c47d937 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 15 Apr 2026 20:23:55 +1000
Subject: [PATCH v12 2/3] Cache sequence information in the sequence sync
worker.
Previously, the sequence sync worker would fetch sequence metadata from
the catalog each time it needed to synchronize sequences. This could be
inefficient when many sequences are involved, as the worker would need
to repeatedly open and scan pg_subscription_rel.
To improve this, introduce a cache for sequence information in the sequence sync
worker. The cache is populated on first use and kept across synchronization
cycles. It is invalidated when pg_subscription_rel is modified, ensuring that
changes to subscription relations are reflected promptly.
---
.../replication/logical/sequencesync.c | 93 +++++++++++++------
1 file changed, 66 insertions(+), 27 deletions(-)
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 22e32dbba57..352fd0e742b 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -99,6 +99,19 @@ typedef enum CopySeqResult
static MemoryContext SequenceSyncContext = NULL;
+/*
+ * Cached list of sequence information (LogicalRepSequenceInfo) for the current
+ * subscription. The cache is invalidated when pg_subscription_rel is modified.
+ *
+ * Note: To avoid the cost of searching for a specific sequence on relcache
+ * invalidation, we do not invalidate the cache immediately when a sequence is
+ * altered (e.g., renamed or moved to another namespace). Instead, we validate
+ * the sequence name and namespace when next attempting to sync it, at which
+ * point we verify the local sequence state.
+ */
+static List *sequence_infos = NIL;
+static bool sequence_infos_valid = false;
+
/*
* Apply worker determines whether a sequence sync worker is needed.
*
@@ -503,6 +516,9 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
+ if (seqinfos == NIL)
+ return false;
+
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -728,24 +744,44 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
}
/*
- * Identifies sequences that require synchronization and initiates the
- * synchronization process.
+ * Callback from syscache invalidation.
+ */
+static void
+invalidate_syncing_sequence_infos(Datum arg, SysCacheIdentifier cacheid,
+ uint32 hashvalue)
+{
+ sequence_infos_valid = false;
+}
+
+/*
+ * Get the list of sequence information for the current subscription.
*
- * Returns true if sequences have been updated.
+ * Return cached sequence states if valid; otherwise fetches them from the
+ * catalog, caches the result, and return it.
*/
-static bool
-LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
+static List *
+fetch_sequence_infos(void)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- bool sequence_copied = false;
- List *seqinfos = NIL;
- MemoryContext oldctx;
+ List *tmp_seqinfos = NIL;
- Assert(SequenceSyncContext);
+ if (sequence_infos_valid)
+ return sequence_infos;
+
+ /* Free the existing invalid cache entries */
+ foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ {
+ pfree(seqinfo->nspname);
+ pfree(seqinfo->seqname);
+ pfree(seqinfo);
+ }
+
+ list_free(sequence_infos);
+ sequence_infos = NIL;
StartTransactionCommand();
@@ -766,6 +802,7 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
char relstate;
+ MemoryContext oldctx;
CHECK_FOR_INTERRUPTS();
@@ -788,17 +825,14 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
- /*
- * Worker needs to process sequences across transaction boundary, so
- * allocate them under SequenceSyncContext.
- */
- oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+ /* Cache the information in a permanent memory context */
+ oldctx = MemoryContextSwitchTo(CacheMemoryContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
- seqinfos = lappend(seqinfos, seq);
+ tmp_seqinfos = lappend(tmp_seqinfos, seq);
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -808,18 +842,12 @@ LogicalRepSyncSequences(WalReceiverConn *conn, bool update_lsn)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- CommitTransactionCommand();
+ sequence_infos = tmp_seqinfos;
+ sequence_infos_valid = true;
- /*
- * Exit early if no catalog entries found, likely due to concurrent drops.
- */
- if (!seqinfos)
- return false;
-
- /* Process sequences */
- sequence_copied = copy_sequences(conn, seqinfos, update_lsn);
+ CommitTransactionCommand();
- return sequence_copied;
+ return sequence_infos;
}
/*
@@ -834,6 +862,14 @@ start_sequence_sync(void)
{
Assert(am_sequencesync_worker());
+ /*
+ * Setup callback for syscache so that we know when something changes in
+ * the subscription relation state.
+ */
+ CacheRegisterSyscacheCallback(SUBSCRIPTIONRELMAP,
+ invalidate_syncing_sequence_infos,
+ (Datum) 0);
+
PG_TRY();
{
char *err;
@@ -879,6 +915,7 @@ start_sequence_sync(void)
bool sequence_copied = false;
MemoryContext oldctx;
bool update_lsn;
+ List *seqinfos;
TimestampTz now = GetCurrentTimestamp();
CHECK_FOR_INTERRUPTS();
@@ -910,8 +947,10 @@ start_sequence_sync(void)
/*
* Synchronize all sequences (both READY and INIT states).
*/
- sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
- update_lsn);
+ seqinfos = fetch_sequence_infos();
+
+ sequence_copied = copy_sequences(LogRepWorkerWalRcvConn, seqinfos,
+ update_lsn);
MemoryContextReset(SequenceSyncContext);
MemoryContextSwitchTo(oldctx);
--
2.47.3
[application/octet-stream] v12-0003-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch (15.7K, 4-v12-0003-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch)
download | inline diff:
From f1b314e2341c35115eebcc407060d80633c95665 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Wed, 15 Apr 2026 20:26:01 +1000
Subject: [PATCH v12 3/3] Synchronize sequences directly in REFRESH SEQUENCES
command.
The ALTER SUBSCRIPTION ... REFRESH SEQUENCES command currently sets all
sequence states in pg_subscription_rel to INIT and relies on the sequence sync
worker to perform the actual synchronization and update states to READY.
With the recent change making the sequence sync worker long-lived, most
sequences are now synchronized in the background, reducing the need for
REFRESH SEQUENCES. However, the command remains necessary for sequences that
haven't been synchronized.
This commit enhances REFRESH SEQUENCES to synchronize sequences directly within
the command itself, eliminating the overhead of launching a worker and updating
catalog entries unnecessarily.
---
doc/src/sgml/logical-replication.sgml | 5 +-
src/backend/commands/subscriptioncmds.c | 27 ++--
.../replication/logical/sequencesync.c | 148 +++++++++++++-----
src/include/replication/logicalworker.h | 5 +
src/test/subscription/t/036_sequences.pl | 49 ++++++
5 files changed, 176 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 430d2581699..99746c410cb 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1791,8 +1791,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands. The worker will
- remain running for the life of the subscription, periodically
+ after executing <command>CREATE SUBSCRIPTION</command> or
+ <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command> command. The
+ worker will remain running for the life of the subscription, periodically
synchronizing all published sequences.
</para>
<para>
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 3a30d8ddb1f..91b3d05aaf8 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1288,25 +1288,20 @@ AlterSubscription_refresh_seq(Subscription *sub)
PG_TRY();
{
- List *subrel_states;
-
check_publications_origin_sequences(wrconn, sub->publications, true,
sub->origin, NULL, 0, sub->name);
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
+ /*
+ * Stop the sequencesync worker to prevent concurrent updates. This
+ * avoids a race condition where the sequence value could be updated
+ * by this command and then immediately moved backward by a
+ * concurrently running worker. Stopping the worker is safe even if it
+ * attempts to restart, as it will wait on the subscription lock
+ * already held by this ALTER SUBSCRIPTION command.
+ */
+ logicalrep_worker_stop(WORKERTYPE_SEQUENCESYNC, sub->oid, InvalidOid);
+
+ AlterSubSyncSequences(wrconn, sub->oid, sub->name, sub->runasowner);
}
PG_FINALLY();
{
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 352fd0e742b..e1fc556dd51 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -217,7 +217,7 @@ get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx, List *seqinfos)
+ List *missing_seqs_idx, List *seqinfos, char *subname)
{
StringInfoData seqstr;
@@ -263,7 +263,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
ereport(ERROR,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("logical replication sequence synchronization failed for subscription \"%s\"",
- MySubscription->name));
+ subname));
}
/*
@@ -420,10 +420,9 @@ check_seq_privileges_and_drift(LogicalRepSequenceInfo *seqinfo,
*/
static CopySeqResult
copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
- bool update_lsn)
+ Oid subid, bool run_as_owner, bool update_lsn)
{
UserContext ucxt;
- bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
CopySeqResult result;
bool need_lsn_update = false;
@@ -467,8 +466,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
XLogRecPtr local_page_lsn;
/* Get the local page LSN for comparison with the remote value */
- (void) GetSubscriptionRelState(MySubscription->oid,
- RelationGetRelid(sequence_rel),
+ (void) GetSubscriptionRelState(subid, RelationGetRelid(sequence_rel),
&local_page_lsn);
need_lsn_update = (local_page_lsn != seqinfo->page_lsn);
@@ -480,7 +478,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
* cycle (update_lsn is true).
*/
if (seqinfo->relstate == SUBREL_STATE_INIT || need_lsn_update)
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ UpdateSubscriptionRelState(subid, seqoid, SUBREL_STATE_READY,
seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
@@ -499,7 +497,8 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
* Returns true/false if any sequences were actually copied.
*/
static bool
-copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
+copy_sequences(WalReceiverConn *conn, List *seqinfos, Oid subid, char *subname,
+ bool runasowner, bool update_lsn)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -531,11 +530,16 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
+ bool started_tx = false;
WalRcvExecResult *res;
TupleTableSlot *slot;
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
for (int idx = cur_batch_base_index; idx < n_seqinfos; idx++)
{
@@ -627,14 +631,15 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
&seqinfo, &seqidx, seqinfos);
if (sync_status == COPYSEQ_ALLOWED)
- sync_status = copy_sequence(seqinfo, sequence_rel, update_lsn);
+ sync_status = copy_sequence(seqinfo, sequence_rel, subid,
+ runasowner, update_lsn);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
"logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
- seqinfo->nspname, seqinfo->seqname, MySubscription->name);
+ seqinfo->nspname, seqinfo->seqname, subname);
batch_succeeded_count++;
sequence_copied = true;
break;
@@ -707,13 +712,17 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
elog(DEBUG1,
"logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
- MySubscription->name,
+ subname,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
- /* Commit this batch, and prepare for next batch */
- CommitTransactionCommand();
+ /*
+ * Commit this batch if started a transaction, and prepare for next
+ * batch.
+ */
+ if (started_tx)
+ CommitTransactionCommand();
if (batch_missing_count)
{
@@ -738,7 +747,7 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos, bool update_lsn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx, seqinfos);
+ missing_seqs_idx, seqinfos, subname);
return sequence_copied;
}
@@ -754,37 +763,28 @@ invalidate_syncing_sequence_infos(Datum arg, SysCacheIdentifier cacheid,
}
/*
- * Get the list of sequence information for the current subscription.
+ * Get the list of sequence information for the given subscription from
+ * catalog.
*
- * Return cached sequence states if valid; otherwise fetches them from the
- * catalog, caches the result, and return it.
+ * All entries in the returned list are allocated in the specified memory
+ * context.
*/
static List *
-fetch_sequence_infos(void)
+fetch_sequences_from_catalog(Oid subid, MemoryContext ctx)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
- Oid subid = MyLogicalRepWorker->subid;
- List *tmp_seqinfos = NIL;
+ List *seqinfos = NIL;
+ bool started_tx = false;
- if (sequence_infos_valid)
- return sequence_infos;
-
- /* Free the existing invalid cache entries */
- foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ if (!IsTransactionState())
{
- pfree(seqinfo->nspname);
- pfree(seqinfo->seqname);
- pfree(seqinfo);
+ StartTransactionCommand();
+ started_tx = true;
}
- list_free(sequence_infos);
- sequence_infos = NIL;
-
- StartTransactionCommand();
-
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
/* Scan for all sequences belonging to this subscription */
@@ -825,14 +825,14 @@ fetch_sequence_infos(void)
Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
- /* Cache the information in a permanent memory context */
- oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ /* Cache the information in the given memory context */
+ oldctx = MemoryContextSwitchTo(ctx);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
seq->relstate = relstate;
- tmp_seqinfos = lappend(tmp_seqinfos, seq);
+ seqinfos = lappend(seqinfos, seq);
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -842,10 +842,38 @@ fetch_sequence_infos(void)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- sequence_infos = tmp_seqinfos;
- sequence_infos_valid = true;
+ if (started_tx)
+ CommitTransactionCommand();
- CommitTransactionCommand();
+ return seqinfos;
+}
+
+/*
+ * Get the list of sequence information for the current subscription.
+ *
+ * Return cached sequence states if valid; otherwise fetches them from the
+ * catalog, caches the result, and return it.
+ */
+static List *
+fetch_sequence_infos(void)
+{
+ if (sequence_infos_valid)
+ return sequence_infos;
+
+ /* Free the existing invalid cache entries */
+ foreach_ptr(LogicalRepSequenceInfo, seqinfo, sequence_infos)
+ {
+ pfree(seqinfo->nspname);
+ pfree(seqinfo->seqname);
+ pfree(seqinfo);
+ }
+
+ list_free(sequence_infos);
+ sequence_infos = NIL;
+
+ sequence_infos = fetch_sequences_from_catalog(MySubscription->oid,
+ CacheMemoryContext);
+ sequence_infos_valid = true;
return sequence_infos;
}
@@ -950,6 +978,9 @@ start_sequence_sync(void)
seqinfos = fetch_sequence_infos();
sequence_copied = copy_sequences(LogRepWorkerWalRcvConn, seqinfos,
+ MySubscription->oid,
+ MySubscription->name,
+ MySubscription->runasowner,
update_lsn);
MemoryContextReset(SequenceSyncContext);
@@ -1019,3 +1050,40 @@ SequenceSyncWorkerMain(Datum main_arg)
FinishSyncWorker();
}
+
+/*
+ * Wrapper for LogicalRepSyncSequences to synchronize all sequences of a
+ * subscription from outside the sequencesync worker.
+ */
+void
+AlterSubSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
+{
+ List *seqinfos;
+
+ Assert(!SequenceSyncContext);
+
+ /*
+ * Fetch sequences directly from the catalog rather than using the
+ * sequence cache, which is maintained only for the sequence sync
+ * worker.
+ */
+ seqinfos = fetch_sequences_from_catalog(subid, CurrentMemoryContext);
+
+ PG_TRY();
+ {
+ /*
+ * Use the current memory context for synchronization. Since this should
+ * be short-lived command context that will be cleaned up automatically,
+ * we can simply assign it as the synchronization context.
+ */
+ SequenceSyncContext = CurrentMemoryContext;
+
+ (void) copy_sequences(conn, seqinfos, subid, subname, runasowner, true);
+ }
+ PG_FINALLY();
+ {
+ SequenceSyncContext = NULL;
+ }
+ PG_END_TRY();
+}
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..73afd7853d0 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,6 +14,8 @@
#include <signal.h>
+#include "replication/walreceiver.h"
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
extern void ApplyWorkerMain(Datum main_arg);
@@ -31,4 +33,7 @@ extern void LogicalRepWorkersWakeupAtCommit(Oid subid);
extern void AtEOXact_LogicalRepWorkers(bool isCommit);
+extern void AlterSubSyncSequences(WalReceiverConn *conn, Oid subid,
+ char *subname, bool runasowner);
+
#endif /* LOGICALWORKER_H */
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index af190713b2b..8d25ac40ce0 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -176,4 +176,53 @@ $node_subscriber->wait_for_log(
qr/WARNING: ( [A-Z0-9]+:)? missing sequence on publisher \("public.regress_s4"\)/,
$log_offset);
+##########
+# ALTER SUBSCRIPTION ... REFRESH SEQUENCES synchronizes sequences online,
+# eliminating the need to launch a sequencesync worker.
+##########
+
+# Reduce max_logical_replication_workers to disallow sequence worker from running
+$node_subscriber->append_conf('postgresql.conf',
+ qq(max_logical_replication_workers = 0));
+$node_subscriber->restart;
+
+# Verify there is no logical replication apply worker running
+$result = $node_subscriber->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'logical replication apply worker'");
+
+is($result, '0', 'no logical replication worker is running');
+
+# Increment sequence on publisher
+$node_publisher->safe_psql('postgres',
+ qq(SELECT nextval('regress_s1');));
+
+# The command should fail due to missing sequence ('regress_s4')
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+like(
+ $stderr,
+ qr/WARNING: missing sequence on publisher \("public.regress_s4"\)/,
+ "output the wanring for the missing sequence regress_s4");
+
+like(
+ $stderr,
+ qr/ERROR: logical replication sequence synchronization failed for subscription \"regress_seq_sub\"/,
+ "the command failed due to the missing sequence regress_s4");
+
+# Refresh the publication to remove the missing sequence
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION;");
+
+# Sync the sequence regress_s1
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+# Get the current sequence value on subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ qq(SELECT last_value FROM regress_s1;));
+
+is($result, '201', 'sequence regress_s1 is synced now');
+
done_testing();
--
2.47.3
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-06 08:52 ` Chao Li <[email protected]>
2026-03-06 10:27 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: Chao Li @ 2026-03-06 08:52 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: shveta malik <[email protected]>; Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
> On Mar 5, 2026, at 19:34, Zhijie Hou (Fujitsu) <[email protected]> wrote:
>
> On Thursday, March 5, 2026 1:48 PM shveta malik <[email protected]> wrote:
>>
>> On Thu, Mar 5, 2026 at 9:35 AM shveta malik <[email protected]>
>> wrote:
>>>
>>> On Thu, Mar 5, 2026 at 8:16 AM Zhijie Hou (Fujitsu)
>>> <[email protected]> wrote:
>>>>
>>>>
>>>> Here is V10 patch set which addressed all comments.
>>>>
>>>
>>> Thank You. Please find a few comments on 001:
>>>
>>
>> A concern in 002:
>>
>> I realized that below might not be the correct logic to avoid overwriting
>> sequences at sub which are already at latest values.
>>
>> + /*
>> + * Skip synchronization if the local sequence value is already ahead of
>> + * the publisher's value.
>> ...
>> + */
>> + if (local_last_value > seqinfo->last_value) { ereport(WARNING,
>> + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
>> + errmsg("skipped synchronizing the sequence \"%s.%s\"",
>> + seqinfo->nspname, seqinfo->seqname), errdetail("The local
>> + last_value %lld is ahead of the one on publisher",
>> + (long long int) local_last_value));
>> +
>> + return COPYSEQ_NO_DRIFT;
>> + }
>>
>>
>> A sequence could be descending one too and thus we may wrongly end up
>> avoiding synchronization. We should first check if it is descending or ascending
>> (perhaps by checking if increment_by < 0 or >0), then decide to manage
>> conflict.
>
> Thanks for catching this, I changed it to check the increment before reporting warning.
>
> Best Regards,
> Hou zj
Hi,
I just started reviewing this patch and wanted to first discuss the design.
The current approach introduces a long-lived sync worker for any subscription that has at least one sequence. I noticed a previous email suggesting that this approach is “acceptable”, but it still seems like a big runtime cost.
What I had in mind instead is whether we could extend the WAL decoding protocol to send RM_SEQ_ID over the logical replication stream, so that sequence synchronization becomes part of logical replication itself. That would make it essentially event-driven and close to zero cost at runtime, rather than relying on periodic polling.
There is also one case I haven’t seen discussed yet. Suppose the standby side inserts a tuple into a table that is under logical replication. This might not immediately cause a tuple-level replication conflict, but it could advance the sequence locally. In that case, the standby sequence could diverge from the primary sequence and remain out of sync indefinitely. How should that situation be handled?
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 05:48 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 08:52 ` Re: [PATCH] Support automatic sequence replication Chao Li <[email protected]>
@ 2026-03-06 10:27 ` Zhijie Hou (Fujitsu) <[email protected]>
0 siblings, 0 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-06 10:27 UTC (permalink / raw)
To: Chao Li <[email protected]>; +Cc: shveta malik <[email protected]>; Amit Kapila <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Friday, March 6, 2026 4:52 PM Chao Li <[email protected]> wrote:
>
> I just started reviewing this patch and wanted to first discuss the design.
>
Thanks for reviewing.
> The current approach introduces a long-lived sync worker for any subscription
> that has at least one sequence. I noticed a previous email suggesting that
> this approach is “acceptable”, but it still seems like a big runtime cost.
Currently in the patch, the sequence sync worker operates at a dynamic interval
(between 2 and 30 seconds), adjusting based on how frequently the sequence is
updated, and only synchronizes when the sequence has actually diverged from the
publisher. So while it's a long-lived worker, I think its runtime impact is
acceptable.
> What I had in mind instead is whether we could extend the WAL decoding
> protocol to send RM_SEQ_ID over the logical replication stream, so that
> sequence synchronization becomes part of logical replication itself.
That approach was explored in depth during earlier discussions, but it was
ultimately set aside due to the complexity and correctness challenges it would
introduce into logical decoding. Adding sequence information to the replication
stream would require significant changes to the decoding machinery, and given
that the primary use case is to support upgrades, the trade-off didn't seem
justified. You can find a detailed breakdown of the design considerations and
the reasoning behind the decision in [1].
> That would make it essentially event-driven and close to zero cost at runtime,
> rather than relying on periodic polling.
I don't think so. Even with an event-driven model, there's still a cost,
decoding sequence changes would add overhead to walsenders on the publisher that
needs to replicate sequences. So it's just a different distribution of the
overhead.
> There is also one case I haven’t seen discussed yet. Suppose the standby side
> inserts a tuple into a table that is under logical replication. This might not
> immediately cause a tuple-level replication conflict, but it could advance the
> sequence locally. In that case, the standby sequence could diverge from the
> primary sequence and remain out of sync indefinitely. How should that
> situation be handled?
This scenario was also considered in earlier discussions, see [2]. The
divergence issue you mentioned is not introduced by the sequence sync worker; It
is considered as a logical conflict and resolving it would require user
intervention or a future enhancement to handle such conflicts automatically.
[1] https://www.postgresql.org/message-id/CAA4eK1LC%2BKJiAkSrpE_NwvNdidw9F2os7GERUeSxSKv71gXysQ%40mail.g...
[2] https://www.postgresql.org/message-id/CAA4eK1LLkxqeZ_GDjquzxY3bwN3yV8Nq7brvgyOBiWOXtWt4Jg%40mail.gma...
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
@ 2026-03-05 10:46 ` Amit Kapila <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 1 reply; 58+ messages in thread
From: Amit Kapila @ 2026-03-05 10:46 UTC (permalink / raw)
To: shveta malik <[email protected]>; +Cc: Zhijie Hou (Fujitsu) <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 5, 2026 at 9:35 AM shveta malik <[email protected]> wrote:
>
>
> 3)
> + /* Sleep for the configured interval */
> + (void) WaitLatch(MyLatch,
> + WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
> + sleep_ms,
> + WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
>
> I don't think this wait-event is appropriate. Unlike tablesync, we are
> not waiting for any state change here. Shall we add a new one for our
> case? How about WAIT_EVENT_LOGICAL_SEQSYNC_MAIN? Thoughts?
>
+1 for a new wait event. Few other minor comments:
1.
+ * Check if the subscription includes sequences and start a sequencesync
+ * worker if one is not already running. The active sequencesync worker will
+ * handle all pending sequence synchronization. If any sequences remain
+ * unsynchronized after it exits, a new worker can be started in the next
+ * iteration.
*
- * Start a sequencesync worker if one is not already running. The active
- * sequencesync worker will handle all pending sequence synchronization. If any
- * sequences remain unsynchronized after it exits, a new worker can be started
- * in the next iteration.
Why did this comment change? The earlier one sounds okay to me.
2.
break;
+
case COPYSEQ_INSUFFICIENT_PERM:
Why does this patch add additional new lines? We use both styles
(existing and what this patch does) in the code but it seems
unnecessary to change for this patch.
3.
- ProcessSequencesForSync();
+
+ /* Check if sequence worker needs to be started */
+ MaybeLaunchSequenceSyncWorker();
No need for an additional line and a comment here.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 10:46 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
@ 2026-03-05 11:34 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-09 03:16 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-13 07:12 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
0 siblings, 2 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-05 11:34 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; shveta malik <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thursday, March 5, 2026 6:47 PM Amit Kapila <[email protected]> wrote:
> On Thu, Mar 5, 2026 at 9:35 AM shveta malik <[email protected]>
> wrote:
> >
> >
> > 3)
> > + /* Sleep for the configured interval */
> > + (void) WaitLatch(MyLatch,
> > + WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH, sleep_ms,
> > + WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE);
> >
> > I don't think this wait-event is appropriate. Unlike tablesync, we are
> > not waiting for any state change here. Shall we add a new one for our
> > case? How about WAIT_EVENT_LOGICAL_SEQSYNC_MAIN? Thoughts?
> >
>
> +1 for a new wait event. Few other minor comments:
Added.
>
> 1.
> + * Check if the subscription includes sequences and start a
> + sequencesync
> + * worker if one is not already running. The active sequencesync worker
> + will
> + * handle all pending sequence synchronization. If any sequences remain
> + * unsynchronized after it exits, a new worker can be started in the
> + next
> + * iteration.
> *
> - * Start a sequencesync worker if one is not already running. The active
> - * sequencesync worker will handle all pending sequence synchronization. If
> any
> - * sequences remain unsynchronized after it exits, a new worker can be
> started
> - * in the next iteration.
>
> Why did this comment change? The earlier one sounds okay to me.
I think either version is fine, so reverted this change now.
>
> 2.
> break;
> +
> case COPYSEQ_INSUFFICIENT_PERM:
>
> Why does this patch add additional new lines? We use both styles (existing
> and what this patch does) in the code but it seems unnecessary to change for
> this patch.
Removed.
>
> 3.
> - ProcessSequencesForSync();
> +
> + /* Check if sequence worker needs to be started */
> + MaybeLaunchSequenceSyncWorker();
>
> No need for an additional line and a comment here.
Removed.
Here is the V11 patch which addressed all above comments and [1][2].
[1] https://www.postgresql.org/message-id/CAJpy0uAfu-VPqCknLLvJ%2BPUx_cyoR-b70xUNT6Pyv8N-odKizQ%40mail.g...
[2] https://www.postgresql.org/message-id/CAJpy0uBeAdz6-3P26Eryeq3TyjA-GiKY3z0hFMxzZD%3DAYGqQ3Q%40mail.g...
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v11-0002-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch (18.5K, 2-v11-0002-Synchronize-sequences-directly-in-REFRESH-SEQUEN.patch)
download | inline diff:
From cce0c367a0c925ec9111947362e5c4585df3b780 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <[email protected]>
Date: Thu, 5 Mar 2026 17:17:41 +0800
Subject: [PATCH v11 2/2] Synchronize sequences directly in REFRESH SEQUENCES
command
The ALTER SUBSCRIPTION ... REFRESH SEQUENCES command currently sets all
sequence states in pg_subscription_rel to INIT and relies on the sequence sync
worker to perform the actual synchronization and update states to READY.
With the recent change making the sequence sync worker long-lived, most
sequences are now synchronized in the background, reducing the need for
REFRESH SEQUENCES. However, the command remains necessary for sequences that
haven't been synchronized.
This commit enhances REFRESH SEQUENCES to synchronize sequences directly within
the command itself, eliminating the overhead of launching a worker and updating
catalog entries unnecessarily.
---
doc/src/sgml/logical-replication.sgml | 6 +-
src/backend/commands/subscriptioncmds.c | 17 +-
.../replication/logical/sequencesync.c | 175 +++++++++++++++---
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/replication/logicalworker.h | 5 +
src/test/subscription/t/036_sequences.pl | 49 +++++
6 files changed, 206 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index bb523af5d37..323d5b7c7a0 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,9 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands. The worker will
- remain running for the life of the subscription, periodically
- synchronizing all published sequences.
+ after executing <command>ALTER SUBSCRIPTION ... REFRESH PUBLICATION</command>
+ command. The worker will remain running for the life of the subscription,
+ periodically synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 5e3c0964d38..0a5acfda0ff 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -1245,25 +1245,10 @@ AlterSubscription_refresh_seq(Subscription *sub)
PG_TRY();
{
- List *subrel_states;
-
check_publications_origin_sequences(wrconn, sub->publications, true,
sub->origin, NULL, 0, sub->name);
- /* Get local sequence list. */
- subrel_states = GetSubscriptionRelations(sub->oid, false, true, false);
- foreach_ptr(SubscriptionRelState, subrel, subrel_states)
- {
- Oid relid = subrel->relid;
-
- UpdateSubscriptionRelState(sub->oid, relid, SUBREL_STATE_INIT,
- InvalidXLogRecPtr, false);
- ereport(DEBUG1,
- errmsg_internal("sequence \"%s.%s\" of subscription \"%s\" set to INIT state",
- get_namespace_name(get_rel_namespace(relid)),
- get_rel_name(relid),
- sub->name));
- }
+ AlterSubSyncSequences(wrconn, sub->oid, sub->name, sub->runasowner);
}
PG_FINALLY();
{
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index cde9eaba474..cc72f9ed0fc 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -207,7 +207,7 @@ get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx, List *seqinfos)
+ List *missing_seqs_idx, List *seqinfos, char *subname)
{
StringInfo seqstr;
@@ -253,7 +253,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
ereport(ERROR,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("logical replication sequence synchronization failed for subscription \"%s\"",
- MySubscription->name));
+ subname));
}
/*
@@ -281,6 +281,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
+ LOCKMODE lockmode;
*seqidx = DatumGetInt32(slot_getattr(slot, ++col, &isnull));
Assert(!isnull);
@@ -326,8 +327,22 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
Assert(col == REMOTE_SEQ_COL_COUNT);
seqinfo_local->found_on_pub = true;
+ seqinfo_local->increment = (remote_increment > 0);
- *sequence_rel = try_table_open(seqinfo_local->localrelid, RowExclusiveLock);
+ /*
+ * We take a stronger lock during DDL commands (currently only ALTER
+ * SUBSCRIPTION ... REFRESH SEQUENCES) to prevent concurrent sequencesync
+ * workers from updating the page_lsn while the DDL is also updating the
+ * same sequence. This ensures we can always fetch the latest page_lsn to
+ * determine whether the remote sequence value should be synchronized (see
+ * validate_seqsync_state).
+ */
+ if (IsLogicalWorker())
+ lockmode = RowExclusiveLock;
+ else
+ lockmode = ShareRowExclusiveLock;
+
+ *sequence_rel = try_table_open(seqinfo_local->localrelid, lockmode);
/* Sequence was concurrently dropped? */
if (!*sequence_rel)
@@ -366,7 +381,8 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
* whether the sequence has drifted.
*/
static CopySeqResult
-validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ XLogRecPtr local_page_lsn)
{
AclResult aclresult;
Oid seqoid = seqinfo->localrelid;
@@ -377,6 +393,24 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
int64 local_last_value;
bool local_is_called;
+ /*
+ * Allow synchronization if the remote sequence data has a more recent
+ * LSN than the local state, or if no local LSN exists yet.
+ */
+ if (!XLogRecPtrIsValid(local_page_lsn) ||
+ local_page_lsn < seqinfo->page_lsn)
+ return COPYSEQ_ALLOWED;
+
+ /*
+ * Skip synchronization if we are processing outdated sequence info
+ * based on the LSN. This occurs when the sequence has been updated to
+ * more recent data concurrently (via either ALTER SUBSCRIPTION ...
+ * REFRESH SEQUENCES or the sequencesync worker).
+ */
+ if (XLogRecPtrIsValid(local_page_lsn) &&
+ local_page_lsn > seqinfo->page_lsn)
+ return COPYSEQ_NO_DRIFT;
+
/*
* Skip synchronization if the current user does not have sufficient
* privileges to read the sequence data.
@@ -384,6 +418,40 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
if (!GetSequence(sequence_rel, &local_last_value, &local_is_called))
return COPYSEQ_INSUFFICIENT_PERM;
+ /*
+ * Skip synchronization if applying the remote value would cause the
+ * local sequence to move backward. This can happen when:
+ *
+ * - For incremental sequences, the remote value is behind the local one
+ * - For decremental sequences, the remote value is ahead of the local
+ * one
+ *
+ * XXX This occurs not only when the local sequence has been
+ * synchronized to a newer value from the publisher (where skipping is
+ * necessary to avoid backward movement), but also when the local
+ * sequence has been manually updated by the user on the subscriber. The
+ * latter could be considered a replication conflict, and overwriting
+ * the user's update might be acceptable. However, since we cannot
+ * easily distinguish between these two scenarios, we choose to skip
+ * synchronization in all cases and emit a WARNING to notify the user to
+ * manually resolve the conflict.
+ */
+ if ((seqinfo->increment && local_last_value > seqinfo->last_value) ||
+ (!seqinfo->increment && local_last_value < seqinfo->last_value))
+ {
+ ereport(WARNING,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("skipped synchronizing the sequence \"%s.%s\"",
+ seqinfo->nspname, seqinfo->seqname),
+ seqinfo->increment
+ ? errdetail("The local last_value %lld is ahead of the one on publisher",
+ (long long int) local_last_value)
+ : errdetail("The local last_value %lld is behind of the one on publisher",
+ (long long int) local_last_value));
+
+ return COPYSEQ_NO_DRIFT;
+ }
+
/*
* Skip synchronization if the sequence has not drifted from the
* publisher's value.
@@ -408,16 +476,15 @@ validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* sequences already in READY state, synchronize only if drift is detected.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel,
+ Oid subid, bool run_as_owner)
{
UserContext ucxt;
- bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
CopySeqResult result;
XLogRecPtr local_page_lsn;
- (void) GetSubscriptionRelState(MySubscription->oid,
- RelationGetRelid(sequence_rel),
+ (void) GetSubscriptionRelState(subid, RelationGetRelid(sequence_rel),
&local_page_lsn);
/*
@@ -427,7 +494,7 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
if (!run_as_owner)
SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- result = validate_seqsync_state(seqinfo, sequence_rel);
+ result = validate_seqsync_state(seqinfo, sequence_rel, local_page_lsn);
if (result != COPYSEQ_ALLOWED)
{
@@ -472,7 +539,8 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
* Returns true/false if any sequences were actually copied.
*/
static bool
-copy_sequences(WalReceiverConn *conn, List *seqinfos)
+copy_sequences(WalReceiverConn *conn, List *seqinfos, Oid subid, char *subname,
+ bool runasowner)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -498,11 +566,16 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
+ bool started_tx = false;
WalRcvExecResult *res;
TupleTableSlot *slot;
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
for (int idx = cur_batch_base_index; idx < n_seqinfos; idx++)
{
@@ -594,23 +667,23 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
&seqinfo, &seqidx, seqinfos);
if (sync_status == COPYSEQ_ALLOWED)
- sync_status = copy_sequence(seqinfo, sequence_rel);
+ sync_status = copy_sequence(seqinfo, sequence_rel,
+ subid, runasowner);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
"logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
- seqinfo->nspname, seqinfo->seqname, MySubscription->name);
+ seqinfo->nspname, seqinfo->seqname, subname);
batch_succeeded_count++;
sequence_copied = true;
break;
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in SequenceSyncContext
- * since these will be used after the transaction is
- * committed.
+ * Remember mismatched sequences in SequenceSyncContext since
+ * these will be used after the transaction is committed.
*/
oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
@@ -674,13 +747,17 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
elog(DEBUG1,
"logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
- MySubscription->name,
+ subname,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
- /* Commit this batch, and prepare for next batch */
- CommitTransactionCommand();
+ /*
+ * Commit this batch if started a transaction, and prepare for next
+ * batch.
+ */
+ if (started_tx)
+ CommitTransactionCommand();
if (batch_missing_count)
{
@@ -705,7 +782,7 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx, seqinfos);
+ missing_seqs_idx, seqinfos, subname);
return sequence_copied;
}
@@ -717,20 +794,23 @@ copy_sequences(WalReceiverConn *conn, List *seqinfos)
* Returns true if sequences have been updated.
*/
static bool
-LogicalRepSyncSequences(WalReceiverConn *conn)
+LogicalRepSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
{
Relation rel;
HeapTuple tup;
ScanKeyData skey[1];
SysScanDesc scan;
- Oid subid = MyLogicalRepWorker->subid;
bool sequence_copied = false;
List *seqinfos = NIL;
MemoryContext oldctx;
+ bool started_tx = false;
- Assert(SequenceSyncContext);
-
- StartTransactionCommand();
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ started_tx = true;
+ }
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
@@ -791,7 +871,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
systable_endscan(scan);
table_close(rel, AccessShareLock);
- CommitTransactionCommand();
+ if (started_tx)
+ CommitTransactionCommand();
/*
* Exit early if no catalog entries found, likely due to concurrent drops.
@@ -800,7 +881,8 @@ LogicalRepSyncSequences(WalReceiverConn *conn)
return false;
/* Process sequences */
- sequence_copied = copy_sequences(conn, seqinfos);
+ sequence_copied = copy_sequences(conn, seqinfos, subid, subname,
+ runasowner);
return sequence_copied;
}
@@ -876,7 +958,10 @@ start_sequence_sync(void)
/*
* Synchronize all sequences (both READY and INIT states).
*/
- sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn,
+ MySubscription->oid,
+ MySubscription->name,
+ MySubscription->runasowner);
MemoryContextReset(SequenceSyncContext);
MemoryContextSwitchTo(oldctx);
@@ -938,3 +1023,37 @@ SequenceSyncWorkerMain(Datum main_arg)
FinishSyncWorker();
}
+
+/*
+ * Wrapper for LogicalRepSyncSequences to synchronize all sequences of a
+ * subscription from outside the sequencesync worker
+ */
+void
+AlterSubSyncSequences(WalReceiverConn *conn, Oid subid, char *subname,
+ bool runasowner)
+{
+ /*
+ * Init the SequenceSyncContext which we clean up after the sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(CurrentMemoryContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ MemoryContext oldctx;
+
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ LogicalRepSyncSequences(conn, subid, subname, runasowner);
+
+ MemoryContextSwitchTo(oldctx);
+ }
+ PG_FINALLY();
+ {
+ MemoryContextDelete(SequenceSyncContext);
+ SequenceSyncContext = NULL;
+ }
+ PG_END_TRY();
+}
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 86574b69169..2c4e60c8f52 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -102,6 +102,7 @@ typedef struct LogicalRepSequenceInfo
XLogRecPtr page_lsn;
int64 last_value;
bool is_called;
+ bool increment;
/*
* True if the sequence identified by nspname + seqname exists on the
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..73afd7853d0 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -14,6 +14,8 @@
#include <signal.h>
+#include "replication/walreceiver.h"
+
extern PGDLLIMPORT volatile sig_atomic_t ParallelApplyMessagePending;
extern void ApplyWorkerMain(Datum main_arg);
@@ -31,4 +33,7 @@ extern void LogicalRepWorkersWakeupAtCommit(Oid subid);
extern void AtEOXact_LogicalRepWorkers(bool isCommit);
+extern void AlterSubSyncSequences(WalReceiverConn *conn, Oid subid,
+ char *subname, bool runasowner);
+
#endif /* LOGICALWORKER_H */
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index af190713b2b..8d25ac40ce0 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -176,4 +176,53 @@ $node_subscriber->wait_for_log(
qr/WARNING: ( [A-Z0-9]+:)? missing sequence on publisher \("public.regress_s4"\)/,
$log_offset);
+##########
+# ALTER SUBSCRIPTION ... REFRESH SEQUENCES synchronizes sequences online,
+# eliminating the need to launch a sequencesync worker.
+##########
+
+# Reduce max_logical_replication_workers to disallow sequence worker from running
+$node_subscriber->append_conf('postgresql.conf',
+ qq(max_logical_replication_workers = 0));
+$node_subscriber->restart;
+
+# Verify there is no logical replication apply worker running
+$result = $node_subscriber->safe_psql(
+ 'postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'logical replication apply worker'");
+
+is($result, '0', 'no logical replication worker is running');
+
+# Increment sequence on publisher
+$node_publisher->safe_psql('postgres',
+ qq(SELECT nextval('regress_s1');));
+
+# The command should fail due to missing sequence ('regress_s4')
+my ($cmdret, $stdout, $stderr) = $node_subscriber->psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+like(
+ $stderr,
+ qr/WARNING: missing sequence on publisher \("public.regress_s4"\)/,
+ "output the wanring for the missing sequence regress_s4");
+
+like(
+ $stderr,
+ qr/ERROR: logical replication sequence synchronization failed for subscription \"regress_seq_sub\"/,
+ "the command failed due to the missing sequence regress_s4");
+
+# Refresh the publication to remove the missing sequence
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION;");
+
+# Sync the sequence regress_s1
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;");
+
+# Get the current sequence value on subscriber
+$result = $node_subscriber->safe_psql('postgres',
+ qq(SELECT last_value FROM regress_s1;));
+
+is($result, '201', 'sequence regress_s1 is synced now');
+
done_testing();
--
2.51.1.windows.1
[application/octet-stream] v11-0001-Support-automatic-sequence-replication.patch (42.5K, 3-v11-0001-Support-automatic-sequence-replication.patch)
download | inline diff:
From 09a5cb9c7cf6abb4ff8238eb9d989b38a475ebab Mon Sep 17 00:00:00 2001
From: Ajin Cherian <[email protected]>
Date: Tue, 24 Feb 2026 21:37:01 +1100
Subject: [PATCH v11 1/2] Support automatic sequence replication.
Currently, sequence values are synchronized from publisher to subscriber only
when the user manually runs ALTER SUBSCRIPTION ... REFRESH PUBLICATION (which
affects only newly subscribed sequences) or REFRESH SEQUENCES. The sequence sync
worker exits immediately after completing each synchronization round.
The primary use case for sequence replication is during upgrades, where it's
recommended that users ensure sequences are in sync by running REFRESH SEQUENCES
before finishing the upgrade. However, this command can be slow when
synchronizing a large number of sequences, potentially increasing downtime.
To address this, this commit makes the sequence sync worker long-lived,
continuously monitoring sequences and resynchronizing them when drift is
detected. The worker uses an adaptive sleep interval: it starts at 2 seconds,
doubles up to a maximum of 30 seconds when no drift is observed, and resets to
the minimum interval once drift is found.
With this change, most sequences are silently synchronized in the background,
eliminating the need to run REFRESH SEQUENCES for the majority of cases.
However, frequently updated sequences may still lag behind, requiring a final
REFRESH SEQUENCES before upgrade completion. Users can monitor progress by
checking whether sequence states transition from INIT to READY in
pg_subscription_rel.
The REFRESH SEQUENCES command is retained for this final synchronization step,
though it currently updates all sequence states to INIT, which has room for
improvement. A future patch will enhance this command to synchronize sequences
directly without launching a worker, reducing catalog overhead.
Author: Ajin Cherian <[email protected]>
Author: Zhijie Hou <[email protected]>
Reviewed-by: Shveta Malik <[email protected]>
Reviewed-by: Peter Smith <[email protected]>
Reviewed-by: Ashutosh Sharma <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
---
doc/src/sgml/logical-replication.sgml | 23 +-
doc/src/sgml/ref/alter_subscription.sgml | 5 -
src/backend/commands/sequence.c | 41 ++
.../replication/logical/sequencesync.c | 362 +++++++++++++-----
src/backend/replication/logical/syncutils.c | 46 +--
src/backend/replication/logical/worker.c | 11 +
.../utils/activity/wait_event_names.txt | 1 +
src/include/catalog/pg_subscription_rel.h | 1 +
src/include/commands/sequence.h | 1 +
src/include/replication/worker_internal.h | 2 +-
src/test/subscription/t/026_stats.pl | 2 +
src/test/subscription/t/036_sequences.pl | 81 +---
12 files changed, 375 insertions(+), 201 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 5028fe9af09..bb523af5d37 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1787,8 +1787,9 @@ Publications:
<para>
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands, and will exit once the
- sequences are synchronized.
+ after executing any of the above subscriber commands. The worker will
+ remain running for the life of the subscription, periodically
+ synchronizing all published sequences.
</para>
<para>
The ability to launch a sequence synchronization worker is limited by the
@@ -1817,7 +1818,7 @@ Publications:
<sect2 id="sequences-out-of-sync">
<title>Refreshing Out-of-Sync Sequences</title>
<para>
- Subscriber sequence values will become out of sync as the publisher
+ Subscriber sequence values can become out of sync as the publisher
advances them.
</para>
<para>
@@ -2335,15 +2336,13 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
- Incremental sequence changes are not replicated. Although the data in
- serial or identity columns backed by sequences will be replicated as part
- of the table, the sequences themselves do not replicate ongoing changes.
- On the subscriber, a sequence will retain the last value it synchronized
- from the publisher. If the subscriber is used as a read-only database,
- then this should typically not be a problem. If, however, some kind of
- switchover or failover to the subscriber database is intended, then the
- sequences would need to be updated to the latest values, either by
- executing <link linkend="sql-altersubscription-params-refresh-sequences">
+ Incremental sequence changes are continuously replicated. If, however,
+ some kind of switchover or failover to the subscriber database is
+ intended, then the sequences replication could be lagging behind and
+ the sequences on the subscriber should be compared with that of the
+ publisher to make sure that they are up to date, if not they
+ need to be updated to the latest values, either by executing
+ <link linkend="sql-altersubscription-params-refresh-sequences">
<command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>
or by copying the current data from the publisher (perhaps using
<command>pg_dump</command>) or by determining a sufficiently high value
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index 5318998e80c..5a7b4f3c2c2 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -190,11 +190,6 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
Previously subscribed tables are not copied, even if a table's row
filter <literal>WHERE</literal> clause has since been modified.
</para>
- <para>
- Previously subscribed sequences are not re-synchronized. To do that,
- use <link linkend="sql-altersubscription-params-refresh-sequences">
- <command>ALTER SUBSCRIPTION ... REFRESH SEQUENCES</command></link>.
- </para>
<para>
See <xref linkend="sequence-definition-mismatches"/> for recommendations on how
to handle any warnings about sequence definition differences between
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..92ae5aab488 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -929,6 +929,47 @@ lastval(PG_FUNCTION_ARGS)
PG_RETURN_INT64(result);
}
+/*
+ * Read the current sequence values (last_value and is_called)
+ *
+ * This is a read-only operation used by logical replication sequence
+ * synchronization to detect drift. The caller must hold a lock on the sequence.
+ *
+ * Return false if the caller does not have sufficient privileges to access the
+ * sequence, true otherwise.
+ */
+bool
+GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
+{
+ Buffer buf;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Oid relid = RelationGetRelid(seqrel);
+
+ /* Confirm that the relation is a sequence and is locked */
+ Assert(seqrel->rd_rel->relkind == RELKIND_SEQUENCE);
+ Assert(CheckRelationLockedByMe(seqrel, AccessShareLock, true));
+
+ if (pg_class_aclcheck(relid, GetUserId(), ACL_SELECT) != ACLCHECK_OK)
+ {
+ *last_value = 0;
+ *is_called = false;
+ return false;
+ }
+
+ /* Read the sequence tuple */
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ /* Extract the values */
+ *last_value = seq->last_value;
+ *is_called = seq->is_called;
+
+ /* Release buffer */
+ UnlockReleaseBuffer(buf);
+
+ return true;
+}
+
/*
* Main internal procedure that handles 2 & 3 arg forms of SETVAL.
*
diff --git a/src/backend/replication/logical/sequencesync.c b/src/backend/replication/logical/sequencesync.c
index 9c92fddd624..cde9eaba474 100644
--- a/src/backend/replication/logical/sequencesync.c
+++ b/src/backend/replication/logical/sequencesync.c
@@ -19,10 +19,6 @@
* CREATE SUBSCRIPTION
* ALTER SUBSCRIPTION ... REFRESH PUBLICATION
*
- * Executing the following command resets all sequences in the subscription to
- * state INIT, triggering re-synchronization:
- * ALTER SUBSCRIPTION ... REFRESH SEQUENCES
- *
* The apply worker periodically scans pg_subscription_rel for sequences in
* INIT state. When such sequences are found, it spawns a sequencesync worker
* to handle synchronization.
@@ -36,8 +32,24 @@
* local subscriber, and finally marks each sequence as READY upon successful
* synchronization.
*
+ * The sequencesync worker then fetches all sequences that are
+ * in the READY state, queries the publisher for current sequence values, and
+ * updates any sequences that have drifted and then goes to sleep. The sleep
+ * interval starts as SEQSYNC_MIN_SLEEP_MS and doubles after each wake cycle
+ * (up to SEQSYNC_MAX_SLEEP_MS). When drift is detected, the interval resets to
+ * the minimum to ensure timely updates.
+ *
+ * After CREATE SUBSCRIPTION, sequences begin in the INIT state. Sequences
+ * added through ALTER SUBSCRIPTION.. REFRESH PUBLICATION also start in the INIT
+ * state. All INIT sequences are synchronized unconditionally, then transition
+ * to the READY state. Once in the READY state, sequences are checked for drift
+ * from the publisher and synchronized only when drift is detected.
+ *
* Sequence state transitions follow this pattern:
- * INIT -> READY
+ * INIT --> READY ->-+
+ * ^ | (check/synchronize)
+ * | |
+ * +--<---+
*
* To avoid creating too many transactions, up to MAX_SEQUENCES_SYNC_PER_BATCH
* sequences are synchronized per transaction. The locks on the sequence
@@ -60,6 +72,7 @@
#include "postmaster/interrupt.h"
#include "replication/logicalworker.h"
#include "replication/worker_internal.h"
+#include "storage/latch.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
@@ -76,25 +89,35 @@
typedef enum CopySeqResult
{
COPYSEQ_SUCCESS,
+ COPYSEQ_ALLOWED,
COPYSEQ_MISMATCH,
COPYSEQ_INSUFFICIENT_PERM,
- COPYSEQ_SKIPPED
+ COPYSEQ_SKIPPED,
+ COPYSEQ_NO_DRIFT,
} CopySeqResult;
-static List *seqinfos = NIL;
+/* Sleep intervals for sync */
+#define SEQSYNC_MIN_SLEEP_MS 2000 /* 2 seconds */
+#define SEQSYNC_MAX_SLEEP_MS 30000 /* 30 seconds */
+
+static MemoryContext SequenceSyncContext = NULL;
/*
- * Apply worker determines if sequence synchronization is needed.
+ * Apply worker determines whether a sequence sync worker is needed.
*
* Start a sequencesync worker if one is not already running. The active
* sequencesync worker will handle all pending sequence synchronization. If any
* sequences remain unsynchronized after it exits, a new worker can be started
* in the next iteration.
+ *
+ * The pointer to the sequencesync worker is cached to avoid scanning the
+ * workers array each time via logicalrep_worker_find().
*/
void
-ProcessSequencesForSync(void)
+MaybeLaunchSequenceSyncWorker(void)
{
- LogicalRepWorker *sequencesync_worker;
+ static LogicalRepWorker *sequencesync_worker = NULL;
+
int nsyncworkers;
bool has_pending_sequences;
bool started_tx;
@@ -112,6 +135,19 @@ ProcessSequencesForSync(void)
LWLockAcquire(LogicalRepWorkerLock, LW_SHARED);
+ /*
+ * Quick exit if the sequence sync worker for the current subscription is
+ * already alive.
+ */
+ if (sequencesync_worker &&
+ sequencesync_worker->proc &&
+ isSequenceSyncWorker(sequencesync_worker) &&
+ sequencesync_worker->subid == MyLogicalRepWorker->subid)
+ {
+ LWLockRelease(LogicalRepWorkerLock);
+ return;
+ }
+
/* Check if there is a sequencesync worker already running? */
sequencesync_worker = logicalrep_worker_find(WORKERTYPE_SEQUENCESYNC,
MyLogicalRepWorker->subid,
@@ -144,7 +180,7 @@ ProcessSequencesForSync(void)
* for the given list of sequence indexes.
*/
static void
-get_sequences_string(List *seqindexes, StringInfo buf)
+get_sequences_string(List *seqindexes, List *seqinfos, StringInfo buf)
{
resetStringInfo(buf);
foreach_int(seqidx, seqindexes)
@@ -171,7 +207,7 @@ get_sequences_string(List *seqindexes, StringInfo buf)
*/
static void
report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
- List *missing_seqs_idx)
+ List *missing_seqs_idx, List *seqinfos)
{
StringInfo seqstr;
@@ -183,7 +219,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (mismatched_seqs_idx)
{
- get_sequences_string(mismatched_seqs_idx, seqstr);
+ get_sequences_string(mismatched_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("mismatched or renamed sequence on subscriber (%s)",
@@ -194,7 +230,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (insuffperm_seqs_idx)
{
- get_sequences_string(insuffperm_seqs_idx, seqstr);
+ get_sequences_string(insuffperm_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("insufficient privileges on sequence (%s)",
@@ -205,7 +241,7 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
if (missing_seqs_idx)
{
- get_sequences_string(missing_seqs_idx, seqstr);
+ get_sequences_string(missing_seqs_idx, seqinfos, seqstr);
ereport(WARNING,
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_plural("missing sequence on publisher (%s)",
@@ -229,7 +265,8 @@ report_sequence_errors(List *mismatched_seqs_idx, List *insuffperm_seqs_idx,
*/
static CopySeqResult
get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
- LogicalRepSequenceInfo **seqinfo, int *seqidx)
+ LogicalRepSequenceInfo **seqinfo, int *seqidx,
+ List *seqinfos)
{
bool isnull;
int col = 0;
@@ -240,7 +277,7 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
int64 remote_min;
int64 remote_max;
bool remote_cycle;
- CopySeqResult result = COPYSEQ_SUCCESS;
+ CopySeqResult result = COPYSEQ_ALLOWED;
HeapTuple tup;
Form_pg_sequence local_seq;
LogicalRepSequenceInfo *seqinfo_local;
@@ -325,32 +362,79 @@ get_and_validate_seq_info(TupleTableSlot *slot, Relation *sequence_rel,
}
/*
- * Apply remote sequence state to local sequence and mark it as
- * synchronized (READY).
+ * Check whether the user has required privileges on the sequence and
+ * whether the sequence has drifted.
*/
static CopySeqResult
-copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
+validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
{
- UserContext ucxt;
AclResult aclresult;
+ Oid seqoid = seqinfo->localrelid;
+
+ /* Perform drift check if it's not the initial sync */
+ if (seqinfo->relstate == SUBREL_STATE_READY)
+ {
+ int64 local_last_value;
+ bool local_is_called;
+
+ /*
+ * Skip synchronization if the current user does not have sufficient
+ * privileges to read the sequence data.
+ */
+ if (!GetSequence(sequence_rel, &local_last_value, &local_is_called))
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ /*
+ * Skip synchronization if the sequence has not drifted from the
+ * publisher's value.
+ */
+ if (local_last_value == seqinfo->last_value &&
+ local_is_called == seqinfo->is_called)
+ return COPYSEQ_NO_DRIFT;
+ }
+
+ /* Verify that the current user can update the sequence */
+ aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+
+ if (aclresult != ACLCHECK_OK)
+ return COPYSEQ_INSUFFICIENT_PERM;
+
+ return COPYSEQ_ALLOWED;
+}
+
+/*
+ * Apply remote sequence state to local sequence. For sequences in INIT state,
+ * always synchronize and then move them to READY state upon completion. For
+ * sequences already in READY state, synchronize only if drift is detected.
+ */
+static CopySeqResult
+copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
+{
+ UserContext ucxt;
bool run_as_owner = MySubscription->runasowner;
Oid seqoid = seqinfo->localrelid;
+ CopySeqResult result;
+ XLogRecPtr local_page_lsn;
+
+ (void) GetSubscriptionRelState(MySubscription->oid,
+ RelationGetRelid(sequence_rel),
+ &local_page_lsn);
/*
* If the user did not opt to run as the owner of the subscription
* ('run_as_owner'), then copy the sequence as the owner of the sequence.
*/
if (!run_as_owner)
- SwitchToUntrustedUser(seqowner, &ucxt);
+ SwitchToUntrustedUser(sequence_rel->rd_rel->relowner, &ucxt);
- aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
+ result = validate_seqsync_state(seqinfo, sequence_rel);
- if (aclresult != ACLCHECK_OK)
+ if (result != COPYSEQ_ALLOWED)
{
if (!run_as_owner)
RestoreUserContext(&ucxt);
- return COPYSEQ_INSUFFICIENT_PERM;
+ return result;
}
/*
@@ -368,19 +452,27 @@ copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
/*
* Record the remote sequence's LSN in pg_subscription_rel and mark the
- * sequence as READY.
+ * sequence as READY if updating a sequence that is in INIT state.
*/
- UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
- seqinfo->page_lsn, false);
+ if (seqinfo->relstate == SUBREL_STATE_INIT ||
+ seqinfo->page_lsn != local_page_lsn)
+ UpdateSubscriptionRelState(MySubscription->oid, seqoid, SUBREL_STATE_READY,
+ seqinfo->page_lsn, false);
return COPYSEQ_SUCCESS;
}
/*
* Copy existing data of sequences from the publisher.
+ *
+ * If the state of sequence is SUBREL_STATE_READY, only synchronize sequences
+ * that have drifted from their publisher values. Otherwise, synchronize all
+ * sequences.
+ *
+ * Returns true/false if any sequences were actually copied.
*/
-static void
-copy_sequences(WalReceiverConn *conn)
+static bool
+copy_sequences(WalReceiverConn *conn, List *seqinfos)
{
int cur_batch_base_index = 0;
int n_seqinfos = list_length(seqinfos);
@@ -390,13 +482,10 @@ copy_sequences(WalReceiverConn *conn)
StringInfo seqstr = makeStringInfo();
StringInfo cmd = makeStringInfo();
MemoryContext oldctx;
+ bool sequence_copied = false;
#define MAX_SEQUENCES_SYNC_PER_BATCH 100
- elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - total unsynchronized: %d",
- MySubscription->name, n_seqinfos);
-
while (cur_batch_base_index < n_seqinfos)
{
Oid seqRow[REMOTE_SEQ_COL_COUNT] = {INT8OID, INT8OID,
@@ -406,6 +495,7 @@ copy_sequences(WalReceiverConn *conn)
int batch_mismatched_count = 0;
int batch_skipped_count = 0;
int batch_insuffperm_count = 0;
+ int batch_no_drift = 0;
int batch_missing_count;
Relation sequence_rel = NULL;
@@ -501,28 +591,28 @@ copy_sequences(WalReceiverConn *conn)
}
sync_status = get_and_validate_seq_info(slot, &sequence_rel,
- &seqinfo, &seqidx);
- if (sync_status == COPYSEQ_SUCCESS)
- sync_status = copy_sequence(seqinfo,
- sequence_rel->rd_rel->relowner);
+ &seqinfo, &seqidx, seqinfos);
+
+ if (sync_status == COPYSEQ_ALLOWED)
+ sync_status = copy_sequence(seqinfo, sequence_rel);
switch (sync_status)
{
case COPYSEQ_SUCCESS:
elog(DEBUG1,
- "logical replication synchronization for subscription \"%s\", sequence \"%s.%s\" has finished",
- MySubscription->name, seqinfo->nspname,
- seqinfo->seqname);
+ "logical replication synchronization has updated sequence \"%s.%s\" in subscription \"%s\"",
+ seqinfo->nspname, seqinfo->seqname, MySubscription->name);
batch_succeeded_count++;
+ sequence_copied = true;
break;
case COPYSEQ_MISMATCH:
/*
- * Remember mismatched sequences in a long-lived memory
- * context since these will be used after the transaction
- * is committed.
+ * Remember mismatched sequences in SequenceSyncContext
+ * since these will be used after the transaction is
+ * committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
mismatched_seqs_idx = lappend_int(mismatched_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -531,11 +621,11 @@ copy_sequences(WalReceiverConn *conn)
case COPYSEQ_INSUFFICIENT_PERM:
/*
- * Remember sequences with insufficient privileges in a
- * long-lived memory context since these will be used
- * after the transaction is committed.
+ * Remember sequences with insufficient privileges in
+ * SequenceSyncContext since these will be used after the
+ * transaction is committed.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
insuffperm_seqs_idx = lappend_int(insuffperm_seqs_idx,
seqidx);
MemoryContextSwitchTo(oldctx);
@@ -558,6 +648,13 @@ copy_sequences(WalReceiverConn *conn)
batch_skipped_count++;
}
break;
+ case COPYSEQ_NO_DRIFT:
+ /* Nothing to do */
+ batch_no_drift++;
+ break;
+ default:
+ elog(ERROR, "unrecognized Sequence replication result: %d", (int) sync_status);
+
}
if (sequence_rel)
@@ -572,14 +669,15 @@ copy_sequences(WalReceiverConn *conn)
batch_missing_count = batch_size - (batch_succeeded_count +
batch_mismatched_count +
batch_insuffperm_count +
- batch_skipped_count);
+ batch_skipped_count +
+ batch_no_drift);
elog(DEBUG1,
- "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped",
+ "logical replication sequence synchronization for subscription \"%s\" - batch #%d = %d attempted, %d succeeded, %d mismatched, %d insufficient permission, %d missing from publisher, %d skipped, %d no drift",
MySubscription->name,
(cur_batch_base_index / MAX_SEQUENCES_SYNC_PER_BATCH) + 1,
batch_size, batch_succeeded_count, batch_mismatched_count,
- batch_insuffperm_count, batch_missing_count, batch_skipped_count);
+ batch_insuffperm_count, batch_missing_count, batch_skipped_count, batch_no_drift);
/* Commit this batch, and prepare for next batch */
CommitTransactionCommand();
@@ -607,47 +705,50 @@ copy_sequences(WalReceiverConn *conn)
/* Report mismatches, permission issues, or missing sequences */
report_sequence_errors(mismatched_seqs_idx, insuffperm_seqs_idx,
- missing_seqs_idx);
+ missing_seqs_idx, seqinfos);
+
+ return sequence_copied;
}
/*
* Identifies sequences that require synchronization and initiates the
* synchronization process.
+ *
+ * Returns true if sequences have been updated.
*/
-static void
-LogicalRepSyncSequences(void)
+static bool
+LogicalRepSyncSequences(WalReceiverConn *conn)
{
- char *err;
- bool must_use_password;
Relation rel;
HeapTuple tup;
- ScanKeyData skey[2];
+ ScanKeyData skey[1];
SysScanDesc scan;
Oid subid = MyLogicalRepWorker->subid;
- StringInfoData app_name;
+ bool sequence_copied = false;
+ List *seqinfos = NIL;
+ MemoryContext oldctx;
+
+ Assert(SequenceSyncContext);
StartTransactionCommand();
rel = table_open(SubscriptionRelRelationId, AccessShareLock);
+ /* Scan for all sequences belonging to this subscription */
ScanKeyInit(&skey[0],
Anum_pg_subscription_rel_srsubid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(subid));
- ScanKeyInit(&skey[1],
- Anum_pg_subscription_rel_srsubstate,
- BTEqualStrategyNumber, F_CHAREQ,
- CharGetDatum(SUBREL_STATE_INIT));
-
scan = systable_beginscan(rel, InvalidOid, false,
- NULL, 2, skey);
+ NULL, 1, skey);
+
while (HeapTupleIsValid(tup = systable_getnext(scan)))
{
Form_pg_subscription_rel subrel;
LogicalRepSequenceInfo *seq;
Relation sequence_rel;
- MemoryContext oldctx;
+ char relstate;
CHECK_FOR_INTERRUPTS();
@@ -666,18 +767,21 @@ LogicalRepSyncSequences(void)
continue;
}
+ relstate = subrel->srsubstate;
+
+ Assert(relstate == SUBREL_STATE_INIT || relstate == SUBREL_STATE_READY);
+
/*
* Worker needs to process sequences across transaction boundary, so
- * allocate them under long-lived context.
+ * allocate them under SequenceSyncContext.
*/
- oldctx = MemoryContextSwitchTo(ApplyContext);
-
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
seq = palloc0_object(LogicalRepSequenceInfo);
seq->localrelid = subrel->srrelid;
seq->nspname = get_namespace_name(RelationGetNamespace(sequence_rel));
seq->seqname = pstrdup(RelationGetRelationName(sequence_rel));
+ seq->relstate = relstate;
seqinfos = lappend(seqinfos, seq);
-
MemoryContextSwitchTo(oldctx);
table_close(sequence_rel, NoLock);
@@ -693,36 +797,16 @@ LogicalRepSyncSequences(void)
* Exit early if no catalog entries found, likely due to concurrent drops.
*/
if (!seqinfos)
- return;
+ return false;
- /* Is the use of a password mandatory? */
- must_use_password = MySubscription->passwordrequired &&
- !MySubscription->ownersuperuser;
+ /* Process sequences */
+ sequence_copied = copy_sequences(conn, seqinfos);
- initStringInfo(&app_name);
- appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
- MySubscription->oid, GetSystemIdentifier());
-
- /*
- * Establish the connection to the publisher for sequence synchronization.
- */
- LogRepWorkerWalRcvConn =
- walrcv_connect(MySubscription->conninfo, true, true,
- must_use_password,
- app_name.data, &err);
- if (LogRepWorkerWalRcvConn == NULL)
- ereport(ERROR,
- errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
- MySubscription->name, err));
-
- pfree(app_name.data);
-
- copy_sequences(LogRepWorkerWalRcvConn);
+ return sequence_copied;
}
/*
- * Execute the initial sync with error handling. Disable the subscription,
+ * Execute the sequence sync with error handling. Disable the subscription,
* if required.
*
* Note that we don't handle FATAL errors which are probably because of system
@@ -735,8 +819,92 @@ start_sequence_sync(void)
PG_TRY();
{
- /* Call initial sync. */
- LogicalRepSyncSequences();
+ char *err;
+ bool must_use_password;
+ StringInfoData app_name;
+ long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+
+ /* Is the use of a password mandatory? */
+ must_use_password = MySubscription->passwordrequired &&
+ !MySubscription->ownersuperuser;
+
+ initStringInfo(&app_name);
+ appendStringInfo(&app_name, "pg_%u_sequence_sync_" UINT64_FORMAT,
+ MySubscription->oid, GetSystemIdentifier());
+
+ /*
+ * Establish the connection to the publisher for sequence
+ * synchronization.
+ */
+ LogRepWorkerWalRcvConn =
+ walrcv_connect(MySubscription->conninfo, true, true,
+ must_use_password,
+ app_name.data, &err);
+ if (LogRepWorkerWalRcvConn == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("sequencesync worker for subscription \"%s\" could not connect to the publisher: %s",
+ MySubscription->name, err));
+
+ pfree(app_name.data);
+
+ /*
+ * Init the SequenceSyncContext which we clean up after each sequence
+ * synchronization.
+ */
+ SequenceSyncContext = AllocSetContextCreate(ApplyContext,
+ "SequenceSyncContext",
+ ALLOCSET_DEFAULT_SIZES);
+
+ for (;;)
+ {
+ bool sequence_copied = false;
+ MemoryContext oldctx;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Process any invalidation messages that might have accumulated */
+ AcceptInvalidationMessages();
+ maybe_reread_subscription();
+
+ /*
+ * Perform sequence synchronization under SequenceSyncContext and
+ * reset it each cycle to avoid manual memory management.
+ */
+ oldctx = MemoryContextSwitchTo(SequenceSyncContext);
+
+ /*
+ * Synchronize all sequences (both READY and INIT states).
+ */
+ sequence_copied = LogicalRepSyncSequences(LogRepWorkerWalRcvConn);
+
+ MemoryContextReset(SequenceSyncContext);
+ MemoryContextSwitchTo(oldctx);
+
+ /*
+ * Adjust sleep interval based on whether sequences were copied
+ * over
+ */
+ if (sequence_copied)
+ {
+ sleep_ms = SEQSYNC_MIN_SLEEP_MS;
+ }
+ else
+ {
+ /*
+ * Double the sleep time, but not beyond the maximum allowable
+ * value.
+ */
+ sleep_ms = Min(sleep_ms * 2, SEQSYNC_MAX_SLEEP_MS);
+ }
+
+ /* Sleep for the configured interval */
+ (void) WaitLatch(MyLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
+ sleep_ms,
+ WAIT_EVENT_LOGICAL_SEQSYNC_MAIN);
+ ResetLatch(MyLatch);
+ }
}
PG_CATCH();
{
diff --git a/src/backend/replication/logical/syncutils.c b/src/backend/replication/logical/syncutils.c
index ef61ca0437d..3a0dc8669f9 100644
--- a/src/backend/replication/logical/syncutils.c
+++ b/src/backend/replication/logical/syncutils.c
@@ -172,7 +172,7 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
case WORKERTYPE_APPLY:
ProcessSyncingTablesForApply(current_lsn);
- ProcessSequencesForSync();
+ MaybeLaunchSequenceSyncWorker();
break;
case WORKERTYPE_SEQUENCESYNC:
@@ -191,13 +191,13 @@ ProcessSyncingRelations(XLogRecPtr current_lsn)
*
* The pg_subscription_rel catalog is shared by tables and sequences. Changes
* to either sequences or tables can affect the validity of relation states, so
- * we identify non-READY tables and non-READY sequences together to ensure
+ * we identify non-READY tables and sequences (in any state) together to ensure
* consistency.
*
* has_pending_subtables: true if the subscription has one or more tables that
* are not in READY state, otherwise false.
* has_pending_subsequences: true if the subscription has one or more sequences
- * that are not in READY state, otherwise false.
+ * (in any state), otherwise false.
*/
void
FetchRelationStates(bool *has_pending_subtables,
@@ -205,23 +205,22 @@ FetchRelationStates(bool *has_pending_subtables,
bool *started_tx)
{
/*
- * has_subtables and has_subsequences_non_ready are declared as static,
- * since the same value can be used until the system table is invalidated.
+ * has_subtables and has_subsequences are declared as static, since the
+ * same value can be used until the system table is invalidated.
*/
static bool has_subtables = false;
- static bool has_subsequences_non_ready = false;
+ static bool has_subsequences = false;
*started_tx = false;
-
if (relation_states_validity != SYNC_RELATIONS_STATE_VALID)
{
MemoryContext oldctx;
List *rstates;
+ List *seq_states;
SubscriptionRelState *rstate;
relation_states_validity = SYNC_RELATIONS_STATE_REBUILD_STARTED;
- has_subsequences_non_ready = false;
-
+ has_subsequences = false;
/* Clean the old lists. */
list_free_deep(table_states_not_ready);
table_states_not_ready = NIL;
@@ -231,27 +230,28 @@ FetchRelationStates(bool *has_pending_subtables,
StartTransactionCommand();
*started_tx = true;
}
-
- /* Fetch tables and sequences that are in non-READY state. */
- rstates = GetSubscriptionRelations(MySubscription->oid, true, true,
+ /* Fetch tables that are in non-READY state. */
+ rstates = GetSubscriptionRelations(MySubscription->oid, true, false,
true);
-
+ /* Fetch all sequences (regardless of state). */
+ seq_states = GetSubscriptionRelations(MySubscription->oid, false, true,
+ false);
/* Allocate the tracking info in a permanent memory context. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
foreach_ptr(SubscriptionRelState, subrel, rstates)
{
- if (get_rel_relkind(subrel->relid) == RELKIND_SEQUENCE)
- has_subsequences_non_ready = true;
- else
- {
- rstate = palloc_object(SubscriptionRelState);
- memcpy(rstate, subrel, sizeof(SubscriptionRelState));
- table_states_not_ready = lappend(table_states_not_ready,
- rstate);
- }
+ rstate = palloc_object(SubscriptionRelState);
+ memcpy(rstate, subrel, sizeof(SubscriptionRelState));
+ table_states_not_ready = lappend(table_states_not_ready,
+ rstate);
}
+
+ /* Check if there are any sequences. */
+ has_subsequences = (seq_states != NIL);
MemoryContextSwitchTo(oldctx);
+ list_free_deep(seq_states);
+
/*
* Does the subscription have tables?
*
@@ -277,5 +277,5 @@ FetchRelationStates(bool *has_pending_subtables,
*has_pending_subtables = has_subtables;
if (has_pending_subsequences)
- *has_pending_subsequences = has_subsequences_non_ready;
+ *has_pending_subsequences = has_subsequences;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index f9c4b484754..f91c8f9ecde 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -5099,6 +5099,9 @@ maybe_reread_subscription(void)
* worker won't restart if the streaming option's value is changed from
* 'parallel' to any other value or the server decides not to stream the
* in-progress transaction.
+ *
+ * Note: some parameters may not be relevant to the sequence sync worker,
+ * but exit anyway.
*/
if (strcmp(newsub->conninfo, MySubscription->conninfo) != 0 ||
strcmp(newsub->name, MySubscription->name) != 0 ||
@@ -5114,6 +5117,10 @@ maybe_reread_subscription(void)
ereport(LOG,
(errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because of a parameter change",
MySubscription->name)));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ (errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because of a parameter change",
+ MySubscription->name)));
else
ereport(LOG,
(errmsg("logical replication worker for subscription \"%s\" will restart because of a parameter change",
@@ -5132,6 +5139,10 @@ maybe_reread_subscription(void)
ereport(LOG,
errmsg("logical replication parallel apply worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
MySubscription->name));
+ else if (am_sequencesync_worker())
+ ereport(LOG,
+ errmsg("logical replication sequence synchronization worker for subscription \"%s\" will stop because the subscription owner's superuser privileges have been revoked",
+ MySubscription->name));
else
ereport(LOG,
errmsg("logical replication worker for subscription \"%s\" will restart because the subscription owner's superuser privileges have been revoked",
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index 4aa864fe3c3..a91085e7723 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -61,6 +61,7 @@ IO_WORKER_MAIN "Waiting in main loop of IO Worker process."
LOGICAL_APPLY_MAIN "Waiting in main loop of logical replication apply process."
LOGICAL_LAUNCHER_MAIN "Waiting in main loop of logical replication launcher process."
LOGICAL_PARALLEL_APPLY_MAIN "Waiting in main loop of logical replication parallel apply process."
+LOGICAL_SEQSYNC_MAIN "Waiting in main loop of logical replication sequence sync process."
RECOVERY_WAL_STREAM "Waiting in main loop of startup process for WAL to arrive, during streaming recovery."
REPLICATION_SLOTSYNC_MAIN "Waiting in main loop of slot synchronization."
REPLICATION_SLOTSYNC_SHUTDOWN "Waiting for slot sync worker to shut down."
diff --git a/src/include/catalog/pg_subscription_rel.h b/src/include/catalog/pg_subscription_rel.h
index 502640d3018..86574b69169 100644
--- a/src/include/catalog/pg_subscription_rel.h
+++ b/src/include/catalog/pg_subscription_rel.h
@@ -96,6 +96,7 @@ typedef struct LogicalRepSequenceInfo
char *seqname;
char *nspname;
Oid localrelid;
+ char relstate;
/* Sequence information retrieved from the publisher node */
XLogRecPtr page_lsn;
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 2c3c4a3f074..fd4f69bdd1c 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -47,6 +47,7 @@ extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
extern void SequenceChangePersistence(Oid relid, char newrelpersistence);
extern void DeleteSequenceTuple(Oid relid);
extern void ResetSequence(Oid seq_relid);
+extern bool GetSequence(Relation seqrel, int64 *last_value, bool *is_called);
extern void SetSequence(Oid relid, int64 next, bool iscalled);
extern void ResetSequenceCaches(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 4ecbdcfadac..a41cb045f19 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -286,7 +286,7 @@ extern void UpdateTwoPhaseState(Oid suboid, char new_state);
extern void ProcessSyncingTablesForSync(XLogRecPtr current_lsn);
extern void ProcessSyncingTablesForApply(XLogRecPtr current_lsn);
-extern void ProcessSequencesForSync(void);
+extern void MaybeLaunchSequenceSyncWorker(void);
pg_noreturn extern void FinishSyncWorker(void);
extern void InvalidateSyncingRelStates(Datum arg, SysCacheIdentifier cacheid,
diff --git a/src/test/subscription/t/026_stats.pl b/src/test/subscription/t/026_stats.pl
index 5d457060a02..2fe209f461f 100644
--- a/src/test/subscription/t/026_stats.pl
+++ b/src/test/subscription/t/026_stats.pl
@@ -16,6 +16,8 @@ $node_publisher->start;
# Create subscriber node.
my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
$node_subscriber->init;
+$node_subscriber->append_conf('postgresql.conf',
+ "max_logical_replication_workers = 10");
$node_subscriber->start;
diff --git a/src/test/subscription/t/036_sequences.pl b/src/test/subscription/t/036_sequences.pl
index 471780a3585..af190713b2b 100644
--- a/src/test/subscription/t/036_sequences.pl
+++ b/src/test/subscription/t/036_sequences.pl
@@ -75,18 +75,14 @@ is($result, '100|t',
##########
## ALTER SUBSCRIPTION ... REFRESH PUBLICATION should cause sync of new
-# sequences of the publisher, but changes to existing sequences should
-# not be synced.
+# sequences of the publisher.
##########
-# Create a new sequence 'regress_s2', and update existing sequence 'regress_s1'
+# Create a new sequence 'regress_s2'
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s2;
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
-
- -- Existing sequence
- INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
# Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION
@@ -97,19 +93,6 @@ $result = $node_subscriber->safe_psql(
$node_subscriber->poll_query_until('postgres', $synced_query)
or die "Timed out while waiting for subscriber to synchronize data";
-$result = $node_publisher->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'Check sequence value in the publisher');
-
-# Check - existing sequence ('regress_s1') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '100|t', 'REFRESH PUBLICATION will not sync existing sequence');
-
# Check - newly published sequence ('regress_s2') is synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
@@ -119,16 +102,13 @@ is($result, '100|t',
'REFRESH PUBLICATION will sync newly published sequence');
##########
-# Test: REFRESH SEQUENCES and REFRESH PUBLICATION (copy_data = false)
-#
-# 1. ALTER SUBSCRIPTION ... REFRESH SEQUENCES should re-synchronize all
-# existing sequences, but not synchronize newly added ones.
-# 2. ALTER SUBSCRIPTION ... REFRESH PUBLICATION with (copy_data = false) should
-# also not update sequence values for newly added sequences.
+# Test:
+# 1. Automatic update of existing sequence values
+# 2. Newly added sequences are not automatically updated.
##########
-# Create a new sequence 'regress_s3', and update the existing sequence
-# 'regress_s2'.
+# Create a new sequence 'regress_s3', and update the existing sequences
+# 'regress_s2' and 'regress_s1'.
$node_publisher->safe_psql(
'postgres', qq(
CREATE SEQUENCE regress_s3;
@@ -136,53 +116,28 @@ $node_publisher->safe_psql(
-- Existing sequence
INSERT INTO regress_seq_test SELECT nextval('regress_s2') FROM generate_series(1,100);
+ INSERT INTO regress_seq_test SELECT nextval('regress_s1') FROM generate_series(1,100);
));
-# 1. Do ALTER SUBSCRIPTION ... REFRESH SEQUENCES
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH SEQUENCES;
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
-
# Check - existing sequences ('regress_s1' and 'regress_s2') are synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s1;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s2;
-));
-is($result, '200|t', 'REFRESH SEQUENCES will sync existing sequences');
-# Check - newly published sequence ('regress_s3') is not synced
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- SELECT last_value, is_called FROM regress_s3;
-));
-is($result, '1|f',
- 'REFRESH SEQUENCES will not sync newly published sequence');
+# Poll until regress_s1 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s1;))
+ or die "Timed out while waiting for regress_s1 sequence to sync";
-# 2. Do ALTER SUBSCRIPTION ... REFRESH PUBLICATION with copy_data as false
-$result = $node_subscriber->safe_psql(
- 'postgres', qq(
- ALTER SUBSCRIPTION regress_seq_sub REFRESH PUBLICATION WITH (copy_data = false);
-));
-$node_subscriber->poll_query_until('postgres', $synced_query)
- or die "Timed out while waiting for subscriber to synchronize data";
+# Poll until regress_s2 reflects the updated sequence value
+$node_subscriber->poll_query_until('postgres',
+ qq(SELECT last_value = 200 AND is_called = 't' FROM regress_s2;))
+ or die "Timed out while waiting for regress_s2 sequence to sync";
-# Check - newly published sequence ('regress_s3') is not synced with copy_data
-# as false.
+# Check - newly published sequence ('regress_s3') is not synced
$result = $node_subscriber->safe_psql(
'postgres', qq(
SELECT last_value, is_called FROM regress_s3;
));
is($result, '1|f',
- 'REFRESH PUBLICATION will not sync newly published sequence with copy_data as false'
-);
+ 'Newly published sequences are not synced automatically');
##########
# ALTER SUBSCRIPTION ... REFRESH PUBLICATION should report an error when:
--
2.51.1.windows.1
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 10:46 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-09 03:16 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-09 10:29 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-13 07:12 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 2 replies; 58+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-03-09 03:16 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>; shveta malik <[email protected]>
Dear Hou,
Thanks for uprating the patch. Few comments.
01.
The patch needs to be rebased, because of the missing inclusion of wait_event.h.
It might be affected by the commit 868825aae.
02. start_sequence_sync
Missing ProcessConfigFile() in the loop. Now it's done only in copy_sequences(),
but we can reach there when some sequences need to be synchronized.
03.
Can we describe needed privileges? IIUC, initial sync needs the UPDATE privilege,
additionally periodic synch needs SELECT privilege.
04.
```
+ long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
```
This is used in the for loop, make it the loop-specific variable.
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 58+ messages in thread
* Re: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 10:46 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-09 03:16 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-03-09 10:29 ` shveta malik <[email protected]>
1 sibling, 0 replies; 58+ messages in thread
From: shveta malik @ 2026-03-09 10:29 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Zhijie Hou (Fujitsu) <[email protected]>; Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>; shveta malik <[email protected]>
Few comments for 0002:
1)
+ /*
+ * Allow synchronization if the remote sequence data has a more recent
+ * LSN than the local state, or if no local LSN exists yet.
+ */
+ if (!XLogRecPtrIsValid(local_page_lsn) ||
+ local_page_lsn < seqinfo->page_lsn)
+ return COPYSEQ_ALLOWED;
/*
* Skip synchronization if the current user does not have sufficient
* privileges to read the sequence data.
@@ -384,6 +418,40 @@ validate_seqsync_state(LogicalRepSequenceInfo
*seqinfo, Relation sequence_rel)
if (!GetSequence(sequence_rel, &local_last_value, &local_is_called))
return COPYSEQ_INSUFFICIENT_PERM;
Shouldn't the 'COPYSEQ_INSUFFICIENT_PERM' check be the first one, else
we may end up allowing the copy due to the first check even if the
user has no privileges? Or let me know if I am missing something here.
2)
+ ereport(WARNING,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("skipped synchronizing the sequence \"%s.%s\"",
+ seqinfo->nspname, seqinfo->seqname),
+ seqinfo->increment
+ ? errdetail("The local last_value %lld is ahead of the one on publisher",
+ (long long int) local_last_value)
+ : errdetail("The local last_value %lld is behind of the one on publisher",
+ (long long int) local_last_value));
This error message and DETAIL might not be very clear to the user.
IMO, it may not be easy for the user to deduce that a sequence is
descending and thus it being "behind" is the reason for skipping
synchronization.
Shall we update ERROR msg to say ascending/descending
ERROR: skipped synchronizing ascending sequence <name>
ERROR: skipped synchronizing descending sequence <name>
Since we use descending/ascending in sequence doc many times, I feel
it is okay to use it in ERROR messages too to give more clarity.
Thoughts?
3)
A <firstterm>sequence synchronization worker</firstterm> will be started
- after executing any of the above subscriber commands. The worker will
- remain running for the life of the subscription, periodically
- synchronizing all published sequences.
A <firstterm>sequence synchronization worker</firstterm> will be started
+ after executing <command>ALTER SUBSCRIPTION ... REFRESH
PUBLICATION</command>
+ command. The worker will remain running for the life of the subscription,
+ periodically synchronizing all published sequences.
Why have we changed this to mention REFRESH PUB alone responsible for
starting worker? From user's point of view, even CREATE SUB (which has
sequences) will also start the worker no?
thanks
Shveta
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 10:46 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-09 03:16 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
@ 2026-03-13 07:12 ` Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 0 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-13 07:12 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <[email protected]>; +Cc: Ajin Cherian <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>; Amit Kapila <[email protected]>; shveta malik <[email protected]>
On Monday, March 9, 2026 11:17 AM Kuroda, Hayato/黒田 隼人 <[email protected]> wrote:
> Few comments.
Thanks for the comments.
>
> 03.
> Can we describe needed privileges? IIUC, initial sync needs the UPDATE
> privilege, additionally periodic synch needs SELECT privilege.
I could not find existing doc for the privilege required for table sync or
sequence sync, so I think we can consider that as a separate patch.
>
> 04.
> ```
> + long sleep_ms = SEQSYNC_MIN_SLEEP_MS;
> ```
>
> This is used in the for loop, make it the loop-specific variable.
This value should survive across sync loop as it's dynamically
adjusted in the loop.
Other comments have been addressed in latest version.
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 58+ messages in thread
* RE: [PATCH] Support automatic sequence replication
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-11 03:30 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-12 09:23 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-17 11:21 ` Re: [PATCH] Support automatic sequence replication Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-18 07:06 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-18 10:04 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-20 10:57 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-23 01:26 ` RE: [PATCH] Support automatic sequence replication Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-24 06:17 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-24 11:04 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-26 07:37 ` Re: [PATCH] Support automatic sequence replication Ajin Cherian <[email protected]>
2026-02-27 11:07 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-02-28 08:40 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 10:24 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 02:46 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-03-05 10:46 ` Re: [PATCH] Support automatic sequence replication Amit Kapila <[email protected]>
2026-03-05 11:34 ` RE: [PATCH] Support automatic sequence replication Zhijie Hou (Fujitsu) <[email protected]>
@ 2026-03-13 07:12 ` Zhijie Hou (Fujitsu) <[email protected]>
1 sibling, 0 replies; 58+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-03-13 07:12 UTC (permalink / raw)
To: Chao Li <[email protected]>; +Cc: Amit Kapila <[email protected]>; shveta malik <[email protected]>; Ajin Cherian <[email protected]>; Hayato Kuroda (Fujitsu) <[email protected]>; Ashutosh Sharma <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tuesday, March 10, 2026 2:44 PM Chao Li <[email protected]> wrote:
> Thanks for your clarification in the other email. I have reviewed v11 and here
> comes my comments:
Thanks for the comments.
>
> 1 - 0001
> ```
> static CopySeqResult
> -copy_sequence(LogicalRepSequenceInfo *seqinfo, Oid seqowner)
> +validate_seqsync_state(LogicalRepSequenceInfo *seqinfo, Relation
> sequence_rel)
> {
> - UserContext ucxt;
> AclResult aclresult;
> + Oid seqoid = seqinfo->localrelid;
> +
> + /* Perform drift check if it's not the initial sync */
> + if (seqinfo->relstate == SUBREL_STATE_READY)
> + {
> + int64 local_last_value;
> + bool local_is_called;
> +
> + /*
> + * Skip synchronization if the current user does not have
> sufficient
> + * privileges to read the sequence data.
> + */
> + if (!GetSequence(sequence_rel, &local_last_value,
> &local_is_called))
> + return COPYSEQ_INSUFFICIENT_PERM;
> +
> + /*
> + * Skip synchronization if the sequence has not drifted from
> the
> + * publisher's value.
> + */
> + if (local_last_value == seqinfo->last_value &&
> + local_is_called == seqinfo->is_called)
> + return COPYSEQ_NO_DRIFT;
> + }
> +
> + /* Verify that the current user can update the sequence */
> + aclresult = pg_class_aclcheck(seqoid, GetUserId(), ACL_UPDATE);
> +
> + if (aclresult != ACLCHECK_OK)
> + return COPYSEQ_INSUFFICIENT_PERM;
> +
> + return COPYSEQ_ALLOWED;
> +}
> ```
>
> I think we can move pg_class_aclcheck() to before the drift check, because it’s
> a local check, should be cheaper than a remote check. If the local check fails,
> we don’t need to touch remote.
It makes more sense to structure the permission check around the actual
requirement. Since we only update the sequence when a drift is detected, the
permission check should naturally follow that order. If the permission check
fails, an error will be raised anyway, and in that case, performance is less of
a concern.
>
> 2 - 0001
> ```
> +static CopySeqResult
> +copy_sequence(LogicalRepSequenceInfo *seqinfo, Relation sequence_rel)
> +{
> + UserContext ucxt;
> bool run_as_owner = MySubscription->runasowner;
> Oid seqoid = seqinfo->localrelid;
> + CopySeqResult result;
> + XLogRecPtr local_page_lsn;
> +
> + (void) GetSubscriptionRelState(MySubscription->oid,
> +
> RelationGetRelid(sequence_rel),
> +
> &local_page_lsn);
> ```
>
> I think GetSubscriptionRelState is called to get local_page_lsn. But
> local_page_lsn is only used very late in this function, and there is an early
> return branch, so can we move this call to right before where local_page_lsn
> is used?
Yes, we can do that. Changed.
>
> 3 - 0001
> ```
> /*
> * Record the remote sequence's LSN in pg_subscription_rel and mark
> the
> - * sequence as READY.
> + * sequence as READY if updating a sequence that is in INIT state.
> */
> - UpdateSubscriptionRelState(MySubscription->oid, seqoid,
> SUBREL_STATE_READY,
> - seqinfo->page_lsn,
> false);
> + if (seqinfo->relstate == SUBREL_STATE_INIT ||
> + seqinfo->page_lsn != local_page_lsn)
> + UpdateSubscriptionRelState(MySubscription->oid, seqoid,
> SUBREL_STATE_READY,
> + seqinfo-
> >page_lsn, false);
> ```
>
> In the comment, I think you don’t have to add “if updating .. that is in INIT
> state”, but if you do, then you should also mention the lsn condition.
Changed.
>
> 4 - 0001
> ```
> + else
> + {
> + /*
> + * Double the sleep time, but not beyond the
> maximum allowable
> + * value.
> + */
> + sleep_ms = Min(sleep_ms * 2,
> SEQSYNC_MAX_SLEEP_MS);
> + }
> ```
>
> Double the sleep time when no drift is an optimization. But looks like the
> doubling happens only when all sequences have no drift. Say, there are 1000
> sequences, and only one is hot, then the it will still fetch the 1000 sequences
> from remote every 2 seconds, making the optimization much less efficient. I
> think the worker can wake up every 2 seconds, but next fetch time should be
> per sequence.
I'm unsure whether the complexity of per-sequence interval adjustment is
justified. We could consider this as a future enhancement based on user
feedback.
> 5 - 0001
> ```
> + * If the state of sequence is SUBREL_STATE_READY, only synchronize
> sequences
> ```
>
> “The state of sequence is SUBREL_STATE_READY” is inaccurate, I think it
> should be “the state of sequence sync is SUBREL_STATE_READY”.
I slightly reworded the comments.
> 6 - 0001
>
> start_sequence_sync runs an infinite loop to periodically sync sequences. I
> don’t it has an auto reconnect mechanism. When something wrong happens,
> the sync worker will exit, how can the worker
The comment seems incomplete.
>
> 7 - 0001
>
> Say a major version upgrade use case that uses logical replication. Before
> shutdown the publication side (old version), if there are 1000 sequences, how
> can a user decide if all sequences have been synced? From this perspective,
> would it make sense to log a INFO message when all sequences have no drift?
> If next round still no drift, then don’t repeat the message. In other words,
> when the states between (all no drift) and (any drift) switch, log a INFO
> message.
To determine whether a sequence needs to be resynchronized, users should check
srsublsn in pg_subscription_rel, as documented in [1]. To ensure all sequences
are fully synced, they can simply execute REFRESH SEQUENCE as the final step.
>
> 8 - 0001
> ```
> +bool
> +GetSequence(Relation seqrel, int64 *last_value, bool *is_called)
> ```
>
> This function name feels too general. How about ReadSequenceState or
> GetSequenceLastValueAndIsCalled?
I am not sure if the suggested name is better, considering that we already
have SetSequence().
>
> 9 - 0001
> ```
> + elog(ERROR, "unrecognized Sequence
> replication result: %d", (int) sync_status);
> ```
> Nit: The “S” seems not have to be capital.
Changed.
>
> 10 - 0002
0002 includes significant changes in this version, though I have made an effort to address
the comments that are still applicable.
[1] https://www.postgresql.org/docs/devel/logical-replication-sequences.html#SEQUENCES-OUT-OF-SYNC
Best Regards,
Hou zj
^ permalink raw reply [nested|flat] 58+ messages in thread
end of thread, other threads:[~2026-04-15 10:41 UTC | newest]
Thread overview: 58+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-02-06 09:15 Re: [PATCH] Support automatic sequence replication shveta malik <[email protected]>
2026-02-06 09:17 ` shveta malik <[email protected]>
2026-02-11 03:30 ` shveta malik <[email protected]>
2026-02-12 09:23 ` Ajin Cherian <[email protected]>
2026-02-12 10:02 ` shveta malik <[email protected]>
2026-02-13 05:42 ` Peter Smith <[email protected]>
2026-02-17 11:21 ` Ashutosh Sharma <[email protected]>
2026-02-18 04:48 ` Amit Kapila <[email protected]>
2026-02-18 07:06 ` shveta malik <[email protected]>
2026-02-18 07:41 ` Amit Kapila <[email protected]>
2026-02-18 11:28 ` shveta malik <[email protected]>
2026-02-19 02:15 ` Amit Kapila <[email protected]>
2026-02-19 04:19 ` shveta malik <[email protected]>
2026-02-18 10:04 ` shveta malik <[email protected]>
2026-02-18 21:56 ` Peter Smith <[email protected]>
2026-02-19 02:22 ` Amit Kapila <[email protected]>
2026-02-19 04:05 ` shveta malik <[email protected]>
2026-02-20 10:57 ` Ajin Cherian <[email protected]>
2026-02-23 01:26 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-02-23 05:44 ` Amit Kapila <[email protected]>
2026-02-24 05:36 ` Amit Kapila <[email protected]>
2026-02-24 06:17 ` Amit Kapila <[email protected]>
2026-02-24 11:04 ` Ajin Cherian <[email protected]>
2026-02-24 11:25 ` Amit Kapila <[email protected]>
2026-02-24 13:18 ` Dilip Kumar <[email protected]>
2026-02-25 03:02 ` Amit Kapila <[email protected]>
2026-02-25 03:57 ` shveta malik <[email protected]>
2026-02-26 07:37 ` Ajin Cherian <[email protected]>
2026-02-26 14:13 ` Nisha Moond <[email protected]>
2026-02-27 06:01 ` Amit Kapila <[email protected]>
2026-02-27 11:07 ` Amit Kapila <[email protected]>
2026-02-28 08:40 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-02 07:58 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-04 04:20 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-05 02:45 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 12:57 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-04 10:24 ` shveta malik <[email protected]>
2026-03-05 02:46 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 04:05 ` shveta malik <[email protected]>
2026-03-05 05:48 ` shveta malik <[email protected]>
2026-03-05 11:34 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-06 05:25 ` shveta malik <[email protected]>
2026-03-09 03:13 ` shveta malik <[email protected]>
2026-03-13 07:13 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-13 08:37 ` Chao Li <[email protected]>
2026-03-13 10:41 ` shveta malik <[email protected]>
2026-03-13 11:35 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-16 06:58 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-26 04:25 ` Ajin Cherian <[email protected]>
2026-04-15 10:41 ` Ajin Cherian <[email protected]>
2026-03-06 08:52 ` Chao Li <[email protected]>
2026-03-06 10:27 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-05 10:46 ` Amit Kapila <[email protected]>
2026-03-05 11:34 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-09 03:16 ` Hayato Kuroda (Fujitsu) <[email protected]>
2026-03-09 10:29 ` shveta malik <[email protected]>
2026-03-13 07:12 ` Zhijie Hou (Fujitsu) <[email protected]>
2026-03-13 07:12 ` Zhijie Hou (Fujitsu) <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox