Subject: pgsql: Add ALTER SUBSCRIPTION ... SKIP.

From: Amit Kapila <[email-masked]>
Date: Tue, 22 Mar 2022 01:56:03 +0000

Add ALTER SUBSCRIPTION ... SKIP.

This feature allows skipping the transaction on subscriber nodes.

If incoming change violates any constraint, logical replication stops
until it's resolved. Currently, users need to either manually resolve the
conflict by updating a subscriber-side database or by using function
pg_replication_origin_advance() to skip the conflicting transaction. This
commit introduces a simpler way to skip the conflicting transactions.

The user can specify LSN by ALTER SUBSCRIPTION ... SKIP (lsn = XXX),
which allows the apply worker to skip the transaction finished at
specified LSN. The apply worker skips all data modification changes within
the transaction.

Author: Masahiko Sawada
Reviewed-by: Takamichi Osumi, Hou Zhijie, Peter Eisentraut, Amit Kapila, Shi Yu, Vignesh C, Greg Nancarrow, Haiying Tang, Euler Taveira
Discussion: https://postgr.es/m/CAD21AoDeScrsHhLyEPYqN3sydg6PxAPVBboK=[email-masked]

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/208c5d65bbd60e33e272964578cb74182ac726a8

Modified Files
--------------
doc/src/sgml/catalogs.sgml                      |  10 +
doc/src/sgml/logical-replication.sgml           |  27 +--
doc/src/sgml/ref/alter_subscription.sgml        |  42 +++++
src/backend/catalog/pg_subscription.c           |   1 +
src/backend/catalog/system_views.sql            |   2 +-
src/backend/commands/subscriptioncmds.c         |  73 ++++++++
src/backend/parser/gram.y                       |   9 +
src/backend/replication/logical/worker.c        | 233 +++++++++++++++++++++++-
src/bin/pg_dump/pg_dump.c                       |   4 +
src/bin/psql/describe.c                         |   8 +-
src/bin/psql/tab-complete.c                     |   5 +-
src/include/catalog/catversion.h                |   2 +-
src/include/catalog/pg_subscription.h           |   5 +
src/include/nodes/parsenodes.h                  |   3 +-
src/test/regress/expected/subscription.out      | 126 +++++++------
src/test/regress/sql/subscription.sql           |  11 ++
src/test/subscription/t/029_disable_on_error.pl |  94 ----------
src/test/subscription/t/029_on_error.pl         | 183 +++++++++++++++++++
18 files changed, 665 insertions(+), 173 deletions(-)

---

From: Andres Freund <[email-masked]>
Date: Tue, 22 Mar 2022 16:59:17 -0700

Hi,

> specified LSN. The apply worker skips all data modification changes within

This was missing an include of xlogdefs.h in pg_subscription.h, thus failing
in headerscheck. See e.g.
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2022-03-22%2022%3A22%3A05

I've pushed the trivial fix for that. I'll propose adding headerscheck to CI /
cfbot.

Greetings,

Andres Freund

---

From: Amit Kapila <[email-masked]>
Date: Wed, 23 Mar 2022 07:26:28 +0530

> I've pushed the trivial fix for that. I'll propose adding headerscheck to CI /

Thanks.

-- 
With Regards,
Amit Kapila.

---

From: Alvaro Herrera <[email-masked]>
Date: Sun, 4 Sep 2022 10:18:21 +0200

> Add ALTER SUBSCRIPTION ... SKIP.

There are two messages here that seem oddly worded.

msgid "start skipping logical replication transaction finished at %X/%X"
msgid "done skipping logical replication transaction finished at %X/%X"

Two complaints here.  First, the phrases "start / finished" and "done /
finished" look very strange.  It took me a while to realize that
"finished" refers to the LSN, not to the skipping operation.  Do we ever
talk about a transaction "finished at XYZ" as opposed to a transaction
whose LSN is XYZ?  (This became particularly strange when I realized
that the LSN might come from a PREPARE.)

Second, "logical replication transaction".  Is it not a regular
transaction that we happen to be processing via logical replication?

I think they should say something like

"logical replication starts skipping transaction with LSN %X/%X"
"logical replication completed skipping transaction with LSN %X/%X"

Other ideas?

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
Officer Krupke, what are we to do?
Gee, officer Krupke, Krup you! (West Side Story, "Gee, Officer Krupke")

---

From: Amit Kapila <[email-masked]>
Date: Sun, 4 Sep 2022 17:08:55 +0530

> that the LSN might come from a PREPARE.)

The reason to add "finished at ..." was to be explicit about whether
it is a starting LSN or an end LSN of a transaction. We do have such
differentiation in ReorderBufferTXN (first_lsn ... end_lsn).

> "logical replication completed skipping transaction with LSN %X/%X"

This looks better to me. If you find the above argument to
differentiate between the start and end LSN convincing then we can
think of replacing "with" in the above messages with "finished at". I
see your point related to using "finished at" for PREPARE may not be a
good idea but I don't have better ideas for the same.

-- 
With Regards,
Amit Kapila.

---

From: Masahiko Sawada <[email-masked]>
Date: Tue, 6 Sep 2022 23:10:04 +0900

On Sun, Sep 4, 2022 at 8:39 PM Amit Kapila <[email-masked]> wrote:
> This looks better to me.

+1

> If you find the above argument to

Given that the user normally doesn't need to be aware of the
difference between start LSN and end LSN in the context of using this
feature, I think we can use "with LSN %X/%X".

Regards,

-- 
Masahiko Sawada

---

From: Amit Kapila <[email-masked]>
Date: Wed, 7 Sep 2022 09:16:54 +0530

> feature, I think we can use "with LSN %X/%X".

Fair enough.

Alvaro, would you like to push your proposed change? Otherwise, I am
happy to take care of this.

-- 
With Regards,
Amit Kapila.