From: Amit Kapila <akapila@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Allow users to skip logical replication of data having origin.
Date: Thu, 21 Jul 2022 03:29:39 +0000
Message-ID: <E1oEMsc-000W9e-AX@gemulon.postgresql.org> (raw)
Allow users to skip logical replication of data having origin.
This patch adds a new SUBSCRIPTION parameter "origin". It specifies
whether the subscription will request the publisher to only send changes
that don't have an origin or send changes regardless of origin. Setting it
to "none" means that the subscription will request the publisher to only
send changes that have no origin associated. Setting it to "any" means
that the publisher sends changes regardless of their origin. The default
is "any".
Usage:
CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=postgres port=9999'
PUBLICATION pub1 WITH (origin = none);
This can be used to avoid loops (infinite replication of the same data)
among replication nodes.
This feature allows filtering only the replication data originating from
WAL but for initial sync (initial copy of table data) we don't have such a
facility as we can only distinguish the data based on origin from WAL. As
a follow-up patch, we are planning to forbid the initial sync if the
origin is specified as none and we notice that the publication tables were
also replicated from other publishers to avoid duplicate data or loops.
We forbid to allow creating origin with names 'none' and 'any' to avoid
confusion with the same name options.
Author: Vignesh C, Amit Kapila
Reviewed-By: Peter Smith, Amit Kapila, Dilip Kumar, Shi yu, Ashutosh Bapat, Hayato Kuroda
Discussion: https://postgr.es/m/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/366283961ac0ed6d89014444c6090f3fd02fce0a
Modified Files
--------------
contrib/test_decoding/expected/replorigin.out | 10 ++
contrib/test_decoding/sql/replorigin.sql | 5 +
doc/src/sgml/catalogs.sgml | 14 ++
doc/src/sgml/ref/alter_subscription.sgml | 5 +-
doc/src/sgml/ref/create_subscription.sgml | 15 ++
src/backend/catalog/pg_subscription.c | 8 ++
src/backend/catalog/system_views.sql | 4 +-
src/backend/commands/subscriptioncmds.c | 43 +++++-
.../libpqwalreceiver/libpqwalreceiver.c | 5 +
src/backend/replication/logical/origin.c | 22 ++-
src/backend/replication/logical/worker.c | 2 +
src/backend/replication/pgoutput/pgoutput.c | 27 +++-
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 22 +++
src/bin/psql/describe.c | 7 +-
src/bin/psql/tab-complete.c | 7 +-
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_subscription.h | 17 +++
src/include/replication/pgoutput.h | 1 +
src/include/replication/walreceiver.h | 2 +
src/test/regress/expected/subscription.out | 142 +++++++++++--------
src/test/regress/sql/subscription.sql | 10 ++
src/test/subscription/t/030_origin.pl | 155 +++++++++++++++++++++
24 files changed, 463 insertions(+), 78 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: akapila@postgresql.org
Subject: Re: pgsql: Allow users to skip logical replication of data having origin.
In-Reply-To: <E1oEMsc-000W9e-AX@gemulon.postgresql.org>
References: <E1oEMsc-000W9e-AX@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox