public inbox for [email protected]
help / color / mirror / Atom feedFrom: Shlok Kyal <[email protected]>
To: Peter Smith <[email protected]>
Cc: shveta malik <[email protected]>
Cc: Amit Kapila <[email protected]>
Cc: vignesh C <[email protected]>
Cc: Zhijie Hou (Fujitsu) <[email protected]>
Cc: YeXiu <[email protected]>
Cc: Ian Lawrence Barwick <[email protected]>
Cc: Bharath Rupireddy <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Skipping schema changes in publication
Date: Tue, 23 Dec 2025 12:02:49 +0530
Message-ID: <CANhcyEUiwF9w2U6CzkuOsXkLYMOjPs=6O8QyjU_5fDfYWyMswg@mail.gmail.com> (raw)
In-Reply-To: <CAHut+Pu6ameXD4YwbhMXf8kHBhPJXGpOmc21R5o7Lo18hkSKMQ@mail.gmail.com>
References: <CALDaNm3=JrucjhiiwsYQw5-PGtBHFONa6F7hhWCXMsGvh=tamA@mail.gmail.com>
<CALj2ACVOzhs+BD+abFV2x4oKJdsDNd6SgsE7r8UjnZDCKGEckA@mail.gmail.com>
<CAA4eK1K6Kr88d2S0zFdHRMyuoaZeNh+ktU+oigmCuD09_x_-+g@mail.gmail.com>
<CAHut+PsvC-NezO3MJkdyEz=G1QRje2LntjwhQiEeVbmhOQuBMA@mail.gmail.com>
<CALDaNm18VH2j8cTqfELHQ=0ZNognbGBhbHPteJenWQC6C2dueQ@mail.gmail.com>
<CALDaNm0k_0Ccj47wzJzzPFwgQB7w=R5+Q2_nSqYrmMmjhmcRUw@mail.gmail.com>
<CAHut+Pv_0DwyWoGQNMF+G2AGqMuJTzWQKRtmxaC+=zLTPL-Zkw@mail.gmail.com>
<CALDaNm2-GJt2HsYTkLqQ=ecm=R-vOBw1=aM_d2EiYbz39x_cTQ@mail.gmail.com>
<TYCPR01MB8373C3120C2B3112001ED6F1EDCF9@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<CALDaNm0iZZDB300Dez_97S8G6_RW5QpQ8ef6X3wq8tyK-8wnXQ@mail.gmail.com>
<CAHut+PtiomM+iyAZHvb2dzfsPvRru266KuBe49hKy2n2h+m_zA@mail.gmail.com>
<CALDaNm30KDnwX4Czi29fqLb8JBkuwqjbpj9ixwNXXox574NZqQ@mail.gmail.com>
<CALDaNm1PfKRJsEzbKpyt=v4p3bw+_SzE+LFPsMhR5X+qs+0pPw@mail.gmail.com>
<TYCPR01MB83730A2F1D6A5303E9C1416AEDD99@TYCPR01MB8373.jpnprd01.prod.outlook.com>
<CALDaNm0sAU4s1KTLOEWv=rYo5dQK6uFTJn_0FKj3XG1Nv4D-qw@mail.gmail.com>
<CALDaNm3CLRa95tpas6tEj8x58MUNDShxBNoYS+P8Uq5cryoAOw@mail.gmail.com>
<CALDaNm0EKC3o=v+F7GneGibuCULGKkBWXmNaVB4GR9HoqD066A@mail.gmail.com>
<CALDaNm1Z1Rmqj9s6P9ZzmrVA9F_vZ_DwwhYAJmsjqmY6dS3-hA@mail.gmail.com>
<CAB8KJ=jJGuW=ozKmXZzKDUHZ_-J2ZYGOtJo=i2cnNbSu6=KuYg@mail.gmail.com>
<CALDaNm1mbFP8fxHU_H1Ex4cT2Aq3n8FE79tq0TO5ThvFnDUYMA@mail.gmail.com>
<CAB8KJ=jq4RwTs8K7pokmXQwQppP2ChVJLMSAdXaxAX+c1r+mdg@mail.gmail.com>
<CALDaNm1mJvLni8GODebKBmyegXuZ18bLoG-Pz6H1MCX=vphCYA@mail.gmail.com>
<CALDaNm3dWZCYDih55qTNAYsjCvYXMFv=46UsDWmfCnXMt3kPCg@mail.gmail.com>
<CALDaNm1AQZYgT0tALRrkvpP1Q+8+e7vkGCUjQ-jim1C0q3e=zA@mail.gmail.com>
<CAA4eK1KRdAPC=5=7tQ1GW0cRwD=zaDMi+T4u_k4GxPhPY6e8BQ@mail.gmail.com>
<OS3PR01MB5718C8BE84B862E7E0CEC29B94BD2@OS3PR01MB5718.jpnprd01.prod.outlook.com>
<CAA4eK1KYQz7cf46_D=6VkZ4J6Y8vJ88MMi=6zm2TJXDP+V1mLg@mail.gmail.com>
<CANhcyEXZq4mP5dNgg7u=sMPwvxA4_ZN9U92uZEuzs=0xTu+8Yg@mail.gmail.com>
<CANhcyEXspT3v5-Tdop9uqQV2HWBvZoN5P0BxXQ6Md6Mr7GXK9A@mail.gmail.com>
<CAHut+PuiaLOCkiAx9nPnjk6wTbPFvnm9T5svTuKbgwJwTdea8w@mail.gmail.com>
<CANhcyEV_MePxgftHY65et1WdOAk70M0C7PZ1STPUO8PXHVB1YA@mail.gmail.com>
<CAHut+Ps0hSNqrjv_jT1AuXxO-CrZue3ixE0jKsxVhtArMrkujQ@mail.gmail.com>
<CANhcyEXX3viVpYcGHD_fzhf_f6CDQWr2+VBywrJf5zm_XiB4tg@mail.gmail.com>
<CAHut+PsXP_61ZXuVOx5u9FZGK3oH4taaA59oOzgqyygZx8ezWw@mail.gmail.com>
<CANhcyEU+aPu6iAH2cTA0cDtn3pd6c_njBONCt3FubYZoEEnm8Q@mail.gmail.com>
<CAHut+Pv2P6dJ7hZj_fmzN+=xzjvpOpgkAJvDZg3TD2xpvmY1NQ@mail.gmail.com>
<CANhcyEW2LK4diNeCG862DE40yQoV3VAgf59kXUq2TuR8fnw5vQ@mail.gmail.com>
<CAHut+PuSHScrODVGCM7P53Mv1HE2N6ThzkH4+gQ1eFXVeD-OCA@mail.gmail.com>
<CANhcyEUtYV-9ujtxLasnxN_peT+3LuZjcRx1xUECh1CCmANB8w@mail.gmail.com>
<CAHut+PuviFA6C7qps=+kDYfe3P99as8NCjbR=SYxoi0o96ipoA@mail.gmail.com>
<CANhcyEXkeg3sjkS3DS9yU1ckz4ozUBNZ+RmrWaRNSSVCR8RquA@mail.gmail.com>
<CAHut+PsHavMy_KJ0MwR9J6q0BTTty54TxS-KBZc7X6tb4u7rfA@mail.gmail.com>
<CANhcyEU=k4+0BqOu25N76g738XKUwfLGGdf8e+ssGiRKHC4RwQ@mail.gmail.com>
<CAHut+Ptdi8txJbMc+5Sp2uB4QLGOp-rZdkBbXyhAkxim0iAhBQ@mail.gmail.com>
<CANhcyEW+uJB_bvQLEaZCgoRTc1=i+QnrPPHxZ2=0SBSCyj9pkg@mail.gmail.com>
<CAHut+Ptq0-tMYUOvG3yR34AvuEzR9vUH=muqV_=uEO3zCuA6rA@mail.gmail.com>
<CANhcyEUEMWSkTfGc7Q3B+UiOzSiOmOGLgK-+C5DXwtCGOnDBhg@mail.gmail.com>
<CAHut+PuAgSDr3kbSxPMYEyCeGiJ5hgaT1JUvuiYPRT=Q+--O-Q@mail.gmail.com>
<CANhcyEU_uuiKMRrd_E_DeYsyCvwY_u995E-Do3d66J7tQnzdzw@mail.gmail.com>
<CANhcyEWK9sj9UY4uaV36Q8qxUv=8Joch0o98RCN5U3xLjUxAag@mail.gmail.com>
<CALDaNm32XQDR4qsOhPQeophVbZ8r+ShJSSssoVfdPcwG6joPHQ@mail.gmail.com>
<CANhcyEVt2CBnG7MOktaPPV4rYapHR-VHe5=qoziTZh1L9SVc6w@mail.gmail.com>
<CAHut+PtGu2j72yV_as_TVKfWr3ctd18svReGEx3xa=q5BtKyKA@mail.gmail.com>
<CANhcyEUh9ki36VTXyYf8UqUrfLX9ZhfP_f2LjpvvycgqWLQqqQ@mail.gmail.com>
<CAHut+Pv96u0VciuSx2B99jDHEvn7svVJynCmw-qYb=z4Kc2knA@mail.gmail.com>
<CANhcyEWGiWwGt1-v6d9bCAae9Np7cNPt+iRV9PXBZ0z=75XEVw@mail.gmail.com>
<CANhcyEWAQHtUfgNPA2m-+okEh7pXaK5irBm+yzyNVJXL2LUTXw@mail.gmail.com>
<CAHut+PuzcKXheZwgNvDJkwK5txd1kzNRxCmcJcbr=_9mGHjKtA@mail.gmail.com>
<CANhcyEXCKPCAdoqBLAhxt64Nwf+7T52dd8daE3qvhBNTvro13Q@mail.gmail.com>
<CAHut+Pu50yWjMR5Mswhi6uVKQmn3hO9o0ocRAgXyUUf4cnVTwg@mail.gmail.com>
<CANhcyEV7ewT+nfLM2owquxW-_6m8Ju+P93y=acoS=JCBHoT-MQ@mail.gmail.com>
<CAJpy0uDdQH7b=LRn_HevbmFACpvq9=c32Vb3tRvjSOnDsQd74w@mail.gmail.com>
<CAA4eK1KZ1Sb0soHp3HH2htwJ3=qka-eQjW35vOW3+4VeWw4VoQ@mail.gmail.com>
<CANhcyEXwLrQsec6g+1dqWTKyJQMQMh=getj28C+zLL14BjuumA@mail.gmail.com>
<CAJpy0uD35+YmDahVNUm+NZsjisV_k2hUDBkiFOJDwAJ1o1CT9A@mail.gmail.com>
<CANhcyEV_EVi5cgJ6WPvmeVAqjCS7Of+VAWuRHZtsVf8PQb_z7g@mail.gmail.com>
<CAHut+Pu5Ukuvd_5oK_mTEjSOEAYupbEXzRqXwSPoLe_sDkA_fg@mail.gmail.com>
<CANhcyEWg2WbEW_fFwk0D3J2KBrUF7th6VrE+gvESgkUKP9VpZg@mail.gmail.com>
<CAJpy0uBegaytfG=AS5VUb-6jAEDzC374-1icn-hP5AnRoMJ+Lg@mail.gmail.com>
<CAJpy0uD+orAxUWKq9Ogf5FEWtXcwkQXb_YZOvXqDc8b15nJe9A@mail.gmail.com>
<CANhcyEVDTPVRWDm-BdJe0yaWFuz1ozTUFNHKvzX2YuiUHh=ArA@mail.gmail.com>
<CAHut+Pu6ameXD4YwbhMXf8kHBhPJXGpOmc21R5o7Lo18hkSKMQ@mail.gmail.com>
On Mon, 22 Dec 2025 at 11:37, Peter Smith <[email protected]> wrote:
>
> Hi Shlok.
>
> Some review comments for patch v33-0001 (code part)
>
> ======
> src/backend/catalog/pg_publication.c
>
> GetPublicationRelationsInternal:
>
> 1.
> Static function names should be snake_case.
>
> ~~~
>
> GetPublicationIncludedRelations:
>
> 2.
> +/*
> + * Return the list of relation OIDs for a publication.
> + *
> + * For a FOR TABLE publication, this returns the list of relations explicitly
> + * included in the publication.
> + *
> + * Publications declared with FOR ALL TABLES or FOR ALL SEQUENCES should use
> + * GetAllPublicationRelations() to obtain the complete set of tables covered by
> + * the publication.
> + */
> +List *
> +GetPublicationIncludedRelations(Oid pubid, PublicationPartOpt pub_partopt)
> +{
> + Assert(!GetPublication(pubid)->alltables);
> +
> + return GetPublicationRelationsInternal(pubid, pub_partopt, false);
> +}
>
> Why isn't the Assert also saying something about puballsequences, as
> mentioned in the function comment?
>
I reported a similar kind of issue in HEAD in [1].
As per the latest discussion, I understood that it is ok to call this
function for ALL SEQUENCES.
I have updated the comment.
> ~~~
>
> GetAllPublicationRelations:
>
> 3.
> + * root partitioned tables. The list also excludes tables that are
> + * explicitly excluded via the EXCEPT TABLE clause of the publication
> + * identified by pubid. Neither of these rules applies to FOR ALL SEQUENCES
> + * publications.
>
> 3.
> It seems wrong to say "FOR ALL SEQUENCES" ... that seems to assume the
> "FOR ALL SEQUENCES" and "FOR ALL TABLES" cannot co-exist. Did you mean
> "Neither of ... to published sequences"?
>
I have modified the comment.
> ~
>
> 4.
> -GetAllPublicationRelations(char relkind, bool pubviaroot)
> +GetAllPublicationRelations(Oid pubid, char relkind, bool pubviaroot)
>
> There are tricky rules about relation vs sequences and the
> publish_via_partition_root parameter value. It would be better if you
> encapsulate all this within this function. Specifically, it would be
> simpler if you passed the 'Publication' arg instead of the pubid. Then
> you can get the pubviaroot value from that (within the function)
> instead of passing around "fake" values of false when you are looking
> at RELKIND_SEQUENCE.
> ======
> src/backend/commands/publicationcmds.c
>
> ObjectsInAllPublicationToOids:
>
> 5.
> + foreach_ptr(PublicationAllObjSpec, puballobj, puballobjspec_list)
> + {
> + if (puballobj->pubobjtype != PUBLICATION_ALL_TABLES)
> + continue;
> +
> + foreach_ptr(PublicationObjSpec, pubobj, puballobj->except_tables)
> + {
> + pubobj->pubtable->except = true;
> + *rels = lappend(*rels, pubobj->pubtable);
> + }
> + }
>
> I think it's tidier to code this like below:
>
> if (puballobj->pubobjtype == PUBLICATION_ALL_TABLES)
> {
> foreach_ptr...
> }
>
> ~~~
>
> pub_contains_invalid_column:
>
> 6.
> bool
> pub_contains_invalid_column(Oid pubid, Relation relation, List *ancestors,
> bool pubviaroot, char pubgencols_type,
> - bool *invalid_column_list,
> - bool *invalid_gen_col)
> + bool *invalid_column_list, bool *invalid_gen_col)
>
> Why does this change even exist at all in this patch?
This change is not required. I have reverted it.
>
> ~~~
>
> CreatePublication:
>
> 7.
> + /*
> + * If the publication is for ALL TABLES and 'relations' is not empty, it
> + * indicates that some relations should be excluded from the publication.
> + * Add those excluded relations to the publication with 'prexcept' set to
> + * true. Otherwise, 'relations' contains the list of relations to be
> + * explicitly included in the publication.
> + */
> + if (relations != NIL)
> + {
> + List *rels;
> +
> + rels = OpenTableList(relations);
> + TransformPubWhereClauses(rels, pstate->p_sourcetext,
> + publish_via_partition_root);
> +
> + CheckPubRelationColumnList(stmt->pubname, rels,
> + schemaidlist != NIL,
> + publish_via_partition_root);
> +
> + PublicationAddTables(puboid, rels, true, NULL);
> + CloseTableList(rels);
> + }
> +
>
> The comment and the code don't match. The comment is talking about
> rules for FOR ALL TABLES, but puballtables is not part of any
> condition here (??). Was all this supposed to be within the "if
> (stmt->for_all_tables)" code block?
>
For both ALL TABLES publication and non-ALL TABLES publication we need
the same code block.
Setting of prexcept flag will be handled in PublicationAddTables.
This comment clarifies what the list 'relations' would mean for ALL
TABLES publication and non-ALL TABLES publication
> ======
> src/bin/pg_dump/pg_dump.c
>
> 8.
> - "SELECT tableoid, oid, prpubid, prrelid, "
> + "SELECT tableoid, oid, prpubid, prrelid,\n"
> "pg_catalog.pg_get_expr(prqual, prrelid) AS prrelqual, "
> "(CASE\n"
> " WHEN pr.prattrs IS NOT NULL THEN\n"
> @@ -4868,6 +4929,9 @@ getPublicationTables(Archive *fout, TableInfo
> tblinfo[], int numTables)
> " WHERE attrelid = pr.prrelid AND attnum = prattrs[s])\n"
> " ELSE NULL END) prattrs "
> "FROM pg_catalog.pg_publication_rel pr");
> + if (fout->remoteVersion >= 190000)
> + appendPQExpBufferStr(query, " WHERE prexcept = false");
>
> 8a
> Isn't it better to qualify everything here with the alias 'pr'?
>
It is an existing code. So I prefer not to modify it in this patch. I
have added the alias for the column added by this patch.
> ~
>
> 8b.
> Also "WHERE NOT pr.prexcept;" might be more conssitent with other code
> I saw in describe.c
>
> ======
> src/bin/pg_dump/pg_dump.h
>
> 9.
> PublishGencolsType pubgencols_type;
> + SimplePtrList excepttbls;
> } PublicationInfo;
>
> How about "tables instead of "tbls" (e.g. "excepttables" or
> "except_tables") here? That would also be more consistent with the
> other puballtables member.
>
> ======
> src/test/regress/sql/publication.sql
>
> 10.
> RESET client_min_messages;
> \dRp+ testpub3
> \dRp+ testpub4
> +\dRp+ testpub5
> +\dRp+ testpub6
> +\dRp+ testpub7
>
>
> I feel it would be better to keep each \dRp+ together with the test it
> belongs to, rather than have a bunch of different tests which are then
> followed by a bunch of different \dRp+. Note: this same comment
> applies to other place of places -- not just here. Check everywhere
> you do \dRp+
>
I have addressed the remaining comments, did some cosmetic changes and
addressed the comment shared by Shveta in [2].
[1]: https://www.postgresql.org/message-id/[email protected]...
[2]: https://www.postgresql.org/message-id/CAJpy0uCf5tXvqyVS3GQzU9J5HdSLAxX6Lxt1UKY4HJ8qnimCAw%40mail.gma...
Thanks,
Shlok Kyal
Attachments:
[application/octet-stream] v34-0001-Skip-publishing-the-tables-specified-in-EXCEPT-T.patch (64.4K, 2-v34-0001-Skip-publishing-the-tables-specified-in-EXCEPT-T.patch)
download | inline diff:
From 57cb41080989077653ed3185912acedbe8171c96 Mon Sep 17 00:00:00 2001
From: Shlok Kyal <[email protected]>
Date: Tue, 9 Dec 2025 22:41:23 +0530
Subject: [PATCH v34] Skip publishing the tables specified in EXCEPT TABLE.
A new "EXCEPT TABLE" clause for CREATE PUBLICATION allows one or more
tables to be excluded. The publisher will not send the data of excluded
tables to the subscriber.
The new syntax allows specifying excluded relations when creating a
publication. For example:
CREATE PUBLICATION pub1 FOR ALL TABLES EXCEPT TABLE (t1,t2);
A new column "prexcept" is added to table "pg_publication_rel", to flag
the relations that the user wants to exclude from the publications.
pg_dump is updated to identify and dump the excluded tables of the publications.
The psql \d family of commands can now display excluded tables.
Bump catalog version.
---
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/logical-replication.sgml | 6 +-
doc/src/sgml/ref/create_publication.sgml | 55 ++++-
doc/src/sgml/ref/psql-ref.sgml | 8 +-
src/backend/catalog/pg_publication.c | 156 ++++++++++---
src/backend/commands/publicationcmds.c | 106 ++++++---
src/backend/commands/tablecmds.c | 4 +-
src/backend/parser/gram.y | 33 ++-
src/backend/replication/pgoutput/pgoutput.c | 22 +-
src/backend/utils/cache/relcache.c | 21 +-
src/bin/pg_dump/pg_dump.c | 64 +++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 30 +++
src/bin/psql/describe.c | 87 ++++++-
src/bin/psql/tab-complete.in.c | 12 +-
src/include/catalog/pg_publication.h | 13 +-
src/include/catalog/pg_publication_rel.h | 1 +
src/include/nodes/parsenodes.h | 3 +
src/test/regress/expected/publication.out | 77 +++++-
src/test/regress/sql/publication.sql | 34 ++-
src/test/subscription/meson.build | 1 +
.../t/037_rep_changes_except_table.pl | 219 ++++++++++++++++++
22 files changed, 851 insertions(+), 112 deletions(-)
create mode 100644 src/test/subscription/t/037_rep_changes_except_table.pl
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 2fc63442980..9e847152b44 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -6581,6 +6581,16 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
if there is no publication qualifying condition.</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>prexcept</structfield> <type>bool</type>
+ </para>
+ <para>
+ True if the relation is excluded from the publication. See
+ <link linkend="sql-createpublication-params-for-except-table"><literal>EXCEPT TABLE</literal></link>.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>prattrs</structfield> <type>int2vector</type>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index b3faaa675ef..ba454b78cbc 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -116,7 +116,11 @@
<literal>FOR TABLES IN SCHEMA</literal>, <literal>FOR ALL TABLES</literal>,
or <literal>FOR ALL SEQUENCES</literal>. Unlike tables, sequences can be
synchronized at any time. For more information, see
- <xref linkend="logical-replication-sequences"/>.
+ <xref linkend="logical-replication-sequences"/>. When a publication is
+ created with <literal>FOR ALL TABLES</literal>, tables can be explicitly
+ excluded from publication using the
+ <link linkend="sql-createpublication-params-for-except-table"><literal>EXCEPT TABLE</literal></link>
+ clause.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index 75a508bebfa..77bca21ce92 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -32,12 +32,16 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
<phrase>and <replaceable class="parameter">publication_all_object</replaceable> is one of:</phrase>
- ALL TABLES
+ ALL TABLES [ EXCEPT [ TABLE ] ( <replaceable class="parameter">except_table_object</replaceable> [, ... ] ) ]
ALL SEQUENCES
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
[ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ ( <replaceable class="parameter">column_name</replaceable> [, ... ] ) ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ]
+
+<phrase>and <replaceable class="parameter">except_table_object</replaceable> is:</phrase>
+
+ [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ]
</synopsis>
</refsynopsisdiv>
@@ -164,7 +168,8 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
<listitem>
<para>
Marks the publication as one that replicates changes for all tables in
- the database, including tables created in the future.
+ the database, including tables created in the future. Tables listed in
+ EXCEPT TABLE are excluded from the publication.
</para>
</listitem>
</varlistentry>
@@ -184,6 +189,35 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry id="sql-createpublication-params-for-except-table">
+ <term><literal>EXCEPT TABLE</literal></term>
+ <listitem>
+ <para>
+ This clause specifies a list of tables to be excluded from the
+ publication.
+ </para>
+ <para>
+ For inherited tables, if <literal>ONLY</literal> is specified before the
+ table name, only that table is excluded from the publication. If
+ <literal>ONLY</literal> is not specified, the table and all its descendant
+ tables (if any) are excluded. Optionally, <literal>*</literal> can be
+ specified after the table name to explicitly indicate that descendant
+ tables are excluded. This does not apply to a partitioned table, however.
+ </para>
+ <para>
+ For partitioned tables, when <literal>publish_via_partition_root</literal>
+ is set to <literal>true</literal>, specifying a root partitioned table in
+ <literal>EXCEPT TABLE</literal> excludes it and all its partitions from
+ replication. Specifying a leaf partition has no effect, as its changes are
+ still replicated via the root partitioned table. When
+ <literal>publish_via_partition_root</literal> is set to
+ <literal>false</literal>, specifying a root partitioned table has no
+ effect, as changes are replicated via the leaf partitions. Specifying a
+ leaf partition excludes only that partition from replication.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createpublication-params-with">
<term><literal>WITH ( <replaceable class="parameter">publication_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
@@ -487,6 +521,23 @@ CREATE PUBLICATION all_sequences FOR ALL SEQUENCES;
all sequences for synchronization:
<programlisting>
CREATE PUBLICATION all_tables_sequences FOR ALL TABLES, ALL SEQUENCES;
+</programlisting>
+ </para>
+
+ <para>
+ Create a publication that publishes all changes in all tables except
+ <structname>users</structname> and <structname>departments</structname>:
+<programlisting>
+CREATE PUBLICATION all_tables_except FOR ALL TABLES EXCEPT (users, departments);
+</programlisting>
+ </para>
+
+ <para>
+ Create a publication that publishes all sequences for synchronization, and
+ all changes in all tables except <structname>users</structname> and
+ <structname>departments</structname>:
+<programlisting>
+CREATE PUBLICATION all_sequences_tables_except FOR ALL SEQUENCES, ALL TABLES EXCEPT (users, departments);
</programlisting>
</para>
</refsect1>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index f56c70263e0..c8fcb126c8b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -1301,7 +1301,8 @@ SELECT $1 \parse stmt1
special attributes such as <literal>NOT NULL</literal> or defaults.
Associated indexes, constraints, rules, and triggers are
also shown. For foreign tables, the associated foreign
- server is shown as well.
+ server is shown as well. For a table, the associated publications and
+ the publications from which the table is excluded are also shown.
(<quote>Matching the pattern</quote> is defined in
<xref linkend="app-psql-patterns"/> below.)
</para>
@@ -2103,8 +2104,9 @@ SELECT $1 \parse stmt1
listed.
If <literal>x</literal> is appended to the command name, the results
are displayed in expanded mode.
- If <literal>+</literal> is appended to the command name, the tables and
- schemas associated with each publication are shown as well.
+ If <literal>+</literal> is appended to the command name, the tables,
+ excluded tables, and schemas associated with each publication are shown
+ as well.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 7aa3f179924..c92ff928878 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -366,11 +366,25 @@ GetTopMostAncestorInPublication(Oid puboid, List *ancestors, int *ancestor_level
foreach(lc, ancestors)
{
Oid ancestor = lfirst_oid(lc);
- List *apubids = GetRelationPublications(ancestor);
+ List *apubids = NIL;
+ List *aexceptpubids = NIL;
List *aschemaPubids = NIL;
+ GetRelationPublications(ancestor, &apubids, &aexceptpubids);
+
level++;
+ /*
+ * This step is not strictly necessary, but is kept to allow skipping
+ * the ancestor if it is part of the publication's EXCEPT TABLE list,
+ * avoiding an expensive schema-mapping check later.
+ */
+ if (list_member_oid(aexceptpubids, puboid))
+ {
+ list_free(aexceptpubids);
+ continue;
+ }
+
if (list_member_oid(apubids, puboid))
{
topmost_relid = ancestor;
@@ -391,6 +405,7 @@ GetTopMostAncestorInPublication(Oid puboid, List *ancestors, int *ancestor_level
}
list_free(apubids);
+ list_free(aexceptpubids);
list_free(aschemaPubids);
}
@@ -466,6 +481,26 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
RelationGetRelationName(targetrel), pub->name)));
}
+ /*
+ * Handle the case where a partition is excluded by EXCEPT TABLE while
+ * publish_via_partition_root = true.
+ */
+ if (pub->alltables && pub->pubviaroot && pri->except &&
+ targetrel->rd_rel->relispartition)
+ ereport(WARNING,
+ (errmsg("partition \"%s\" might be replicated as publish_via_partition_root is \"%s\"",
+ RelationGetRelationName(targetrel), "true")));
+
+ /*
+ * Handle the case where a partitioned table is excluded by EXCEPT TABLE
+ * while publish_via_partition_root = false.
+ */
+ if (pub->alltables && !pub->pubviaroot && pri->except &&
+ targetrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ ereport(WARNING,
+ (errmsg("partitioned table \"%s\" might be replicated as publish_via_partition_root is \"%s\"",
+ RelationGetRelationName(targetrel), "false")));
+
check_publication_add_relation(targetrel);
/* Validate and translate column names into a Bitmapset of attnums. */
@@ -482,6 +517,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
ObjectIdGetDatum(pubid);
values[Anum_pg_publication_rel_prrelid - 1] =
ObjectIdGetDatum(relid);
+ values[Anum_pg_publication_rel_prexcept - 1] =
+ BoolGetDatum(pri->except);
/* Add qualifications, if available */
if (pri->whereClause != NULL)
@@ -749,38 +786,58 @@ publication_add_schema(Oid pubid, Oid schemaid, bool if_not_exists)
return myself;
}
-/* Gets list of publication oids for a relation */
-List *
-GetRelationPublications(Oid relid)
+/*
+ * Get the list of publication oids associated with a specified relation.
+ *
+ * Parameter 'pubids' returns the OIDs of the publications the relation is part
+ * of. Parameter 'except_pubids' returns the OIDs of publications the relation
+ * is excluded from.
+ *
+ * This function returns true if the relation is part of any publication.
+ */
+bool
+GetRelationPublications(Oid relid, List **pubids, List **except_pubids)
{
- List *result = NIL;
CatCList *pubrellist;
- int i;
+ bool found = false;
/* Find all publications associated with the relation. */
pubrellist = SearchSysCacheList1(PUBLICATIONRELMAP,
ObjectIdGetDatum(relid));
- for (i = 0; i < pubrellist->n_members; i++)
+ for (int i = 0; i < pubrellist->n_members; i++)
{
HeapTuple tup = &pubrellist->members[i]->tuple;
- Oid pubid = ((Form_pg_publication_rel) GETSTRUCT(tup))->prpubid;
+ Form_pg_publication_rel pubrel = (Form_pg_publication_rel) GETSTRUCT(tup);
+ Oid pubid = pubrel->prpubid;
- result = lappend_oid(result, pubid);
+ if (pubrel->prexcept)
+ {
+ if (except_pubids)
+ *except_pubids = lappend_oid(*except_pubids, pubid);
+ }
+ else
+ {
+ if (pubids)
+ *pubids = lappend_oid(*pubids, pubid);
+ found = true;
+ }
}
ReleaseSysCacheList(pubrellist);
- return result;
+ return found;
}
/*
- * Gets list of relation oids for a publication.
+ * Internal function to get the list of relation OIDs for a publication.
*
- * This should only be used FOR TABLE publications, the FOR ALL TABLES/SEQUENCES
- * should use GetAllPublicationRelations().
+ * If except_flag is true, returns the list of relations excluded from the
+ * publication; otherwise, returns the list of relations included in the
+ * publication.
*/
-List *
-GetPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt)
+static List *
+get_publication_relations_internal(Oid pubid, PublicationPartOpt pub_partopt,
+ bool except_flag)
{
List *result;
Relation pubrelsrel;
@@ -805,8 +862,10 @@ GetPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt)
Form_pg_publication_rel pubrel;
pubrel = (Form_pg_publication_rel) GETSTRUCT(tup);
- result = GetPubPartitionOptionRelations(result, pub_partopt,
- pubrel->prrelid);
+
+ if (except_flag == pubrel->prexcept)
+ result = GetPubPartitionOptionRelations(result, pub_partopt,
+ pubrel->prrelid);
}
systable_endscan(scan);
@@ -819,6 +878,36 @@ GetPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt)
return result;
}
+/*
+ * Return the list of relation OIDs for a publication.
+ *
+ * For a FOR TABLE publication, this returns the list of relations explicitly
+ * included in the publication.
+ *
+ * Publications declared with FOR ALL TABLES should use
+ * GetAllPublicationRelations() to obtain the complete set of tables covered by
+ * the publication.
+ */
+List *
+GetPublicationIncludedRelations(Oid pubid, PublicationPartOpt pub_partopt)
+{
+ Assert(!GetPublication(pubid)->alltables);
+
+ return get_publication_relations_internal(pubid, pub_partopt, false);
+}
+
+/*
+ * Return the list of relation OIDs excluded from a publication.
+ * This is only applicable for FOR ALL TABLES publications.
+ */
+List *
+GetPublicationExcludedRelations(Oid pubid, PublicationPartOpt pub_partopt)
+{
+ Assert(GetPublication(pubid)->alltables);
+
+ return get_publication_relations_internal(pubid, pub_partopt, true);
+}
+
/*
* Gets list of publication oids for publications marked as FOR ALL TABLES.
*/
@@ -864,18 +953,29 @@ GetAllTablesPublications(void)
* partitioned tables, we must exclude partitions in favor of including the
* root partitioned tables. This is not applicable to FOR ALL SEQUENCES
* publication.
+ *
+ * FOR ALL TABLES publication, the list excludes the tables that are explicitly
+ * mentioned in EXCEPT TABLE clause.
*/
List *
-GetAllPublicationRelations(char relkind, bool pubviaroot)
+GetAllPublicationRelations(Publication *pub, char relkind)
{
Relation classRel;
ScanKeyData key[1];
TableScanDesc scan;
HeapTuple tuple;
List *result = NIL;
+ List *exceptlist = NIL;
+ bool pubviaroot = pub->pubviaroot;
+ Oid pubid = pub->oid;
Assert(!(relkind == RELKIND_SEQUENCE && pubviaroot));
+ if (relkind == RELKIND_RELATION)
+ exceptlist = GetPublicationExcludedRelations(pubid, pubviaroot ?
+ PUBLICATION_PART_ALL :
+ PUBLICATION_PART_ROOT);
+
classRel = table_open(RelationRelationId, AccessShareLock);
ScanKeyInit(&key[0],
@@ -891,7 +991,8 @@ GetAllPublicationRelations(char relkind, bool pubviaroot)
Oid relid = relForm->oid;
if (is_publishable_class(relid, relForm) &&
- !(relForm->relispartition && pubviaroot))
+ !(relForm->relispartition && pubviaroot) &&
+ !list_member_oid(exceptlist, relid))
result = lappend_oid(result, relid);
}
@@ -912,7 +1013,8 @@ GetAllPublicationRelations(char relkind, bool pubviaroot)
Oid relid = relForm->oid;
if (is_publishable_class(relid, relForm) &&
- !relForm->relispartition)
+ !relForm->relispartition &&
+ !list_member_oid(exceptlist, relid))
result = lappend_oid(result, relid);
}
@@ -1168,17 +1270,17 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
* those. Otherwise, get the partitioned table itself.
*/
if (pub_elem->alltables)
- pub_elem_tables = GetAllPublicationRelations(RELKIND_RELATION,
- pub_elem->pubviaroot);
+ pub_elem_tables = GetAllPublicationRelations(pub_elem,
+ RELKIND_RELATION);
else
{
List *relids,
*schemarelids;
- relids = GetPublicationRelations(pub_elem->oid,
- pub_elem->pubviaroot ?
- PUBLICATION_PART_ROOT :
- PUBLICATION_PART_LEAF);
+ relids = GetPublicationIncludedRelations(pub_elem->oid,
+ pub_elem->pubviaroot ?
+ PUBLICATION_PART_ROOT :
+ PUBLICATION_PART_LEAF);
schemarelids = GetAllSchemaPublicationRelations(pub_elem->oid,
pub_elem->pubviaroot ?
PUBLICATION_PART_ROOT :
@@ -1367,7 +1469,7 @@ pg_get_publication_sequences(PG_FUNCTION_ARGS)
publication = GetPublicationByName(pubname, false);
if (publication->allsequences)
- sequences = GetAllPublicationRelations(RELKIND_SEQUENCE, false);
+ sequences = GetAllPublicationRelations(publication, RELKIND_SEQUENCE);
funcctx->user_fctx = sequences;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index a1983508950..84a3d95ea8a 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -170,6 +170,30 @@ parse_publication_options(ParseState *pstate,
}
}
+/*
+ * Convert the PublicationObjSpec list which is part of
+ * PublicationAllObjSpecType list into PublicationTable list.
+ */
+static void
+ObjectsInAllPublicationToOids(List *puballobjspec_list,
+ ParseState *pstate, List **rels)
+{
+ if (!puballobjspec_list)
+ return;
+
+ foreach_ptr(PublicationAllObjSpec, puballobj, puballobjspec_list)
+ {
+ if (puballobj->pubobjtype == PUBLICATION_ALL_TABLES)
+ {
+ foreach_ptr(PublicationObjSpec, pubobj, puballobj->except_tables)
+ {
+ pubobj->pubtable->except = true;
+ *rels = lappend(*rels, pubobj->pubtable);
+ }
+ }
+ }
+}
+
/*
* Convert the PublicationObjSpecType list into schema oid list and
* PublicationTable list.
@@ -193,7 +217,12 @@ ObjectsInPublicationToOids(List *pubobjspec_list, ParseState *pstate,
switch (pubobj->pubobjtype)
{
+ case PUBLICATIONOBJ_EXCEPT_TABLE:
+ pubobj->pubtable->except = true;
+ *rels = lappend(*rels, pubobj->pubtable);
+ break;
case PUBLICATIONOBJ_TABLE:
+ pubobj->pubtable->except = false;
*rels = lappend(*rels, pubobj->pubtable);
break;
case PUBLICATIONOBJ_TABLES_IN_SCHEMA:
@@ -514,8 +543,8 @@ InvalidatePubRelSyncCache(Oid pubid, bool puballtables)
* a target. However, WAL records for TRUNCATE specify both a root and
* its leaves.
*/
- relids = GetPublicationRelations(pubid,
- PUBLICATION_PART_ALL);
+ relids = GetPublicationIncludedRelations(pubid,
+ PUBLICATION_PART_ALL);
schemarelids = GetAllSchemaPublicationRelations(pubid,
PUBLICATION_PART_ALL);
@@ -925,14 +954,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Associate objects with the publication. */
if (stmt->for_all_tables)
- {
- /*
- * Invalidate relcache so that publication info is rebuilt. Sequences
- * publication doesn't require invalidation, as replica identity
- * checks don't apply to them.
- */
- CacheInvalidateRelcacheAll();
- }
+ ObjectsInAllPublicationToOids(stmt->pubobjects, pstate, &relations);
+
else if (!stmt->for_all_sequences)
{
ObjectsInPublicationToOids(stmt->pubobjects, pstate, &relations,
@@ -944,22 +967,6 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser to create FOR TABLES IN SCHEMA publication"));
- if (relations != NIL)
- {
- List *rels;
-
- rels = OpenTableList(relations);
- TransformPubWhereClauses(rels, pstate->p_sourcetext,
- publish_via_partition_root);
-
- CheckPubRelationColumnList(stmt->pubname, rels,
- schemaidlist != NIL,
- publish_via_partition_root);
-
- PublicationAddTables(puboid, rels, true, NULL);
- CloseTableList(rels);
- }
-
if (schemaidlist != NIL)
{
/*
@@ -971,8 +978,37 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * If the publication is for ALL TABLES and 'relations' is not empty, it
+ * indicates that some relations should be excluded from the publication.
+ * Add those excluded relations to the publication with 'prexcept' set to
+ * true. Otherwise, 'relations' contains the list of relations to be
+ * explicitly included in the publication.
+ */
+ if (relations != NIL)
+ {
+ List *rels;
+
+ rels = OpenTableList(relations);
+ TransformPubWhereClauses(rels, pstate->p_sourcetext,
+ publish_via_partition_root);
+
+ CheckPubRelationColumnList(stmt->pubname, rels,
+ schemaidlist != NIL,
+ publish_via_partition_root);
+
+ PublicationAddTables(puboid, rels, true, NULL);
+ CloseTableList(rels);
+ }
+
table_close(rel, RowExclusiveLock);
+ if (stmt->for_all_tables)
+ {
+ /* Invalidate relcache so that publication info is rebuilt. */
+ CacheInvalidateRelcacheAll();
+ }
+
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
if (wal_level != WAL_LEVEL_LOGICAL)
@@ -1040,8 +1076,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
LockDatabaseObject(PublicationRelationId, pubform->oid, 0,
AccessShareLock);
- root_relids = GetPublicationRelations(pubform->oid,
- PUBLICATION_PART_ROOT);
+ root_relids = GetPublicationIncludedRelations(pubform->oid,
+ PUBLICATION_PART_ROOT);
foreach(lc, root_relids)
{
@@ -1160,8 +1196,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
* trees, not just those explicitly mentioned in the publication.
*/
if (root_relids == NIL)
- relids = GetPublicationRelations(pubform->oid,
- PUBLICATION_PART_ALL);
+ relids = GetPublicationIncludedRelations(pubform->oid,
+ PUBLICATION_PART_ALL);
else
{
/*
@@ -1246,8 +1282,8 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
PublicationDropTables(pubid, rels, false);
else /* AP_SetObjects */
{
- List *oldrelids = GetPublicationRelations(pubid,
- PUBLICATION_PART_ROOT);
+ List *oldrelids = GetPublicationIncludedRelations(pubid,
+ PUBLICATION_PART_ROOT);
List *delrels = NIL;
ListCell *oldlc;
@@ -1348,6 +1384,7 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
oldrel = palloc_object(PublicationRelInfo);
oldrel->whereClause = NULL;
oldrel->columns = NIL;
+ oldrel->except = false;
oldrel->relation = table_open(oldrelid,
ShareUpdateExclusiveLock);
delrels = lappend(delrels, oldrel);
@@ -1398,7 +1435,8 @@ AlterPublicationSchemas(AlterPublicationStmt *stmt,
ListCell *lc;
List *reloids;
- reloids = GetPublicationRelations(pubform->oid, PUBLICATION_PART_ROOT);
+ reloids = GetPublicationIncludedRelations(pubform->oid,
+ PUBLICATION_PART_ROOT);
foreach(lc, reloids)
{
@@ -1761,6 +1799,7 @@ OpenTableList(List *tables)
pub_rel->relation = rel;
pub_rel->whereClause = t->whereClause;
pub_rel->columns = t->columns;
+ pub_rel->except = t->except;
rels = lappend(rels, pub_rel);
relids = lappend_oid(relids, myrelid);
@@ -1833,6 +1872,7 @@ OpenTableList(List *tables)
/* child inherits column list from parent */
pub_rel->columns = t->columns;
+ pub_rel->except = t->except;
rels = lappend(rels, pub_rel);
relids = lappend_oid(relids, childrelid);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 6b1a00ed477..3ea95ae1a26 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -8687,7 +8687,7 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName,
* expressions.
*/
if (attgenerated == ATTRIBUTE_GENERATED_VIRTUAL &&
- GetRelationPublications(RelationGetRelid(rel)) != NIL)
+ GetRelationPublications(RelationGetRelid(rel), NULL, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ALTER TABLE / SET EXPRESSION is not supported for virtual generated columns in tables that are part of a publication"),
@@ -18882,7 +18882,7 @@ ATPrepChangePersistence(AlteredTableInfo *tab, Relation rel, bool toLogged)
* UNLOGGED, as UNLOGGED tables can't be published.
*/
if (!toLogged &&
- GetRelationPublications(RelationGetRelid(rel)) != NIL)
+ GetRelationPublications(RelationGetRelid(rel), NULL, NULL))
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot change table \"%s\" to unlogged because it is part of a publication",
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 28f4e11e30f..bcf0bc57aa4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -455,6 +455,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
TriggerTransitions TriggerReferencing
vacuum_relation_list opt_vacuum_relation_list
drop_option_list pub_obj_list pub_all_obj_type_list
+ pub_except_obj_list opt_pub_except_clause
%type <retclause> returning_clause
%type <node> returning_option
@@ -592,6 +593,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <node> var_value zone_value
%type <rolespec> auth_ident RoleSpec opt_granted_by
%type <publicationobjectspec> PublicationObjSpec
+%type <publicationobjectspec> PublicationExceptObjSpec
%type <publicationallobjectspec> PublicationAllObjSpec
%type <keyword> unreserved_keyword type_func_name_keyword
@@ -10787,7 +10789,7 @@ AlterOwnerStmt: ALTER AGGREGATE aggregate_with_argtypes OWNER TO RoleSpec
*
* pub_all_obj_type is one of:
*
- * TABLES
+ * TABLES [EXCEPT [TABLE] ( table [, ...] )]
* SEQUENCES
*
* CREATE PUBLICATION FOR pub_obj [, ...] [WITH options]
@@ -10813,6 +10815,7 @@ CreatePublicationStmt:
CreatePublicationStmt *n = makeNode(CreatePublicationStmt);
n->pubname = $3;
+ n->pubobjects = $5;
preprocess_pub_all_objtype_list($5, &n->for_all_tables,
&n->for_all_sequences,
yyscanner);
@@ -10853,6 +10856,7 @@ PublicationObjSpec:
$$->pubtable->relation = $2;
$$->pubtable->columns = $3;
$$->pubtable->whereClause = $4;
+ $$->location = @1;
}
| TABLES IN_P SCHEMA ColId
{
@@ -10928,11 +10932,19 @@ pub_obj_list: PublicationObjSpec
{ $$ = lappend($1, $3); }
;
+opt_pub_except_clause:
+ EXCEPT opt_table '(' pub_except_obj_list ')' { $$ = $4; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
PublicationAllObjSpec:
- ALL TABLES
+ ALL TABLES opt_pub_except_clause
{
$$ = makeNode(PublicationAllObjSpec);
$$->pubobjtype = PUBLICATION_ALL_TABLES;
+ $$->except_tables = $3;
+ if($$->except_tables != NULL)
+ preprocess_pubobj_list($$->except_tables, yyscanner);
$$->location = @1;
}
| ALL SEQUENCES
@@ -10949,6 +10961,23 @@ pub_all_obj_type_list: PublicationAllObjSpec
{ $$ = lappend($1, $3); }
;
+PublicationExceptObjSpec:
+ relation_expr
+ {
+ $$ = makeNode(PublicationObjSpec);
+ $$->pubobjtype = PUBLICATIONOBJ_EXCEPT_TABLE;
+ $$->pubtable = makeNode(PublicationTable);
+ $$->pubtable->except = true;
+ $$->pubtable->relation = $1;
+ $$->location = @1;
+ }
+ ;
+
+pub_except_obj_list: PublicationExceptObjSpec
+ { $$ = list_make1($1); }
+ | pub_except_obj_list ',' PublicationExceptObjSpec
+ { $$ = lappend($1, $3); }
+ ;
/*****************************************************************************
*
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 787998abb8a..88174bd299a 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -2084,7 +2084,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
if (!entry->replicate_valid)
{
Oid schemaId = get_rel_namespace(relid);
- List *pubids = GetRelationPublications(relid);
+ List *pubids = NIL;
+ List *exceptpubids = NIL;
/*
* We don't acquire a lock on the namespace system table as we build
@@ -2099,6 +2100,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
char relkind = get_rel_relkind(relid);
List *rel_publications = NIL;
+ GetRelationPublications(relid, &pubids, &exceptpubids);
+
/* Reload publications if needed before use. */
if (!publications_valid)
{
@@ -2205,9 +2208,21 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
if (pub->pubviaroot && am_partition)
{
List *ancestors = get_partition_ancestors(relid);
+ int level = 0;
+
+ foreach_oid(ancestor, ancestors)
+ {
+ List *aexceptpubids = NIL;
- pub_relid = llast_oid(ancestors);
- ancestor_level = list_length(ancestors);
+ level++;
+ GetRelationPublications(ancestor, NULL, &aexceptpubids);
+
+ if (!list_member_oid(aexceptpubids, pub->oid))
+ {
+ pub_relid = ancestor;
+ ancestor_level = level;
+ }
+ }
}
}
@@ -2322,6 +2337,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
list_free(pubids);
list_free(schemaPubids);
+ list_free(exceptpubids);
list_free(rel_publications);
entry->replicate_valid = true;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 2d0cb7bcfd4..bc5f9495923 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5793,7 +5793,9 @@ RelationGetExclusionInfo(Relation indexRelation,
void
RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
{
- List *puboids;
+ List *puboids = NIL;
+ List *exceptpuboids = NIL;
+ List *alltablespuboids;
ListCell *lc;
MemoryContext oldcxt;
Oid schemaid;
@@ -5831,7 +5833,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->gencols_valid_for_delete = true;
/* Fetch the publication membership info. */
- puboids = GetRelationPublications(relid);
+ GetRelationPublications(relid, &puboids, &exceptpuboids);
schemaid = RelationGetNamespace(relation);
puboids = list_concat_unique_oid(puboids, GetSchemaPublications(schemaid));
@@ -5843,16 +5845,25 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
foreach(lc, ancestors)
{
Oid ancestor = lfirst_oid(lc);
+ List *ancestor_puboids = NIL;
+ List *ancestor_exceptpuboids = NIL;
- puboids = list_concat_unique_oid(puboids,
- GetRelationPublications(ancestor));
+ GetRelationPublications(ancestor, &ancestor_puboids,
+ &ancestor_exceptpuboids);
+
+ puboids = list_concat_unique_oid(puboids, ancestor_puboids);
schemaid = get_rel_namespace(ancestor);
puboids = list_concat_unique_oid(puboids,
GetSchemaPublications(schemaid));
+ exceptpuboids = list_concat_unique_oid(exceptpuboids,
+ ancestor_exceptpuboids);
}
}
- puboids = list_concat_unique_oid(puboids, GetAllTablesPublications());
+ alltablespuboids = GetAllTablesPublications();
+ puboids = list_concat_unique_oid(puboids,
+ list_difference_oid(alltablespuboids,
+ exceptpuboids));
foreach(lc, puboids)
{
Oid pubid = lfirst_oid(lc);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 27f6be3f0f8..0021a48a7b6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4634,9 +4634,48 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
pubinfo[i].pubgencols_type =
*(PQgetvalue(res, i, i_pubgencols));
+ pubinfo[i].except_tables = (SimplePtrList)
+ {
+ NULL, NULL
+ };
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
+
+ if (fout->remoteVersion >= 190000)
+ {
+ int ntbls;
+ PGresult *res_tbls;
+
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT prrelid\n"
+ "FROM pg_catalog.pg_publication_rel\n"
+ "WHERE prpubid = %u and prexcept = true",
+ pubinfo[i].dobj.catId.oid);
+
+ res_tbls = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntbls = PQntuples(res_tbls);
+ if (ntbls == 0)
+ continue;
+
+ for (int j = 0; j < ntbls; j++)
+ {
+ Oid prrelid;
+ TableInfo *tbinfo;
+
+ prrelid = atooid(PQgetvalue(res_tbls, j, 0));
+
+ tbinfo = findTableByOid(prrelid);
+ if (tbinfo == NULL)
+ continue;
+
+ simple_ptr_list_append(&pubinfo[i].except_tables, tbinfo);
+ }
+
+ PQclear(res_tbls);
+ }
}
cleanup:
@@ -4676,7 +4715,28 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
if (pubinfo->puballtables && pubinfo->puballsequences)
appendPQExpBufferStr(query, " FOR ALL TABLES, ALL SEQUENCES");
else if (pubinfo->puballtables)
+ {
+ bool first_tbl = true;
+
appendPQExpBufferStr(query, " FOR ALL TABLES");
+
+ /* Include exception tables if the publication has EXCEPT TABLEs */
+ for (SimplePtrListCell *cell = pubinfo->except_tables.head; cell; cell = cell->next)
+ {
+ TableInfo *tbinfo = (TableInfo *) cell->ptr;
+
+ if (first_tbl)
+ {
+ appendPQExpBufferStr(query, " EXCEPT TABLE (");
+ first_tbl = false;
+ }
+ else
+ appendPQExpBufferStr(query, ", ");
+ appendPQExpBuffer(query, "ONLY %s", fmtQualifiedDumpable(tbinfo));
+ }
+ if (!first_tbl)
+ appendPQExpBufferStr(query, ")");
+ }
else if (pubinfo->puballsequences)
appendPQExpBufferStr(query, " FOR ALL SEQUENCES");
@@ -4856,6 +4916,7 @@ getPublicationTables(Archive *fout, TableInfo tblinfo[], int numTables)
/* Collect all publication membership info. */
if (fout->remoteVersion >= 150000)
+ {
appendPQExpBufferStr(query,
"SELECT tableoid, oid, prpubid, prrelid, "
"pg_catalog.pg_get_expr(prqual, prrelid) AS prrelqual, "
@@ -4868,6 +4929,9 @@ getPublicationTables(Archive *fout, TableInfo tblinfo[], int numTables)
" WHERE attrelid = pr.prrelid AND attnum = prattrs[s])\n"
" ELSE NULL END) prattrs "
"FROM pg_catalog.pg_publication_rel pr");
+ if (fout->remoteVersion >= 190000)
+ appendPQExpBufferStr(query, " WHERE NOT pr.prexcept");
+ }
else
appendPQExpBufferStr(query,
"SELECT tableoid, oid, prpubid, prrelid, "
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 72a00e1bc20..a19e77d149e 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -676,6 +676,7 @@ typedef struct _PublicationInfo
bool pubtruncate;
bool pubviaroot;
PublishGencolsType pubgencols_type;
+ SimplePtrList except_tables;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e33aa95f6ff..a3fcf3c2b0a 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -3170,6 +3170,36 @@ my %tests = (
like => { %full_runs, section_post_data => 1, },
},
+ 'CREATE PUBLICATION pub8' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub8 FOR ALL TABLES EXCEPT (dump_test.test_table);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub8 FOR ALL TABLES EXCEPT TABLE (ONLY dump_test.test_table) WITH (publish = 'insert, update, delete, truncate');\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
+ 'CREATE PUBLICATION pub9' => {
+ create_order => 50,
+ create_sql =>
+ 'CREATE PUBLICATION pub9 FOR ALL TABLES EXCEPT TABLE (dump_test.test_table, dump_test.test_second_table);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub9 FOR ALL TABLES EXCEPT TABLE (ONLY dump_test.test_table, ONLY dump_test.test_second_table) WITH (publish = 'insert, update, delete, truncate');\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
+ 'CREATE PUBLICATION pub10' => {
+ create_order => 92,
+ create_sql =>
+ 'CREATE PUBLICATION pub10 FOR ALL TABLES EXCEPT TABLE (dump_test.test_inheritance_parent);',
+ regexp => qr/^
+ \QCREATE PUBLICATION pub10 FOR ALL TABLES EXCEPT TABLE (ONLY dump_test.test_inheritance_parent, ONLY dump_test.test_inheritance_child) WITH (publish = 'insert, update, delete, truncate');\E
+ /xm,
+ like => { %full_runs, section_post_data => 1, },
+ },
+
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 36f24502842..8cd73b3ad53 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -3073,17 +3073,34 @@ describeOneTableDetails(const char *schemaname,
" WHERE attrelid = pr.prrelid AND attnum = prattrs[s])\n"
" ELSE NULL END) "
"FROM pg_catalog.pg_publication p\n"
- " JOIN pg_catalog.pg_publication_rel pr ON p.oid = pr.prpubid\n"
- " JOIN pg_catalog.pg_class c ON c.oid = pr.prrelid\n"
- "WHERE pr.prrelid = '%s'\n"
+ " JOIN pg_catalog.pg_publication_rel pr ON p.oid = pr.prpubid\n"
+ " JOIN pg_catalog.pg_class c ON c.oid = pr.prrelid\n"
+ "WHERE pr.prrelid = '%s'\n",
+ oid, oid, oid);
+
+ if (pset.sversion >= 190000)
+ appendPQExpBufferStr(&buf, " AND NOT pr.prexcept\n");
+
+ appendPQExpBuffer(&buf,
"UNION\n"
"SELECT pubname\n"
- " , NULL\n"
- " , NULL\n"
+ " , NULL\n"
+ " , NULL\n"
"FROM pg_catalog.pg_publication p\n"
- "WHERE p.puballtables AND pg_catalog.pg_relation_is_publishable('%s')\n"
- "ORDER BY 1;",
- oid, oid, oid, oid);
+ "WHERE p.puballtables AND pg_catalog.pg_relation_is_publishable('%s')\n",
+ oid);
+
+ if (pset.sversion >= 190000)
+ appendPQExpBuffer(&buf,
+ " AND NOT EXISTS (\n"
+ " SELECT 1\n"
+ " FROM pg_catalog.pg_publication_rel pr\n"
+ " JOIN pg_catalog.pg_class pc\n"
+ " ON pr.prrelid = pc.oid\n"
+ " WHERE pr.prrelid = '%s' AND pr.prpubid = p.oid)\n",
+ oid);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
}
else
{
@@ -3134,6 +3151,35 @@ describeOneTableDetails(const char *schemaname,
PQclear(result);
}
+ /* Print publication the relation is excluded explicitly */
+ if (pset.sversion >= 190000)
+ {
+ printfPQExpBuffer(&buf,
+ "SELECT pubname\n"
+ "FROM pg_catalog.pg_publication p\n"
+ "JOIN pg_catalog.pg_publication_rel pr ON p.oid = pr.prpubid\n"
+ "WHERE pr.prrelid = '%s'\n AND pr.prexcept\n"
+ "ORDER BY 1;", oid);
+
+ result = PSQLexec(buf.data);
+ if (!result)
+ goto error_return;
+ else
+ tuples = PQntuples(result);
+
+ if (tuples > 0)
+ printTableAddFooter(&cont, _("Except Publications:"));
+
+ /* Might be an empty set - that's ok */
+ for (i = 0; i < tuples; i++)
+ {
+ printfPQExpBuffer(&buf, " \"%s\"", PQgetvalue(result, i, 0));
+
+ printTableAddFooter(&cont, buf.data);
+ }
+ PQclear(result);
+ }
+
/*
* If verbose, print NOT NULL constraints.
*/
@@ -6753,8 +6799,12 @@ describePublications(const char *pattern)
" pg_catalog.pg_publication_rel pr\n"
"WHERE c.relnamespace = n.oid\n"
" AND c.oid = pr.prrelid\n"
- " AND pr.prpubid = '%s'\n"
- "ORDER BY 1,2", pubid);
+ " AND pr.prpubid = '%s'\n", pubid);
+
+ if (pset.sversion >= 190000)
+ appendPQExpBuffer(&buf, " AND NOT pr.prexcept\n");
+
+ appendPQExpBuffer(&buf, "ORDER BY 1,2");
if (!addFooterToPublicationDesc(&buf, _("Tables:"), false, &cont))
goto error_return;
@@ -6772,6 +6822,23 @@ describePublications(const char *pattern)
goto error_return;
}
}
+ else
+ {
+ if (pset.sversion >= 190000)
+ {
+ /* Get the excluded tables for the specified publication */
+ printfPQExpBuffer(&buf,
+ "SELECT concat(c.relnamespace::regnamespace, '.', c.relname)\n"
+ "FROM pg_catalog.pg_class c\n"
+ " JOIN pg_catalog.pg_publication_rel pr ON c.oid = pr.prrelid\n"
+ "WHERE pr.prpubid = '%s'\n"
+ " AND pr.prexcept\n"
+ "ORDER BY 1", pubid);
+ if (!addFooterToPublicationDesc(&buf, _("Except tables:"),
+ true, &cont))
+ goto error_return;
+ }
+ }
printTable(&cont, pset.queryFout, false, pset.logfile);
printTableCleanup(&cont);
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index ab2712216b5..97c34379556 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -3657,7 +3657,17 @@ match_previous_words(int pattern_id,
else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL"))
COMPLETE_WITH("TABLES", "SEQUENCES");
else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES"))
- COMPLETE_WITH("WITH (");
+ COMPLETE_WITH("EXCEPT TABLE (", "WITH (");
+ else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "EXCEPT"))
+ COMPLETE_WITH("TABLE (");
+ else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "EXCEPT", "TABLE"))
+ COMPLETE_WITH("(");
+ else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "EXCEPT", "TABLE", "("))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_tables);
+ else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "EXCEPT", "TABLE", "(", MatchAnyN) && ends_with(prev_wd, ','))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_tables);
+ else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "EXCEPT", "TABLE", "(", MatchAnyN) && !ends_with(prev_wd, ','))
+ COMPLETE_WITH(")");
else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "TABLES"))
COMPLETE_WITH("IN SCHEMA");
else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "TABLE", MatchAny) && !ends_with(prev_wd, ','))
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 22f48bb8975..bd094113c7a 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -146,14 +146,16 @@ typedef struct PublicationRelInfo
Relation relation;
Node *whereClause;
List *columns;
+ bool except;
} PublicationRelInfo;
extern Publication *GetPublication(Oid pubid);
extern Publication *GetPublicationByName(const char *pubname, bool missing_ok);
-extern List *GetRelationPublications(Oid relid);
+extern bool GetRelationPublications(Oid relid, List **pubids, List **except_pubids);
/*---------
- * Expected values for pub_partopt parameter of GetPublicationRelations(),
+ * Expected values for pub_partopt parameter of
+ * GetPublicationIncludedRelations(), and GetPublicationExcludedRelations(),
* which allows callers to specify which partitions of partitioned tables
* mentioned in the publication they expect to see.
*
@@ -168,9 +170,12 @@ typedef enum PublicationPartOpt
PUBLICATION_PART_ALL,
} PublicationPartOpt;
-extern List *GetPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt);
+extern List *GetPublicationIncludedRelations(Oid pubid,
+ PublicationPartOpt pub_partopt);
+extern List *GetPublicationExcludedRelations(Oid pubid,
+ PublicationPartOpt pub_partopt);
extern List *GetAllTablesPublications(void);
-extern List *GetAllPublicationRelations(char relkind, bool pubviaroot);
+extern List *GetAllPublicationRelations(Publication *pub, char relkind);
extern List *GetPublicationSchemas(Oid pubid);
extern List *GetSchemaPublications(Oid schemaid);
extern List *GetSchemaPublicationRelations(Oid schemaid,
diff --git a/src/include/catalog/pg_publication_rel.h b/src/include/catalog/pg_publication_rel.h
index 92cc36dfdf6..e7d7f3ba85c 100644
--- a/src/include/catalog/pg_publication_rel.h
+++ b/src/include/catalog/pg_publication_rel.h
@@ -31,6 +31,7 @@ CATALOG(pg_publication_rel,6106,PublicationRelRelationId)
Oid oid; /* oid */
Oid prpubid BKI_LOOKUP(pg_publication); /* Oid of the publication */
Oid prrelid BKI_LOOKUP(pg_class); /* Oid of the relation */
+ bool prexcept BKI_DEFAULT(f); /* exclude the relation */
#ifdef CATALOG_VARLEN /* variable-length fields start here */
pg_node_tree prqual; /* qualifications */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index bc7adba4a0f..ca1055051f9 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -4299,6 +4299,7 @@ typedef struct PublicationTable
RangeVar *relation; /* relation to be published */
Node *whereClause; /* qualifications */
List *columns; /* List of columns in a publication table */
+ bool except; /* exclude the relation */
} PublicationTable;
/*
@@ -4307,6 +4308,7 @@ typedef struct PublicationTable
typedef enum PublicationObjSpecType
{
PUBLICATIONOBJ_TABLE, /* A table */
+ PUBLICATIONOBJ_EXCEPT_TABLE, /* A table to be excluded */
PUBLICATIONOBJ_TABLES_IN_SCHEMA, /* All tables in schema */
PUBLICATIONOBJ_TABLES_IN_CUR_SCHEMA, /* All tables in first element of
* search_path */
@@ -4335,6 +4337,7 @@ typedef struct PublicationAllObjSpec
{
NodeTag type;
PublicationAllObjType pubobjtype; /* type of this publication object */
+ List *except_tables; /* List of tables to be excluded */
ParseLoc location; /* token location, or -1 if unknown */
} PublicationAllObjSpec;
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index e72d1308967..30073f59706 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -213,33 +213,88 @@ Not-null constraints:
regress_publication_user | t | f | t | t | f | f | none | f
(1 row)
-DROP TABLE testpub_tbl2;
-DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema, testpub_for_tbl_schema;
-CREATE TABLE testpub_tbl3 (a int);
-CREATE TABLE testpub_tbl3a (b text) INHERITS (testpub_tbl3);
SET client_min_messages = 'ERROR';
-CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
-CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
+-- Exclude tables using FOR ALL TABLES EXCEPT TABLE (tablelist)
+CREATE PUBLICATION testpub_foralltables_excepttable FOR ALL TABLES EXCEPT TABLE (testpub_tbl1, testpub_tbl2);
+\dRp+ testpub_foralltables_excepttable
+ Publication testpub_foralltables_excepttable
+ Owner | All tables | All sequences | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
+--------------------------+------------+---------------+---------+---------+---------+-----------+-------------------+----------
+ regress_publication_user | t | f | t | t | t | t | none | f
+Except tables:
+ "public.testpub_tbl1"
+ "public.testpub_tbl2"
+
+-- Exclude tables using FOR ALL TABLES EXCEPT (tablelist)
+CREATE PUBLICATION testpub_foralltables_excepttable1 FOR ALL TABLES EXCEPT (testpub_tbl1);
+\dRp+ testpub_foralltables_excepttable1
+ Publication testpub_foralltables_excepttable1
+ Owner | All tables | All sequences | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
+--------------------------+------------+---------------+---------+---------+---------+-----------+-------------------+----------
+ regress_publication_user | t | f | t | t | t | t | none | f
+Except tables:
+ "public.testpub_tbl1"
+
RESET client_min_messages;
+DROP TABLE testpub_tbl2;
+DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema, testpub_for_tbl_schema, testpub_foralltables_excepttable, testpub_foralltables_excepttable1;
+CREATE TABLE testpub_tbl_parent (a int);
+CREATE TABLE testpub_tbl_child (b text) INHERITS (testpub_tbl_parent);
+SET client_min_messages = 'ERROR';
+CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl_parent;
\dRp+ testpub3
Publication testpub3
Owner | All tables | All sequences | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
--------------------------+------------+---------------+---------+---------+---------+-----------+-------------------+----------
regress_publication_user | f | f | t | t | t | t | none | f
Tables:
- "public.testpub_tbl3"
- "public.testpub_tbl3a"
+ "public.testpub_tbl_child"
+ "public.testpub_tbl_parent"
+CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl_parent;
\dRp+ testpub4
Publication testpub4
Owner | All tables | All sequences | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
--------------------------+------------+---------------+---------+---------+---------+-----------+-------------------+----------
regress_publication_user | f | f | t | t | t | t | none | f
Tables:
- "public.testpub_tbl3"
+ "public.testpub_tbl_parent"
-DROP TABLE testpub_tbl3, testpub_tbl3a;
-DROP PUBLICATION testpub3, testpub4;
+-- Exclude parent table, omitting both of 'ONLY' and '*'
+CREATE PUBLICATION testpub5 FOR ALL TABLES EXCEPT TABLE (testpub_tbl_parent);
+\dRp+ testpub5
+ Publication testpub5
+ Owner | All tables | All sequences | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
+--------------------------+------------+---------------+---------+---------+---------+-----------+-------------------+----------
+ regress_publication_user | t | f | t | t | t | t | none | f
+Except tables:
+ "public.testpub_tbl_child"
+ "public.testpub_tbl_parent"
+
+-- EXCEPT with asterisk: exclude table and all descendants
+CREATE PUBLICATION testpub6 FOR ALL TABLES EXCEPT TABLE (testpub_tbl_parent *);
+\dRp+ testpub6
+ Publication testpub6
+ Owner | All tables | All sequences | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
+--------------------------+------------+---------------+---------+---------+---------+-----------+-------------------+----------
+ regress_publication_user | t | f | t | t | t | t | none | f
+Except tables:
+ "public.testpub_tbl_child"
+ "public.testpub_tbl_parent"
+
+-- EXCEPT with ONLY: exclude table but not descendants
+CREATE PUBLICATION testpub7 FOR ALL TABLES EXCEPT TABLE (ONLY testpub_tbl_parent);
+\dRp+ testpub7
+ Publication testpub7
+ Owner | All tables | All sequences | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
+--------------------------+------------+---------------+---------+---------+---------+-----------+-------------------+----------
+ regress_publication_user | t | f | t | t | t | t | none | f
+Except tables:
+ "public.testpub_tbl_parent"
+
+RESET client_min_messages;
+DROP TABLE testpub_tbl_parent, testpub_tbl_child;
+DROP PUBLICATION testpub3, testpub4, testpub5, testpub6, testpub7;
--- Tests for publications with SEQUENCES
CREATE SEQUENCE regress_pub_seq0;
CREATE SEQUENCE pub_test.regress_pub_seq1;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index 00390aecd47..920da5360ad 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -105,20 +105,38 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
\d+ testpub_tbl2
\dRp+ testpub_foralltables
+SET client_min_messages = 'ERROR';
+-- Exclude tables using FOR ALL TABLES EXCEPT TABLE (tablelist)
+CREATE PUBLICATION testpub_foralltables_excepttable FOR ALL TABLES EXCEPT TABLE (testpub_tbl1, testpub_tbl2);
+\dRp+ testpub_foralltables_excepttable
+-- Exclude tables using FOR ALL TABLES EXCEPT (tablelist)
+CREATE PUBLICATION testpub_foralltables_excepttable1 FOR ALL TABLES EXCEPT (testpub_tbl1);
+\dRp+ testpub_foralltables_excepttable1
+
+RESET client_min_messages;
DROP TABLE testpub_tbl2;
-DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema, testpub_for_tbl_schema;
+DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema, testpub_for_tbl_schema, testpub_foralltables_excepttable, testpub_foralltables_excepttable1;
-CREATE TABLE testpub_tbl3 (a int);
-CREATE TABLE testpub_tbl3a (b text) INHERITS (testpub_tbl3);
+CREATE TABLE testpub_tbl_parent (a int);
+CREATE TABLE testpub_tbl_child (b text) INHERITS (testpub_tbl_parent);
SET client_min_messages = 'ERROR';
-CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
-CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
-RESET client_min_messages;
+CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl_parent;
\dRp+ testpub3
+CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl_parent;
\dRp+ testpub4
+-- Exclude parent table, omitting both of 'ONLY' and '*'
+CREATE PUBLICATION testpub5 FOR ALL TABLES EXCEPT TABLE (testpub_tbl_parent);
+\dRp+ testpub5
+-- EXCEPT with asterisk: exclude table and all descendants
+CREATE PUBLICATION testpub6 FOR ALL TABLES EXCEPT TABLE (testpub_tbl_parent *);
+\dRp+ testpub6
+-- EXCEPT with ONLY: exclude table but not descendants
+CREATE PUBLICATION testpub7 FOR ALL TABLES EXCEPT TABLE (ONLY testpub_tbl_parent);
+\dRp+ testpub7
-DROP TABLE testpub_tbl3, testpub_tbl3a;
-DROP PUBLICATION testpub3, testpub4;
+RESET client_min_messages;
+DROP TABLE testpub_tbl_parent, testpub_tbl_child;
+DROP PUBLICATION testpub3, testpub4, testpub5, testpub6, testpub7;
--- Tests for publications with SEQUENCES
CREATE SEQUENCE regress_pub_seq0;
diff --git a/src/test/subscription/meson.build b/src/test/subscription/meson.build
index 85d10a89994..b8e5c54c314 100644
--- a/src/test/subscription/meson.build
+++ b/src/test/subscription/meson.build
@@ -46,6 +46,7 @@ tests += {
't/034_temporal.pl',
't/035_conflicts.pl',
't/036_sequences.pl',
+ 't/037_rep_changes_except_table.pl',
't/100_bugs.pl',
],
},
diff --git a/src/test/subscription/t/037_rep_changes_except_table.pl b/src/test/subscription/t/037_rep_changes_except_table.pl
new file mode 100644
index 00000000000..2a53aae7fbe
--- /dev/null
+++ b/src/test/subscription/t/037_rep_changes_except_table.pl
@@ -0,0 +1,219 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Logical replication tests for EXCEPT TABLE publications
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Initialize publisher node
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+# Initialize subscriber node
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+# ============================================
+# EXCEPT TABLE test cases for normal tables
+# ============================================
+# Create schemas and tables on publisher
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE SCHEMA sch1;
+ CREATE TABLE sch1.tab1 AS SELECT generate_series(1,10) AS a;
+));
+
+# Create schemas and tables on subscriber
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE SCHEMA sch1;
+ CREATE TABLE sch1.tab1 (a int);
+));
+
+# Setup logical replication
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub_schema FOR ALL TABLES EXCEPT TABLE (sch1.tab1)"
+);
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connstr' PUBLICATION tap_pub_schema"
+);
+
+# Wait for initial table sync to finish
+$node_subscriber->wait_for_subscription_sync($node_publisher,
+ 'tap_sub_schema');
+
+# Check the table data does not sync for excluded table
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM sch1.tab1");
+is($result, qq(0||),
+ 'check there is no initial data copied for the excluded table');
+
+# Verify that data inserted to the excluded table is not replicated.
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO sch1.tab1 VALUES(generate_series(11,20))");
+
+$node_publisher->wait_for_catchup('tap_sub_schema');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM sch1.tab1");
+is($result, qq(0||), 'check replicated inserts on subscriber');
+
+# cleanup
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION tap_pub_schema");
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_schema");
+
+# ============================================
+# EXCEPT TABLE test cases for partitioned tables
+# Check behavior of EXCEPT TABLE with publish_via_partition_root on a
+# partitioned table and its partitions.
+# ============================================
+# Setup partitioned table and partitions on the publisher that map to normal
+# tables on the subscriber
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE sch1.t1(a int) PARTITION BY RANGE(a);
+ CREATE TABLE sch1.part1 PARTITION OF sch1.t1 FOR VALUES FROM (0) TO (5);
+ CREATE TABLE sch1.part2 PARTITION OF sch1.t1 FOR VALUES FROM (6) TO (10);
+));
+
+$node_subscriber->safe_psql(
+ 'postgres', qq(
+ CREATE TABLE sch1.t1(a int);
+ CREATE TABLE sch1.part1(a int);
+ CREATE TABLE sch1.part2(a int);
+));
+
+# EXCEPT TABLE (sch1.part1) with publish_via_partition_root = false
+# Excluding a partition while publish_via_partition_root = false prevents
+# replication of rows inserted into the partitioned table for that particular
+# partition.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE PUBLICATION tap_pub_part FOR ALL TABLES EXCEPT TABLE (sch1.part1) WITH (publish_via_partition_root = false);
+ INSERT INTO sch1.t1 VALUES (1), (6);
+));
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub_part CONNECTION '$publisher_connstr' PUBLICATION tap_pub_part"
+);
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'tap_sub_part');
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO sch1.t1 VALUES (2), (7);");
+$node_publisher->wait_for_catchup('tap_sub_part');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.t1");
+is($result, qq(), 'check rows on partitioned table');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part1");
+is($result, qq(), 'check rows on excluded partition');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part2");
+is( $result, qq(6
+7), 'check rows on other partition');
+
+$node_publisher->safe_psql('postgres', "TRUNCATE sch1.t1");
+$node_publisher->wait_for_catchup('tap_sub_part');
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_part");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION tap_pub_part;");
+
+# EXCEPT TABLE (sch1.t1) with publish_via_partition_root = false
+# Excluding the partitioned table still allows rows inserted into the
+# partitioned table to be replicated via its partitions.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE PUBLICATION tap_pub_part FOR ALL TABLES EXCEPT TABLE (sch1.t1) WITH (publish_via_partition_root = false);
+ INSERT INTO sch1.t1 VALUES (1), (6);
+));
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub_part CONNECTION '$publisher_connstr' PUBLICATION tap_pub_part"
+);
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'tap_sub_part');
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO sch1.t1 VALUES (2), (7);");
+$node_publisher->wait_for_catchup('tap_sub_part');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.t1");
+is($result, qq(), 'check rows on partitioned table');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part1");
+is( $result, qq(1
+2), 'check rows on first partition');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part2");
+is( $result, qq(6
+7), 'check rows on second partition');
+
+$node_publisher->safe_psql('postgres', "TRUNCATE sch1.t1");
+$node_publisher->wait_for_catchup('tap_sub_part');
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_part");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION tap_pub_part;");
+
+# EXCEPT TABLE (sch1.t1) with publish_via_partition_root = true
+# When the partitioned table is excluded and publish_via_partition_root is true,
+# no rows from the table or its partitions are replicated.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE PUBLICATION tap_pub_part FOR ALL TABLES EXCEPT TABLE (sch1.t1) WITH (publish_via_partition_root);
+ INSERT INTO sch1.t1 VALUES (1), (6);
+));
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub_part CONNECTION '$publisher_connstr' PUBLICATION tap_pub_part"
+);
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'tap_sub_part');
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO sch1.t1 VALUES (2), (7);");
+$node_publisher->wait_for_catchup('tap_sub_part');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.t1");
+is($result, qq(), 'check rows on partitioned table');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part1");
+is($result, qq(), 'check rows on first partition');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part2");
+is($result, qq(), 'check rows on second partition');
+
+$node_publisher->safe_psql('postgres', "TRUNCATE sch1.t1");
+$node_publisher->wait_for_catchup('tap_sub_part');
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION tap_sub_part");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION tap_pub_part;");
+
+# EXCEPT TABLE (sch1.part1) with publish_via_partition_root = true
+# When a partition is excluded but publish_via_partition_root is true,
+# rows published through the partitioned table can still be replicated.
+$node_publisher->safe_psql(
+ 'postgres', qq(
+ CREATE PUBLICATION tap_pub_part FOR ALL TABLES EXCEPT TABLE (sch1.part1) WITH (publish_via_partition_root);
+ INSERT INTO sch1.t1 VALUES (1), (6)
+));
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub_part CONNECTION '$publisher_connstr' PUBLICATION tap_pub_part"
+);
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'tap_sub_part');
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO sch1.t1 VALUES (2), (7);");
+$node_publisher->wait_for_catchup('tap_sub_part');
+
+$result =
+ $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.t1 ORDER BY a");
+is( $result, qq(1
+2
+6
+7), 'check rows on partitioned table');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part1");
+is($result, qq(), 'check rows on excluded partition');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.part2");
+is($result, qq(), 'check rows on other partition');
+
+$node_subscriber->stop('fast');
+$node_publisher->stop('fast');
+
+done_testing();
--
2.34.1
view thread (377+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Skipping schema changes in publication
In-Reply-To: <CANhcyEUiwF9w2U6CzkuOsXkLYMOjPs=6O8QyjU_5fDfYWyMswg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox