pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Amit Kapila <akapila@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Avoid syncing data twice for the 'publish_via_partition_root' op
Date: Wed, 29 Mar 2023 05:38:08 +0000
Message-ID: <E1phOVa-000GnZ-TD@gemulon.postgresql.org> (raw)

Avoid syncing data twice for the 'publish_via_partition_root' option.

When there are multiple publications for a subscription and one of those
publishes via the parent table by using publish_via_partition_root and the
other one directly publishes the child table, we end up copying the same
data twice during initial synchronization. The reason for this was that we
get both the parent and child tables from the publisher and try to copy
the data for both of them.

This patch extends the function pg_get_publication_tables() to take a
publication list as its input parameter. This allows us to exclude a
partition table whose ancestor is published by the same publication list.

This problem does exist in back-branches but we decide to fix it there in
a separate commit if required. The fix for back-branches requires quite
complicated changes to fetch the required table information from the
publisher as we can't update the function pg_get_publication_tables() in
back-branches. We are not sure whether we want to deviate and complicate
the code in back-branches for this problem as there are no field reports
yet.

Author: Wang wei
Reviewed-by: Peter Smith, Jacob Champion, Kuroda Hayato, Vignesh C, Osumi Takamichi, Amit Kapila
Discussion: https://postgr.es/m/OS0PR01MB57167F45D481F78CDC5986F794B99@OS0PR01MB5716.jpnprd01.prod.outlook.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/062a8444242404242f7c2b814fed37b329639408

Modified Files
--------------
doc/src/sgml/ref/create_publication.sgml   |  10 ++
src/backend/catalog/pg_publication.c       | 211 +++++++++++++++++++----------
src/backend/commands/subscriptioncmds.c    |  59 ++++++--
src/include/catalog/catversion.h           |   2 +-
src/include/catalog/pg_proc.dat            |  12 +-
src/test/regress/expected/rules.out        |   2 +-
src/test/subscription/t/013_partition.pl   |  25 ++--
src/test/subscription/t/028_row_filter.pl  |  42 +++++-
src/test/subscription/t/031_column_list.pl |   9 +-
src/tools/pgindent/typedefs.list           |   1 +
10 files changed, 274 insertions(+), 99 deletions(-)



Message-ID: <E1phOVa-000GnZ-TD@gemulon.postgresql.org>
Permalink:  ../E1phOVa-000GnZ-TD@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1phOVa-000GnZ-TD@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-committers@postgresql.org
  Cc: akapila@postgresql.org, pgsql-committers@lists.postgresql.org
  Subject: Re: pgsql: Avoid syncing data twice for the 'publish_via_partition_root' op
  In-Reply-To: <E1phOVa-000GnZ-TD@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox