pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Amit Kapila <akapila@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix data loss in logical replication.
Date: Thu, 10 Apr 2025 08:01:47 +0000
Message-ID: <E1u2mr5-003ZBB-1j@gemulon.postgresql.org> (raw)

Fix data loss in logical replication.

Data loss can happen when the DDLs like ALTER PUBLICATION ... ADD TABLE ...
or ALTER TYPE ...  that don't take a strong lock on table happens
concurrently to DMLs on the tables involved in the DDL. This happens
because logical decoding doesn't distribute invalidations to concurrent
transactions and those transactions use stale cache data to decode the
changes. The problem becomes bigger because we keep using the stale cache
even after those in-progress transactions are finished and skip the
changes required to be sent to the client.

This commit fixes the issue by distributing invalidation messages from
catalog-modifying transactions to all concurrent in-progress transactions.
This allows the necessary rebuild of the catalog cache when decoding new
changes after concurrent DDL.

We observed performance regression primarily during frequent execution of
*publication DDL* statements that modify the published tables. The
regression is minor or nearly nonexistent for DDLs that do not affect the
published tables or occur infrequently, making this a worthwhile cost to
resolve a longstanding data loss issue.

An alternative approach considered was to take a strong lock on each
affected table during publication modification. However, this would only
address issues related to publication DDLs (but not the ALTER TYPE ...)
and require locking every relation in the database for publications
created as FOR ALL TABLES, which is impractical.

The bug exists in all supported branches, but we are backpatching till 14.
The fix for 13 requires somewhat bigger changes than this fix, so the fix
for that branch is still under discussion.

Reported-by: hubert depesz lubaczewski <depesz@depesz.com>
Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Tested-by: Benoit Lobréau <benoit.lobreau@dalibo.com>
Backpatch-through: 14
Discussion: https://postgr.es/m/de52b282-1166-1180-45a2-8d8917ca74c6@enterprisedb.com
Discussion: https://postgr.es/m/CAD21AoAenVqiMjpN-PvGHL1N9DWnHSq673bfgr6phmBUzx=kLQ@mail.gmail.com

Branch
------
REL_17_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/cadaf0ac46374360e178f4ac120a7e29438b4dc0

Modified Files
--------------
contrib/test_decoding/Makefile                     |  2 +-
.../expected/invalidation_distrubution.out         | 20 +++++++
contrib/test_decoding/meson.build                  |  1 +
.../specs/invalidation_distrubution.spec           | 32 +++++++++++
src/backend/replication/logical/reorderbuffer.c    | 23 ++++++++
src/backend/replication/logical/snapbuild.c        | 67 +++++++++++++++++-----
src/include/replication/reorderbuffer.h            |  4 ++
7 files changed, 134 insertions(+), 15 deletions(-)



view thread (6+ messages)  latest in thread

Message-ID: <E1u2mr5-003ZBB-1j@gemulon.postgresql.org>
Permalink:  ../E1u2mr5-003ZBB-1j@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1u2mr5-003ZBB-1j@gemulon.postgresql.org

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-committers@postgresql.org
  Cc: akapila@postgresql.org, pgsql-committers@lists.postgresql.org
  Subject: Re: pgsql: Fix data loss in logical replication.
  In-Reply-To: <E1u2mr5-003ZBB-1j@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox