pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Amit Kapila <akapila@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix the logical replication timeout during large transactions.
Date: Wed, 11 May 2022 05:51:07 +0000
Message-ID: <E1nofFa-0007g2-Vm@gemulon.postgresql.org> (raw)

Fix the logical replication timeout during large transactions.

The problem is that we don't send keep-alive messages for a long time
while processing large transactions during logical replication where we
don't send any data of such transactions. This can happen when the table
modified in the transaction is not published or because all the changes
got filtered. We do try to send the keep_alive if necessary at the end of
the transaction (via WalSndWriteData()) but by that time the
subscriber-side can timeout and exit.

To fix this we try to send the keepalive message if required after
processing certain threshold of changes.

Reported-by: Fabrice Chapuis
Author: Wang wei and Amit Kapila
Reviewed By: Masahiko Sawada, Euler Taveira, Hou Zhijie, Hayato Kuroda
Backpatch-through: 10
Discussion: https://postgr.es/m/CAA5-nLARN7-3SLU_QUxfy510pmrYK6JJb=bk3hcgemAM_pAv+w@mail.gmail.com

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/a4015ec0375da35e999e86696571ff354a8f706b

Modified Files
--------------
src/backend/replication/logical/logical.c   |  8 ++++++
src/backend/replication/pgoutput/pgoutput.c | 38 ++++++++++++++++++++++++++++-
src/backend/replication/walsender.c         | 38 +++++++++++++++++++++++++----
src/include/replication/logical.h           |  1 +
4 files changed, 79 insertions(+), 6 deletions(-)



view thread (7+ messages)  latest in thread

Message-ID: <E1nofFa-0007g2-Vm@gemulon.postgresql.org>
Permalink:  ../E1nofFa-0007g2-Vm@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1nofFa-0007g2-Vm@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: akapila@postgresql.org
  Subject: Re: pgsql: Fix the logical replication timeout during large transactions.
  In-Reply-To: <E1nofFa-0007g2-Vm@gemulon.postgresql.org>
  References: <E1nofFa-0007g2-Vm@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox