pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Amit Kapila <akapila@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix the logical replication timeout during large DDLs.
Date: Wed, 08 Feb 2023 02:46:37 +0000
Message-ID: <E1pPaTk-0000Zr-BZ@gemulon.postgresql.org> (raw)

Fix the logical replication timeout during large DDLs.

The DDLs like Refresh Materialized views that generate lots of temporary
data due to rewrite rules may not be processed by output plugins (for
example pgoutput). So, we won't send keep-alive messages for a long time
while processing such commands and that can lead the subscriber side to
timeout. We have previously fixed a similar case for large transactions in
commit f95d53eded where the output plugin filters all or most of the
changes but missed to handle the DDLs.

We decided not to backpatch this as this adds a new callback in the
existing exposed structure and moreover, users can increase the
wal_sender_timeout and wal_receiver_timeout to avoid this problem.

Author: Wang wei, Hou Zhijie
Reviewed-by: Peter Smith, Ashutosh Bapat, Shi yu, Amit Kapila
Discussion: https://postgr.es/m/OS3PR01MB6275478E5D29E4A563302D3D9E2B9@OS3PR01MB6275.jpnprd01.prod.outlook.com
Discussion: https://postgr.es/m/CAA5-nLARN7-3SLU_QUxfy510pmrYK6JJb=bk3hcgemAM_pAv+w@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/8c58624df46222d4d09c5655d8350f3b037880c8

Modified Files
--------------
src/backend/replication/logical/logical.c       | 50 +++++++++++++++++++++++
src/backend/replication/logical/reorderbuffer.c | 20 +++++++++
src/backend/replication/pgoutput/pgoutput.c     | 54 +++----------------------
src/include/replication/reorderbuffer.h         | 12 ++++++
src/tools/pgindent/typedefs.list                |  1 +
5 files changed, 89 insertions(+), 48 deletions(-)



view thread (7+ messages)

Message-ID: <E1pPaTk-0000Zr-BZ@gemulon.postgresql.org>
Permalink:  ../E1pPaTk-0000Zr-BZ@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1pPaTk-0000Zr-BZ@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: akapila@postgresql.org
  Subject: Re: pgsql: Fix the logical replication timeout during large DDLs.
  In-Reply-To: <E1pPaTk-0000Zr-BZ@gemulon.postgresql.org>
  References: <E1pPaTk-0000Zr-BZ@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox