pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Duncan Sands <duncan.sands@deepbluecap.com>
To: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Subject: Re: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5
Date: Mon, 2 Jun 2025 09:12:50 +0200
Message-ID: <61ccd3cd-3185-4b07-9ce4-738c91546b54@deepbluecap.com> (raw)
In-Reply-To: <OSCPR01MB149667B316377CE0615E15138F567A@OSCPR01MB14966.jpnprd01.prod.outlook.com>
References: <680bdaf6-f7d1-4536-b580-05c2760c67c6@deepbluecap.com>
	<CAA4eK1JwJw6JOnfDxtGtSRF7kM0LbEVPRmNxWeJa5+wyoG05Xg@mail.gmail.com>
	<CAA4eK1LMgqeT_bPZ3MH-VKvwOqpZyfJmF7knZhu1rqt2Pqsnwg@mail.gmail.com>
	<CAD21AoCjpj-DoASrhKCiEKuxh=0q3rC1RnAeugEwEJ=xwk=n6g@mail.gmail.com>
	<CAA4eK1LVcHD6VpH7m=8F9ru7Qdj9hsXE0gXtS93RmtTxf6L1NA@mail.gmail.com>
	<CAD21AoCeM2nni1P7Z6KXzLM=6zCdShC82sOvuvu0_hBuJkm9Qw@mail.gmail.com>
	<OS7PR01MB14968B3C263074A2DEB77DB58F565A@OS7PR01MB14968.jpnprd01.prod.outlook.com>
	<CAA4eK1+PGTdkTJwK8Necq8GrAJWP4T8cp_ponzbtogkFYvbK2w@mail.gmail.com>
	<CAD21AoBCn7RR0EYbK+1n5UTksc3CVn5AKvxBRSr7zR2eWqTTOw@mail.gmail.com>
	<OSCPR01MB149667B316377CE0615E15138F567A@OSCPR01MB14966.jpnprd01.prod.outlook.com>

Dear Hayato Kuroda, I tried 
v4-PG17-0001-Avoid-distributing-invalidation-messages-sev.patch and I can 
confirm that it also resolves my original issue.

Best wishes, Duncan.

On 28/05/2025 14:27, Hayato Kuroda (Fujitsu) wrote:
> Dear Sawada-san, Amit,
> 
>>> It can impact the performance for large transactions with fewer
>>> invalidations, especially the ones which has spilled changes because
>>> it needs to traverse the entire list of changes again at the end.
>>
>> Agreed.
>>
>>> The
>>> other idea would be to add new member(s) in ReorderBufferTXN to
>>> receive distributed invalidations. For adding the new member in
>>> ReorderBufferTXN: (a) in HEAD, it should be okay, (b) for
>>> backbranches, we may be able to add at the end, but we should check if
>>> there are any extensions using sizeof(ReorderBufferTxn) and if they
>>> are using what we need to do.
>>
>> If we can make sure that that change won't break the existing
>> extensions, I think this would be the most reasonable solution.
> 
> Based on the discussion, I created PoC for master/PG17. Please see attached.
> The basic idea is to introduce the new queue which only contains distributed inval
> messages. Contents are consumed at end of transactions. I feel some of codes can
> be re-used so that internal functions are introduced. At least, it could pass
> regression tests and workloads discussed here.
> 
> Best regards,
> Hayato Kuroda
> FUJITSU LIMITED
> 






view thread (94+ messages)  latest in thread

Message-ID: <61ccd3cd-3185-4b07-9ce4-738c91546b54@deepbluecap.com>
Permalink:  ../61ccd3cd-3185-4b07-9ce4-738c91546b54@deepbluecap.com/
Also on:    postgresql.org/message-id/61ccd3cd-3185-4b07-9ce4-738c91546b54@deepbluecap.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: duncan.sands@deepbluecap.com, kuroda.hayato@fujitsu.com, sawada.mshk@gmail.com, amit.kapila16@gmail.com, pgsql-bugs@lists.postgresql.org
  Subject: Re: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5
  In-Reply-To: <61ccd3cd-3185-4b07-9ce4-738c91546b54@deepbluecap.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox