Message-ID: <61ccd3cd-3185-4b07-9ce4-738c91546b54@deepbluecap.com>
Date: Mon, 2 Jun 2025 09:12:50 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Logical replication 'invalid memory alloc request size
 1585837200' after upgrading to 17.5
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>,
 'Masahiko Sawada' <sawada.mshk@gmail.com>,
 Amit Kapila <amit.kapila16@gmail.com>
Cc: "pgsql-bugs@lists.postgresql.org" <pgsql-bugs@lists.postgresql.org>
References: <680bdaf6-f7d1-4536-b580-05c2760c67c6@deepbluecap.com>
 <CAA4eK1JwJw6JOnfDxtGtSRF7kM0LbEVPRmNxWeJa5+wyoG05Xg@mail.gmail.com>
 <CAA4eK1LMgqeT_bPZ3MH-VKvwOqpZyfJmF7knZhu1rqt2Pqsnwg@mail.gmail.com>
 <CAD21AoCjpj-DoASrhKCiEKuxh=0q3rC1RnAeugEwEJ=xwk=n6g@mail.gmail.com>
 <CAA4eK1LVcHD6VpH7m=8F9ru7Qdj9hsXE0gXtS93RmtTxf6L1NA@mail.gmail.com>
 <CAD21AoCeM2nni1P7Z6KXzLM=6zCdShC82sOvuvu0_hBuJkm9Qw@mail.gmail.com>
 <OS7PR01MB14968B3C263074A2DEB77DB58F565A@OS7PR01MB14968.jpnprd01.prod.outlook.com>
 <CAA4eK1+PGTdkTJwK8Necq8GrAJWP4T8cp_ponzbtogkFYvbK2w@mail.gmail.com>
 <CAD21AoBCn7RR0EYbK+1n5UTksc3CVn5AKvxBRSr7zR2eWqTTOw@mail.gmail.com>
 <OSCPR01MB149667B316377CE0615E15138F567A@OSCPR01MB14966.jpnprd01.prod.outlook.com>
Content-Language: en-US
From: Duncan Sands <duncan.sands@deepbluecap.com>
In-Reply-To: 
 <OSCPR01MB149667B316377CE0615E15138F567A@OSCPR01MB14966.jpnprd01.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/61ccd3cd-3185-4b07-9ce4-738c91546b54%40deepbluecap.com>
Precedence: bulk

Dear Hayato Kuroda, I tried 
v4-PG17-0001-Avoid-distributing-invalidation-messages-sev.patch and I can 
confirm that it also resolves my original issue.

Best wishes, Duncan.

On 28/05/2025 14:27, Hayato Kuroda (Fujitsu) wrote:
> Dear Sawada-san, Amit,
> 
>>> It can impact the performance for large transactions with fewer
>>> invalidations, especially the ones which has spilled changes because
>>> it needs to traverse the entire list of changes again at the end.
>>
>> Agreed.
>>
>>> The
>>> other idea would be to add new member(s) in ReorderBufferTXN to
>>> receive distributed invalidations. For adding the new member in
>>> ReorderBufferTXN: (a) in HEAD, it should be okay, (b) for
>>> backbranches, we may be able to add at the end, but we should check if
>>> there are any extensions using sizeof(ReorderBufferTxn) and if they
>>> are using what we need to do.
>>
>> If we can make sure that that change won't break the existing
>> extensions, I think this would be the most reasonable solution.
> 
> Based on the discussion, I created PoC for master/PG17. Please see attached.
> The basic idea is to introduce the new queue which only contains distributed inval
> messages. Contents are consumed at end of transactions. I feel some of codes can
> be re-used so that internal functions are introduced. At least, it could pass
> regression tests and workloads discussed here.
> 
> Best regards,
> Hayato Kuroda
> FUJITSU LIMITED
>