MIME-Version: 1.0
References: <CAD1FGCT2sYrP_70RTuo56QTizyc+J3wJdtn2gtO3VttQFpdMZg@mail.gmail.com>
 <OSCPR01MB149660B4FB4ED7B34D4340C6AF575A@OSCPR01MB14966.jpnprd01.prod.outlook.com>
In-Reply-To: <OSCPR01MB149660B4FB4ED7B34D4340C6AF575A@OSCPR01MB14966.jpnprd01.prod.outlook.com>
From: Max Madden <maxmmadden@gmail.com>
Date: Wed, 11 Jun 2025 11:57:51 +0100
Message-ID: <CAD1FGCQmgLK9XqHfmkAR2+79oLBsJRYPQj0W-hirbZJRgzLVkg@mail.gmail.com>
Subject: Re: Logical Replication Memory Allocation Error - "invalid memory
 alloc request size"
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Cc: "pgsql-general@lists.postgresql.org" <pgsql-general@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000e0eb67063749af42"
Archived-At: <https://www.postgresql.org/message-id/CAD1FGCQmgLK9XqHfmkAR2%2B79oLBsJRYPQj0W-hirbZJRgzLVkg%40mail.gmail.com>
Precedence: bulk

--000000000000e0eb67063749af42
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Hayato,

Thank you for your reply.

We have rewritten as many of our transactions as possible to avoid using
temporary tables, and so far, that seems to have resolved the problem.

Thank you for your help.

Many thanks,

Max

On Wed, Jun 11, 2025 at 3:31=E2=80=AFAM Hayato Kuroda (Fujitsu) <
kuroda.hayato@fujitsu.com> wrote:

> Dear Max,
>
> Thanks for the report.
>
> > The initial snapshot and data copy complete successfully for all tables=
.
> However, anywhere from 5
> > minutes to 2 hours after the initial sync, the subscription consistentl=
y
> fails with memory allocation errors like:
> >
> > ```
> > 2025-06-10 14:14:56.800 UTC [299] ERROR: could not receive data from WA=
L
> stream: ERROR: invalid memory alloc request size 1238451248
> > 2025-06-10 14:14:56.805 UTC [1] LOG: background worker "logical
> replication worker" (PID 299) exited with exit code 1
> > ```
>
> I think this is a known postgres bug which has been also reported at [1].
> We are discussing
> how we fix. Typically this can happen when there are lots of concurrent
> transactions
> and they have DDLs. IIUC there are no good workaround for now - any
> parameters can't
> avoid the failure. Only you can reduce them.
>
> I'm happy if you apply the patch posted at [1] and confirms the issue can
> be solved, but...
> seems difficult because you are in the managed env.
>
> [1]:
> https://www.postgresql.org/message-id/CALDaNm0TaTPuza7Fa%2BDRMzL%2BmqK3%2=
B7RVEvFiRoDJbU2vkJESwg%40mail.gmail.com
>
> Best regards,
> Hayato Kuroda
> FUJITSU LIMITED
>
>

--000000000000e0eb67063749af42
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">Hi Hayato,=C2=A0<div><br></div><div>Thank=
 you for your reply.</div><div><br></div><div>We have rewritten as many of =
our transactions as possible to avoid using temporary tables, and so far, t=
hat seems to have resolved the problem.=C2=A0</div><div><br></div><div>Than=
k you for your help.=C2=A0</div><div><br></div><div>Many thanks,</div><div>=
=C2=A0</div><div>Max</div></div><br><div class=3D"gmail_quote gmail_quote_c=
ontainer"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jun 11, 2025 at 3:3=
1=E2=80=AFAM Hayato Kuroda (Fujitsu) &lt;<a href=3D"mailto:kuroda.hayato@fu=
jitsu.com">kuroda.hayato@fujitsu.com</a>&gt; wrote:<br></div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex">Dear Max,<br>
<br>
Thanks for the report.<br>
<br>
&gt; The initial snapshot and data copy complete successfully for all table=
s. However, anywhere from 5<br>
&gt; minutes to 2 hours after the initial sync, the subscription consistent=
ly fails with memory allocation errors like:<br>
&gt;<br>
&gt; ```<br>
&gt; 2025-06-10 14:14:56.800 UTC [299] ERROR: could not receive data from W=
AL stream: ERROR: invalid memory alloc request size 1238451248 <br>
&gt; 2025-06-10 14:14:56.805 UTC [1] LOG: background worker &quot;logical r=
eplication worker&quot; (PID 299) exited with exit code 1<br>
&gt; ```<br>
<br>
I think this is a known postgres bug which has been also reported at [1]. W=
e are discussing<br>
how we fix. Typically this can happen when there are lots of concurrent tra=
nsactions<br>
and they have DDLs. IIUC there are no good workaround for now - any paramet=
ers can&#39;t<br>
avoid the failure. Only you can reduce them.<br>
<br>
I&#39;m happy if you apply the patch posted at [1] and confirms the issue c=
an be solved, but...<br>
seems difficult because you are in the managed env.<br>
<br>
[1]: <a href=3D"https://www.postgresql.org/message-id/CALDaNm0TaTPuza7Fa%2B=
DRMzL%2BmqK3%2B7RVEvFiRoDJbU2vkJESwg%40mail.gmail.com" rel=3D"noreferrer" t=
arget=3D"_blank">https://www.postgresql.org/message-id/CALDaNm0TaTPuza7Fa%2=
BDRMzL%2BmqK3%2B7RVEvFiRoDJbU2vkJESwg%40mail.gmail.com</a><br>
<br>
Best regards,<br>
Hayato Kuroda<br>
FUJITSU LIMITED<br>
<br>
</blockquote></div></div>

--000000000000e0eb67063749af42--