MIME-Version: 1.0
References: <18354-140864c09686b5a6@postgresql.org> <20240222.114600.1019580904613326727.horikyota.ntt@gmail.com>
In-Reply-To: <20240222.114600.1019580904613326727.horikyota.ntt@gmail.com>
From: Tender Wang <tndrwang@gmail.com>
Date: Thu, 22 Feb 2024 11:49:29 +0800
Message-ID: <CAHewXN=ZEL1Rvw2fJmt8k3sO_3fb58gijv7GR-JnY8ujgGP0UA@mail.gmail.com>
Subject: Re: BUG #18354: Aborted transaction aborted during cleanup when
 temp_file_limit exceeded
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: exclusion@gmail.com, pgsql-bugs@lists.postgresql.org
Content-Type: multipart/alternative; boundary="0000000000004652c00611f0554e"
Archived-At: <https://www.postgresql.org/message-id/CAHewXN%3DZEL1Rvw2fJmt8k3sO_3fb58gijv7GR-JnY8ujgGP0UA%40mail.gmail.com>
Precedence: bulk

--0000000000004652c00611f0554e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

When the first error was reported, we entered AbortTransaction(),
proc->xid was changed to InvalidTransactionId
in ProcArrayEndTransactionInternal().
After AbortTransaction() is done, we enter  into CleanupTransaction(), and
error would be reported again. So we enter AbortTransaction() again because
the transaction block state is still TBLOCK_STARTED. MyProc->xid now is
InvalidTransactionId, so the assert failed.

Changing the behavior of tuplestore can work for this issue,  but I'm not
sure if this change will affect other components which depend on
BufFile(like nodeMaterial)

Kyotaro Horiguchi <horikyota.ntt@gmail.com> =E4=BA=8E2024=E5=B9=B42=E6=9C=
=8822=E6=97=A5=E5=91=A8=E5=9B=9B 10:46=E5=86=99=E9=81=93=EF=BC=9A

> At Wed, 21 Feb 2024 12:00:01 +0000, PG Bug reporting form <
> noreply@postgresql.org> wrote in
> > triggers two errors, a warning, and an assertion failure:
> > ERROR:  temporary file size exceeds temp_file_limit (100kB)
> > WARNING:  AbortTransaction while in ABORT state
> > ERROR:  temporary file size exceeds temp_file_limit (100kB)
> > server closed the connection unexpectedly
> ...
> > For the second error:
> > 2024-02-21 11:40:07.001 UTC|law|regression|65d5e116.13cfcb|ERROR:
> temporary
> > file size exceeds temp_file_limit (100kB)
> > 2024-02-21 11:40:07.001 UTC|law|regression|65d5e116.13cfcb|BACKTRACE:
> > FileWrite at fd.c:2183:5
> > BufFileDumpBuffer at buffile.c:537:18
> > BufFileFlush at buffile.c:723:3
> > BufFileClose at buffile.c:419:9
> > tuplestore_end at tuplestore.c:459:5
> > MemoryContextSwitchTo at palloc.h:142:23
> >  (inlined by) PortalDrop at portalmem.c:587:3
> > AtCleanup_Portals at portalmem.c:907:3
>
> Therefore, BufFileClose should not flush the content during error
> handling.  In the first place, tuplestore doesn't need to flush the
> underlying files in _end and _clear. In this case, I would choose to
> change the general behavior of tuplestore. The attached PoC patch
> fixes the issue for me. It introduces a new "extended" function to
> control flushing, avoiding the addition of an unnatural parameter to
> BufFileClose. I suspect that it is usable in some other places, but I
> haven't checked that.
>
> regards.
>
> --
> Kyotaro Horiguchi
> NTT Open Source Software Center
>


--=20
Tender Wang
OpenPie:  https://en.openpie.com/

--0000000000004652c00611f0554e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">When the first error was reported, we entered=C2=A0AbortTr=
ansaction(),=C2=A0 proc-&gt;xid was changed to InvalidTransactionId in=C2=
=A0ProcArrayEndTransactionInternal().<div>After AbortTransaction() is done,=
 we enter=C2=A0 into CleanupTransaction(), and error would be reported agai=
n. So we enter AbortTransaction() again because</div><div>the transaction b=
lock state is still TBLOCK_STARTED. MyProc-&gt;xid now is InvalidTransactio=
nId, so the assert failed.=C2=A0</div><div><br></div><div>Changing the beha=
vior of tuplestore can work=C2=A0for this issue,=C2=A0 but I&#39;m not sure=
 if this change will affect other components which depend on BufFile(like n=
odeMaterial)</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla=
ss=3D"gmail_attr">Kyotaro Horiguchi &lt;<a href=3D"mailto:horikyota.ntt@gma=
il.com">horikyota.ntt@gmail.com</a>&gt; =E4=BA=8E2024=E5=B9=B42=E6=9C=8822=
=E6=97=A5=E5=91=A8=E5=9B=9B 10:46=E5=86=99=E9=81=93=EF=BC=9A<br></div><bloc=
kquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:=
1px solid rgb(204,204,204);padding-left:1ex">At Wed, 21 Feb 2024 12:00:01 +=
0000, PG Bug reporting form &lt;<a href=3D"mailto:noreply@postgresql.org" t=
arget=3D"_blank">noreply@postgresql.org</a>&gt; wrote in <br>
&gt; triggers two errors, a warning, and an assertion failure:<br>
&gt; ERROR:=C2=A0 temporary file size exceeds temp_file_limit (100kB)<br>
&gt; WARNING:=C2=A0 AbortTransaction while in ABORT state<br>
&gt; ERROR:=C2=A0 temporary file size exceeds temp_file_limit (100kB)<br>
&gt; server closed the connection unexpectedly<br>
...<br>
&gt; For the second error:<br>
&gt; 2024-02-21 11:40:07.001 UTC|law|regression|65d5e116.13cfcb|ERROR:=C2=
=A0 temporary<br>
&gt; file size exceeds temp_file_limit (100kB)<br>
&gt; 2024-02-21 11:40:07.001 UTC|law|regression|65d5e116.13cfcb|BACKTRACE:=
=C2=A0 <br>
&gt; FileWrite at fd.c:2183:5<br>
&gt; BufFileDumpBuffer at buffile.c:537:18<br>
&gt; BufFileFlush at buffile.c:723:3<br>
&gt; BufFileClose at buffile.c:419:9<br>
&gt; tuplestore_end at tuplestore.c:459:5<br>
&gt; MemoryContextSwitchTo at palloc.h:142:23<br>
&gt;=C2=A0 (inlined by) PortalDrop at portalmem.c:587:3<br>
&gt; AtCleanup_Portals at portalmem.c:907:3<br>
<br>
Therefore, BufFileClose should not flush the content during error<br>
handling.=C2=A0 In the first place, tuplestore doesn&#39;t need to flush th=
e<br>
underlying files in _end and _clear. In this case, I would choose to<br>
change the general behavior of tuplestore. The attached PoC patch<br>
fixes the issue for me. It introduces a new &quot;extended&quot; function t=
o<br>
control flushing, avoiding the addition of an unnatural parameter to<br>
BufFileClose. I suspect that it is usable in some other places, but I<br>
haven&#39;t checked that.<br>
<br>
regards.<br>
<br>
-- <br>
Kyotaro Horiguchi<br>
NTT Open Source Software Center<br>
</blockquote></div><br clear=3D"all"><div><br></div><span class=3D"gmail_si=
gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d=
iv dir=3D"ltr"><div>Tender Wang<br></div>OpenPie:=C2=A0=C2=A0<a href=3D"htt=
ps://en.openpie.com/" target=3D"_blank">https://en.openpie.com/</a></div></=
div>

--0000000000004652c00611f0554e--