MIME-Version: 1.0
In-Reply-To: <10df46d9.7dc2.19cd740a640.Coremail.jiye_sw@126.com>
References: <10df46d9.7dc2.19cd740a640.Coremail.jiye_sw@126.com>
From: "David G. Johnston" <david.g.johnston@gmail.com>
Date: Tue, 10 Mar 2026 22:30:03 -0700
Message-ID: <CAKFQuwYcEvfna4HUkG9VkX0cR_xbRRhT_0qDaAHhrJ4UX0EW-g@mail.gmail.com>
Subject: Re: FDW connection drops with "Connection timed out" during async
 append query due to TCP receive buffer filling up
To: jiye <jiye_sw@126.com>
Cc: "pgsql-bugs@lists.postgresql.org" <pgsql-bugs@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="00000000000096a868064cb8edbf"
Archived-At: <https://www.postgresql.org/message-id/CAKFQuwYcEvfna4HUkG9VkX0cR_xbRRhT_0qDaAHhrJ4UX0EW-g%40mail.gmail.com>
Precedence: bulk

--00000000000096a868064cb8edbf
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tuesday, March 10, 2026, jiye <jiye_sw@126.com> wrote:
>
> This is a minimal working example. In practice, if the local table scan
> takes too long and the foreign table has sufficiently wide rows, this iss=
ue
> may reproduce.
> In my understanding, when performing a local sequential scan, the
> PostgreSQL backend fetches data from the local plan without fetching any
> data from the FDW. As a result, the TCP receive buffer may become full,
> causing the FDW connection to be disconnected.
> I believe this is a minor issue. How can I resolve this problem?
>

Do not establish a timeout that the execution of the query cannot beat.
Or, I think, at least ensure the non-async portion of the query can produce
a row within the allotted time so the async node is polled within the
timeout.  IIUC, the general loop flow is:  begin append, begin async, poll
async, poll non-async, poll async, poll non-async, etc=E2=80=A6. There will=
 usually
be some lag between async polls.  The tcp timeout has to be large enough to
accommodate your reality.  No different than if you used a statement
timeout.

I don=E2=80=99t actually know whether or if =E2=80=9Cbuffer filling up=E2=
=80=9D is accurate or
relevant here.  It doesn=E2=80=99t seem that way.  You haven=E2=80=99t demo=
nstrated that
scenario here, just a timeout being reached.

https://github.com/postgres/postgres/blob/82467f627bd478569de04f4a3f1993098=
e80c812/src/backend/executor/nodeAppend.c#L342

And since the main design point of async is that any of them may be polled
at any time it is necessary for all such scans to be initialized before any
polling begins.  Starting the clock on all of them.

If you don=E2=80=99t want a connection timeout to happen do not set one.  T=
hat=E2=80=99s
the resolution here so far as I can tell.

David J.

--00000000000096a868064cb8edbf
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tuesday, March 10, 2026, jiye &lt;<a href=3D"mailto:jiye_sw@126.com">jiy=
e_sw@126.com</a>&gt; wrote:<blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"lin=
e-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div style=3D"=
margin:0px;text-indent:2em">This is a minimal working example. In practice,=
 if the local table scan takes too long and the foreign table has sufficien=
tly wide rows, this issue may reproduce.
</div><div style=3D"margin:0px;text-indent:2em">In my understanding, when p=
erforming a local sequential scan, the PostgreSQL backend fetches data from=
 the local plan without fetching any data from the FDW. As a result, the TC=
P receive buffer may become full, causing the FDW connection to be disconne=
cted.
</div><div style=3D"margin:0px;text-indent:2em">I believe this is a minor i=
ssue. How can I resolve this problem?</div></div></blockquote><div><br></di=
v><div>Do not establish a timeout that the execution of the query cannot be=
at.=C2=A0 Or, I think, at least ensure the non-async portion of the query c=
an produce a row within the allotted time so the async node is polled withi=
n the timeout.=C2=A0 IIUC, the general loop flow is: =C2=A0begin append, be=
gin async, poll async, poll non-async, poll async, poll non-async, etc=E2=
=80=A6. There will usually be some lag between async polls.=C2=A0 The tcp t=
imeout has to be large enough to accommodate your reality.=C2=A0 No differe=
nt than if you used a statement timeout.</div><div><br></div><div>I don=E2=
=80=99t actually know whether or if =E2=80=9Cbuffer filling up=E2=80=9D is =
accurate or relevant here.=C2=A0 It doesn=E2=80=99t seem that way.=C2=A0 Yo=
u haven=E2=80=99t demonstrated that scenario here, just a timeout being rea=
ched.</div><div><br></div><div><a href=3D"https://github.com/postgres/postg=
res/blob/82467f627bd478569de04f4a3f1993098e80c812/src/backend/executor/node=
Append.c#L342">https://github.com/postgres/postgres/blob/82467f627bd478569d=
e04f4a3f1993098e80c812/src/backend/executor/nodeAppend.c#L342</a><br></div>=
<div><br></div><div>And since the main design point of async is that any of=
 them may be polled at any time it is necessary for all such scans to be in=
itialized before any polling begins.=C2=A0 Starting the clock on all of the=
m.</div><div><br></div><div>If you don=E2=80=99t want a connection timeout =
to happen do not set one.=C2=A0 That=E2=80=99s the resolution here so far a=
s I can tell.</div><div><br></div><div>David J.</div><div><br></div>

--00000000000096a868064cb8edbf--