MIME-Version: 1.0
References: 
 <CA+mi_8ZH-feQP0+k_j7_joygE=hMZCFnaNMv8Z1KCLKB9YLb0w@mail.gmail.com>
 <CAP50yQ_Mt=RtR0sXYoazJTuarphR1Kv8A6YTevCMiv+xZeD9=Q@mail.gmail.com>
 <20200330074917.GA28993@campbell-lange.net>
In-Reply-To: <20200330074917.GA28993@campbell-lange.net>
From: Stefan Knecht <knecht.stefan@gmail.com>
Date: Mon, 30 Mar 2020 16:14:59 +0700
Message-ID: 
 <CAP50yQ9WSB6s_rfXe2S0V-8H8ByCRNSw6ninrGdn5NgLzWZ8bA@mail.gmail.com>
Subject: Re: psycopg3: a first report
To: Rory Campbell-Lange <rory@campbell-lange.net>
Cc: Daniele Varrazzo <daniele.varrazzo@gmail.com>, psycopg@postgresql.org
Content-Type: multipart/alternative; boundary="0000000000004f84a505a20ee3dc"
Precedence: bulk

--0000000000004f84a505a20ee3dc
Content-Type: text/plain; charset="UTF-8"

Rory, this is about established connections, not new connections - psycopg2
already offers a connection timeout, but that is a different thing. I don't
want to drift too far off topic - but we are already using pgbouncer, and
the problem isn't detected by it, either. I'm not a developer but I believe
the problem is the generic nature of some blocking socket calls, which may
hang under some odd circumstances, and they remain hanging until some odd
ssl timeout is reached (15 minutes+ which is a very long time for any
application to be hanging in limbo, but more so for our own monitoring
tools which are written in Python).

Having the client provide general handling for in-flight timeouts would be
the overall best solution imho.

On Mon, Mar 30, 2020 at 2:48 PM Rory Campbell-Lange <rory@campbell-lange.net>
wrote:

> On 30/03/20, Stefan Knecht (knecht.stefan@gmail.com) wrote:
> > If I may, one thing that has been a constant troublemaker for us -
> running
> > primarily on AWS - is timeouts.
>
> If the issue is disappearing hosts or general connectivity problems,
> isn't that problem best dealt with by pg_bouncer?
>
> https://www.pgbouncer.org/config.html#connection-sanity-checks-timeouts
>
> Specifically you may wish to refer to server_connect_timeout.
>
> Rory
>
>

-- 
//
zztat - The Next-Gen Oracle Performance Monitoring and Reaction Framework!
Visit us at zztat.net | @zztat_oracle | fb.me/zztat | zztat.net/blog/

--0000000000004f84a505a20ee3dc
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Rory, this is about established connections, not new =
connections - psycopg2 already offers a connection timeout, but that is a d=
ifferent thing.

I don&#39;t want to drift too far off topic - but we are already using pgbo=
uncer, and the problem isn&#39;t detected by it, either. I&#39;m not a deve=
loper but I believe the problem is the generic nature of some blocking sock=
et calls, which may hang under some odd circumstances, and they remain hang=
ing until some odd ssl timeout is reached (15 minutes+ which is a very long=
 time for any application to be hanging in limbo, but more so for our own m=
onitoring tools which are written in Python). <br></div><div><br></div><div=
>Having the client provide general handling for in-flight timeouts would be=
 the overall best solution imho. <br></div></div><br><div class=3D"gmail_qu=
ote"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Mar 30, 2020 at 2:48 PM =
Rory Campbell-Lange &lt;<a href=3D"mailto:rory@campbell-lange.net">rory@cam=
pbell-lange.net</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pad=
ding-left:1ex">On 30/03/20, Stefan Knecht (<a href=3D"mailto:knecht.stefan@=
gmail.com" target=3D"_blank">knecht.stefan@gmail.com</a>) wrote:<br>
&gt; If I may, one thing that has been a constant troublemaker for us - run=
ning<br>
&gt; primarily on AWS - is timeouts. <br>
<br>
If the issue is disappearing hosts or general connectivity problems,<br>
isn&#39;t that problem best dealt with by pg_bouncer?<br>
<br>
<a href=3D"https://www.pgbouncer.org/config.html#connection-sanity-checks-t=
imeouts" rel=3D"noreferrer" target=3D"_blank">https://www.pgbouncer.org/con=
fig.html#connection-sanity-checks-timeouts</a><br>
<br>
Specifically you may wish to refer to server_connect_timeout.<br>
<br>
Rory<br>
<br>
</blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g=
mail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div>//</div><span s=
tyle=3D"font-size:small">zztat - The Next-Gen Oracle Performance Monitoring=
 and Reaction Framework!</span><div style=3D"font-size:small">Visit us at=
=C2=A0<a href=3D"http://zztat.net/" style=3D"color:rgb(17,85,204)" target=
=3D"_blank">zztat.net</a>=C2=A0| @zztat_oracle | <a href=3D"http://fb.me/zz=
tat" target=3D"_blank">fb.me/zztat</a> | <a href=3D"http://zztat.net/blog/"=
 target=3D"_blank">zztat.net/blog/</a></div></div></div></div></div>

--0000000000004f84a505a20ee3dc--