public inbox for [email protected]
help / color / mirror / Atom feedFrom: Daniele Varrazzo <[email protected]>
To: Magnus Hagander <[email protected]>
Cc: [email protected]
Subject: Re: Designing a better connection pool for psycopg3
Date: Mon, 18 Jan 2021 15:29:34 +0100
Message-ID: <CA+mi_8a83=Qy-Ym+_X4tLPcD1YHVVVYF2Ki5jeHU4eDz_etG5w@mail.gmail.com> (raw)
In-Reply-To: <CABUevExMQQB3=xShPie+XR04kctiR8-P_BEUz1BmCyDLhU_yhg@mail.gmail.com>
References: <CA+mi_8aZQXzB8Kr4V0nps3aMQ74N72uWC=pZF0P1H7AW4GLasA@mail.gmail.com>
<[email protected]>
<CA+mi_8a-0Wx62eziOBwp-=ryBynsqrgx4AcBGW5+pSr3vZXjHg@mail.gmail.com>
<CABUevExMQQB3=xShPie+XR04kctiR8-P_BEUz1BmCyDLhU_yhg@mail.gmail.com>
On Mon, 18 Jan 2021 at 15:05, Magnus Hagander <[email protected]> wrote:
>
> On Mon, Jan 18, 2021 at 2:50 PM Daniele Varrazzo
> <[email protected]> wrote:
> >
> > On Mon, 18 Jan 2021 at 14:13, Karsten Hilbert <[email protected]> wrote:
> >
> > > I would strongly advise against making sys.exit() the default
> > > for pool.terminate() unless I misunderstand something.
> >
> > How would you terminate the program if a maintenance thread, not the
> > main one, thinks that the program is not in working state?
>
> Why would it be OK for a maintenance thread to terminate the program
> at all? And certainly by default?
>
> Wouldn't the reasonable thing to do be to flag the pool as broken, and
> then just stop trying. Then whenever the application makes an attempt
> to use the pool, it can he thrown an exception saying that this
> happened.
I'm trying to imagine what happens in a case such as a network
partition or reconfiguration, and the app server doesn't see the
database anymore. This node is arguably broken.
If the reconnection thread fails to obtain new connections, and the
ones currently in the pool are discarded as detected broken,
eventually the pool is depleted.
The requesting threads (e.g. web requests handlers) would try to
obtain a connection, time out after e.g. 30 seconds, and receive an
error. The error would result in a 500 for the user, probably a sentry
exception and log in a file, but the program would likely not die.
The program could remain in this condition for an arbitrary long time,
until someone notices by looking at the logs and scratches their head
to understand how to fix the problem.
If the program dies, its manager would try to restart it, insisting
until the configuration makes the database visible again. A service
down in a crash loop is more visible than a service up and running but
not serving.
Anyway I appreciate that the default of terminating a program is
probably too aggressive. So I would remove the `terminate()` function
and base implementation and leave a `connection_failed()` handler,
with a default no-op implementation, which people preferring their
program to terminate can subclass (with `sys.exit(1)` or whatever
termination strategy they find useful).
-- Daniele
view thread (19+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: Designing a better connection pool for psycopg3
In-Reply-To: <CA+mi_8a83=Qy-Ym+_X4tLPcD1YHVVVYF2Ki5jeHU4eDz_etG5w@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox