MIME-Version: 1.0
References: 
 <CAP50yQ-PUwP80+awO-U3Ud45HCZHZ=fjK3L2iWh4CfcEPF1p6A@mail.gmail.com>
In-Reply-To: 
 <CAP50yQ-PUwP80+awO-U3Ud45HCZHZ=fjK3L2iWh4CfcEPF1p6A@mail.gmail.com>
From: Daniele Varrazzo <daniele.varrazzo@gmail.com>
Date: Mon, 17 Oct 2022 12:55:53 +0100
Message-ID: 
 <CA+mi_8aX4FsE6=HWF08tf579G8Wgg9cQOtN2BkmtTehyxAb5FA@mail.gmail.com>
Subject: Re: Puzzling situation with threads and psycopg2 on RDS
To: Stefan Knecht <knecht.stefan@gmail.com>
Cc: psycopg@postgresql.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: 
 <https://www.postgresql.org/message-id/CA%2Bmi_8aX4FsE6%3DHWF08tf579G8Wgg9cQOtN2BkmtTehyxAb5FA%40mail.gmail.com>
Precedence: bulk

Hi Stefan,

On Mon, 17 Oct 2022 at 12:25, Stefan Knecht <knecht.stefan@gmail.com> wrote:
>
> Hello
>
> I have this situation now, that I can't wrap my head around and I'm hoping someone can shed some light onto what is going on, before I spend hours trying to reproduce this by taking the code apart into a test case.
>
> A few facts:
>
> - I've got a piece of code split into several classes.
> - It initializes a dynamic number of threads. For the sake of this topic, I'll stick to just 2 threads to illustrate.
> - Each thread creates its own database connection (psycopg2.connect()) when it's started
> - Each thread, for each task it does, prints out its own name and the PID of the database session.
...

> Perhaps you can already see the problem. One thread randomly disconnects.

I understand this happens on RDS. Does it happen on normal Postgres
too, from a normal process (not some serverless thing?)

Are there processes/forks involved? Which version of psycopg are you
using? Some old one have problems with closing connections in
multiprocess environments. That was fixed in psycopg 2.8 (#829).

Cheers

-- Daniele