Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1okPL1-0004Ty-Qs for psycopg@arkaria.postgresql.org; Mon, 17 Oct 2022 12:35:23 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1okPL0-0002yU-Kp for psycopg@arkaria.postgresql.org; Mon, 17 Oct 2022 12:35:22 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1okPL0-0002yK-Df for psycopg@lists.postgresql.org; Mon, 17 Oct 2022 12:35:22 +0000 Received: from mail-yb1-xb32.google.com ([2607:f8b0:4864:20::b32]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1okPKy-0000Qb-HC for psycopg@postgresql.org; Mon, 17 Oct 2022 12:35:22 +0000 Received: by mail-yb1-xb32.google.com with SMTP id y205so13050990yby.13 for ; Mon, 17 Oct 2022 05:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=vf+WP4ixSf7fCxwyPZdpqmkRmLrER24tBiWpPXFL0Ug=; b=bbQKUiyY2w1UOHgkjwTOmaO9GDvJXesn0HsKZ02ruS+ggsddB3KILihuVLOpLiExDZ YrbD7SyPc9uomjsbrlJlbLtoFgA2efrQoDPhROyYf8zbynJs5e/53v9nzB8Oy/tO3hCO J+PB/pCv4o7g+holgegkpqVl+GEpzSSSkseIY2AMDIsZ+4+j7jDOHWcmnKp4EA0CUEeT Eh4eZMUAvFZavSkTmiCH+Uid+CTQzoEx1S3uMou2txTtYlJX/N1BSjJQ5xw6CGQ5z5V5 MF58a62GxrihhN35y6h9uE88wWVZyYdtJTKW4HyBB0aJ/SpJ4PiH8lf6WTK7klWfwzZH Jfiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vf+WP4ixSf7fCxwyPZdpqmkRmLrER24tBiWpPXFL0Ug=; b=d3i2xW1KCFyQ5XkPRo5sPqxM4vrunxqDxGnTx3ILTE+b/nj1nWoaV1MEwjXFqVXwUd TYY2XP29qZNAHyNWdvvXtQa0t65ZwqQEJotZkmq4gP+eLlCQ0kyl8uWYXQTtDhcA4RVt hqsETL/vef/vqJunjarOKlG9YGYzujZv9iHv4PKSOdWpv+R4pxm2w+356REu5tbbO9Dg bp6O7c0u5DXHyyfcU3t4HSVZLwo9BNaZWGpNLS+BgVdcTG+5CHVFpAPv0Bh0OakeyeVz xf/ZbXnuqdNFfiWtqz1K+hO1+N9RwfaZZhXvnmR9wftzDAb92ioweG9psKvxntq4Oo2N 3DgA== X-Gm-Message-State: ACrzQf1pzSf6JsMypvkbs9juDQqZ5CGDPRvUFS3XmKPNEHaFJyV3NC92 uSClbrVuAUOeFlrIUeeW+TopJsMLPmVlkE5a1uc= X-Google-Smtp-Source: AMsMyM7mfQdsDdVVE+1N/Qaur3OagJc9ViKCv2Wdsvbf07Ncow5QKxT7z+lCj0GPgaLjPF5sY8B+mm/NcVLtFLGywvs= X-Received: by 2002:a25:ff11:0:b0:6b2:8bc4:c1a6 with SMTP id c17-20020a25ff11000000b006b28bc4c1a6mr8616876ybe.606.1666010119213; Mon, 17 Oct 2022 05:35:19 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Stefan Knecht Date: Mon, 17 Oct 2022 19:35:10 +0700 Message-ID: Subject: Re: Puzzling situation with threads and psycopg2 on RDS To: Daniele Varrazzo Cc: psycopg@postgresql.org Content-Type: multipart/alternative; boundary="0000000000005916ff05eb3a3545" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000005916ff05eb3a3545 Content-Type: text/plain; charset="UTF-8" Ah, the error was (of course) between the keyboard and the chair. A database handle was defined on the thread class - not the instance. And for some reason everything was still working, just with very unexpected behavior :) Have a good one Daniele On Mon, Oct 17, 2022 at 7:13 PM Stefan Knecht wrote: > Exact same behavior with psycopg2-binary 2.9.4. > > > > On Mon, Oct 17, 2022 at 7:08 PM Stefan Knecht > wrote: > >> Ciao Daniele >> >> Thanks for looking at this. >> >> >> >>> I understand this happens on RDS. Does it happen on normal Postgres >>> too, from a normal process (not some serverless thing?) >>> >>> >> I'll test this now. >> >> >>> Are there processes/forks involved? >> >> >> No, just Python's basic threading. >> >> >>> Which version of psycopg are you >>> using? Some old one have problems with closing connections in >>> multiprocess environments. That was fixed in psycopg 2.8 (#829). >>> >>> >> This is psycopg 3.1.3 >> >> And I just realized that all the other threading code I've got is on >> psycopg2-binary 2.8. Has there something changed with threads in v3 ? >> >> >> >> Stefan >> >> --0000000000005916ff05eb3a3545 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Ah, the error was (of course) between the keyboard an= d the chair.

A database handle was defined on the = thread class - not the instance. And for some reason everything was still w= orking, just with very unexpected behavior :)


=
Have a good one Daniele


<= /div>
O= n Mon, Oct 17, 2022 at 7:13 PM Stefan Knecht <knecht.stefan@gmail.com> wrote:
Exact same be= havior with psycopg2-binary 2.9.4.



On Mo= n, Oct 17, 2022 at 7:08 PM Stefan Knecht <knecht.stefan@gmail.com> wrote:
Ciao Daniele

Thanks for looking = at this.



I understand this happens on RDS. Does it happen on normal Postgres
too, from a normal process (not some serverless thing?)


I'll test this now.
=C2=A0
Are there processes/forks involved?

No, ju= st Python's basic threading.
=C2=A0
Which version of psycopg are you
using? Some old one have problems with closing connections in
multiprocess environments. That was fixed in psycopg 2.8 (#829).


This is psycopg 3.1.3
=C2=A0=
And I just realized that all the other threading code I've g= ot is on psycopg2-binary 2.8. Has there something changed with threads in v= 3 ?



Stefan

=
--0000000000005916ff05eb3a3545--