Subject: PgBouncer 1.24.0 - New upstream version

From: Bradford Boyle <[email-masked]>
Date: Sat, 11 Jan 2025 00:46:30 -0800

Hi All,

PgBouncer 1.24.0 was recently released [0]. I've updated the Debian
package with this new version but I am running into some issues with the
autopkgtests. The autopkgtests passes on Salsa's CI but is failing on
my both my local system and jengus [1].

On my local system, the test fails to create a new regress cluster
because port 5432 is already in use. I recently switched my sbuild
configuration from schroot to unshare. It looks like the unshare backend
doesn't create a new network namespace when running the autopkgtests.
If I stop the main postgres cluster on my machine before running
sbuild/autopkgtest, then the test passes. After reviewing the
autopkgtest docs [2] and the test script, I think we might need to add
isolation-container to the test's restrictions. On my local system, this
causes the test to be skipped since the unshare backend doesn't provide
this capability.

I will try to look in to this some more in the next few days, but I
wanted to check and see if anyone had run into similar issues with
building and testing other packages.

-- Bradford

[0]: https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer_1_24_0
[1]: https://jengus.postgresql.org/job/pgbouncer-binaries/101/architecture=amd64,distribution=sid/
[2]: https://people.debian.org/~eriberto/README.package-tests.html

---

From: Brian Cosgrove <[email-masked]>
Date: Sun, 12 Jan 2025 06:47:05 -0600

Hey Bradford,

I haven't run into that issue myself but will poke at it today and see if I
can reproduce it.

Unrelated to autopkgtest, I noticed in October that during debbuild the
package does not run the main pytest suite for pgbouncer? Is that
intentional? It appears that before the tests were rewritten in python that
the package ran the shell-based suite properly.

I submitted a patch that should be able to be incorporated as-is:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1085109

On Sat, Jan 11, 2025 at 2:46 AM Bradford Boyle <[email-masked]>
wrote:

> [1]:
> https://jengus.postgresql.org/job/pgbouncer-binaries/101/architecture=amd64,distribution=sid/

-- 
Brian Cosgrove

---

From: Bradford Boyle <[email-masked]>
Date: Sun, 12 Jan 2025 20:13:51 -0800

Hi Brian,

On Sun, Jan 12, 2025 at 4:47 AM Brian Cosgrove <[email-masked]> wrote:
>
> Unrelated to autopkgtest, I noticed in October that during debbuild
> the package does not run the main pytest suite for pgbouncer? Is that
> intentional? It appears that before the tests were rewritten in python
> that the package ran the shell-based suite properly.

Looking at the history for d/rules, I think disabling the main test
suite during the build was intended to be a temporary fix for a missing
file in the upstream source archive [0],[1]. It looks like the tests
weren't re-enabled when the issue was fixed upstream.

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1085109

I've applied your patch to the package and pushed the changes to Salsa.
I did have to drop the b-d on python3-contextlib2 since this package is
not available (and not needed) in unstable. I also had to add
tzdata-legacy in order for the tests to pass. We'll probably need to add
some tweaks for the pgdg packages to build on older distributions.

[0]: https://salsa.debian.org/postgresql/pgbouncer/-/commit/74f05a0#8756c63497c8dc39f7773438edf53b220c773f67_22_25
[1]: https://github.com/pgbouncer/pgbouncer/issues/431

---

From: Bradford Boyle <[email-masked]>
Date: Mon, 20 Jan 2025 20:42:44 -0800

Hi Brian,

I was looking at the package for pgBouncer 1.24.0 again today and it
looks like the tests are failing [0]. After reading through the logs, it
looks like psycopg in testing and unstable was updated to 3.2.4 this
past Friday. I am able to reproduce the test failures in dev environment
by updating the version in requirements.txt. Should we disable the main
test suite for now to unblock the upload of 1.24.0?

Regards,

--Bradford

[0]: https://salsa.debian.org/postgresql/pgbouncer/-/jobs/6962351

---

From: Brian Cosgrove <[email-masked]>
Date: Mon, 20 Jan 2025 23:25:37 -0600

Yes, please do. A Postgres client package (psycopg) update causing failing
integration tests seems highly unlikely to be a genuine issue for the
pgbouncer.

I also reproduced locally the failures you're seeing on Salsa. I notice
that there are relevant changes in psycopg 3.2.4 and I think that there are
some assertions needing updates in the upstream project.

I'll file an issue with pgbouncer but for now please unblock the upload.

-- 
Brian Cosgrove

---

From: Brian Cosgrove <[email-masked]>
Date: Mon, 20 Jan 2025 23:38:25 -0600

I think this note in the psycopg 3.2.4 news is what we're seeing in the
pgbouncer tests:
https://github.com/psycopg/psycopg/compare/3.2.3...3.2.4#diff-87f7a7bb7fe8dbed76e16b67b87f27bda7b779b56e7d93ed3350d77ee4a3f217R20-R23

On Mon, Jan 20, 2025 at 11:25 PM Brian Cosgrove <[email-masked]> wrote:

> On Mon, Jan 20, 2025 at 10:42 PM Bradford Boyle <
> [email-masked]> wrote:
>
>> Hi Brian,
>>

-- 
Brian Cosgrove

---

From: Brian Cosgrove <[email-masked]>
Date: Mon, 20 Jan 2025 23:55:17 -0600

Ah, funny, we (pgbouncer) just started pinning this in requirements.txt
because of this failure:
https://github.com/pgbouncer/pgbouncer/commit/0cf59fadc0671a7363afa51086631b87a26893dc

On Mon, Jan 20, 2025 at 11:38 PM Brian Cosgrove <[email-masked]> wrote:

>> Yes, please do. A Postgres client package (psycopg) update causing
>> failing integration tests seems highly unlikely to be a genuine issue for
>> the pgbouncer.
>>

-- 
Brian Cosgrove

---

From: Brian Cosgrove <[email-masked]>
Date: Tue, 21 Jan 2025 00:11:16 -0600

Thanks for the heads up, Bradford.
https://github.com/pgbouncer/pgbouncer/pull/1250

-- 
Brian Cosgrove

---

From: Bradford Boyle <[email-masked]>
Date: Tue, 21 Jan 2025 22:47:43 -0800

Hi Christoph,

I've updated the package for pgbouncer to include a patch based on [0]
to get the main test suite passing on salsa.debian.org. With this, the
package should be ready for review and upload.

I've also attached an initial patch to generate-pgdg-source that should
add the required dependency on python3-contextlib2 and remove the
dependency on tzdata-legacy to get the pacakge building on older
distributions.

Regards,

-- Bradford

[0]: https://github.com/pgbouncer/pgbouncer/pull/1250

---

From: Christoph Berg <[email-masked]>
Date: Tue, 4 Feb 2025 16:05:58 +0100

Re: Bradford Boyle
> package should be ready for review and upload.

Hi Bradford,

thanks!

> dependency on tzdata-legacy to get the pacakge building on older

Unfortunately that needs more work as it looks like:

bookworm and others:

14:54:22 E   AttributeError: 'PGconn' object has no attribute 'close_prepared'

https://jengus.postgresql.org/job/pgbouncer-binaries/106/architecture=amd64,distribution=bookworm/console

bullseye doesn't have python3-psycopg (and building that doesn't look
trivial). Do we think we could go without or with less tests there?

Christoph

---

From: Bradford Boyle <[email-masked]>
Date: Tue, 4 Feb 2025 23:40:16 -0800

> https://jengus.postgresql.org/job/pgbouncer-binaries/106/architecture=amd64,distribution=bookworm/console

Yeah -- it looks like close_prepared was added in psycopg 3.2.0 and
bookworm has 3.1.7. I could patch the tests to include a skipif based on
the version of psycopg.

> trivial). Do we think we could go without or with less tests there?

I think we could go without tests on bullseye since it seems like we are
nearing the trixie release and bullseye will become old old stable.
(Correct me if I am wrong on this point, I'm making this assumption
based on the trixie freeze dates). There's probably little value in
creating extra work here.

It looks like d/rules for pgbouncer will skip the tests if nocheck is in
DEB_BUILD_OPTIONS -- is this something that can be configured via the Jenkins
YAML? I am not very familiar with the Jenkins job builder YAML and I didn't see
any usages of that in the current set of jobs. Or would I need to update
generate-pgdg-source to include an additional tweak to disable the tests on
older platforms?

Thanks,
-- Bradford

---

From: Christoph Berg <[email-masked]>
Date: Wed, 5 Feb 2025 10:09:51 +0100

Re: Bradford Boyle
> creating extra work here.

Or we just don't update the package on bullseye, leaving it at 1.23.1.
But there is still the autopkgtest, so we still have test coverage
there.

> generate-pgdg-source to include an additional tweak to disable the tests on

Any of these would work.

DEB_BUILD_OPTIONS=nocheck would have to be set in the sbuild-package
script. That's probably the easiest way forward.

The extra test dependencies need annotating with <!nocheck> so the
build doesn't try to install them.

Christoph

---

From: Brian Cosgrove <[email-masked]>
Date: Wed, 5 Feb 2025 06:40:41 -0600

Skipping this release for bullseye is not entirely unreasonable with Trixie
pending.

Skipping the entire suite feels unreasonable if the Postgres Debian team
releases 1.24.

I am working up a patch that will skip only these tests that will likely be
accepted upstream as well, should Debian choose to release 1.24 for
bullseye.

https://github.com/pgbouncer/pgbouncer/pull/1266

-- 
Brian Cosgrove

---

From: Brian Cosgrove <[email-masked]>
Date: Wed, 5 Feb 2025 11:01:39 -0600

Attached in DEP-3 format, I think, if it helps at all.

On Wed, Feb 5, 2025 at 6:40 AM Brian Cosgrove <[email-masked]> wrote:

> Skipping this release for bullseye is not entirely unreasonable with
> Trixie pending.
>
> I am working up a patch that will skip only these tests that will likely
> be accepted upstream as well, should Debian choose to release 1.24 for
> bullseye.
>

-- 
Brian Cosgrove

---

From: Bradford Boyle <[email-masked]>
Date: Sun, 16 Feb 2025 17:53:03 -0800

> Attached in DEP-3 format, I think, if it helps at all.

Thanks. I've applied that and pushed a new commit to s.d.o with this
patch and bumping to the package version. Attached is a patch for pgapt
to skip pgbouncer for bullseye -- this seemed like the least amount of
work and there were no objections to leaving bullseye on 1.23.1.

I think with an upload of 1.24.0-2 and skipping bullseye, this should
could get pgbouncer updated in PGDG.

Regards,
-- Bradford