Subject: libpq 11.10 / 11.11 are not propertly released for bionic

From: Saif Hakim <[email-masked]>
Date: Fri, 12 Feb 2021 15:54:07 -0800

Hey there,

I noticed that version 11.11 of libpq5 and libpq-dev were released
yesterday, but the release seems incomplete.

The new releases are found at
https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/ , e.g.:
libpq-dev_11.11-1.pgdg18.04+1_amd64.deb            11-Feb-2021 10:39
       165700
libpq-dev_11.11-1.pgdg18.04+1_arm64.deb            11-Feb-2021 10:40
       154792
libpq-dev_11.11-1.pgdg18.04+1_i386.deb             11-Feb-2021 10:39
       175292
libpq-dev_11.11-1.pgdg18.04+1_ppc64el.deb          11-Feb-2021 10:40
       179288

However, in
https://apt.postgresql.org/pub/repos/apt/dists/bionic-pgdg/11/binary-amd64/
, Packages was last updated on 12-Nov-2020 12:10, listing filenames for the
two packages for the *11.10* release:
pool/11/p/postgresql-11/libpq-dev_11.10-1.pgdg18.04+1_amd64.deb
pool/11/p/postgresql-11/libpq5_11.10-1.pgdg18.04+1_amd64.deb

Furthermore, the referenced files are *not* found in the pool. The only
11.10 files are as follows:
libpq-dev_11.10-1.pgdg110+1+b1_amd64.deb           20-Nov-2020 09:24
       164420
libpq-dev_11.10-1.pgdg110+1+b1_arm64.deb           20-Nov-2020 09:24
       161744
libpq-dev_11.10-1.pgdg110+1+b1_ppc64el.deb         20-Nov-2020 09:24
       179880
libpq-dev_11.10-1.pgdg110+1+b2_amd64.deb           04-Jan-2021 12:10
       164424
libpq-dev_11.10-1.pgdg110+1+b2_arm64.deb           04-Jan-2021 12:10
       161788

This prevents me from installing libpq 11 on bionic, as reproduced via
`docker run -it ubuntu:18.04 bash` below:

root@62d4e1ddd8bf:/# apt-get update -qq && apt-get install -qq curl gnupg2
...
root@62d4e1ddd8bf:/# curl -sf
https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
OK
root@62d4e1ddd8bf:/# echo 'deb http://apt.postgresql.org/pub/repos/apt/
bionic-pgdg 11' >> /etc/apt/sources.list.d/postgresql.list
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
  Installed: (none)
  Candidate: 10.15-0ubuntu0.18.04.1
  Version table:
     10.15-0ubuntu0.18.04.1 500
        500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64
Packages
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64
Packages
     10.3-1 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get update -qq
root@62d4e1ddd8bf:/# apt-cache policy libpq5
libpq5:
  Installed: (none)
  Candidate: 11.10-1.pgdg18.04+1
  Version table:
     11.10-1.pgdg18.04+1 500
        500 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64
Packages
     10.15-0ubuntu0.18.04.1 500
        500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64
Packages
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64
Packages
     10.3-1 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@62d4e1ddd8bf:/# apt-get install libpq5=11\*
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '11.10-1.pgdg18.04+1' (PostgreSQL for Debian/Ubuntu
repository:bionic-pgdg [amd64]) for 'libpq5'
The following NEW packages will be installed:
  libpq5
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 169 kB of archives.
After this operation, 767 kB of additional disk space will be used.
Err:1 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/11 amd64 libpq5
amd64 11.10-1.pgdg18.04+1
  404  Not Found [IP: 87.238.57.227 80]
E: Failed to fetch
http://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq5_11.10-1.pgdg18.04+1_amd64.deb
 404  Not Found [IP: 87.238.57.227 80]
E: Unable to fetch some archives, maybe run apt-get update or try with
--fix-missing?

I *think* I can manually install these packages via the following, but I'm
not sure if there are any other release-related issues:

curl https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq-dev_11.11-1.pgdg18.04%2B1_amd64.deb
-o /tmp/libpq-dev.debcurl
https://apt.postgresql.org/pub/repos/apt/pool/11/p/postgresql-11/libpq5_11.11-1.pgdg18.04%2B1_amd64.deb
-o /tmp/libpq.debdpkg -i /tmp/libpq-dev.deb
dpkg -i /tmp/libpq.deb

Thanks,
Saif

---

From: Christoph Berg <[email-masked]>
Date: Sun, 14 Feb 2021 21:53:43 +0100

Re: Saif Hakim
> yesterday, but the release seems incomplete.

Hi Saif,

> two packages for the *11.10* release:

that is indeed stale, thanks for spotting that. The problem is present
in the other distributions as well. I have now manually updated
bionic-pgdg, but I will leave the other untouched until I can figure
out why reprepro isn't maintaining those files properly by itself.

> bionic-pgdg 11' >> /etc/apt/sources.list.d/postgresql.list

That said, you shouldn't worry about libpq5 11 - using the 13 version
from the "main" repository component is the recommended way. I guess
the fact that no one has spotted that problem before means no one is
actually using the "11" and the other versioned components.

So the recommendation would be to use "bionic-pgdg main" in
sources.list.

Christoph

---

From: Saif Hakim <[email-masked]>
Date: Sun, 14 Feb 2021 14:58:46 -0800

Hi Christoph,

> bionic-pgdg, but I will leave the other untouched until I can figure

Thanks!, I've confirmed that my repro no longer works and version 11 is
installed.

> from the "main" repository component is the recommended way.

For my own understanding, what are the ramifications of using the
non-"main" repository component / what are the differences?

I was also under the impression that since the version number tracks the
postgres release it is compatible with, that I should be choosing the
version that matches the version of postgres I'm running, i.e. since I'm
running Postgres 11, I should be using libpq5-11.xx. I'm primarily
installing libpq for python; however, taking a closer look at psycopg2 docs
(https://www.psycopg.org/docs/install.html#runtime-requirements), I see
that they are in agreement with what you're saying:

> Note: Whatever version of libpq psycopg2 is compiled with, it will
> be possible to connect to PostgreSQL servers of any supported
> version: just install the most recent libpq version or the most
> practical, without trying to match it to the version of the
> PostgreSQL server you will have to connect to.

Out of curiosity, if we were running libpq5-11 with Postgres 13, would we
see any issues or does postgres maintain backwards compatibility with these
libs as well?

Thanks again!

Best,
Saif

---

From: Aaron Pavely <[email-masked]>
Date: Sun, 14 Feb 2021 20:53:44 -0600

> actually using the "11" and the other versioned components.

Actually, I received a few complaints the other day from developers for the
bionic-pgdg/9.6 component when 9.6.21 was pushed, so Saif's issue was not
an isolated incident. At the time, I saw that the 'Packages' file was
similarly last updated in November. Everything looks good now, though.

While I recommend to my teams use the 'main' component for libpq, there
continues to be fear among many that libpq versions mismatched from the
parent database cluster version may cause unexpected issues — even after I
point out that the database cluster itself is using the most recent release
of libpq from the 'main' component.

Aaron

---

From: Christoph Berg <[email-masked]>
Date: Mon, 15 Feb 2021 12:10:46 +0100

Re: Saif Hakim
> Thanks!, I've confirmed that my repro no longer works and version 11 is

In the meantime I found the issue: to speed up getting the different
PG major versions out for the release, I had told reprepro not to
"export" the Packages files except on the last version processed.
Normally this is fine, since the last run would export all the files
touched, but as the extra components are only touched by a single PG
major version each, the last run had only exported the extra files for
that PG version, leaving the other versions in the old state.

I have now exported all the dists manually again and things will be
back on track in a few minutes with the next mirror run.

> non-"main" repository component / what are the differences?

The non-main components contain the lib packages only, so you can't
actually install postgresql-11 from there since that is contained in
"main".

If you want libpq5 only, you can of course go with "11" only (or
"main 11"), but I have yet to hear from a case where libpq5 version 13
isn't just working as well.

> see any issues or does postgres maintain backwards compatibility with these

On the client side, you should be able to see much of a difference,
the protocol is the same. Possibly there are some functions in libpq5
that are only available with 13, but dpkg's dependency tracking will
tell you if a newer version is required.

Re: Aaron Pavely
> similarly last updated in November. Everything looks good now, though.

Nod.

> of libpq from the 'main' component.

Right. (But note that the PG server itself doesn't actually use libpq,
it's only required for things like postgres_fdw and dblink.)

Christoph