Message-ID: <378df4dd8eac41fcfe78a552e5e08ebb7c041455.camel@j-davis.com>
Subject: Re: Collation version tracking for macOS
From: Jeff Davis <pgsql@j-davis.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, 
 Joe Conway <mail@joeconway.com>, Peter Eisentraut
 <peter.eisentraut@enterprisedb.com>, Jeremy Schneider
 <schneider@ardentperf.com>, Peter Geoghegan <pg@bowt.ie>, "Nasby, Jim"
 <nasbyj@amazon.com>, Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers
 <pgsql-hackers@postgresql.org>
Date: Mon, 12 Feb 2024 12:25:09 -0800
In-Reply-To: <CA+TgmoZ0AtG0y7NHh1Aax+X=n0U5hdzV5gdfpth_3wbGY8mE5g@mail.gmail.com>
References: <b37c2acd-24f4-07c9-da6e-6fec66137f69@enterprisedb.com>
	 <D9EBBA4F-C92C-4B26-AB4D-6038BC52793D@ardentperf.com>
	 <398aabd1-ad95-ba2d-d70a-dd5d90bf6e07@enterprisedb.com>
	 <CA+hUKGL36vXMfcaDq+U1ZkoSsdfFnNx7GxhGM7aYzEbKs1W0=Q@mail.gmail.com>
	 <CA+hUKGKq=iLH3bY+nK7v8b2zBCuKOk-fe0cP0it2RxNaWFVxYA@mail.gmail.com>
	 <606bd2baa6d65b38fee6eb23bba40c5da210255b.camel@j-davis.com>
	 <CA+hUKG+OSQtrRAk-bHwMJmuvqp-b-LGuxsF2PoDBVyQcT+VEAQ@mail.gmail.com>
	 <9f8e9b5a3352478d4cf7d6c0a5dd7e82496be4b6.camel@j-davis.com>
	 <CA+hUKGJxg6AbKC9RJ7r1ByVLtvVkThQV+RZO6BKVWYESPCp3Ug@mail.gmail.com>
	 <346f836208a39009c1998ed5a41c7f1a0be36911.camel@j-davis.com>
	 <CA+hUKGJhPeW3iRqO3XQB=+BTMx7O6zTCFKTfeSV9TXBtdyMqbQ@mail.gmail.com>
	 <CA+hUKGLr=9d+-k8PVv8e__TOxuq=n0SKNDpqCzbGrK5EbDyxAg@mail.gmail.com>
	 <d8656ac98e1957401a21428949e98c840badf674.camel@j-davis.com>
	 <CA+hUKGKWX6q_UXsDhVbhYns2+VsrPLW5o6ohqO6C=cZ6r23G7w@mail.gmail.com>
	 <dae66ea33c91c597692522d2eabfee46ae799b54.camel@j-davis.com>
	 <776e9ddb-a0e6-3f29-9368-a194be30d1a6@joeconway.com>
	 <CA+hUKGLEqMhnpZrgAcisoUeYFGz8W6EWdhtK2h-4QN0iOSFRqw@mail.gmail.com>
	 <c4fda90ec6a7568a896f243a38eb273c3b5c3d93.camel@j-davis.com>
	 <f5e4eb630a37f55770131b29887c28ee5c4ee6a4.camel@j-davis.com>
	 <6f4a8c01a5cb1edf3a07d204c371fbddaef252f9.camel@j-davis.com>
	 <CALDaNm0dOJ1dBwVUH7i8Ak8Eaot+iK1iJeGy4vOi1tB=WK1E=A@mail.gmail.com>
	 <ab26a4ec4f307010877bbef2aa82dea281b2e71c.camel@j-davis.com>
	 <CA+TgmoYwqh5=6UsafL2ELBp7asOtZ=vm=bR76M7XvBfLy0aA1Q@mail.gmail.com>
	 <69ce2595450464ff1bcd8da1d5b0613b1d504b22.camel@j-davis.com>
	 <CA+TgmoZ0AtG0y7NHh1Aax+X=n0U5hdzV5gdfpth_3wbGY8mE5g@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/378df4dd8eac41fcfe78a552e5e08ebb7c041455.camel%40j-davis.com>
Precedence: bulk

On Sun, 2024-02-11 at 22:04 +0530, Robert Haas wrote:
> 1. Here's what we think your OS package manager is probably going to
> do.
> 2. That's going to interact with PostgreSQL in this way that I will
> now describe.
> 3. See, that sucks, because of the stuff I said above about needing
> stable collations!
> 4. But if you installed this module instead, then you could prevent
> the things I said under #2 from happening.
> 5. Instead, you'd get this other behavior, which would make you
> happy.

I like that framing, thank you. I'll try to come up with something
there.

> I feel like I can almost piece together in my head how this is
> supposed to work -- I think it's like "we expect the OS package
> manager to drop all the ICU versions in the same directory via side
> by
> side installs, and that works well for other programs because ... for
> some mysterious reason they can latch onto the specific version they
> were linked against ... but we can't or don't do that because ... I
> guess we're dumber than those other pieces of software or
> something???? ...=C2=A0

Postgres can and does latch on to the version of ICU it was compiled
against. It's a normal shared library dependency.

The problem is that databases -- and the file structures -- outlive a
particular version of Postgres. So if Postgres 16 is compiled against
ICU X and Postgres 17 is compiled against ICU Y, how do you upgrade
from 16 to 17? Postgres 17 will try to access the old file structures
using ICU Y, and they'll be corrupt.

What we want is the file structures that depend on ICU X to continue to
find ICU X even after you upgrade to Postgres 17, yet allow new
structures to be created using ICU Y. In other words, "multi-lib",
meaning that the same Postgres binary is linking to multiple versions
of ICU and the different versions for different structures. That would
allow users to recreate one index at a time to use ICU Y, until nothing
depends on ICU X any longer.

I should say this is not an easy process even if something like
icu_multilib is available. We don't have all of the information needed
in the catalog to track which structures depend on which versions of a
collation library, collation library version is itself not easy to
define, and it still involves rebuilding (or at least re-validating) a
lot of structures. This is a "make hard things possible" tool, and I
suspect only a handful of users would use it successfully to migrate to
new ICU versions.

More simply, some users might just want to lock down the version of ICU
to X, and just use that forever until they have a reason to change it.
icu_multilib can also facilitate that, though it's still not trivial.

> "icu_multilib must be loaded via shared_preload_libraries.
> icu_multilib ignores any ICU library with a major version greater
> than
> that with which PostgreSQL was built."
>=20
> It's not clear from reading this whether the second sentence here is
> a
> regrettable implementation restriction or design behavior. If it's
> design behavior, what's the point of it?

That restriction came from Thomas's (uncommitted) work on the same
problem. I believe the reasoning was that we don't know whether future
versions of ICU might break something that we're doing, though perhaps
there's a better way.

Regards,
	Jeff Davis