Subject: Re: Collation version tracking for macOS

From: Jeff Davis <[email-masked]>
Date: Fri, 20 Jan 2023 12:54:28 -0800

On Thu, 2023-01-19 at 00:11 -0800, Jeff Davis wrote:
> Attached are a new set of patches, including a major enhancement: the
> icu_multilib contrib module.

Attached rebased v8.

[ It looks like my email client truncated the last email somehow, in
case someone was wondering why it just stopped. ]

The big change is the introduction of the icu_multilib contrib module
which provides a lot of the functionality requested in this thread:

   * icu version stability, which allows you to "lock down" ICU to a
specific major and minor version (or major version only)
   * multi-lib ICU, which (if a GUC is set) will enable the "search by
collversion" behavior. Some doubts were raised about the wisdom of this
approach, but it's the only multi-lib solution we have without doing
some significant catalog work.

I rendered the HTML docs for icu_multilib and attached to this email to
make it easier to view.

icu_multilib assumes that the various ICU library versions are already
available in a single location, most likely installed with a package
manager. That location can be the same as the built-in ICU, or a
different location. Ideally, packagers would start to offer a few
"stable" versions of ICU that would be available for a long time, but
it will take a while for that to happen. So for now, it's up to the
user to figure out how to get the right versions of ICU on their system
and keep them there.

Automated tests of icu_multilib are a problem unless the one running
the tests is willing to compile the right versions of ICU (like I did).
But I at least have automated tests for the hooks by using the test
module test_collator_lib_hooks.

The v7 patches in this thread are dependent on the pure refactoring
patches in this CF entry:

   https://commitfest.postgresql.org/41/3935/
  
https://postgr.es/m/[email-masked]

The requested functionality _not_ offered by icu_multilib is tying a
specific collation to a specific ICU version. A few variants were
proposed, the latest is to tie a collation to the library file itself
through the provider. That needs to be done with proper catalog support
in core. But I believe the work I've done here has made a lot of
progress in that direction, and also shows the versatility of the new
hook to solve at least some problems.

-- 
Jeff Davis
PostgreSQL Contributor Team - AWS

From 1f305419c350a6f0db8245cdc4630c4942a2e1a9 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email-masked]>
Date: Sat, 14 Jan 2023 12:17:05 -0800
Subject: [PATCH v8 4/4] icu_multilib: contrib extension for managing ICU
 libraries.

The icu_multilib module provides control over the version (or
versions) of the ICU provider library used by PostgreSQL, which can be
different from the version of ICU with which it was built.

Control over the specific ICU library is important, because any
differences in collation may result in corrupt indexes, and possibly
other problems.

The primary use case is the ability to stabilize the ICU version
("lock" it to a specific version), even if the built-in ICU library
changes. It can be "locked" to a specific major and minor version (the
safest option); or to a major version only, enabling non-disruptive
minor version updates of ICU (albeit with some risk of subtle
differences in collation behavior).

If enabled, icu_multilib also provides support for multiple ICU
libraries by using "search by collator version" logic. This logic
assumes that identical collator versions (as provided by ICU) mean
identical behavior, which also carries some risk.

Author: Jeff Davis, Thomas Munro
---
 contrib/Makefile                           |   6 +
 contrib/icu_multilib/Makefile              |  21 +
 contrib/icu_multilib/icu_multilib--1.0.sql |  43 +
 contrib/icu_multilib/icu_multilib.c        | 880 +++++++++++++++++++++
 contrib/icu_multilib/icu_multilib.control  |   7 +
 contrib/icu_multilib/meson.build           |  24 +
 contrib/meson.build                        |   3 +
 doc/src/sgml/contrib.sgml                  |   1 +
 doc/src/sgml/filelist.sgml                 |   1 +
 doc/src/sgml/icu-multilib.sgml             | 496 ++++++++++++
 10 files changed, 1482 insertions(+)
 create mode 100644 contrib/icu_multilib/Makefile
 create mode 100644 contrib/icu_multilib/icu_multilib--1.0.sql
 create mode 100644 contrib/icu_multilib/icu_multilib.c
 create mode 100644 contrib/icu_multilib/icu_multilib.control
 create mode 100644 contrib/icu_multilib/meson.build
 create mode 100644 doc/src/sgml/icu-multilib.sgml

diff --git a/contrib/Makefile b/contrib/Makefile
index bbf220407b..06447d9346 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -65,6 +65,12 @@ else
 ALWAYS_SUBDIRS += uuid-ossp
 endif
 
+ifeq ($(with_icu),yes)
+SUBDIRS += icu_multilib
+else
+ALWAYS_SUBDIRS += icu_multilib
+endif
+
 ifeq ($(with_libxml),yes)
 SUBDIRS += xml2
 else
diff --git a/contrib/icu_multilib/Makefile b/contrib/icu_multilib/Makefile
new file mode 100644
index 0000000000..4823007abe
--- /dev/null
+++ b/contrib/icu_multilib/Makefile
@@ -0,0 +1,21 @@
+# contrib/icu_multilib/Makefile
+
+MODULE_big = icu_multilib
+OBJS = \
+	$(WIN32RES) \
+	icu_multilib.o
+
+EXTENSION = icu_multilib
+DATA = icu_multilib--1.0.sql
+PGFILEDESC = "icu_multilib -- support multiple ICU library versions"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/icu_multilib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/icu_multilib/icu_multilib--1.0.sql b/contrib/icu_multilib/icu_multilib--1.0.sql
new file mode 100644
index 0000000000..6751242bb4
--- /dev/null
+++ b/contrib/icu_multilib/icu_multilib--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/icu_multilib/icu_multilib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION icu_multilib" to load this file. \quit
+
+CREATE FUNCTION library_versions(
+    icu_version OUT TEXT,
+    unicode_version OUT TEXT,
+    cldr_version OUT TEXT,
+    libicui18n_name OUT TEXT,
+    libicuuc_name OUT TEXT
+  )
+  RETURNS SETOF RECORD LANGUAGE C
+  AS 'MODULE_PATHNAME', 'library_versions';
+
+CREATE FUNCTION library_collators(
+    major_version IN TEXT DEFAULT NULL,
+    locale OUT TEXT,
+    icu_version OUT TEXT,
+    uca_version OUT TEXT,
+    collator_version OUT TEXT
+  )
+  RETURNS SETOF RECORD LANGUAGE C
+  AS 'MODULE_PATHNAME', 'library_collators';
+
+CREATE FUNCTION collator_version_search (
+    locale IN TEXT,
+    requested_version IN TEXT DEFAULT NULL,
+    log_ok IN BOOLEAN DEFAULT FALSE,
+    icu_version OUT TEXT,
+    uca_version OUT TEXT,
+    collator_version OUT TEXT)
+  RETURNS RECORD LANGUAGE C
+  AS 'MODULE_PATHNAME', 'collator_version_search';
+
+CREATE FUNCTION collator_versions (
+    locale IN TEXT,
+    icu_version OUT TEXT,
+    uca_version OUT TEXT,
+    collator_version OUT TEXT
+  )
+  RETURNS SETOF RECORD LANGUAGE C
+  AS 'MODULE_PATHNAME', 'collator_versions';
diff --git a/contrib/icu_multilib/icu_multilib.c b/contrib/icu_multilib/icu_multilib.c
new file mode 100644
index 0000000000..2fde2a85d5
--- /dev/null
+++ b/contrib/icu_multilib/icu_multilib.c
@@ -0,0 +1,880 @@
+
+#include "postgres.h"
+
+#include <dlfcn.h>
+#include <limits.h>
+
+#include "common/string.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/pg_locale_internal.h"
+
+#ifndef USE_ICU
+#error "ICU support is required to build icu_multilib"
+#endif
+
+/*
+ * We don't want to call into dlopen'd ICU libraries that are newer than the
+ * one we were compiled and linked against, just in case there is an
+ * incompatible API change.
+ */
+#define PG_MAX_ICU_MAJOR U_ICU_VERSION_MAJOR_NUM
+
+/*
+ * The oldest ICU release we're likely to encounter, and that has all the
+ * funcitons required.
+ */
+#define PG_MIN_ICU_MAJOR 50
+
+/*
+ * Enough to hold entries for minimum to maximum supported ICU versions, and
+ * also the builtin ICU (if icu_multilib.include_builtin_icu is set).
+ */
+#define ICU_LIB_TABLESIZE (PG_MAX_ICU_MAJOR - PG_MIN_ICU_MAJOR + 2)
+
+typedef struct icu_version
+{
+	int major;
+	int minor;
+} icu_version;
+
+static const struct config_enum_entry log_level_options[] = {
+	{"debug5", DEBUG5, false},
+	{"debug4", DEBUG4, false},
+	{"debug3", DEBUG3, false},
+	{"debug2", DEBUG2, false},
+	{"debug1", DEBUG1, false},
+	{"debug", DEBUG2, true},
+	{"log", LOG, false},
+	{"info", INFO, true},
+	{"notice", NOTICE, false},
+	{"warning", WARNING, false},
+	{"error", ERROR, false},
+	{NULL, 0, false}
+};
+
+PG_MODULE_MAGIC;
+
+static get_icu_library_hook_type prev_icu_library_hook = NULL;
+
+static pg_icu_library *icu_library_table[ICU_LIB_TABLESIZE] = {};
+
+static char *icu_library_path = "";
+static char *default_icu_version = "";
+static icu_version default_version = { .major = -1, .minor = -1 };
+static bool search_by_collator_version = false;
+static bool include_builtin = true;
+static int version_mismatch_log_level = WARNING;
+static int library_search_log_level = DEBUG1;
+
+static void initialize_gucs_1(void);
+static void initialize_gucs_2(void);
+static pg_icu_library *icu_multilib_hook(Oid collid, const char *locale,
+										 const char *version);
+static pg_icu_library *icu_multilib_search(Oid collid, const char *locale,
+										   const char *version,
+										   bool logOk);
+static void load_all_libraries(void);
+static pg_icu_library *load_icu_library(int major);
+static pg_icu_library *get_icu_by_major_version(int major);
+static void get_library_version(const pg_icu_library *lib, int *major,
+								int *minor);
+
+static bool check_string_version(char **newval, void **extra, GucSource source);
+static void assign_default_version(const char *newval, void *extra);
+
+void
+_PG_init()
+{
+	/*TODO: error messages */
+	if (!process_shared_preload_libraries_in_progress)
+		elog(ERROR, "icu_multilib must be loaded by shared_preload_libraries");
+
+	prev_icu_library_hook = get_icu_library_hook;
+	get_icu_library_hook = icu_multilib_hook;
+
+	initialize_gucs_1();
+	load_all_libraries();
+	initialize_gucs_2();
+}
+
+/* initialize GUCs before loading ICU libraries */
+static void
+initialize_gucs_1()
+{
+	/*
+	 * The library search path is initialized once per server start and not
+	 * changable -- this keeps the memory usage in TopMemoryContext bounded.
+	 */
+	DefineCustomStringVariable("icu_multilib.library_path",
+							   "Filesystem path where ICU libraries are installed.",
+							   NULL,
+							   &icu_library_path,
+							   "",
+							   PGC_POSTMASTER,
+							   0, NULL, NULL, NULL);
+	DefineCustomEnumVariable("icu_multilib.version_mismatch_log_level",
+							 "Level of log message when a collator version mismatch is detected.",
+							 NULL,
+							 &version_mismatch_log_level,
+							 WARNING,
+							 log_level_options,
+							 PGC_SUSET,
+							 0, NULL, NULL, NULL);
+	DefineCustomEnumVariable("icu_multilib.library_search_log_level",
+							 "Level of log messages related to searching for an ICU library.",
+							 NULL,
+							 &library_search_log_level,
+							 DEBUG1,
+							 log_level_options,
+							 PGC_SUSET,
+							 0, NULL, NULL, NULL);
+}
+
+/* initialize GUCs after loading ICU libraries */
+static void
+initialize_gucs_2()
+{
+	DefineCustomStringVariable("icu_multilib.default_icu_version",
+							   "The version of the default ICU library.",
+							   "Can be specified with major and minor versions, or major version only.",
+							   &default_icu_version,
+							   "",
+							   PGC_SUSET,
+							   0, check_string_version, assign_default_version, NULL);
+	DefineCustomBoolVariable("icu_multilib.include_builtin",
+							 "Include built-in ICU library when listing or searching libraries.",
+							 NULL,
+							 &include_builtin,
+							 true,
+							 PGC_SUSET,
+							 0, NULL, NULL, NULL);
+	DefineCustomBoolVariable("icu_multilib.search_by_collator_version",
+							 "Enable searching for the ICU library based on the collator version.",
+							 NULL,
+							 &search_by_collator_version,
+							 false,
+							 PGC_SUSET,
+							 0, NULL, NULL, NULL);
+}
+
+static bool
+parse_icu_version(const char *version, icu_version *out_version)
+{
+	size_t	input_len = strlen(version);
+	int		parse_len;
+	int		parse_major;
+	int		parse_minor;
+
+	if (sscanf(version, "%d.%d%n",
+			   &parse_major, &parse_minor, &parse_len) == 2 &&
+		input_len == parse_len)
+	{
+		out_version->major = parse_major;
+		out_version->minor = parse_minor;
+		return true;
+	}
+	else if (sscanf(version, "%d%n", &parse_major, &parse_len) == 1 &&
+			 input_len == parse_len)
+	{
+		out_version->major = parse_major;
+		out_version->minor = -1;
+		return true;
+	}
+
+	return false;
+}
+
+static bool
+check_string_version(char **newval, void **extra, GucSource source)
+{
+	pg_icu_library	*lib;
+	icu_version		*myextra;
+	icu_version		 parsed_version;
+
+	if (*newval == NULL || *newval[0] == '\0')
+		return true;
+
+	if (!parse_icu_version(*newval, &parsed_version))
+	{
+		GUC_check_errmsg("error parsing ICU version \"%s\"", *newval);
+		GUC_check_errhint("The ICU version should be either in the form \"major.minor\", "
+						  "or just be a plain major version number.");
+		return false;
+	}
+
+	if (parsed_version.major < PG_MIN_ICU_MAJOR ||
+		parsed_version.major > PG_MAX_ICU_MAJOR)
+	{
+		GUC_check_errmsg("major version %d is out of range",
+						 parsed_version.major);
+		GUC_check_errhint("ICU major version must be between %d and %d.",
+						  PG_MIN_ICU_MAJOR, PG_MAX_ICU_MAJOR);
+		return false;
+	}
+
+	lib = get_icu_by_major_version(parsed_version.major);
+	if (!lib)
+	{
+		GUC_check_errmsg("ICU library with major version %d not found",
+						 parsed_version.major);
+		return false;
+	}
+
+	if (parsed_version.minor != -1 && lib->minor_version != parsed_version.minor)
+	{
+		GUC_check_errmsg("ICU library with major version %d has minor version %d; "
+						 "expected minor version %d",
+						 lib->major_version, lib->minor_version,
+						 parsed_version.minor);
+		return false;
+	}
+
+	myextra = guc_malloc(ERROR, sizeof(icu_version));
+	*myextra = parsed_version;
+	*extra = myextra;
+
+	return true;
+}
+
+static void
+assign_default_version(const char *newval, void *extra)
+{
+	icu_version *myextra = (icu_version *) extra;
+
+	if (myextra == NULL)
+		return;
+
+	default_version = *myextra;
+}
+
+static void
+load_all_libraries()
+{
+	pg_icu_library *builtin = get_builtin_icu_library();
+
+	icu_library_table[ICU_LIB_TABLESIZE - 1] = builtin;
+
+	ereport(LOG, (errmsg("icu_multilib: retrieved built-in ICU version %d.%d",
+						 builtin->major_version, builtin->minor_version)));
+
+	for (int major = PG_MAX_ICU_MAJOR; major >= PG_MIN_ICU_MAJOR; major--)
+		icu_library_table[major - PG_MIN_ICU_MAJOR] = load_icu_library(major);
+}
+
+static pg_icu_library *
+get_icu_by_major_version(int major)
+{
+	pg_icu_library *lib;
+	if (major < PG_MIN_ICU_MAJOR || major > PG_MAX_ICU_MAJOR)
+		ereport(ERROR, (errmsg("major version %d is out of range", major)));
+
+	lib = icu_library_table[major - PG_MIN_ICU_MAJOR];
+	return lib;
+}
+
+static void
+get_library_version(const pg_icu_library *lib, int *major, int *minor)
+{
+	UVersionInfo version_info;
+	lib->getICUVersion(version_info);
+	*major = version_info[0];
+	*minor = version_info[1];
+	return;
+}
+
+/*
+ * Fill in out_version (which must have U_MAX_VERSION_STRING_LENGTH bytes
+ * available) with the collator version of the given locale in the given
+ * library. Return false if the collator is not found.
+ */
+static bool
+lib_collator_version(pg_icu_library *lib, const char *locale,
+					 char *out_version)
+{
+	UCollator		*collator;
+	UVersionInfo	 version_info;
+	UErrorCode		 status;
+
+	status = U_ZERO_ERROR;
+	collator = lib->openCollator(locale, &status);
+
+	if (!collator)
+		return false;
+
+	lib->getCollatorVersion(collator, version_info);
+	lib->versionToString(version_info, out_version);
+	lib->closeCollator(collator);
+
+	return true;
+}
+
+/*
+ * Find the right ICU library for the given locale and version. The resulting
+ * library may or may not provide a collator with an exactly-matching
+ * version.
+ *
+ * If search_by_collator_version is set, scan the table (first the built-in
+ * ICU library, then descending order of major versions) to find the first
+ * library that provides a collator of the given locale with a matching
+ * version.
+ *
+ * If no exactly matching version is found, and default_major_version is set,
+ * return the default library.
+ *
+ * Otherwise fall back to the built-in library.
+ */
+static pg_icu_library *
+icu_multilib_hook(Oid collid, const char *locale,
+				  const char *requested_version)
+{
+	return icu_multilib_search(collid, locale, requested_version, true);
+}
+
+/*
+ *
+ */
+static pg_icu_library *
+icu_multilib_search(Oid collid, const char *locale,
+					const char *requested_version, bool logOk)
+{
+	char			 actual_version[U_MAX_VERSION_STRING_LENGTH];
+	pg_icu_library	*found_lib = NULL;
+
+	/*
+	 * If another hook was set first, defer to that unless it returns NULL or
+	 * a library that doesn't contain the given collator at all. This may
+	 * result in a mismatching collator version, but we don't want to
+	 * speculate about what's better or worse in the presence of other hooks.
+	 */
+	if (prev_icu_library_hook)
+	{
+		pg_icu_library *tmp_lib;
+		tmp_lib = prev_icu_library_hook(collid, locale, requested_version);
+		if (tmp_lib && lib_collator_version(tmp_lib, locale, actual_version))
+			found_lib = tmp_lib;
+	}
+
+	if (!found_lib && search_by_collator_version && requested_version != NULL)
+	{
+		/*
+		 * Search from newest library to oldest for a matching version of the
+		 * collator with the given name.
+		 */
+		for (int i = ICU_LIB_TABLESIZE - 1; i >= 0; i--)
+		{
+			char			 tmp_version[U_MAX_VERSION_STRING_LENGTH];
+			pg_icu_library	*tmp_lib = icu_library_table[i];
+
+			if (tmp_lib == NULL)
+				continue;
+
+			if (!include_builtin && i == ICU_LIB_TABLESIZE - 1)
+				continue;
+
+			if (lib_collator_version(tmp_lib, locale, tmp_version) &&
+				strcmp(requested_version, tmp_version) == 0)
+			{
+				strcpy(actual_version, tmp_version);
+				found_lib = tmp_lib;
+				break;
+			}
+		}
+	}
+
+	if (!found_lib && default_version.major != -1)
+	{
+		pg_icu_library *tmp_lib;
+		tmp_lib = get_icu_by_major_version(default_version.major);
+		if (!tmp_lib)
+			ereport(ERROR,
+					(errmsg("icu_multilib: default major version %d not found",
+							default_version.major)));
+		if (lib_collator_version(tmp_lib, locale, actual_version))
+			found_lib = tmp_lib;
+		else if (logOk)
+			ereport(library_search_log_level,
+					(errmsg("icu_multilib: found default ICU %d.%d, but collator \"%s\" not found",
+							tmp_lib->major_version, tmp_lib->minor_version, locale)));
+	}
+
+	if (!found_lib && include_builtin)
+	{
+		pg_icu_library *tmp_lib;
+		tmp_lib = icu_library_table[ICU_LIB_TABLESIZE - 1];
+		if (lib_collator_version(tmp_lib, locale, actual_version))
+			found_lib = tmp_lib;
+		else if (logOk)
+			ereport(library_search_log_level,
+					(errmsg("icu_multilib: found built-in ICU %d.%d, but collator \"%s\" not found",
+							tmp_lib->major_version, tmp_lib->minor_version, locale)));
+	}
+
+	/* if not found, fall back to built-in or other hook */
+	if (!found_lib)
+		return NULL;
+
+	if (logOk)
+		ereport(library_search_log_level,
+				(errmsg("icu_multilib: found ICU version %d.%d providing collator version \"%s\" for locale \"%s\"",
+						found_lib->major_version, found_lib->minor_version,
+						actual_version, locale)));
+
+	/*
+	 * This is somewhat redundant with a similar warning in pg_locale.c, but
+	 * it provides details about the locale name and ICU version, which is
+	 * helpful when multiple ICU libraries are in use.
+	 */
+	if (requested_version && logOk &&
+		strcmp(requested_version, actual_version) != 0)
+	{
+		ereport(version_mismatch_log_level,
+				(errmsg("icu_multilib: collator version mismatch detected for locale \"%s\"",
+						locale),
+				 errdetail("ICU %d.%d provides collator version \"%s\" for locale \"%s\"; expected version \"%s\".",
+						   found_lib->major_version, found_lib->minor_version,
+						   actual_version, locale, requested_version)));
+	}
+
+	return found_lib;
+}
+
+PG_FUNCTION_INFO_V1(library_versions);
+Datum
+library_versions(PG_FUNCTION_ARGS)
+{
+#define LIBRARY_VERSIONS_COLS 5
+	ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+	Datum           values[LIBRARY_VERSIONS_COLS];
+	bool            nulls[LIBRARY_VERSIONS_COLS];
+
+	InitMaterializedSRF(fcinfo, 0);
+
+	for (int i = ICU_LIB_TABLESIZE - 1; i >= 0; i--)
+	{
+		UErrorCode      status;
+		UVersionInfo version_info;
+		char            version_string[U_MAX_VERSION_STRING_LENGTH];
+		pg_icu_library	*lib = icu_library_table[i];
+
+		if (lib == NULL)
+			continue;
+
+		if (!include_builtin && i == ICU_LIB_TABLESIZE - 1)
+			continue;
+
+		lib->getICUVersion(version_info);
+		lib->versionToString(version_info, version_string);
+		values[0] = PointerGetDatum(cstring_to_text(version_string));
+		nulls[0] = false;
+
+		lib->getUnicodeVersion(version_info);
+		lib->versionToString(version_info, version_string);
+		values[1] = PointerGetDatum(cstring_to_text(version_string));
+		nulls[1] = false;
+		status = U_ZERO_ERROR;
+		lib->getCLDRVersion(version_info, &status);
+		if (U_SUCCESS(status))
+		{
+			lib->versionToString(version_info, version_string);
+			values[2] = PointerGetDatum(cstring_to_text(version_string));
+			nulls[2] = false;
+		}
+		else
+		{
+			nulls[2] = true;
+		}
+
+		values[3] = PointerGetDatum(cstring_to_text(lib->libicui18n_name));
+		nulls[3] = false;
+
+		values[4] = PointerGetDatum(cstring_to_text(lib->libicuuc_name));
+		nulls[4] = false;
+
+		tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+	}
+
+	return (Datum) 0;
+}
+
+#define COLLATOR_DETAIL_COLS 3
+static void
+make_collator_detail_record(pg_icu_library *lib, const char *locale,
+							Datum *values, bool *nulls)
+{
+	UErrorCode		 status;
+	UCollator		*collator;
+	UVersionInfo	 version_info;
+	char			 version_string[U_MAX_VERSION_STRING_LENGTH];
+
+	status = U_ZERO_ERROR;
+	collator = lib->openCollator(locale, &status);
+
+	lib->getICUVersion(version_info);
+	lib->versionToString(version_info, version_string);
+	values[0] = PointerGetDatum(cstring_to_text(version_string));
+	nulls[0] = false;
+
+	if (collator)
+	{
+		lib->getUCAVersion(collator, version_info);
+		lib->versionToString(version_info, version_string);
+		values[1] = PointerGetDatum(cstring_to_text(version_string));
+		nulls[1] = false;
+	}
+	else
+		nulls[1] = true;
+
+	if (collator)
+	{
+		lib->getCollatorVersion(collator, version_info);
+		lib->versionToString(version_info, version_string);
+		values[2] = PointerGetDatum(cstring_to_text(version_string));
+		nulls[2] = false;
+	}
+	else
+		nulls[2] = true;
+
+	if (collator)
+		lib->closeCollator(collator);
+
+	return;
+}
+
+PG_FUNCTION_INFO_V1(library_collators);
+Datum
+library_collators(PG_FUNCTION_ARGS)
+{
+#define LIBRARY_COLLATORS_COLS (COLLATOR_DETAIL_COLS + 1)
+	pg_icu_library	*lib  = NULL;
+	ReturnSetInfo	*rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+	Datum			 values[LIBRARY_COLLATORS_COLS];
+	bool			 nulls[LIBRARY_COLLATORS_COLS];
+
+	if (PG_ARGISNULL(0))
+	{
+		lib = icu_library_table[ICU_LIB_TABLESIZE - 1];
+	}
+	else
+	{
+		char *major_str = text_to_cstring(PG_GETARG_TEXT_PP(0));
+		int major = atoi(major_str); /* TODO? */
+
+		if (major < PG_MIN_ICU_MAJOR || major > PG_MAX_ICU_MAJOR)
+			ereport(ERROR, (errmsg("icu_multilib: major version %d is out of range", major)));
+		lib = get_icu_by_major_version(major);
+		if (!lib)
+			ereport(ERROR, (errmsg("icu_multilib: major version %d not found", major)));
+	}
+
+	InitMaterializedSRF(fcinfo, 0);
+
+	for (int i = -1; i < lib->countAvailable(); i++)
+	{
+		const char	*name;
+		const char	*locale;
+		UErrorCode	 status;
+		char		 langtag[ULOC_FULLNAME_CAPACITY];
+
+		if (i == -1)
+			name = "";		/* ICU root locale */
+		else
+			name = lib->getAvailable(i);
+
+		status = U_ZERO_ERROR;
+		lib->toLanguageTag(name, langtag, sizeof(langtag), true, &status);
+
+		if (U_FAILURE(status))
+			continue;
+
+		locale = U_ICU_VERSION_MAJOR_NUM >= 54 ? langtag : name;
+
+		if (!pg_is_ascii(langtag) || !pg_is_ascii(locale))
+			continue;
+
+		values[0] = PointerGetDatum(cstring_to_text(locale));
+		nulls[0] = false;
+
+		make_collator_detail_record(lib, locale, values + 1, nulls + 1);
+		tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+	}
+
+	return (Datum) 0;
+}
+
+PG_FUNCTION_INFO_V1(collator_version_search);
+Datum
+collator_version_search(PG_FUNCTION_ARGS)
+{
+#define COLLATOR_VERSION_SEARCH_COLS COLLATOR_DETAIL_COLS
+	const char		*locale;
+	const char		*requested_version = NULL;
+	bool			 logOk;
+	pg_icu_library	*lib;
+	int				 major, minor;
+	TupleDesc		 tupdesc;
+	HeapTuple		 tuple;
+	Datum			 values[COLLATOR_VERSION_SEARCH_COLS];
+	bool			 nulls[COLLATOR_VERSION_SEARCH_COLS];
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	if (PG_ARGISNULL(0))
+		ereport(ERROR, (errmsg("locale argument must be non-NULL")));
+	if (PG_ARGISNULL(2))
+		ereport(ERROR, (errmsg("log_ok argument must be non-NULL")));
+
+	locale = text_to_cstring(PG_GETARG_TEXT_PP(0));
+	if (!PG_ARGISNULL(1))
+		requested_version = text_to_cstring(PG_GETARG_TEXT_PP(1));
+	logOk = PG_GETARG_BOOL(2);
+
+	lib = icu_multilib_search(InvalidOid, locale, requested_version, logOk);
+	get_library_version(lib, &major, &minor);
+
+	make_collator_detail_record(lib, locale, values, nulls);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+}
+
+PG_FUNCTION_INFO_V1(collator_versions);
+Datum
+collator_versions(PG_FUNCTION_ARGS)
+{
+#define COLLATOR_VERSIONS_COLS COLLATOR_DETAIL_COLS
+	const char *locale;
+	ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+	Datum           values[COLLATOR_VERSIONS_COLS];
+	bool            nulls[COLLATOR_VERSIONS_COLS];
+
+	if (PG_ARGISNULL(0))
+		ereport(ERROR, (errmsg("locale argument must be non-NULL")));
+	locale = text_to_cstring(PG_GETARG_TEXT_PP(0));
+
+	InitMaterializedSRF(fcinfo, 0);
+
+	for (int i = ICU_LIB_TABLESIZE - 1; i >= 0; i--)
+	{
+		pg_icu_library	*lib = icu_library_table[i];
+
+		if (lib == NULL)
+			continue;
+
+		if (!include_builtin && i == ICU_LIB_TABLESIZE - 1)
+			continue;
+
+		make_collator_detail_record(lib, locale, values, nulls);
+
+		tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+	}
+
+	return (Datum) 0;
+}
+
+static void
+make_icu_library_names(int major, char **libicui18n, char **libicuuc)
+{
+	char libicui18n_tmp[MAXPGPATH];
+	char libicuuc_tmp[MAXPGPATH];
+
+	/*
+	 * See
+	 * https://unicode-org.github.io/icu/userguide/icu4c/packaging.html#icu-versions
+	 * for conventions on library naming on POSIX and Windows systems.  Apple
+	 * isn't mentioned but varies in the usual way.
+	 *
+	 * Format 1 is expected to be a major version-only symlink pointing to a
+	 * specific minor version (or on Windows it may be the actual library).
+	 * Format 2 is expected to be an actual library.
+	 */
+#ifdef WIN32
+#define ICU_LIBRARY_NAME_FORMAT1 "%s%sicu%s%d" DLSUFFIX
+#elif defined(__darwin__)
+#define ICU_LIBRARY_NAME_FORMAT1 "%s%slibicu%s.%d" DLSUFFIX
+#else
+#define ICU_LIBRARY_NAME_FORMAT1 "%s%slibicu%s" DLSUFFIX ".%d"
+#endif
+
+#ifdef WIN32
+#define PATH_SEPARATOR "\\"
+#define ICU_I18N "in"
+#define ICU_UC "uc"
+#else
+#define PATH_SEPARATOR "/"
+#define ICU_I18N "i18n"
+#define ICU_UC "uc"
+#endif
+
+	snprintf(libicui18n_tmp,
+			 MAXPGPATH,
+			 ICU_LIBRARY_NAME_FORMAT1,
+			 icu_library_path, icu_library_path[0] ? PATH_SEPARATOR : "",
+			 "i18n", major);
+	snprintf(libicuuc_tmp,
+			 MAXPGPATH,
+			 ICU_LIBRARY_NAME_FORMAT1,
+			 icu_library_path, icu_library_path[0] ? PATH_SEPARATOR : "",
+			 "uc", major);
+
+	*libicui18n = pstrdup(libicui18n_tmp);
+	*libicuuc = pstrdup(libicuuc_tmp);
+}
+
+#define MAX_SYMBOL_LEN 128
+static void *
+load_icu_function(void *handle, const char *function, int major)
+{
+	char     function_with_version[MAX_SYMBOL_LEN];
+	int		 ret;
+	void    *result;
+
+	/*
+	 * Try to look up the symbol with the library major version as a suffix.
+	 */
+	ret = snprintf(function_with_version, sizeof(function_with_version),
+				   "%s_%d", function, major);
+	if (ret >= sizeof(function_with_version))
+		elog(ERROR, "icu_multilib: cannot construct symbol name");
+	result = dlsym(handle, function_with_version);
+
+	/*
+	 * Library may have been configured with --disable-renaming, try without
+	 * major version suffix.
+	 */
+	if (result == NULL)
+		result = dlsym(handle, function);
+
+	return result;
+}
+
+#define LOAD_FUNC(DEST, LIBHANDLE, LIBNAME, FNAME)	\
+	do {										\
+		DEST = load_icu_function((LIBHANDLE), (FNAME), major);	\
+		if (DEST == NULL)						\
+		{										\
+			ereport(WARNING,					\
+					(errmsg("icu_multilib: could not find symbol \"%s\" in library \"%s\"",	\
+							(FNAME), (LIBNAME))));	\
+			goto err;							\
+		}										\
+	} while(0)
+
+#define LOAD_FUNC_I18N(DEST, FNAME) \
+	LOAD_FUNC(DEST, libicui18n_handle, libicui18n, FNAME)
+#define LOAD_FUNC_UC(DEST, FNAME) \
+	LOAD_FUNC(DEST, libicuuc_handle, libicuuc, FNAME)
+
+static pg_icu_library *
+load_icu_library(int major)
+{
+	UVersionInfo	 version_info;
+	pg_icu_library	*lib;
+	char			*libicui18n;
+	char			*libicuuc;
+	void			*libicui18n_handle = NULL;
+	void			*libicuuc_handle   = NULL;
+
+	make_icu_library_names(major, &libicui18n, &libicuuc);
+
+	lib = MemoryContextAllocZero(TopMemoryContext, sizeof(pg_icu_library));
+
+	libicui18n_handle = dlopen(libicui18n, RTLD_NOW | RTLD_LOCAL);
+	if (!libicui18n_handle)
+		return NULL;
+
+	/* Load the common library. */
+	libicuuc_handle = dlopen(libicuuc, RTLD_NOW | RTLD_LOCAL);
+	if (!libicui18n_handle)
+	{
+		elog(WARNING, "found ICU library \"%s\" but not companion library \"%s\"",
+			 libicui18n, libicuuc);
+		dlclose(libicui18n_handle);
+		return NULL;
+	}
+
+	/*
+	 * We only allocate the pg_icu_library object after successfully
+	 * opening the libraries to minimize the work done in the ENOENT case,
+	 * when probing a range of versions.  That means we might need to
+	 * clean up on allocation failure.
+	 */
+	lib = MemoryContextAllocExtended(TopMemoryContext, sizeof(*lib),
+									 MCXT_ALLOC_NO_OOM);
+	lib->libicui18n_name = MemoryContextAllocExtended(
+		TopMemoryContext, strlen(libicui18n) + 1, MCXT_ALLOC_NO_OOM);
+	lib->libicuuc_name = MemoryContextAllocExtended(
+		TopMemoryContext, strlen(libicuuc) + 1, MCXT_ALLOC_NO_OOM);
+
+	if (!lib || !lib->libicui18n_name || !lib->libicuuc_name)
+	{
+		dlclose(libicui18n_handle);
+		dlclose(libicuuc_handle);
+		elog(ERROR, "out of memory");
+	}
+
+	strcpy(lib->libicui18n_name, libicui18n);
+	strcpy(lib->libicuuc_name, libicuuc);
+
+	pfree(libicui18n);
+	pfree(libicuuc);
+
+	/* try to find all the symbols we need from the i18n library */
+	LOAD_FUNC_I18N(lib->getICUVersion, "u_getVersion");
+	LOAD_FUNC_I18N(lib->getUnicodeVersion, "u_getUnicodeVersion");
+	LOAD_FUNC_I18N(lib->getCLDRVersion, "ulocdata_getCLDRVersion");
+	LOAD_FUNC_I18N(lib->openCollator, "ucol_open");
+	LOAD_FUNC_I18N(lib->closeCollator, "ucol_close");
+	LOAD_FUNC_I18N(lib->getCollatorVersion, "ucol_getVersion");
+	LOAD_FUNC_I18N(lib->getUCAVersion, "ucol_getUCAVersion");
+	LOAD_FUNC_I18N(lib->versionToString, "u_versionToString");
+	LOAD_FUNC_I18N(lib->strcoll, "ucol_strcoll");
+	LOAD_FUNC_I18N(lib->strcollUTF8, "ucol_strcollUTF8");
+	LOAD_FUNC_I18N(lib->getSortKey, "ucol_getSortKey");
+	LOAD_FUNC_I18N(lib->nextSortKeyPart, "ucol_nextSortKeyPart");
+	LOAD_FUNC_I18N(lib->setUTF8, "uiter_setUTF8");
+	LOAD_FUNC_I18N(lib->errorName, "u_errorName");
+	LOAD_FUNC_I18N(lib->setAttribute, "ucol_setAttribute");
+
+	/* try to find all the symbols we need from the uc library */
+	LOAD_FUNC_UC(lib->strToUpper, "u_strToUpper");
+	LOAD_FUNC_UC(lib->strToLower, "u_strToLower");
+	LOAD_FUNC_UC(lib->strToTitle, "u_strToTitle");
+	LOAD_FUNC_UC(lib->openConverter, "ucnv_open");
+	LOAD_FUNC_UC(lib->closeConverter, "ucnv_close");
+	LOAD_FUNC_UC(lib->fromUChars, "ucnv_fromUChars");
+	LOAD_FUNC_UC(lib->toUChars, "ucnv_toUChars");
+	LOAD_FUNC_UC(lib->toLanguageTag, "uloc_toLanguageTag");
+	LOAD_FUNC_UC(lib->getDisplayName, "uloc_getDisplayName");
+	LOAD_FUNC_UC(lib->countAvailable, "uloc_countAvailable");
+	LOAD_FUNC_UC(lib->getAvailable, "uloc_getAvailable");
+
+	lib->getICUVersion(version_info);
+	lib->major_version = version_info[0];
+	lib->minor_version = version_info[1];
+
+	if (lib->major_version != major)
+		ereport(version_mismatch_log_level,
+				(errmsg("loaded library for major version %d, but library reports major version %d",
+						major, lib->major_version)));
+
+	ereport(LOG, (errmsg("icu_multilib: loaded ICU version %d.%d",
+						 lib->major_version, lib->minor_version)));
+
+	return lib;
+
+err:
+	dlclose(libicui18n_handle);
+	dlclose(libicuuc_handle);
+	pfree(lib->libicui18n_name);
+	pfree(lib->libicuuc_name);
+	pfree(lib);
+	return NULL;
+}
diff --git a/contrib/icu_multilib/icu_multilib.control b/contrib/icu_multilib/icu_multilib.control
new file mode 100644
index 0000000000..8f2c546781
--- /dev/null
+++ b/contrib/icu_multilib/icu_multilib.control
@@ -0,0 +1,7 @@
+# icu_multlib extension
+comment = 'support multiple ICU library versions'
+default_version = '1.0'
+module_pathname = '$libdir/icu_multilib'
+relocatable = false
+schema = icu_multilib
+trusted = true
diff --git a/contrib/icu_multilib/meson.build b/contrib/icu_multilib/meson.build
new file mode 100644
index 0000000000..f2ea4088e4
--- /dev/null
+++ b/contrib/icu_multilib/meson.build
@@ -0,0 +1,24 @@
+# Copyright (c) 2022-2023, PostgreSQL Global Development Group
+
+icu_multilib_sources = files(
+  'icu_multilib.c',
+)
+
+if host_system == 'windows'
+  icu_multilib_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'icu_multilib',
+    '--FILEDESC', 'icu_multilib -- support multiple ICU library versions',])
+endif
+
+icu_multilib = shared_module('icu_multilib',
+  icu_multilib_sources,
+  kwargs: contrib_mod_args,
+)
+contrib_targets += icu_multilib
+
+install_data(
+  'icu_multilib.control',
+  'icu_multilib--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
diff --git a/contrib/meson.build b/contrib/meson.build
index bd4a57c43c..03f72147cf 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -28,6 +28,9 @@ subdir('fuzzystrmatch')
 subdir('hstore')
 subdir('hstore_plperl')
 subdir('hstore_plpython')
+if icu.found()
+   subdir('icu_multilib')
+endif
 subdir('intagg')
 subdir('intarray')
 subdir('isn')
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 4e7b87a42f..469a1746b9 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -113,6 +113,7 @@ CREATE EXTENSION <replaceable>module_name</replaceable>;
  &file-fdw;
  &fuzzystrmatch;
  &hstore;
+ &icu-multilib;
  &intagg;
  &intarray;
  &isn;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 0d6be9a2fa..091ce0a60b 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -131,6 +131,7 @@
 <!ENTITY file-fdw        SYSTEM "file-fdw.sgml">
 <!ENTITY fuzzystrmatch   SYSTEM "fuzzystrmatch.sgml">
 <!ENTITY hstore          SYSTEM "hstore.sgml">
+<!ENTITY icu-multilib    SYSTEM "icu-multilib.sgml">
 <!ENTITY intagg          SYSTEM "intagg.sgml">
 <!ENTITY intarray        SYSTEM "intarray.sgml">
 <!ENTITY isn             SYSTEM "isn.sgml">
diff --git a/doc/src/sgml/icu-multilib.sgml b/doc/src/sgml/icu-multilib.sgml
new file mode 100644
index 0000000000..838872c880
--- /dev/null
+++ b/doc/src/sgml/icu-multilib.sgml
@@ -0,0 +1,496 @@
+<!-- doc/src/sgml/icu-multilib.sgml -->
+
+<sect1 id="icu-multilib" xreflabel="icu_multilib">
+ <title>icu_multilib</title>
+
+ <indexterm zone="icu-multilib">
+  <primary>icu_multilib</primary>
+ </indexterm>
+
+ <para>
+  The <filename>icu_multilib</filename> module provides control over the
+  version (or versions) of the ICU provider library used by
+  <productname>PostgreSQL</productname>, which can be different from the
+  version of ICU with which it was built.
+ </para>
+
+ <para>
+  Collations are a product of natural language, and natural language evolves
+  over time; but <productname>PostgreSQL</productname> depends on stable
+  ordering for structures such as indexes. Newer versions of ICU update the
+  provided collators to adapt to changes in natural language, so it's
+  important to control when and how those new versions of ICU are used to
+  prevent problems such as index corruption.
+ </para>
+
+ <para>
+  This module assumes that the necessary versions of ICU are already
+  available, such as through the operating system's package manager; and
+  already properly installed in a single location accessible to
+  <productname>PostgreSQL</productname>. The configration variable
+  <literal>icu_multilib.library_path</literal> should be set to the location
+  where these ICU library versions are installed.
+ </para>
+
+ <para>
+  <filename>icu_multilib</filename> must be loaded via
+  <literal>shared_preload_libraries</literal>.
+  <filename>icu_multilib</filename> ignores any ICU library with a major
+  version greater than that with which <productname>PostgreSQL</productname>
+  was built.
+ </para>
+
+ <sect2 id="icu-multilib-version-stability">
+  <title>ICU Version Stability</title>
+
+  <para>
+   The simplest way to use <filename>icu_multilib</filename> is to force ICU
+   version stability. Version stability means that
+   <productname>PostgreSQL</productname> will use the same version of ICU for
+   all collations, even if the operating system or
+   <productname>PostgreSQL</productname> itself is upgraded or moved. The
+   stable version may or may not be the same as that with which
+   <productname>PostgreSQL</productname> was built.
+  </para>
+
+  <para>
+   Version stability may be based on both the major and minor version of ICU,
+   such as <literal>68.2</literal>; or on the major version alone, such as
+   <literal>68</literal>. Specifying both the major and minor versions more
+   closely controls exactly which ICU library is loaded, but applying minor
+   version upgrades of the ICU library may interfere with normal
+   operation. Specifying the major version alone makes applying ICU minor
+   version updates simpler, but carries a risk of subtle differences in
+   collation order if the library is updated.
+  </para>
+
+  <para>
+   To configure for version stability, first make sure that
+   <varname>icu_multilib.library_path</varname> is set correctly, which you
+   can observe by executing the
+   <literal>icu_multilib.library_versions()</literal> function. Then set
+   <varname>icu_multilib.default_icu_version</varname> to the stable version,
+   and set <varname>icu_multilib.search_by_collator_version</varname> to
+   <literal>false</literal>. These configuration variables should be set in
+   <filename>postgresql.conf</filename> to ensure that the stable version of
+   ICU is used consistently.
+  </para>
+
+  <note>
+   <para>
+    Ensure that the stable version of the ICU library is always available,
+    including on all replicas and through operating system and
+    <productname>PostgreSQL</productname> upgrades.
+   </para>
+   <para>
+    Similarly, keep the settings in <filename>postgresql.conf</filename>, and
+    keep them consistent across replicas and upgrades.
+   </para>
+  </note>
+
+  <para>
+   For best results, stabilize the ICU version immediately after
+   <command>initdb</command> is run, to ensure that initial collations are
+   loaded from the stabilized version of the ICU library. To do this, use the
+   option <xref linkend="app-initdb-option-no-import-collations"/> for
+   <command>initdb</command>, import the collations manually, and then refresh
+   the versions for the default collations. For instance:
+  </para>
+
+<programlisting>
+$ initdb --no-import-collations -D data --locale-provider=icu --icu-locale="en_US" --locale="en_US.UTF-8"
+$ cat - >> data/postgresql.conf
+shared_preload_libraries = 'icu_multilib'
+icu_multilib.library_path = '/path/to/icu/lib'
+icu_multilib.default_icu_version = '65.1'
+icu_multilib.search_by_collator_version = false
+icu_multilib.include_builtin = false
+^D
+$ pg_ctl -D data -l logfile start
+$ psql template1
+WARNING:  icu_multilib: collator version mismatch detected for locale "en_US"
+DETAIL:  ICU 65.1 provides collator version "153.97" for locale "en_US"; expected version "153.112".
+WARNING:  icu_multilib: collator version mismatch detected for locale "en_US"
+DETAIL:  ICU 65.1 provides collator version "153.97" for locale "en_US"; expected version "153.112".
+WARNING:  database "template1" has a collation version mismatch
+DETAIL:  The database was created using collation version 153.112, but the operating system provides version 153.97.
+HINT:  Rebuild all objects in this database that use the default collation and run ALTER DATABASE template1 REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.
+=# ALTER DATABASE template1 REFRESH COLLATION VERSION;
+NOTICE:  changing version from 153.112 to 153.97
+ALTER DATABASE
+=# SELECT pg_import_system_collations('pg_catalog');
+ pg_import_system_collations
+-----------------------------
+                         792
+(1 row)
+
+=# \q
+$ psql postgres
+WARNING:  icu_multilib: collator version mismatch detected for locale "en_US"
+DETAIL:  ICU 65.1 provides collator version "153.97" for locale "en_US"; expected version "153.112".
+WARNING:  icu_multilib: collator version mismatch detected for locale "en_US"
+DETAIL:  ICU 65.1 provides collator version "153.97" for locale "en_US"; expected version "153.112".
+WARNING:  database "postgres" has a collation version mismatch
+DETAIL:  The database was created using collation version 153.112, but the operating system provides version 153.97.
+HINT:  Rebuild all objects in this database that use the default collation and run ALTER DATABASE postgres REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.
+=# ALTER DATABASE postgres REFRESH COLLATION VERSION;
+NOTICE:  changing version from 153.112 to 153.97
+ALTER DATABASE
+=# SELECT pg_import_system_collations('pg_catalog');
+ pg_import_system_collations
+-----------------------------
+                         792
+(1 row)
+
+=# \q
+$
+</programlisting>
+
+  <para>
+   The result will be a system based on ICU version <literal>65.1</literal>, and
+   <productname>PostgreSQL</productname> will use that version of ICU for all
+   collations that use the ICU provider.
+  </para>
+ </sect2>
+
+ <sect2 id="icu-multi-version">
+  <title>Multiple ICU Version Support</title>
+  <para>
+   Support for multiple ICU library versions in the same database allows the
+   system to adapt to changes in natural language over time. As these changes
+   are introduced in new versions of ICU, those new versions can be brought
+   into <productname>PostgreSQL</productname> incrementally without
+   interfering with existing data (or structures like indexes).
+  </para>
+  <sect3>
+   <title>Search by Collator Version</title>
+
+   <para>
+    Collators provided by ICU also have a version which is distinct from the
+    version of ICU. For instance, in ICU version <literal>70.1</literal>, the
+    collator for the <literal>en_US</literal> locale has version
+    <literal>153.112</literal>. A collator for a given locale may or may not
+    be assigned a new version when the ICU library version is updated.
+   </para>
+
+   <para>
+    When a collation is created, <productname>PostgreSQL</productname> obtains
+    the version of the collator from the ICU library and records it in the
+    catalog. This recorded version can be updated using <literal>ALTER
+    COLLATION ... REFRESH VERSION</literal>; or <literal>ALTER DATABASE
+    ... REFRESH COLLATION VERSION</literal> if it's the database's default
+    collation. These <literal>ALTER</literal> commands obtain the new collator
+    version from the ICU library identified by
+    <varname>icu_multilib.default_icu_version</varname> if set; otherwise they
+    obtain the new collator version from the built-in ICU library.
+   </para>
+
+   <para>
+    Set <varname>icu_multilib.search_by_collator_version</varname> to
+    <literal>true</literal> to enable <filename>icu_multilib</filename> to
+    search (in descending order of ICU major version) among the
+    available libraries to find the first one that provides a collator with a
+    version that matches the one recorded in the catalog.
+   </para>
+
+   <para>
+    For example, ICU versions <literal>62.2</literal> and
+    <literal>63.2</literal> both provide collator version
+    <literal>153.88</literal> for the locale <literal>en_US</literal>; while
+    ICU version <literal>64.2</literal> provides collator version
+    <literal>153.97</literal> for the same locale. Searching for an ICU
+    library for a collation with a recorded version of
+    <literal>153.88</literal> will find the ICU library with version
+    <literal>63.2</literal>, because that's the ICU library with the highest
+    major version (<literal>63</literal>) that provides collator version
+    <literal>153.88</literal>. The function
+    <function>icu_multilib.collator_version_search</function> simulates this
+    search for the given locale and version:
+   </para>
+<programlisting>
+SELECT * FROM icu_multilib.collator_version_search('en_US', '153.88');
+ icu_version | uca_version | collator_version
+-------------+-------------+------------------
+ 63.2        | 11.0        | 153.88
+(1 row)
+
+</programlisting>
+
+   <note>
+    <para>
+     Searching by collator version relies on the assumption that collators
+     with the same version behave identically regardless of which ICU library
+     provides it. The administrator must weigh the risks of this assumption
+     against the convenience when managing multiple major versions of ICU.
+    </para>
+   </note>
+  </sect3>
+ </sect2>
+
+ <sect2 id="icu-multilib-configuration-parameters">
+  <title>Configuration Parameters</title>
+
+  <variablelist>
+   <varlistentry>
+    <term>
+     <varname>icu_multilib.library_path</varname> (<type>string</type>)
+     <indexterm>
+      <primary><varname>icu_multilib.library_path</varname> configuration parameter</primary>
+     </indexterm>
+    </term>
+    <listitem>
+     <para>
+      The filesystem path in which to search for ICU libraries. Must be set
+      for <filename>icu_multilib</filename> to be useful. Multiple major
+      versions of ICU may be present; but for each major version, only a
+      single minor version can be present.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term>
+     <varname>icu_multilib.version_mismatch_log_level</varname> (<type>string</type>)
+     <indexterm>
+      <primary><varname>icu_multilib.version_mismatch_log_level</varname> configuration parameter</primary>
+     </indexterm>
+    </term>
+    <listitem>
+     <para>
+      A version mismatch happens when the version of a collator provided by
+      the ICU library is different than then collator version recorded when
+      the collation was created (or last refreshed).
+     </para>
+     <para>
+      When a version mismatch is detected, a log message is emitted at this
+      level. <literal>WARNING</literal> is the default. If the level is set to
+      <literal>ERROR</literal>, it may interfere with normal operation.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term>
+     <varname>icu_multilib.library_search_log_level</varname> (<type>string</type>)
+     <indexterm>
+      <primary><varname>icu_multilib.library_search_log_level</varname> configuration parameter</primary>
+     </indexterm>
+    </term>
+    <listitem>
+     <para>
+      <filename>icu_multilib</filename> emits messages at this level while
+      determining which ICU library to choose. The default is
+      <literal>DEBUG1</literal>. This setting is useful to diagnose problems
+      when a suitable ICU library is not found.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term>
+     <varname>icu_multilib.default_icu_version</varname> (<type>string</type>)
+     <indexterm>
+      <primary><varname>icu_multilib.default_icu_version</varname> configuration parameter</primary>
+     </indexterm>
+    </term>
+    <listitem>
+     <para>
+      Identifies the version of the ICU library to choose; or, if <varname>icu_multilib.search_by_collator_version</varname> is <literal>true</literal>, the ICU library to choose if no identical match to the collator version is found. The default is the empty string, meaning to use the built-in ICU library.
+     </para>
+     <para>
+      Valid values are either of the form <literal>major.minor</literal> such
+      as <literal>70.1</literal>; or a major version number alone, such as
+      <literal>70</literal>.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term>
+     <varname>icu_multilib.include_builtin</varname> (<type>string</type>)
+     <indexterm>
+      <primary><varname>icu_multilib.include_builtin</varname> configuration parameter</primary>
+     </indexterm>
+    </term>
+    <listitem>
+     <para>
+      Determines whether the built-in version of ICU is included when
+      searching for a suitable ICU library. The default is
+      <literal>true</literal>.
+     </para>
+     <para>
+      Even if set to <literal>false</literal>, the built-in ICU library may be
+      used if no other suitable ICU library is identified and
+      <varname>icu_multilib.default_icu_version</varname> is set to the empty
+      string.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term>
+     <varname>icu_multilib.search_by_collator_version</varname> (<type>string</type>)
+     <indexterm>
+      <primary><varname>icu_multilib.search_by_collator_version</varname> configuration parameter</primary>
+     </indexterm>
+    </term>
+    <listitem>
+     <para>
+      Causes <filename>icu_multilib</filename> to identify the ICU library
+      with the highest major version that offers the required collator with an
+      exactly-matching collator version. The default is <literal>false</literal>.
+     </para>
+     <para>
+      When set to <literal>true</literal>, <filename>icu_multilib</filename>
+      relies on equal collator versions to produce identical collation order.
+     </para>
+     <para>
+      If no ICU library is found with an exactly-matching collator version,
+      <filename>icu_multilib</filename> will fall back to the
+      <varname>icu_multilib.default_icu_version</varname>; or if not set, to
+      the built-in ICU library.
+     </para>
+    </listitem>
+   </varlistentry>
+  </variablelist>
+ </sect2>
+
+ <sect2>
+  <title>Functions</title>
+
+  <table id="icu-multilib-functions">
+   <title><filename>icu_multilib</filename> Functions</title>
+   <tgroup cols="1">
+    <thead>
+     <row>
+      <entry role="func_table_entry"><para role="func_signature">
+       Function
+      </para>
+      <para>
+       Description
+      </para></entry>
+     </row>
+    </thead>
+
+    <tbody>
+     <row>
+      <entry role="func_table_entry"><para role="func_signature">
+       <function>icu_multilib.library_versions</function> ()
+       <returnvalue>setof record</returnvalue>
+       ( <parameter>icu_version</parameter> <type>text</type>,
+       <parameter>unicode_version</parameter> <type>text</type>,
+       <parameter>cldr_version</parameter> <type>text</type>,
+       <parameter>libicui18n_name</parameter> <type>text</type>,
+       <parameter>libicuuc_name</parameter> <type>text</type>)
+      </para>
+      <para>
+       Returns details for each available ICU library found in
+       <varname>icu_multilib.library_path</varname>. Also includes the
+       built-in ICU library iv <varname>icu_multilib.include_builtin</varname>
+       is <literal>true</literal>.
+      </para></entry>
+     </row>
+     <row>
+      <entry role="func_table_entry"><para role="func_signature">
+       <function>icu_multilib.library_collators</function> ( <parameter>major_version</parameter> <type>text</type> <literal>DEFAULT</literal> <literal>null</literal> )
+       <returnvalue>setof record</returnvalue>
+       ( <parameter>locale</parameter> <type>text</type>,
+       <parameter>icu_version</parameter> <type>text</type>,
+       <parameter>uca_version</parameter> <type>text</type>,
+       <parameter>collator_version</parameter> <type>text</type> )
+      </para>
+      <para>
+       Returns details for all available collators provided by the ICU library
+       with the given <parameter>major_version</parameter>; or by the built-in
+       ICU library if <parameter>major_version</parameter> is
+       <literal>null</literal>.
+      </para></entry>
+     </row>
+     <row>
+      <entry role="func_table_entry"><para role="func_signature">
+       <function>icu_multilib.collator_version_search</function> ( <parameter>locale</parameter> <type>text</type>, <parameter>requested_version</parameter> <type>text</type> <literal>DEFAULT</literal> <literal>null</literal>, <parameter>log_ok</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>false</literal> )
+       <returnvalue>record</returnvalue>
+       ( <parameter>icu_version</parameter> <type>text</type>,
+       <parameter>uca_version</parameter> <type>text</type>,
+       <parameter>collator_version</parameter> <type>text</type> )
+      </para>
+      <para>
+       Performs a search for the appropriate ICU library given the locale name
+       and requested collator version, and returns details about the ICU
+       library.
+      </para>
+      <para>
+       If <parameter>requested_version</parameter> is <literal>null</literal>,
+       it will return the ICU library identified by
+       <varname>icu_multilib.default_icu_version</varname> if set to
+       <literal>true</literal>; otherwise the built-in ICU library.
+      </para>
+      <para>
+       If <parameter>log_ok</parameter> is <literal>true</literal>, the search
+       may emit log messages at the level
+       <varname>icu_multilib.library_search_log_level</varname>, which may be
+       useful for diagnosing misconfiguration.
+      </para></entry>
+     </row>
+     <row>
+      <entry role="func_table_entry"><para role="func_signature">
+       <function>icu_multilib.collator_versions</function> ( <parameter>locale</parameter> <type>text</type> )
+       <returnvalue>setof record</returnvalue>
+       ( <parameter>icu_version</parameter> <type>text</type>,
+       <parameter>uca_version</parameter> <type>text</type>,
+       <parameter>collator_version</parameter> <type>text</type> )
+      </para>
+      <para>
+       Returns details about available collators for the given
+       <parameter>locale</parameter> from all available ICU libraries.
+      </para></entry>
+     </row>
+    </tbody>
+   </tgroup>
+  </table>
+ </sect2>
+
+ <sect2 id="icu-multilib-examples">
+  <title>Examples</title>
+
+<screen>
+SELECT icu_version, unicode_version, cldr_version FROM icu_multilib.library_versions() LIMIT 3;
+ icu_version | unicode_version | cldr_version
+-------------+-----------------+--------------
+ 70.1        | 14.0            | 40.0
+ 69.1        | 13.0            | 39.0
+ 68.2        | 13.0            | 38.1
+(3 rows)
+</screen>
+
+<screen>
+SELECT * FROM icu_multilib.collator_versions('en_US') WHERE icu_version BETWEEN '60.0' and '65.0';
+ icu_version | uca_version | collator_version
+-------------+-------------+------------------
+ 64.2        | 12.1        | 153.97
+ 63.2        | 11.0        | 153.88
+ 62.2        | 11.0        | 153.88
+ 61.2        | 10.0        | 153.80
+ 60.3        | 10.0        | 153.80
+(5 rows)
+</screen>
+
+<screen>
+SELECT * FROM icu_multilib.collator_version_search('en_US', '153.97');
+ icu_version | uca_version | collator_version
+-------------+-------------+------------------
+ 65.1        | 12.1        | 153.97
+(1 row)
+</screen>
+
+ </sect2>
+
+ <sect2 id="icu-multilib-author">
+  <title>Authors</title>
+
+  <para>
+   Jeff Davis <email>[email-masked]</email>, Thomas Munro <email>[email-masked]</email>
+  </para>
+ </sect2>
+
+</sect1>
-- 
2.34.1

From 966cdc29309965c3dc5a5608c38c36faeb9cbb77 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email-masked]>
Date: Wed, 18 Jan 2023 12:32:56 -0800
Subject: [PATCH v8 3/4] Add initdb option --no-import-collations.

In preparation for the icu_multilib contrib module, which allows
greater control over how ICU collations and how different ICU library
versions are managed and used.

By not creating the collations during initdb time, the user will be
able to run pg_import_system_collations() themselves using a specific
version of ICU.
---
 doc/src/sgml/ref/initdb.sgml | 11 +++++++++++
 src/bin/initdb/initdb.c      | 10 +++++++++-
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
index 5b2bdac101..5a2da829fb 100644
--- a/doc/src/sgml/ref/initdb.sgml
+++ b/doc/src/sgml/ref/initdb.sgml
@@ -305,6 +305,17 @@ PostgreSQL documentation
       </listitem>
      </varlistentry>
 
+     <varlistentry id="app-initdb-option-no-import-collations">
+      <term><option>--no-import-collations</option></term>
+      <listitem>
+       <para>
+        Prevents <command>initdb</command> from importing collations from the
+        system. This can be used to more carefully control which collations
+        are initially loaded in each database.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="app-initdb-option-no-sync">
       <term><option>-N</option></term>
       <term><option>--no-sync</option></term>
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 7a58c33ace..acaeb00086 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -135,6 +135,7 @@ static char *lc_time = NULL;
 static char *lc_messages = NULL;
 static char locale_provider = COLLPROVIDER_LIBC;
 static char *icu_locale = NULL;
+static bool import_system_collations = true;
 static const char *default_text_search_config = NULL;
 static char *username = NULL;
 static bool pwprompt = false;
@@ -1491,7 +1492,8 @@ setup_collation(FILE *cmdfd)
 				  BOOTSTRAP_SUPERUSERID, COLLPROVIDER_LIBC, PG_UTF8);
 
 	/* Now import all collations we can find in the operating system */
-	PG_CMD_PUTS("SELECT pg_import_system_collations('pg_catalog');\n\n");
+	if (import_system_collations)
+		PG_CMD_PUTS("SELECT pg_import_system_collations('pg_catalog');\n\n");
 }
 
 /*
@@ -2116,6 +2118,8 @@ usage(const char *progname)
 	printf(_("      --no-locale           equivalent to --locale=C\n"));
 	printf(_("      --locale-provider={libc|icu}\n"
 			 "                            set default locale provider for new databases\n"));
+	printf(_("      --no-import-collations\n"
+			 "                            do not import collations from the system"));
 	printf(_("      --pwfile=FILE         read password for the new superuser from file\n"));
 	printf(_("  -T, --text-search-config=CFG\n"
 			 "                            default text search configuration\n"));
@@ -2767,6 +2771,7 @@ main(int argc, char *argv[])
 		{"discard-caches", no_argument, NULL, 14},
 		{"locale-provider", required_argument, NULL, 15},
 		{"icu-locale", required_argument, NULL, 16},
+		{"no-import-collations", no_argument, NULL, 17},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -2924,6 +2929,9 @@ main(int argc, char *argv[])
 			case 16:
 				icu_locale = pg_strdup(optarg);
 				break;
+			case 17:
+				import_system_collations = false;
+				break;
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
-- 
2.34.1

From 540559f8104943eaea9a657ecf16326b31e7e545 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email-masked]>
Date: Wed, 21 Dec 2022 19:50:10 -0800
Subject: [PATCH v8 2/4] Add test module for icu collation provider hook.

---
 src/test/modules/Makefile                     |   1 +
 src/test/modules/meson.build                  |   1 +
 .../modules/test_collator_lib_hooks/Makefile  |  24 ++
 .../test_collator_lib_hooks/meson.build       |  37 +++
 .../test_collator_lib_hooks/t/001_icu.pl      | 153 ++++++++++++
 .../test_collator_lib_hooks.c                 |  43 ++++
 .../test_collator_lib_hooks.control           |   4 +
 .../test_collator_lib_hooks.h                 |  28 +++
 .../test_collator_lib_hooks/test_icu_hook.c   | 228 ++++++++++++++++++
 9 files changed, 519 insertions(+)
 create mode 100644 src/test/modules/test_collator_lib_hooks/Makefile
 create mode 100644 src/test/modules/test_collator_lib_hooks/meson.build
 create mode 100644 src/test/modules/test_collator_lib_hooks/t/001_icu.pl
 create mode 100644 src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.c
 create mode 100644 src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.control
 create mode 100644 src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.h
 create mode 100644 src/test/modules/test_collator_lib_hooks/test_icu_hook.c

diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index c629cbe383..8a90b206a1 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -15,6 +15,7 @@ SUBDIRS = \
 		  snapshot_too_old \
 		  spgist_name_ops \
 		  test_bloomfilter \
+		  test_collator_lib_hooks \
 		  test_copy_callbacks \
 		  test_custom_rmgrs \
 		  test_ddl_deparse \
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 1baa6b558d..a59cc9a89f 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -11,6 +11,7 @@ subdir('snapshot_too_old')
 subdir('spgist_name_ops')
 subdir('ssl_passphrase_callback')
 subdir('test_bloomfilter')
+subdir('test_collator_lib_hooks')
 subdir('test_copy_callbacks')
 subdir('test_custom_rmgrs')
 subdir('test_ddl_deparse')
diff --git a/src/test/modules/test_collator_lib_hooks/Makefile b/src/test/modules/test_collator_lib_hooks/Makefile
new file mode 100644
index 0000000000..b44c8f5914
--- /dev/null
+++ b/src/test/modules/test_collator_lib_hooks/Makefile
@@ -0,0 +1,24 @@
+# src/test/modules/test_collator_lib_hooks/Makefile
+
+MODULE_big = test_collator_lib_hooks
+OBJS = \
+	$(WIN32RES) \
+	test_collator_lib_hooks.o test_icu_hook.o
+PGFILEDESC = "test_collator_lib_hooks - test collator provider library hooks"
+
+EXTENSION = test_collator_lib_hooks
+
+TAP_TESTS = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_collator_lib_hooks
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
+
+export with_icu
diff --git a/src/test/modules/test_collator_lib_hooks/meson.build b/src/test/modules/test_collator_lib_hooks/meson.build
new file mode 100644
index 0000000000..0190a856e0
--- /dev/null
+++ b/src/test/modules/test_collator_lib_hooks/meson.build
@@ -0,0 +1,37 @@
+# FIXME: prevent install during main install, but not during test :/
+
+test_collator_lib_hooks_sources = files(
+  'test_collator_lib_hooks.c',
+  'test_icu_hook.c',
+)
+
+if host_system == 'windows'
+  test_collator_lib_hooks_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'test_collator_lib_hooks',
+    '--FILEDESC', 'test_collator_lib_hooks - test collation provider library hooks',])
+endif
+
+test_collator_lib_hooks = shared_module('test_collator_lib_hooks',
+  test_collator_lib_hooks_sources,
+  kwargs: pg_mod_args,
+)
+testprep_targets += test_collator_lib_hooks
+
+install_data(
+  'test_collator_lib_hooks.control',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'test_collator_lib_hooks',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'tap': {
+    'tests': [
+      't/001_icu.pl',
+    ],
+    'env': {
+      'with_icu': icu.found() ? 'yes' : 'no',
+    },
+  },
+}
diff --git a/src/test/modules/test_collator_lib_hooks/t/001_icu.pl b/src/test/modules/test_collator_lib_hooks/t/001_icu.pl
new file mode 100644
index 0000000000..d3b5729527
--- /dev/null
+++ b/src/test/modules/test_collator_lib_hooks/t/001_icu.pl
@@ -0,0 +1,153 @@
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+if ($ENV{with_icu} ne 'yes')
+{
+	plan skip_all => 'ICU not supported by this build';
+}
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+
+$node->init;
+$node->append_conf(
+	'postgresql.conf', q{
+shared_preload_libraries = 'test_collator_lib_hooks'
+});
+$node->start;
+
+$node->safe_psql('postgres',
+	q{CREATE DATABASE dbicu LOCALE_PROVIDER icu LOCALE 'C' ICU_LOCALE 'DESC' ENCODING 'UTF8' TEMPLATE template0}
+);
+
+# setup
+$node->safe_psql('dbicu',
+	qq[CREATE COLLATION test_asc (PROVIDER=icu, LOCALE='ASC')]);
+$node->safe_psql('dbicu',
+	qq[CREATE COLLATION test_desc (PROVIDER=icu, LOCALE='DESC')]);
+
+$node->safe_psql('dbicu', qq[CREATE TABLE strings(t text)]);
+$node->safe_psql('dbicu',
+	qq[INSERT INTO strings VALUES ('aBcD'), ('fGhI'), ('wXyZ')]);
+
+# check versions
+
+my $version_db =
+  $node->safe_psql('dbicu',
+	  qq[SELECT datcollversion FROM pg_database WHERE datname='dbicu']);
+is($version_db, '2.72',
+	'database "dbicu" has correct version 2.72'
+);
+
+my $version_asc =
+  $node->safe_psql('dbicu',
+	  qq[SELECT collversion FROM pg_collation WHERE collname='test_asc']);
+is($version_asc, '2.72',
+	'collation "test_asc" has correct version 2.72'
+);
+
+my $version_desc =
+  $node->safe_psql('dbicu',
+	  qq[SELECT collversion FROM pg_collation WHERE collname='test_desc']);
+is($version_desc, '2.72',
+	'collation "test_desc" has correct version 2.72'
+);
+
+my $res_sort_expected = "aBcD
+fGhI
+wXyZ";
+
+my $res_reversesort_expected = "wXyZ
+fGhI
+aBcD";
+
+# test comparison
+
+my $comparison =
+  $node->safe_psql('dbicu',
+	  qq[SELECT 'aBcD' COLLATE test_asc < 'wXyZ' COLLATE test_asc]);
+is($comparison, 't',
+	'correct comparison'
+);
+
+# test reverse comparison (database)
+
+my $dbcomparison_reverse =
+  $node->safe_psql('dbicu', qq[SELECT 'aBcD' < 'wXyZ']);
+is($dbcomparison_reverse, 'f',
+	'correct reverse comparison (database)'
+);
+
+# test reverse comparison
+
+my $comparison_reverse =
+  $node->safe_psql('dbicu',
+	  qq[SELECT 'aBcD' COLLATE test_desc < 'wXyZ' COLLATE test_desc]);
+is($comparison_reverse, 'f',
+	'correct reverse comparison'
+);
+
+# test asc sort
+
+my $res_sort =
+  $node->safe_psql('dbicu',
+	  qq[SELECT t FROM strings ORDER BY t COLLATE test_asc]);
+is($res_sort, $res_sort_expected,
+	'correct ascending sort'
+);
+
+# test desc sort
+
+my $res_db_reversesort =
+  $node->safe_psql('dbicu',
+	  qq[SELECT t FROM strings ORDER BY t]);
+is($res_db_reversesort, $res_reversesort_expected,
+	'correct descending sort (database)'
+);
+
+# test desc sort
+
+my $res_reversesort =
+  $node->safe_psql('dbicu',
+	  qq[SELECT t FROM strings ORDER BY t COLLATE test_desc]);
+is($res_reversesort, $res_reversesort_expected,
+	'correct descending sort'
+);
+
+# test lower/upper
+
+my $tcase =
+  $node->safe_psql('dbicu',
+	  qq[SELECT lower('aBcDfgHiwXyZ' collate test_asc),
+                upper('aBcDfgHiwXyZ' collate test_asc)]);
+is($tcase, 'abcdfghiwxyz|ABCDFGHIWXYZ',
+	'correct lowercase and uppercase'
+);
+
+# test reverse lower/upper (database)
+
+my $tcase_db_reverse =
+  $node->safe_psql('dbicu',
+	  qq[SELECT lower('aBcDfgHiwXyZ'),
+                upper('aBcDfgHiwXyZ')]);
+is($tcase_db_reverse, 'ABCDFGHIWXYZ|abcdfghiwxyz',
+	'correct reverse lowercase and uppercase (database)'
+);
+
+# test reverse lower/upper
+
+my $tcase_reverse =
+  $node->safe_psql('dbicu',
+	  qq[SELECT lower('aBcDfgHiwXyZ' collate test_desc),
+                upper('aBcDfgHiwXyZ' collate test_desc)]);
+is($tcase_reverse, 'ABCDFGHIWXYZ|abcdfghiwxyz',
+	'correct reverse lowercase and uppercase'
+);
+
+$node->stop;
+done_testing();
diff --git a/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.c b/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.c
new file mode 100644
index 0000000000..45a849b52d
--- /dev/null
+++ b/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.c
@@ -0,0 +1,43 @@
+/*--------------------------------------------------------------------------
+ *
+ * test_collator_lib_hooks.c
+ *		Code for testing collation provider library hooks
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *		src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.c
+ *
+ * Test implementation of icu-like collation provider.
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "miscadmin.h"
+
+#include "test_collator_lib_hooks.h"
+
+#ifdef USE_ICU
+static get_icu_library_hook_type prev_get_icu_library_hook = NULL;
+#endif
+
+PG_MODULE_MAGIC;
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+	if (!process_shared_preload_libraries_in_progress)
+		ereport(ERROR, (errmsg("test_collator_lib_hooks must be loaded via shared_preload_libraries")));
+
+#ifdef USE_ICU
+	prev_get_icu_library_hook = get_icu_library_hook;
+	get_icu_library_hook = test_get_icu_library;
+#endif
+}
diff --git a/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.control b/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.control
new file mode 100644
index 0000000000..a0b8e031a4
--- /dev/null
+++ b/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.control
@@ -0,0 +1,4 @@
+comment = 'Test code for collation provider library hooks'
+default_version = '1.0'
+module_pathname = '$libdir/test_collation_lib_hooks'
+
diff --git a/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.h b/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.h
new file mode 100644
index 0000000000..c3a9ff45e2
--- /dev/null
+++ b/src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.h
@@ -0,0 +1,28 @@
+/*--------------------------------------------------------------------------
+ *
+ * test_collator_lib_hooks.h
+ *		Definitions for collation library hooks.
+ *
+ * Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		src/test/modules/test_collator_lib_hooks/test_collator_lib_hooks.h
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#ifndef TEST_COLLATOR_LIB_HOOKS_H
+#define TEST_COLLATOR_LIB_HOOKS_H
+
+#include "postgres.h"
+
+#include "utils/memutils.h"
+#include "utils/pg_locale.h"
+#include "utils/pg_locale_internal.h"
+
+#ifdef USE_ICU
+extern pg_icu_library *test_get_icu_library(Oid collid, const char *locale,
+											const char *version);
+#endif
+
+#endif
diff --git a/src/test/modules/test_collator_lib_hooks/test_icu_hook.c b/src/test/modules/test_collator_lib_hooks/test_icu_hook.c
new file mode 100644
index 0000000000..423ff30b5b
--- /dev/null
+++ b/src/test/modules/test_collator_lib_hooks/test_icu_hook.c
@@ -0,0 +1,228 @@
+/*--------------------------------------------------------------------------
+ *
+ * test_icu_hook.c
+ *		Code for testing collation provider icu hook.
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *		src/test/modules/test_collator_lib_hooks/test_icu_hook.c
+ *
+ * Implements a custom icu-like collation provider library for testing the
+ * hooks. It accepts any collator name requested. All behave exactly like the
+ * "en_US" locale, except for the locale named "DESC", which reverses the sort
+ * order and reverses uppercase/lowercase behavior.
+ *
+ * The version is always reported as 2.72, so loading it will cause a version
+ * mismatch warning.
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "test_collator_lib_hooks.h"
+
+#ifdef USE_ICU
+
+#include <unicode/ucnv.h>
+#include <unicode/ulocdata.h>
+#include <unicode/ustring.h>
+
+#define TEST_LOCALE "en_US"
+
+typedef struct TestUCollator {
+	UCollator	*ucol;
+	bool		 reverse;
+} TestUCollator;
+
+static pg_icu_library *test_icu_library = NULL;
+static const UVersionInfo test_icu_version = { 2, 72 };
+
+static bool
+locale_is_reverse(const char *locale)
+{
+	if (strcmp(locale, "DESC") == 0)
+		return true;
+	else
+		return false;
+}
+
+static UCollator *
+test_openCollator(const char *loc, UErrorCode *status)
+{
+	TestUCollator *testcol = MemoryContextAlloc(TopMemoryContext, sizeof(TestUCollator));
+	UCollator *ucol = ucol_open(TEST_LOCALE, status);
+	testcol->ucol = ucol;
+	testcol->reverse = locale_is_reverse(loc);
+	return (UCollator *)testcol;
+}
+
+static void
+test_closeCollator(UCollator *coll)
+{
+	TestUCollator *testcol = (TestUCollator *) coll;
+	ucol_close(testcol->ucol);
+	pfree(testcol);
+}
+
+static void
+test_setAttribute(UCollator *coll, UColAttribute attr,
+				  UColAttributeValue value, UErrorCode *status)
+{
+	TestUCollator *testcol = (TestUCollator *) coll;
+	ucol_setAttribute(testcol->ucol, attr, value, status);
+}
+
+static void
+test_getCollatorVersion(const UCollator *coll, UVersionInfo info)
+{
+	memcpy(info, test_icu_version, sizeof(UVersionInfo));
+}
+
+static UCollationResult
+test_strcoll(const UCollator *coll, const UChar *source, int32_t sourceLength,
+			 const UChar *target, int32_t targetLength)
+{
+	TestUCollator *testcol = (TestUCollator *) coll;
+	UCollationResult ret = ucol_strcoll(testcol->ucol, source, sourceLength,
+										target, targetLength);
+	if (testcol->reverse)
+		return -ret;
+	else
+		return ret;
+}
+
+static UCollationResult
+test_strcollUTF8(const UCollator *coll, const char *source,
+				 int32_t sourceLength, const char *target,
+				 int32_t targetLength, UErrorCode *status)
+{
+	TestUCollator *testcol = (TestUCollator *) coll;
+	UCollationResult ret = ucol_strcollUTF8(testcol->ucol, source,
+											sourceLength, target,
+											targetLength, status);
+	if (testcol->reverse)
+		return -ret;
+	else
+		return ret;
+}
+
+static int32_t
+test_getSortKey(const UCollator *coll, const UChar *source,
+				int32_t sourceLength, uint8_t *result, int32_t resultLength)
+{
+	TestUCollator *testcol = (TestUCollator *) coll;
+	int32_t ret = ucol_getSortKey(testcol->ucol, source, sourceLength,
+								  result, resultLength);
+	size_t result_size = ret + 1;
+
+	if (resultLength >= result_size)
+	{
+		result[resultLength] = '\0';
+
+		if (testcol->reverse)
+			for (int i = 0; i < result_size; i++)
+				*((unsigned char *) result + i) ^= (unsigned char) 0xff;
+	}
+
+	return result_size;
+}
+
+static int32_t
+test_nextSortKeyPart(const UCollator *coll, UCharIterator *iter,
+					 uint32_t state[2], uint8_t *dest, int32_t count,
+					 UErrorCode *status)
+{
+	TestUCollator *testcol = (TestUCollator *) coll;
+	int32_t ret = ucol_nextSortKeyPart(testcol->ucol, iter, state, dest,
+									   count, status);
+
+	if (testcol->reverse)
+		for (int i = 0; i < ret; i++)
+			*((unsigned char *) dest + i) ^= (unsigned char) 0xff;
+
+	/*
+	 * The following is not correct for cases where we finish precisely on the
+	 * boundary (i.e. count is exactly enough). To fix this we'd need to track
+	 * additional state across calls, which doesn't seem worth it for a test
+	 * case.
+	 */
+	if (count >= ret && ret > 0)
+	{
+		if (testcol->reverse)
+			dest[ret] = 0xff;
+		else
+			dest[ret] = '\0';
+		return ret + 1;
+	}
+
+	return ret;
+}
+
+static int32_t
+test_strToUpper(UChar *dest, int32_t destCapacity, const UChar *src,
+				int32_t srcLength, const char *locale, UErrorCode *pErrorCode)
+{
+	if (locale_is_reverse(locale))
+		return u_strToLower(dest, destCapacity, src, srcLength,
+							TEST_LOCALE, pErrorCode);
+	else
+		return u_strToUpper(dest, destCapacity, src, srcLength,
+							TEST_LOCALE, pErrorCode);
+}
+
+static int32_t
+test_strToLower(UChar *dest, int32_t destCapacity, const UChar *src,
+				int32_t srcLength, const char *locale, UErrorCode *pErrorCode)
+{
+	if (locale_is_reverse(locale))
+		return u_strToUpper(dest, destCapacity, src, srcLength,
+							TEST_LOCALE, pErrorCode);
+	else
+		return u_strToLower(dest, destCapacity, src, srcLength,
+							TEST_LOCALE, pErrorCode);
+}
+
+pg_icu_library *
+test_get_icu_library(Oid collid, const char *locale, const char *version)
+{
+	pg_icu_library *lib;
+
+	if (test_icu_library != NULL)
+		return test_icu_library;
+
+	ereport(LOG, (errmsg("loading custom ICU provider for test_collator_lib_hooks")));
+
+	lib = MemoryContextAlloc(TopMemoryContext, sizeof(pg_icu_library));
+	lib->getICUVersion = u_getVersion;
+	lib->getUnicodeVersion = u_getUnicodeVersion;
+	lib->getCLDRVersion = ulocdata_getCLDRVersion;
+	lib->openCollator = test_openCollator;
+	lib->closeCollator = test_closeCollator;
+	lib->getCollatorVersion = test_getCollatorVersion;
+	lib->getUCAVersion = ucol_getUCAVersion;
+	lib->versionToString = u_versionToString;
+	lib->strcoll = test_strcoll;
+	lib->strcollUTF8 = test_strcollUTF8;
+	lib->getSortKey = test_getSortKey;
+	lib->nextSortKeyPart = test_nextSortKeyPart;
+	lib->setUTF8 = uiter_setUTF8;
+	lib->errorName = u_errorName;
+	lib->strToUpper = test_strToUpper;
+	lib->strToLower = test_strToLower;
+	lib->strToTitle = u_strToTitle;
+	lib->setAttribute = test_setAttribute;
+	lib->openConverter = ucnv_open;
+	lib->closeConverter = ucnv_close;
+	lib->fromUChars = ucnv_fromUChars;
+	lib->toUChars = ucnv_toUChars;
+	lib->toLanguageTag = uloc_toLanguageTag;
+	lib->getDisplayName = uloc_getDisplayName;
+	lib->countAvailable = uloc_countAvailable;
+	lib->getAvailable = uloc_getAvailable;
+
+	test_icu_library = lib;
+	return lib;
+}
+
+#endif				/* USE_ICU */
-- 
2.34.1

From 76c3d824ad777f259ea37c5eb2f6454999d8fb48 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email-masked]>
Date: Wed, 7 Dec 2022 11:07:31 -0800
Subject: [PATCH v8 1/4] Support multiple ICU collation provider libraries.

Introduce structure pg_icu_library, which holds pointers to each
required ICU method, and store this as part of pg_locale_t. Each call
to an ICU function instead goes through this structure, so that it can
more easily be replaced by a non-builtin ICU library.

Offer a hook to allow an extension to control which ICU library is
loaded for a given locale and version.

Author: Thomas Munro, Jeff Davis
---
 src/backend/commands/collationcmds.c   |  17 +-
 src/backend/utils/adt/formatting.c     |  67 ++++--
 src/backend/utils/adt/pg_locale.c      | 316 ++++++++++++++++++-------
 src/include/utils/pg_locale_internal.h | 124 +++++++++-
 4 files changed, 414 insertions(+), 110 deletions(-)

diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index d878be2fb8..217c13f066 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -566,13 +566,14 @@ get_icu_language_tag(const char *localename)
 {
 	char		buf[ULOC_FULLNAME_CAPACITY];
 	UErrorCode	status;
+	pg_icu_library *iculib = get_default_icu_library();
 
 	status = U_ZERO_ERROR;
-	uloc_toLanguageTag(localename, buf, sizeof(buf), true, &status);
+	iculib->toLanguageTag(localename, buf, sizeof(buf), true, &status);
 	if (U_FAILURE(status))
 		ereport(ERROR,
 				(errmsg("could not convert locale name \"%s\" to language tag: %s",
-						localename, u_errorName(status))));
+						localename, iculib->errorName(status))));
 
 	return pstrdup(buf);
 }
@@ -591,11 +592,12 @@ get_icu_locale_comment(const char *localename)
 	int32		len_uchar;
 	int32		i;
 	char	   *result;
+	pg_icu_library *iculib = get_default_icu_library();
 
 	status = U_ZERO_ERROR;
-	len_uchar = uloc_getDisplayName(localename, "en",
-									displayname, lengthof(displayname),
-									&status);
+	len_uchar = iculib->getDisplayName(localename, "en",
+									   displayname, lengthof(displayname),
+									   &status);
 	if (U_FAILURE(status))
 		return NULL;			/* no good reason to raise an error */
 
@@ -921,12 +923,13 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
 #ifdef USE_ICU
 	{
 		int			i;
+		pg_icu_library *iculib = get_default_icu_library();
 
 		/*
 		 * Start the loop at -1 to sneak in the root locale without too much
 		 * code duplication.
 		 */
-		for (i = -1; i < uloc_countAvailable(); i++)
+		for (i = -1; i < iculib->countAvailable(); i++)
 		{
 			const char *name;
 			char	   *langtag;
@@ -937,7 +940,7 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
 			if (i == -1)
 				name = "";		/* ICU root locale */
 			else
-				name = uloc_getAvailable(i);
+				name = iculib->getAvailable(i);
 
 			langtag = get_icu_language_tag(name);
 			iculocstr = U_ICU_VERSION_MAJOR_NUM >= 54 ? langtag : name;
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index f45e2043a1..0a683cd816 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -1566,6 +1566,11 @@ typedef int32_t (*ICU_Convert_Func) (UChar *dest, int32_t destCapacity,
 									 const UChar *src, int32_t srcLength,
 									 const char *locale,
 									 UErrorCode *pErrorCode);
+typedef int32_t (*ICU_Convert_BI_Func) (UChar *dest, int32_t destCapacity,
+										const UChar *src, int32_t srcLength,
+										UBreakIterator *bi,
+										const char *locale,
+										UErrorCode *pErrorCode);
 
 static int32_t
 icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
@@ -1573,6 +1578,7 @@ icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
 {
 	UErrorCode	status;
 	int32_t		len_dest;
+	pg_icu_library *iculib = PG_ICU_LIB(mylocale);
 
 	len_dest = len_source;		/* try first with same length */
 	*buff_dest = palloc(len_dest * sizeof(**buff_dest));
@@ -1590,18 +1596,42 @@ icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
 	}
 	if (U_FAILURE(status))
 		ereport(ERROR,
-				(errmsg("case conversion failed: %s", u_errorName(status))));
+				(errmsg("case conversion failed: %s",
+						iculib->errorName(status))));
 	return len_dest;
 }
 
+/*
+ * Like icu_convert_case, but func takes a break iterator (which we don't
+ * make use of).
+ */
 static int32_t
-u_strToTitle_default_BI(UChar *dest, int32_t destCapacity,
-						const UChar *src, int32_t srcLength,
-						const char *locale,
-						UErrorCode *pErrorCode)
+icu_convert_case_bi(ICU_Convert_BI_Func func, pg_locale_t mylocale,
+					UChar **buff_dest, UChar *buff_source, int32_t len_source)
 {
-	return u_strToTitle(dest, destCapacity, src, srcLength,
-						NULL, locale, pErrorCode);
+	UErrorCode	status;
+	int32_t		len_dest;
+	pg_icu_library *iculib = PG_ICU_LIB(mylocale);
+
+	len_dest = len_source;		/* try first with same length */
+	*buff_dest = palloc(len_dest * sizeof(**buff_dest));
+	status = U_ZERO_ERROR;
+	len_dest = func(*buff_dest, len_dest, buff_source, len_source, NULL,
+					mylocale->ctype, &status);
+	if (status == U_BUFFER_OVERFLOW_ERROR)
+	{
+		/* try again with adjusted length */
+		pfree(*buff_dest);
+		*buff_dest = palloc(len_dest * sizeof(**buff_dest));
+		status = U_ZERO_ERROR;
+		len_dest = func(*buff_dest, len_dest, buff_source, len_source, NULL,
+						mylocale->ctype, &status);
+	}
+	if (U_FAILURE(status))
+		ereport(ERROR,
+				(errmsg("case conversion failed: %s",
+						iculib->errorName(status))));
+	return len_dest;
 }
 
 #endif							/* USE_ICU */
@@ -1667,11 +1697,12 @@ str_tolower(const char *buff, size_t nbytes, Oid collid)
 			int32_t		len_conv;
 			UChar	   *buff_uchar;
 			UChar	   *buff_conv;
+			pg_icu_library *iculib = PG_ICU_LIB(mylocale);
 
-			len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
-			len_conv = icu_convert_case(u_strToLower, mylocale,
+			len_uchar = icu_to_uchar(iculib, &buff_uchar, buff, nbytes);
+			len_conv = icu_convert_case(iculib->strToLower, mylocale,
 										&buff_conv, buff_uchar, len_uchar);
-			icu_from_uchar(&result, buff_conv, len_conv);
+			icu_from_uchar(iculib, &result, buff_conv, len_conv);
 			pfree(buff_uchar);
 			pfree(buff_conv);
 		}
@@ -1789,11 +1820,12 @@ str_toupper(const char *buff, size_t nbytes, Oid collid)
 						len_conv;
 			UChar	   *buff_uchar;
 			UChar	   *buff_conv;
+			pg_icu_library *iculib = PG_ICU_LIB(mylocale);
 
-			len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
-			len_conv = icu_convert_case(u_strToUpper, mylocale,
+			len_uchar = icu_to_uchar(iculib, &buff_uchar, buff, nbytes);
+			len_conv = icu_convert_case(iculib->strToUpper, mylocale,
 										&buff_conv, buff_uchar, len_uchar);
-			icu_from_uchar(&result, buff_conv, len_conv);
+			icu_from_uchar(iculib, &result, buff_conv, len_conv);
 			pfree(buff_uchar);
 			pfree(buff_conv);
 		}
@@ -1912,11 +1944,12 @@ str_initcap(const char *buff, size_t nbytes, Oid collid)
 						len_conv;
 			UChar	   *buff_uchar;
 			UChar	   *buff_conv;
+			pg_icu_library *iculib = PG_ICU_LIB(mylocale);
 
-			len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
-			len_conv = icu_convert_case(u_strToTitle_default_BI, mylocale,
-										&buff_conv, buff_uchar, len_uchar);
-			icu_from_uchar(&result, buff_conv, len_conv);
+			len_uchar = icu_to_uchar(iculib, &buff_uchar, buff, nbytes);
+			len_conv = icu_convert_case_bi(iculib->strToTitle, mylocale,
+										   &buff_conv, buff_uchar, len_uchar);
+			icu_from_uchar(iculib, &result, buff_conv, len_conv);
 			pfree(buff_uchar);
 			pfree(buff_conv);
 		}
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index d3ca3c5011..10b179cb48 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -70,6 +70,8 @@
 
 #ifdef USE_ICU
 #include <unicode/ucnv.h>
+#include <unicode/ulocdata.h>
+#include <unicode/ustring.h>
 #endif
 
 #ifdef __GLIBC__
@@ -110,6 +112,41 @@ char	   *localized_full_days[7 + 1];
 char	   *localized_abbrev_months[12 + 1];
 char	   *localized_full_months[12 + 1];
 
+/*
+ * The precise version of a collation provider library is important, because
+ * subtle changes in collation between library versions can corrupt
+ * indexes. This hook allows control over how collation provider libraries are
+ * loaded.
+ *
+ * If the hook is not set, or if it returns NULL, Postgres constructs the
+ * pg_icu_library structure from the symbols Postgres is linked with at build
+ * time.
+ *
+ * If the hook is set, it can instead construct the pg_icu_library structure
+ * using custom logic. Ordinarily, this custom logic would involve finding a
+ * specific known version of the collation provider library, and dynamically
+ * loading the necessary symbols. If the collation version obtained from the
+ * library does not match the collation version recorded in the catalog,
+ * Postgres will issue a WARNING.
+ *
+ * The returned structure must be allocated in TopMemoryContext, and the
+ * associated symbols must remain valid permanently. It's expected that the
+ * hook will maintain a small finite list of libraries and return a pointer to
+ * the appropraite one, so that resource use remains bounded.
+ *
+ * XXX: For now, the only information the hook has access to is the ICU locale
+ * name, and the collation version as recorded in the catalog. We should
+ * consider what other information can be provided to allow for greater
+ * control over which library is loaded.
+ */
+#ifdef USE_ICU
+get_icu_library_hook_type get_icu_library_hook = NULL;
+#endif
+
+#ifdef USE_ICU
+static pg_icu_library *builtin_icu_library = NULL;
+#endif
+
 /* indicates whether locale information cache is valid */
 static bool CurrentLocaleConvValid = false;
 static bool CurrentLCTimeValid = false;
@@ -136,8 +173,12 @@ static char *IsoLocaleName(const char *);
  * Database default locale.
  */
 static pg_locale_t default_locale = NULL;
+#ifdef USE_ICU
+static pg_icu_library *default_icu = NULL;
+#endif
 
 #ifdef USE_ICU
+
 /*
  * Converter object for converting between ICU's UChar strings and C strings
  * in database encoding.  Since the database encoding doesn't change, we only
@@ -145,13 +186,17 @@ static pg_locale_t default_locale = NULL;
  */
 static UConverter *icu_converter = NULL;
 
-static void init_icu_converter(void);
-static size_t uchar_length(UConverter *converter,
+static void init_icu_converter(pg_icu_library *iculib);
+static size_t uchar_length(pg_icu_library *iculib,
+						   UConverter *converter,
 						   const char *str, size_t len);
-static int32_t uchar_convert(UConverter *converter,
+static int32_t uchar_convert(pg_icu_library *iculib,
+							 UConverter *converter,
 							 UChar *dest, int32_t destlen,
 							 const char *str, size_t srclen);
-static void icu_set_collation_attributes(UCollator *collator, const char *loc);
+static void icu_set_collation_attributes(pg_icu_library *iculib,
+										 UCollator *collator,
+										 const char *loc);
 #endif
 
 /*
@@ -1458,6 +1503,85 @@ report_newlocale_failure(const char *localename)
 }
 #endif							/* HAVE_LOCALE_T */
 
+#ifdef USE_ICU
+pg_icu_library *
+get_builtin_icu_library()
+{
+	UVersionInfo version_info;
+	pg_icu_library *lib;
+
+	if (builtin_icu_library != NULL)
+		return builtin_icu_library;
+
+	/*
+	 * These assignments will fail to compile if an incompatible API change is
+	 * made to some future version of ICU, at which point we might need to
+	 * consider special treatment for different major version ranges, with
+	 * intermediate trampoline functions.
+	 */
+	lib = MemoryContextAllocZero(TopMemoryContext, sizeof(pg_icu_library));
+	lib->getICUVersion = u_getVersion;
+	lib->getUnicodeVersion = u_getUnicodeVersion;
+	lib->getCLDRVersion = ulocdata_getCLDRVersion;
+	lib->openCollator = ucol_open;
+	lib->closeCollator = ucol_close;
+	lib->getCollatorVersion = ucol_getVersion;
+	lib->getUCAVersion = ucol_getUCAVersion;
+	lib->versionToString = u_versionToString;
+	lib->strcoll = ucol_strcoll;
+	lib->strcollUTF8 = ucol_strcollUTF8;
+	lib->getSortKey = ucol_getSortKey;
+	lib->nextSortKeyPart = ucol_nextSortKeyPart;
+	lib->setUTF8 = uiter_setUTF8;
+	lib->errorName = u_errorName;
+	lib->strToUpper = u_strToUpper;
+	lib->strToLower = u_strToLower;
+	lib->strToTitle = u_strToTitle;
+	lib->setAttribute = ucol_setAttribute;
+	lib->openConverter = ucnv_open;
+	lib->closeConverter = ucnv_close;
+	lib->fromUChars = ucnv_fromUChars;
+	lib->toUChars = ucnv_toUChars;
+	lib->toLanguageTag = uloc_toLanguageTag;
+	lib->getDisplayName = uloc_getDisplayName;
+	lib->countAvailable = uloc_countAvailable;
+	lib->getAvailable = uloc_getAvailable;
+
+	lib->getICUVersion(version_info);
+	lib->major_version = version_info[0];
+	lib->minor_version = version_info[1];
+	lib->libicui18n_name = "<builtin>";
+	lib->libicuuc_name = "<builtin>";
+
+	/*
+	 * Also assert the size of a couple of types used as output buffers, as a
+	 * canary to tell us to add extra padding in the (unlikely) event that a
+	 * later release makes these values smaller.
+	 */
+	StaticAssertStmt(U_MAX_VERSION_STRING_LENGTH == 20,
+					 "u_versionToString output buffer size changed incompatibly");
+	StaticAssertStmt(U_MAX_VERSION_LENGTH == 4,
+					 "ucol_getVersion output buffer size changed incompatibly");
+
+	builtin_icu_library = lib;
+	return lib;
+}
+
+static pg_icu_library *
+get_icu_library(Oid collid, const char *locale, const char *version)
+{
+	pg_icu_library *lib = NULL;
+
+	if (get_icu_library_hook != NULL)
+		lib = get_icu_library_hook(collid, locale, version);
+
+	if (!lib)
+		lib = get_builtin_icu_library();
+
+	return lib;
+}
+
+#endif
 
 /*
  * Construct a new pg_locale_t object.
@@ -1477,8 +1601,8 @@ report_newlocale_failure(const char *localename)
  * effectively permanent.
  */
 static pg_locale_t
-pg_newlocale(char provider, bool deterministic, const char *collate,
-			 const char *ctype, const char *version)
+pg_newlocale(Oid collid, char provider, bool deterministic,
+			 const char *collate, const char *ctype, const char *version)
 {
 	pg_locale_t result = MemoryContextAlloc(TopMemoryContext,
 											sizeof(struct pg_locale_struct));
@@ -1554,20 +1678,24 @@ pg_newlocale(char provider, bool deterministic, const char *collate,
 	{
 		UCollator  *collator;
 		UErrorCode	status;
+		pg_icu_library *iculib = get_icu_library(collid, collate, version);
+
+		Assert(strcmp(collate, ctype) == 0);
 
 		/* collator may be leaked if we encounter an error */
 
 		status = U_ZERO_ERROR;
-		collator = ucol_open(collate, &status);
+		collator = iculib->openCollator(collate, &status);
 		if (U_FAILURE(status))
 			ereport(ERROR,
 					(errmsg("could not open collator for locale \"%s\": %s",
-							collate, u_errorName(status))));
+							collate, iculib->errorName(status))));
 
 		if (U_ICU_VERSION_MAJOR_NUM < 54)
-			icu_set_collation_attributes(collator, collate);
+			icu_set_collation_attributes(iculib, collator, collate);
 
 		result->info.icu.ucol = collator;
+		result->info.icu.lib = iculib;
 	}
 #endif
 	else
@@ -1589,6 +1717,14 @@ pg_locale_deterministic(pg_locale_t locale)
 		return locale->deterministic;
 }
 
+#ifdef USE_ICU
+pg_icu_library *
+get_default_icu_library()
+{
+	return default_icu;
+}
+#endif
+
 /*
  * Initialize default database locale.
  */
@@ -1596,6 +1732,10 @@ void
 init_default_locale(char provider, const char *collate, const char *ctype,
 					const char *iculocale, const char *version)
 {
+#ifdef USE_ICU
+	default_icu = get_icu_library(DEFAULT_COLLATION_OID, iculocale, version);
+#endif
+
 	/*
 	 * For the purposes of pg_locale_t, if the provider is ICU, we use
 	 * iculocale for both collate and ctype.
@@ -1613,7 +1753,8 @@ init_default_locale(char provider, const char *collate, const char *ctype,
 	 * locales currently don't support pattern matching, which would break a
 	 * lot of things if applied globally.
 	 */
-	default_locale = pg_newlocale(provider, true, collate, ctype, version);
+	default_locale = pg_newlocale(DEFAULT_COLLATION_OID, provider, true,
+								  collate, ctype, version);
 }
 
 /*
@@ -1699,7 +1840,7 @@ pg_newlocale_from_collation(Oid collid)
 			/* shouldn't happen */
 			elog(ERROR, "unsupported collprovider: %c", collform->collprovider);
 
-		locale = pg_newlocale(collform->collprovider,
+		locale = pg_newlocale(collid, collform->collprovider,
 							  collform->collisdeterministic,
 							  collate, ctype, collversionstr);
 
@@ -1759,17 +1900,19 @@ get_collation_actual_version(char collprovider, const char *collcollate)
 		UErrorCode	status;
 		UVersionInfo versioninfo;
 		char		buf[U_MAX_VERSION_STRING_LENGTH];
+		pg_icu_library	*iculib = get_icu_library(InvalidOid, collcollate,
+												  NULL);
 
 		status = U_ZERO_ERROR;
-		collator = ucol_open(collcollate, &status);
+		collator = iculib->openCollator(collcollate, &status);
 		if (U_FAILURE(status))
 			ereport(ERROR,
 					(errmsg("could not open collator for locale \"%s\": %s",
-							collcollate, u_errorName(status))));
-		ucol_getVersion(collator, versioninfo);
-		ucol_close(collator);
+							collcollate, iculib->errorName(status))));
+		iculib->getCollatorVersion(collator, versioninfo);
+		iculib->closeCollator(collator);
 
-		u_versionToString(versioninfo, buf);
+		iculib->versionToString(versioninfo, buf);
 		collversion = pstrdup(buf);
 	}
 	else
@@ -2021,16 +2164,17 @@ pg_strncoll_icu_no_utf8(const char *arg1, size_t len1,
 	UChar	*uchar1,
 			*uchar2;
 	int		 result;
+	pg_icu_library *iculib = PG_ICU_LIB(locale);
 
 	Assert(locale->provider == COLLPROVIDER_ICU);
 #ifdef HAVE_UCOL_STRCOLLUTF8
 	Assert(GetDatabaseEncoding() != PG_UTF8);
 #endif
 
-	init_icu_converter();
+	init_icu_converter(iculib);
 
-	ulen1 = uchar_length(icu_converter, arg1, len1);
-	ulen2 = uchar_length(icu_converter, arg2, len2);
+	ulen1 = uchar_length(iculib, icu_converter, arg1, len1);
+	ulen2 = uchar_length(iculib, icu_converter, arg2, len2);
 
 	bufsize1 = (ulen1 + 1) * sizeof(UChar);
 	bufsize2 = (ulen2 + 1) * sizeof(UChar);
@@ -2041,12 +2185,12 @@ pg_strncoll_icu_no_utf8(const char *arg1, size_t len1,
 	uchar1 = (UChar *) buf;
 	uchar2 = (UChar *) (buf + bufsize1);
 
-	ulen1 = uchar_convert(icu_converter, uchar1, ulen1 + 1, arg1, len1);
-	ulen2 = uchar_convert(icu_converter, uchar2, ulen2 + 1, arg2, len2);
+	ulen1 = uchar_convert(iculib, icu_converter, uchar1, ulen1 + 1, arg1, len1);
+	ulen2 = uchar_convert(iculib, icu_converter, uchar2, ulen2 + 1, arg2, len2);
 
-	result = ucol_strcoll(locale->info.icu.ucol,
-						  uchar1, ulen1,
-						  uchar2, ulen2);
+	result = iculib->strcoll(locale->info.icu.ucol,
+							 uchar1, ulen1,
+							 uchar2, ulen2);
 
 	if (buf != sbuf)
 		pfree(buf);
@@ -2067,6 +2211,7 @@ pg_strncoll_icu(const char *arg1, size_t len1, const char *arg2, size_t len2,
 				pg_locale_t locale)
 {
 	int result;
+	pg_icu_library *iculib = PG_ICU_LIB(locale);
 
 	Assert(locale->provider == COLLPROVIDER_ICU);
 
@@ -2076,13 +2221,14 @@ pg_strncoll_icu(const char *arg1, size_t len1, const char *arg2, size_t len2,
 		UErrorCode	status;
 
 		status = U_ZERO_ERROR;
-		result = ucol_strcollUTF8(locale->info.icu.ucol,
-								  arg1, len1,
-								  arg2, len2,
-								  &status);
+		result = iculib->strcollUTF8(locale->info.icu.ucol,
+									 arg1, len1,
+									 arg2, len2,
+									 &status);
 		if (U_FAILURE(status))
 			ereport(ERROR,
-					(errmsg("collation failed: %s", u_errorName(status))));
+					(errmsg("collation failed: %s",
+							iculib->errorName(status))));
 	}
 	else
 #endif
@@ -2242,12 +2388,13 @@ pg_strnxfrm_icu(char *dest, const char *src, size_t srclen, size_t destsize,
 	int32_t	 ulen;
 	size_t   uchar_bsize;
 	Size	 result_bsize;
+	pg_icu_library *iculib = PG_ICU_LIB(locale);
 
 	Assert(locale->provider == COLLPROVIDER_ICU);
 
-	init_icu_converter();
+	init_icu_converter(iculib);
 
-	ulen = uchar_length(icu_converter, src, srclen);
+	ulen = uchar_length(iculib, icu_converter, src, srclen);
 
 	uchar_bsize = (ulen + 1) * sizeof(UChar);
 
@@ -2256,11 +2403,11 @@ pg_strnxfrm_icu(char *dest, const char *src, size_t srclen, size_t destsize,
 
 	uchar = (UChar *) buf;
 
-	ulen = uchar_convert(icu_converter, uchar, ulen + 1, src, srclen);
+	ulen = uchar_convert(iculib, icu_converter, uchar, ulen + 1, src, srclen);
 
-	result_bsize = ucol_getSortKey(locale->info.icu.ucol,
-								   uchar, ulen,
-								   (uint8_t *) dest, destsize);
+	result_bsize = iculib->getSortKey(locale->info.icu.ucol,
+									  uchar, ulen,
+									  (uint8_t *) dest, destsize);
 
 	if (buf != sbuf)
 		pfree(buf);
@@ -2289,13 +2436,14 @@ pg_strnxfrm_prefix_icu_no_utf8(char *dest, const char *src, size_t srclen,
 	UChar			*uchar = NULL;
 	size_t			 uchar_bsize;
 	Size			 result_bsize;
+	pg_icu_library	*iculib = PG_ICU_LIB(locale);
 
 	Assert(locale->provider == COLLPROVIDER_ICU);
 	Assert(GetDatabaseEncoding() != PG_UTF8);
 
-	init_icu_converter();
+	init_icu_converter(iculib);
 
-	ulen = uchar_length(icu_converter, src, srclen);
+	ulen = uchar_length(iculib, icu_converter, src, srclen);
 
 	uchar_bsize = (ulen + 1) * sizeof(UChar);
 
@@ -2304,21 +2452,19 @@ pg_strnxfrm_prefix_icu_no_utf8(char *dest, const char *src, size_t srclen,
 
 	uchar = (UChar *) buf;
 
-	ulen = uchar_convert(icu_converter, uchar, ulen + 1, src, srclen);
+	ulen = uchar_convert(iculib, icu_converter, uchar, ulen + 1, src, srclen);
 
 	uiter_setString(&iter, uchar, ulen);
 	state[0] = state[1] = 0;	/* won't need that again */
 	status = U_ZERO_ERROR;
-	result_bsize = ucol_nextSortKeyPart(locale->info.icu.ucol,
-										&iter,
-										state,
-										(uint8_t *) dest,
-										destsize,
-										&status);
+	result_bsize = iculib->nextSortKeyPart(
+		locale->info.icu.ucol, &iter, state,
+		(uint8_t *) dest, destsize, &status);
+
 	if (U_FAILURE(status))
 		ereport(ERROR,
 				(errmsg("sort key generation failed: %s",
-						u_errorName(status))));
+						iculib->errorName(status))));
 
 	return result_bsize;
 }
@@ -2327,6 +2473,7 @@ static size_t
 pg_strnxfrm_prefix_icu(char *dest, const char *src, size_t srclen,
 					   size_t destsize, pg_locale_t locale)
 {
+	pg_icu_library *iculib = PG_ICU_LIB(locale);
 	size_t result;
 
 	Assert(locale->provider == COLLPROVIDER_ICU);
@@ -2337,19 +2484,17 @@ pg_strnxfrm_prefix_icu(char *dest, const char *src, size_t srclen,
 		uint32_t	state[2];
 		UErrorCode	status;
 
-		uiter_setUTF8(&iter, src, srclen);
+		iculib->setUTF8(&iter, src, srclen);
 		state[0] = state[1] = 0;	/* won't need that again */
 		status = U_ZERO_ERROR;
-		result = ucol_nextSortKeyPart(locale->info.icu.ucol,
-									  &iter,
-									  state,
-									  (uint8_t *) dest,
-									  destsize,
-									  &status);
+		result = iculib->nextSortKeyPart(
+			locale->info.icu.ucol, &iter, state,
+			(uint8_t *) dest, destsize, &status);
+
 		if (U_FAILURE(status))
 			ereport(ERROR,
 					(errmsg("sort key generation failed: %s",
-							u_errorName(status))));
+							iculib->errorName(status))));
 	}
 	else
 		result = pg_strnxfrm_prefix_icu_no_utf8(dest, src, srclen, destsize,
@@ -2546,7 +2691,7 @@ pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
 
 #ifdef USE_ICU
 static void
-init_icu_converter(void)
+init_icu_converter(pg_icu_library *iculib)
 {
 	const char *icu_encoding_name;
 	UErrorCode	status;
@@ -2563,11 +2708,11 @@ init_icu_converter(void)
 						pg_encoding_to_char(GetDatabaseEncoding()))));
 
 	status = U_ZERO_ERROR;
-	conv = ucnv_open(icu_encoding_name, &status);
+	conv = iculib->openConverter(icu_encoding_name, &status);
 	if (U_FAILURE(status))
 		ereport(ERROR,
 				(errmsg("could not open ICU converter for encoding \"%s\": %s",
-						icu_encoding_name, u_errorName(status))));
+						icu_encoding_name, iculib->errorName(status))));
 
 	icu_converter = conv;
 }
@@ -2576,14 +2721,15 @@ init_icu_converter(void)
  * Find length, in UChars, of given string if converted to UChar string.
  */
 static size_t
-uchar_length(UConverter *converter, const char *str, size_t len)
+uchar_length(pg_icu_library *iculib, UConverter *converter, const char *str, size_t len)
 {
 	UErrorCode	status = U_ZERO_ERROR;
 	int32_t		ulen;
-	ulen = ucnv_toUChars(converter, NULL, 0, str, len, &status);
+	ulen = iculib->toUChars(converter, NULL, 0, str, len, &status);
 	if (U_FAILURE(status) && status != U_BUFFER_OVERFLOW_ERROR)
 		ereport(ERROR,
-				(errmsg("%s failed: %s", "ucnv_toUChars", u_errorName(status))));
+				(errmsg("%s failed: %s", "ucnv_toUChars",
+						iculib->errorName(status))));
 	return ulen;
 }
 
@@ -2592,16 +2738,17 @@ uchar_length(UConverter *converter, const char *str, size_t len)
  * return the length (in UChars).
  */
 static int32_t
-uchar_convert(UConverter *converter, UChar *dest, int32_t destlen,
-			  const char *src, size_t srclen)
+uchar_convert(pg_icu_library *iculib, UConverter *converter, UChar *dest,
+			  int32_t destlen, const char *src, size_t srclen)
 {
 	UErrorCode	status = U_ZERO_ERROR;
 	int32_t		ulen;
 	status = U_ZERO_ERROR;
-	ulen = ucnv_toUChars(converter, dest, destlen, src, srclen, &status);
+	ulen = iculib->toUChars(converter, dest, destlen, src, srclen, &status);
 	if (U_FAILURE(status))
 		ereport(ERROR,
-				(errmsg("%s failed: %s", "ucnv_toUChars", u_errorName(status))));
+				(errmsg("%s failed: %s", "ucnv_toUChars",
+						iculib->errorName(status))));
 	return ulen;
 }
 
@@ -2618,16 +2765,17 @@ uchar_convert(UConverter *converter, UChar *dest, int32_t destlen,
  * result length instead.
  */
 int32_t
-icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes)
+icu_to_uchar(pg_icu_library *iculib, UChar **buff_uchar, const char *buff,
+			 size_t nbytes)
 {
 	int32_t len_uchar;
 
-	init_icu_converter();
+	init_icu_converter(iculib);
 
-	len_uchar = uchar_length(icu_converter, buff, nbytes);
+	len_uchar = uchar_length(iculib, icu_converter, buff, nbytes);
 
 	*buff_uchar = palloc((len_uchar + 1) * sizeof(**buff_uchar));
-	len_uchar = uchar_convert(icu_converter,
+	len_uchar = uchar_convert(iculib, icu_converter,
 							  *buff_uchar, len_uchar + 1, buff, nbytes);
 
 	return len_uchar;
@@ -2645,30 +2793,32 @@ icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes)
  * The result string is nul-terminated.
  */
 int32_t
-icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
+icu_from_uchar(pg_icu_library *iculib, char **result, const UChar *buff_uchar,
+			   int32_t len_uchar)
 {
 	UErrorCode	status;
 	int32_t		len_result;
 
-	init_icu_converter();
+	init_icu_converter(iculib);
 
 	status = U_ZERO_ERROR;
-	len_result = ucnv_fromUChars(icu_converter, NULL, 0,
-								 buff_uchar, len_uchar, &status);
+	len_result = iculib->fromUChars(icu_converter, NULL, 0,
+									buff_uchar, len_uchar, &status);
 	if (U_FAILURE(status) && status != U_BUFFER_OVERFLOW_ERROR)
 		ereport(ERROR,
 				(errmsg("%s failed: %s", "ucnv_fromUChars",
-						u_errorName(status))));
+						iculib->errorName(status))));
 
 	*result = palloc(len_result + 1);
 
 	status = U_ZERO_ERROR;
-	len_result = ucnv_fromUChars(icu_converter, *result, len_result + 1,
-								 buff_uchar, len_uchar, &status);
+	len_result = iculib->fromUChars(icu_converter, *result,
+									len_result + 1, buff_uchar,
+									len_uchar, &status);
 	if (U_FAILURE(status))
 		ereport(ERROR,
 				(errmsg("%s failed: %s", "ucnv_fromUChars",
-						u_errorName(status))));
+						iculib->errorName(status))));
 
 	return len_result;
 }
@@ -2684,7 +2834,8 @@ icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
  */
 pg_attribute_unused()
 static void
-icu_set_collation_attributes(UCollator *collator, const char *loc)
+icu_set_collation_attributes(pg_icu_library *iculib, UCollator *collator,
+							 const char *loc)
 {
 	char	   *str = asc_tolower(loc, strlen(loc));
 
@@ -2758,7 +2909,7 @@ icu_set_collation_attributes(UCollator *collator, const char *loc)
 				status = U_ILLEGAL_ARGUMENT_ERROR;
 
 			if (status == U_ZERO_ERROR)
-				ucol_setAttribute(collator, uattr, uvalue, &status);
+				iculib->setAttribute(collator, uattr, uvalue, &status);
 
 			/*
 			 * Pretend the error came from ucol_open(), for consistent error
@@ -2767,7 +2918,7 @@ icu_set_collation_attributes(UCollator *collator, const char *loc)
 			if (U_FAILURE(status))
 				ereport(ERROR,
 						(errmsg("could not open collator for locale \"%s\": %s",
-								loc, u_errorName(status))));
+								loc, iculib->errorName(status))));
 		}
 	}
 }
@@ -2783,17 +2934,18 @@ check_icu_locale(const char *icu_locale)
 #ifdef USE_ICU
 	UCollator	*collator;
 	UErrorCode   status;
+	pg_icu_library *iculib = get_icu_library(InvalidOid, icu_locale, NULL);
 
 	status = U_ZERO_ERROR;
-	collator = ucol_open(icu_locale, &status);
+	collator = iculib->openCollator(icu_locale, &status);
 	if (U_FAILURE(status))
 		ereport(ERROR,
 				(errmsg("could not open collator for locale \"%s\": %s",
-						icu_locale, u_errorName(status))));
+						icu_locale, iculib->errorName(status))));
 
 	if (U_ICU_VERSION_MAJOR_NUM < 54)
-		icu_set_collation_attributes(collator, icu_locale);
-	ucol_close(collator);
+		icu_set_collation_attributes(iculib, collator, icu_locale);
+	iculib->closeCollator(collator);
 #else
 	ereport(ERROR,
 			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
diff --git a/src/include/utils/pg_locale_internal.h b/src/include/utils/pg_locale_internal.h
index 33465ad92d..b600604585 100644
--- a/src/include/utils/pg_locale_internal.h
+++ b/src/include/utils/pg_locale_internal.h
@@ -14,6 +14,8 @@
 #define _PG_LOCALE_INTERNAL_
 
 #ifdef USE_ICU
+#include <unicode/ubrk.h>
+#include <unicode/ucnv.h>
 #include <unicode/ucol.h>
 #endif
 
@@ -30,6 +32,108 @@
 #endif
 #endif
 
+#ifdef USE_ICU
+/*
+ * An ICU library version that we're either linked against or have loaded at
+ * runtime.
+ */
+typedef struct pg_icu_library
+{
+	int			major_version;
+	int			minor_version;
+	char	   *libicui18n_name;
+	char	   *libicuuc_name;
+	void		(*getICUVersion) (UVersionInfo info);
+	void		(*getUnicodeVersion) (UVersionInfo into);
+	void		(*getCLDRVersion) (UVersionInfo info, UErrorCode *status);
+	UCollator  *(*openCollator) (const char *loc, UErrorCode *status);
+	void		(*closeCollator) (UCollator *coll);
+	void		(*getCollatorVersion) (const UCollator *coll, UVersionInfo info);
+	void		(*getUCAVersion) (const UCollator *coll, UVersionInfo info);
+	void		(*versionToString) (const UVersionInfo versionArray,
+									char *versionString);
+	UCollationResult (*strcoll) (const UCollator *coll,
+								 const UChar *source,
+								 int32_t sourceLength,
+								 const UChar *target,
+								 int32_t targetLength);
+	UCollationResult (*strcollUTF8) (const UCollator *coll,
+									 const char *source,
+									 int32_t sourceLength,
+									 const char *target,
+									 int32_t targetLength,
+									 UErrorCode *status);
+	int32_t		(*getSortKey) (const UCollator *coll,
+							   const UChar *source,
+							   int32_t sourceLength,
+							   uint8_t *result,
+							   int32_t resultLength);
+	int32_t		(*nextSortKeyPart) (const UCollator *coll,
+									UCharIterator *iter,
+									uint32_t state[2],
+									uint8_t *dest,
+									int32_t count,
+									UErrorCode *status);
+	void		(*setUTF8) (UCharIterator *iter,
+							const char *s,
+							int32_t length);
+	const char *(*errorName) (UErrorCode code);
+	int32_t		(*strToUpper) (UChar *dest,
+							   int32_t destCapacity,
+							   const UChar *src,
+							   int32_t srcLength,
+							   const char *locale,
+							   UErrorCode *pErrorCode);
+	int32_t		(*strToLower) (UChar *dest,
+							   int32_t destCapacity,
+							   const UChar *src,
+							   int32_t srcLength,
+							   const char *locale,
+							   UErrorCode *pErrorCode);
+	int32_t		(*strToTitle) (UChar *dest,
+							   int32_t destCapacity,
+							   const UChar *src,
+							   int32_t srcLength,
+							   UBreakIterator *titleIter,
+							   const char *locale,
+							   UErrorCode *pErrorCode);
+	void		(*setAttribute) (UCollator *coll,
+								 UColAttribute attr,
+								 UColAttributeValue value,
+								 UErrorCode *status);
+	UConverter *(*openConverter) (const char *converterName,
+								  UErrorCode *  	err);
+	void		(*closeConverter) (UConverter *converter);
+	int32_t		(*fromUChars) (UConverter *cnv,
+							   char *dest,
+							   int32_t destCapacity,
+							   const UChar *src,
+							   int32_t srcLength,
+							   UErrorCode *pErrorCode);
+	int32_t		(*toUChars) (UConverter *cnv,
+							 UChar *dest,
+							 int32_t destCapacity,
+							 const char *src,
+							 int32_t srcLength,
+							 UErrorCode *pErrorCode);
+	int32_t		(*toLanguageTag) (const char *localeID,
+								  char *langtag,
+								  int32_t langtagCapacity,
+								  UBool strict,
+								  UErrorCode *err);
+	int32_t		(*getDisplayName) (const char *localeID,
+								   const char *inLocaleID,
+								   UChar *result,
+								   int32_t maxResultSize,
+								   UErrorCode *err);
+	int32_t		(*countAvailable) (void);
+	const char *(*getAvailable) (int32_t n);
+} pg_icu_library;
+
+#define PG_ICU_LIB(x) ((x)->info.icu.lib)
+
+#endif
+
 /*
  * We define our own wrapper around locale_t so we can keep the same
  * function signatures for all builds, while not having to create a
@@ -53,7 +157,8 @@ struct pg_locale_struct
 #ifdef USE_ICU
 		struct
 		{
-			UCollator	*ucol;
+			UCollator		*ucol;
+			pg_icu_library	*lib;
 		}			icu;
 #endif
 		int			dummy;		/* in case we have neither LOCALE_T nor ICU */
@@ -61,8 +166,19 @@ struct pg_locale_struct
 };
 
 #ifdef USE_ICU
-extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
-extern int32_t icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar);
-#endif
+
+typedef pg_icu_library *(*get_icu_library_hook_type)(
+	Oid collid, const char *locale, const char *version);
+
+extern PGDLLIMPORT get_icu_library_hook_type get_icu_library_hook;
+
+extern pg_icu_library *get_default_icu_library(void);
+extern pg_icu_library *get_builtin_icu_library(void);
+extern int32_t icu_to_uchar(pg_icu_library *lib, UChar **buff_uchar,
+							const char *buff, size_t nbytes);
+extern int32_t icu_from_uchar(pg_icu_library *lib, char **result,
+							  const UChar *buff_uchar, int32_t len_uchar);
+
+#endif							/* USE_ICU */
 
 #endif							/* _PG_LOCALE_INTERNAL_ */
-- 
2.34.1

---

From: vignesh C <[email-masked]>
Date: Sat, 20 Jan 2024 07:40:44 +0530

> hook to solve at least some problems.

This thread has been idle for a year now, It has stalled after a lot
of discussion.
@Jeff Davis: Do you want to try to restart the discussion by posting
an updated version and see what happens?

Regards,
Vignesh

---

From: Jeff Davis <[email-masked]>
Date: Sun, 21 Jan 2024 10:57:57 -0800

> an updated version and see what happens?

Thank you for following up. Yes, I'd like to find a path forward here,
but I need some validation from others on my approach.

I rendered the docs I wrote as an HTML page and attached it to this
thread, to make it easier for others to read and comment. It's
basically a tool for experts who are willing to devote effort to
managing their collations and ICU libraries. Is that what we want?

At an implementation level, did I get the extension APIs right? I
considered making the API simpler, but that would require the extension
to do quite a bit more work (including a lot of redundant work) to use
ICU properly.

Regards,
	Jeff Davis

---

From: vignesh C <[email-masked]>
Date: Thu, 1 Feb 2024 20:55:22 +0530

> but I need some validation from others on my approach.

Let's start by posting a rebased version to fix the CFBot patch apply
issue as in [1]:

=== Applying patches on top of PostgreSQL commit ID
402388946fb3ac54f0fd5944d7e177ef7737eab2 ===
=== applying patch
./v8-0001-Support-multiple-ICU-collation-provider-libraries.patch
patching file src/backend/commands/collationcmds.c
Hunk #1 FAILED at 566.
....
1 out of 4 hunks FAILED -- saving rejects to file
src/backend/commands/collationcmds.c.rej
patching file src/backend/utils/adt/formatting.c
Hunk #1 succeeded at 1575 (offset 9 lines).
Hunk #2 succeeded at 1587 (offset 9 lines).
Hunk #3 succeeded at 1605 (offset 9 lines).
Hunk #4 succeeded at 1700 (offset 3 lines).
Hunk #5 succeeded at 1819 (offset -1 lines).
Hunk #6 succeeded at 1939 (offset -5 lines).
patching file src/backend/utils/adt/pg_locale.c
Hunk #1 FAILED at 70.
...
Hunk #31 FAILED at 2886.
Hunk #32 FAILED at 2902.
22 out of 32 hunks FAILED -- saving rejects to file
src/backend/utils/adt/pg_locale.c.rej

[1] - http://cfbot.cputube.org/patch_46_3956.log

Regards,
Vignesh

---

From: Robert Haas <[email-masked]>
Date: Thu, 1 Feb 2024 15:58:01 -0500

> to do quite a bit more work (including a lot of redundant work) to use

Not that I'm the most qualified person to have an opinion on this
topic, but did you intend to attach this stuff to this email, or is it
somewhere else?

-- 
Robert Haas
EDB: http://www.enterprisedb.com

---

From: Jeff Davis <[email-masked]>
Date: Sun, 04 Feb 2024 09:12:28 -0800

> topic, but did you intend to attach this stuff to this email, or is

The previous patch is here:

https://www.postgresql.org/message-id/6f4a8c01a5cb1edf3a07d204c371fbddaef252f9.camel%40j-davis.com

And I attached the rendered HTML doc page, which conveniently renders
in the archives (thanks to web team -- I didn't know if that would
actually work until I tried it):

https://www.postgresql.org/message-id/attachment/142818/icu-multilib.html

For anyone interested in this work, the docs are the best place to
start.

I'm hesitant to put much more work into it (e.g. new patches, etc.)
without more feedback. Your opinion would certainly be valuable -- for
instance, when reading the docs, can you imagine yourself actually
using this if you ran into a collation versioning/migration problem?

Regards,
	Jeff Davis

---

From: Robert Haas <[email-masked]>
Date: Sun, 11 Feb 2024 22:04:11 +0530

> using this if you ran into a collation versioning/migration problem?

I'm having some difficulty understanding what the docs are trying to
tell me. I think there are some issues with ordering and pacing.

"The icu_multilib module provides control over the version (or
versions) of the ICU provider library used by PostgreSQL, which can be
different from the version of ICU with which it was built. Collations
are a product of natural language, and natural language evolves over
time; but PostgreSQL depends on stable ordering for structures such as
indexes. Newer versions of ICU update the provided collators to adapt
to changes in natural language, so it's important to control when and
how those new versions of ICU are used to prevent problems such as
index corruption."

Check. So far, so good.

"This module assumes that the necessary versions of ICU are already
available, such as through the operating system's package manager; and
already properly installed in a single location accessible to
PostgreSQL. The configration variable icu_multilib.library_path should
be set to the location where these ICU library versions are
installed."

Here I feel we've skipped a few steps. I suggest postponing all
discussion of specific GUCs to a later point -- specifically the
configuration parameters section, which I think should actually be
F.19.1, with the use cases following that rather than preceding it. In
this introductory section, I suggest elaborating a bit more on what
problem we're trying to solve at a conceptual level. It feels like
we've gone straight from the very general issue (collation definitions
need to be stable but language isn't) to very specific (here's a GUC
that you can set to a pathname). I feel like the need for this module
should be more specifically motivated. Maybe something like:

1. Here's what we think your OS package manager is probably going to do.
2. That's going to interact with PostgreSQL in this way that I will
now describe.
3. See, that sucks, because of the stuff I said above about needing
stable collations!
4. But if you installed this module instead, then you could prevent
the things I said under #2 from happening.
5. Instead, you'd get this other behavior, which would make you happy.

I feel like I can almost piece together in my head how this is
supposed to work -- I think it's like "we expect the OS package
manager to drop all the ICU versions in the same directory via side by
side installs, and that works well for other programs because ... for
some mysterious reason they can latch onto the specific version they
were linked against ... but we can't or don't do that because ... I
guess we're dumber than those other pieces of software or
something???? ... so this module lets you ask for more sensible
behavior." But I think that could be spelled out a bit more clearly
and directly than this document seems to me to do.

I also wonder if we should be explaining why we don't get this right
out of the box. Like, if the normal behavior categorically sucks, why
do you have to install icu_multilib to get something else? Why not
make the multilib treatment the default? And if the normal behavior is
better for some cases and the icu_multilib behavior is better for
other cases, then maybe we ought to explain which one to use in which
scenario.

"icu_multilib must be loaded via shared_preload_libraries.
icu_multilib ignores any ICU library with a major version greater than
that with which PostgreSQL was built."

It's not clear from reading this whether the second sentence here is a
regrettable implementation restriction or design behavior. If it's
design behavior, what's the point of it?

---

From: Jeff Davis <[email-masked]>
Date: Mon, 12 Feb 2024 12:25:09 -0800

> 5. Instead, you'd get this other behavior, which would make you

I like that framing, thank you. I'll try to come up with something
there.

> something???? ... 

Postgres can and does latch on to the version of ICU it was compiled
against. It's a normal shared library dependency.

The problem is that databases -- and the file structures -- outlive a
particular version of Postgres. So if Postgres 16 is compiled against
ICU X and Postgres 17 is compiled against ICU Y, how do you upgrade
from 16 to 17? Postgres 17 will try to access the old file structures
using ICU Y, and they'll be corrupt.

What we want is the file structures that depend on ICU X to continue to
find ICU X even after you upgrade to Postgres 17, yet allow new
structures to be created using ICU Y. In other words, "multi-lib",
meaning that the same Postgres binary is linking to multiple versions
of ICU and the different versions for different structures. That would
allow users to recreate one index at a time to use ICU Y, until nothing
depends on ICU X any longer.

I should say this is not an easy process even if something like
icu_multilib is available. We don't have all of the information needed
in the catalog to track which structures depend on which versions of a
collation library, collation library version is itself not easy to
define, and it still involves rebuilding (or at least re-validating) a
lot of structures. This is a "make hard things possible" tool, and I
suspect only a handful of users would use it successfully to migrate to
new ICU versions.

More simply, some users might just want to lock down the version of ICU
to X, and just use that forever until they have a reason to change it.
icu_multilib can also facilitate that, though it's still not trivial.

> design behavior, what's the point of it?

That restriction came from Thomas's (uncommitted) work on the same
problem. I believe the reasoning was that we don't know whether future
versions of ICU might break something that we're doing, though perhaps
there's a better way.

Regards,
	Jeff Davis

---

From: Robert Haas <[email-masked]>
Date: Tue, 13 Feb 2024 11:43:46 +0530

> depends on ICU X any longer.

Ah, I see. At least, I think I do. I think some of this material could
be very usefully included into the first section of the doc you're
trying to write. What you say here makes it a lot easier to grasp the
motivation and use case for this code, at least for me.