pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Jeff Davis <pgsql@j-davis.com>
To: Andreas Karlsson <andreas@proxel.se>
To: pgsql-hackers@postgresql.org
Subject: Re: Collation & ctype method table, and extension hooks
Date: Tue, 19 Nov 2024 13:32:47 -0800
Message-ID: <78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com> (raw )
In-Reply-To: <247f18f3-09e5-480d-9222-9d0d9684995f@proxel.se >
References: <2830211e1b6e6a2e26d845780b03e125281ea17b.camel@j-davis.com >
<e9cb235c-c820-49c2-a89e-ab38d7910a54@proxel.se >
<fecea00b4ad9b9a2426cc34e78f30a0c75406ca7.camel@j-davis.com >
<59da7ee4-5e1a-4727-b464-a603c6ed84cd@proxel.se >
<dfff1ca84dd0e8ee03aa2478412134895b0b9681.camel@j-davis.com >
<247f18f3-09e5-480d-9222-9d0d9684995f@proxel.se >
On Fri, 2024-11-01 at 14:08 +0100, Andreas Karlsson wrote:
> I think adding such a small file would make life easier for people
> new
> to the collation part of the code base. It would be a nice symmetry
> between collation providers and where code for them can be found.
Done.
> >
> For me combining them would make the intention of the code easier to
> understand since aren't the casemap functions just a set of
> "ctype_methods"?
Done.
There is a bit of weirdness in libc because:
* Single byte encodings use the single-byte isupper(), toupper(), etc.
* UTF8 encoding uses wide character iswupper(), towupper(), etc.
* Non-UTF8 multibyte encodings use isupper() for pattern matching but
towupper() for case mapping
that weirdness existed before, but it's a bit more obvious what's
happening now.
> > > This commit makes me tempted to handle the ctype_is_c logic for
> > > character classes also in callbacks and remove the if in
> > > functions
> > > like
> > > pg_wc_ispunct(). But this si something that would need to be
> > > benchmarked.
I like this idea, but it can be a follow up.
Attached new patchset.
I also tried some performance tests again. I used smalltext (a table of
10M ~30-character strings) and bigtext (a table of 32768 rows, each
containing the 100KiB source of https://en.wikipedia.org/wiki/Diacritic
). And I then ran the following regex on each:
select count(*) from thetable
where t ~
'[[:digit:]][[:space:]][[:punct:]][[:alpha:]][[:lower:]][[:upper:]]';
for "C", "en_US", and "en-US-x-icu". The timings for smalltext were
indistinguishable between master and the patched version. The timings
for bigtext were pretty noisy so it's hard to tell if there was a
regression or not, but I saw some evidence in the profile that
char_properties has a cost (~1%). I'm not sure if that's a significant
concern or not.
Which API do you think is the right one? Individual functions testing
individual properties, or something like char_properties() that can
test several at once?
Regards,
Jeff Davis
Attachments:
[text/x-patch] v8-0001-Perform-provider-specific-initialization-code-in-.patch (18.5K, ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/2-v8-0001-Perform-provider-specific-initialization-code-in-.patch)
download | inline diff:
From 99f7ab6ccaf6a23e1fccd6eb0c20f2292088155d Mon Sep 17 00:00:00 2001
From: Jeff Davis <jeff@j-davis.com>
Date: Wed, 25 Sep 2024 15:49:32 -0700
Subject: [PATCH v8 1/7] Perform provider-specific initialization code in new
functions.
---
src/backend/utils/adt/Makefile | 1 +
src/backend/utils/adt/meson.build | 1 +
src/backend/utils/adt/pg_locale.c | 155 +++-------------------
src/backend/utils/adt/pg_locale_builtin.c | 70 ++++++++++
src/backend/utils/adt/pg_locale_icu.c | 97 +++++++++++++-
src/backend/utils/adt/pg_locale_libc.c | 74 ++++++++++-
6 files changed, 254 insertions(+), 144 deletions(-)
create mode 100644 src/backend/utils/adt/pg_locale_builtin.c
diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile
index 85e5eaf32e..35e8c01aab 100644
--- a/src/backend/utils/adt/Makefile
+++ b/src/backend/utils/adt/Makefile
@@ -79,6 +79,7 @@ OBJS = \
orderedsetaggs.o \
partitionfuncs.o \
pg_locale.o \
+ pg_locale_builtin.o \
pg_locale_icu.o \
pg_locale_libc.o \
pg_lsn.o \
diff --git a/src/backend/utils/adt/meson.build b/src/backend/utils/adt/meson.build
index f73f294b8f..e86d6dc8e0 100644
--- a/src/backend/utils/adt/meson.build
+++ b/src/backend/utils/adt/meson.build
@@ -66,6 +66,7 @@ backend_sources += files(
'orderedsetaggs.c',
'partitionfuncs.c',
'pg_locale.c',
+ 'pg_locale_builtin.c',
'pg_locale_icu.c',
'pg_locale_libc.c',
'pg_lsn.c',
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index d4e89663ec..ec5f509c4e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -92,8 +92,6 @@
/* pg_locale_icu.c */
#ifdef USE_ICU
extern UCollator *pg_ucol_open(const char *loc_str);
-extern UCollator *make_icu_collator(const char *iculocstr,
- const char *icurules);
extern int strncoll_icu(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
@@ -106,8 +104,9 @@ extern size_t strnxfrm_prefix_icu(char *dest, size_t destsize,
#endif
/* pg_locale_libc.c */
-extern locale_t make_libc_collator(const char *collate,
- const char *ctype);
+extern pg_locale_t create_pg_locale_builtin(Oid collid, MemoryContext context);
+extern pg_locale_t create_pg_locale_icu(Oid collid, MemoryContext context);
+extern pg_locale_t create_pg_locale_libc(Oid collid, MemoryContext context);
extern int strncoll_libc(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
@@ -138,7 +137,7 @@ char *localized_full_months[12 + 1];
/* is the databases's LC_CTYPE the C locale? */
bool database_ctype_is_c = false;
-static struct pg_locale_struct default_locale;
+static pg_locale_t default_locale = NULL;
/* indicates whether locale information cache is valid */
static bool CurrentLocaleConvValid = false;
@@ -1213,7 +1212,6 @@ IsoLocaleName(const char *winlocname)
#endif /* WIN32 && LC_MESSAGES */
-
/*
* Create a new pg_locale_t struct for the given collation oid.
*/
@@ -1226,75 +1224,17 @@ create_pg_locale(Oid collid, MemoryContext context)
Datum datum;
bool isnull;
- result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
-
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for collation %u", collid);
collform = (Form_pg_collation) GETSTRUCT(tp);
- result->provider = collform->collprovider;
- result->deterministic = collform->collisdeterministic;
-
if (collform->collprovider == COLLPROVIDER_BUILTIN)
- {
- const char *locstr;
-
- datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
- locstr = TextDatumGetCString(datum);
-
- result->collate_is_c = true;
- result->ctype_is_c = (strcmp(locstr, "C") == 0);
-
- builtin_validate_locale(GetDatabaseEncoding(), locstr);
-
- result->info.builtin.locale = MemoryContextStrdup(context,
- locstr);
- }
+ result = create_pg_locale_builtin(collid, context);
else if (collform->collprovider == COLLPROVIDER_ICU)
- {
-#ifdef USE_ICU
- const char *iculocstr;
- const char *icurules;
-
- datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
- iculocstr = TextDatumGetCString(datum);
-
- result->collate_is_c = false;
- result->ctype_is_c = false;
-
- datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collicurules, &isnull);
- if (!isnull)
- icurules = TextDatumGetCString(datum);
- else
- icurules = NULL;
-
- result->info.icu.locale = MemoryContextStrdup(context, iculocstr);
- result->info.icu.ucol = make_icu_collator(iculocstr, icurules);
-#else
- /* could get here if a collation was created by a build with ICU */
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("ICU is not supported in this build")));
-#endif
- }
+ result = create_pg_locale_icu(collid, context);
else if (collform->collprovider == COLLPROVIDER_LIBC)
- {
- const char *collcollate;
- const char *collctype;
-
- datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_collcollate);
- collcollate = TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_collctype);
- collctype = TextDatumGetCString(datum);
-
- result->collate_is_c = (strcmp(collcollate, "C") == 0) ||
- (strcmp(collcollate, "POSIX") == 0);
- result->ctype_is_c = (strcmp(collctype, "C") == 0) ||
- (strcmp(collctype, "POSIX") == 0);
-
- result->info.lt = make_libc_collator(collcollate, collctype);
- }
+ result = create_pg_locale_libc(collid, context);
else
/* shouldn't happen */
PGLOCALE_SUPPORT_ERROR(collform->collprovider);
@@ -1354,7 +1294,9 @@ init_database_collation(void)
{
HeapTuple tup;
Form_pg_database dbform;
- Datum datum;
+ pg_locale_t result;
+
+ Assert(default_locale == NULL);
/* Fetch our pg_database row normally, via syscache */
tup = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
@@ -1363,80 +1305,21 @@ init_database_collation(void)
dbform = (Form_pg_database) GETSTRUCT(tup);
if (dbform->datlocprovider == COLLPROVIDER_BUILTIN)
- {
- char *datlocale;
-
- datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
- datlocale = TextDatumGetCString(datum);
-
- builtin_validate_locale(dbform->encoding, datlocale);
-
- default_locale.collate_is_c = true;
- default_locale.ctype_is_c = (strcmp(datlocale, "C") == 0);
-
- default_locale.info.builtin.locale = MemoryContextStrdup(
- TopMemoryContext, datlocale);
- }
+ result = create_pg_locale_builtin(DEFAULT_COLLATION_OID,
+ TopMemoryContext);
else if (dbform->datlocprovider == COLLPROVIDER_ICU)
- {
-#ifdef USE_ICU
- char *datlocale;
- char *icurules;
- bool isnull;
-
- datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
- datlocale = TextDatumGetCString(datum);
-
- default_locale.collate_is_c = false;
- default_locale.ctype_is_c = false;
-
- datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_daticurules, &isnull);
- if (!isnull)
- icurules = TextDatumGetCString(datum);
- else
- icurules = NULL;
-
- default_locale.info.icu.locale = MemoryContextStrdup(TopMemoryContext, datlocale);
- default_locale.info.icu.ucol = make_icu_collator(datlocale, icurules);
-#else
- /* could get here if a collation was created by a build with ICU */
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("ICU is not supported in this build")));
-#endif
- }
+ result = create_pg_locale_icu(DEFAULT_COLLATION_OID,
+ TopMemoryContext);
else if (dbform->datlocprovider == COLLPROVIDER_LIBC)
- {
- const char *datcollate;
- const char *datctype;
-
- datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datcollate);
- datcollate = TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datctype);
- datctype = TextDatumGetCString(datum);
-
- default_locale.collate_is_c = (strcmp(datcollate, "C") == 0) ||
- (strcmp(datcollate, "POSIX") == 0);
- default_locale.ctype_is_c = (strcmp(datctype, "C") == 0) ||
- (strcmp(datctype, "POSIX") == 0);
-
- default_locale.info.lt = make_libc_collator(datcollate, datctype);
- }
+ result = create_pg_locale_libc(DEFAULT_COLLATION_OID,
+ TopMemoryContext);
else
/* shouldn't happen */
PGLOCALE_SUPPORT_ERROR(dbform->datlocprovider);
-
- default_locale.provider = dbform->datlocprovider;
-
- /*
- * Default locale is currently always deterministic. Nondeterministic
- * locales currently don't support pattern matching, which would break a
- * lot of things if applied globally.
- */
- default_locale.deterministic = true;
-
ReleaseSysCache(tup);
+
+ default_locale = result;
}
/*
@@ -1454,7 +1337,7 @@ pg_newlocale_from_collation(Oid collid)
bool found;
if (collid == DEFAULT_COLLATION_OID)
- return &default_locale;
+ return default_locale;
if (!OidIsValid(collid))
elog(ERROR, "cache lookup failed for collation %u", collid);
diff --git a/src/backend/utils/adt/pg_locale_builtin.c b/src/backend/utils/adt/pg_locale_builtin.c
new file mode 100644
index 0000000000..4246971a4d
--- /dev/null
+++ b/src/backend/utils/adt/pg_locale_builtin.c
@@ -0,0 +1,70 @@
+/*-----------------------------------------------------------------------
+ *
+ * PostgreSQL locale utilities for builtin provider
+ *
+ * Portions Copyright (c) 2002-2024, PostgreSQL Global Development Group
+ *
+ * src/backend/utils/adt/pg_locale_builtin.c
+ *
+ *-----------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "catalog/pg_database.h"
+#include "catalog/pg_collation.h"
+#include "mb/pg_wchar.h"
+#include "miscadmin.h"
+#include "utils/builtins.h"
+#include "utils/memutils.h"
+#include "utils/pg_locale.h"
+#include "utils/syscache.h"
+
+extern pg_locale_t create_pg_locale_builtin(Oid collid,
+ MemoryContext context);
+
+pg_locale_t
+create_pg_locale_builtin(Oid collid, MemoryContext context)
+{
+ const char *locstr;
+ pg_locale_t result;
+
+ if (collid == DEFAULT_COLLATION_OID)
+ {
+ HeapTuple tp;
+ Datum datum;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp,
+ Anum_pg_database_datlocale);
+ locstr = TextDatumGetCString(datum);
+ ReleaseSysCache(tp);
+ }
+ else
+ {
+ HeapTuple tp;
+ Datum datum;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", collid);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp,
+ Anum_pg_collation_colllocale);
+ locstr = TextDatumGetCString(datum);
+ ReleaseSysCache(tp);
+ }
+
+ builtin_validate_locale(GetDatabaseEncoding(), locstr);
+
+ result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
+
+ result->info.builtin.locale = MemoryContextStrdup(context, locstr);
+ result->provider = COLLPROVIDER_BUILTIN;
+ result->deterministic = true;
+ result->collate_is_c = true;
+ result->ctype_is_c = (strcmp(locstr, "C") == 0);
+
+ return result;
+}
diff --git a/src/backend/utils/adt/pg_locale_icu.c b/src/backend/utils/adt/pg_locale_icu.c
index 2a87e25dfb..73eb430d75 100644
--- a/src/backend/utils/adt/pg_locale_icu.c
+++ b/src/backend/utils/adt/pg_locale_icu.c
@@ -12,14 +12,20 @@
#include "postgres.h"
#ifdef USE_ICU
-
#include <unicode/ucnv.h>
#include <unicode/ustring.h>
+#endif
+#include "access/htup_details.h"
+#include "catalog/pg_database.h"
#include "catalog/pg_collation.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/formatting.h"
+#include "utils/memutils.h"
#include "utils/pg_locale.h"
+#include "utils/syscache.h"
/*
* Size of stack buffer to use for string transformations, used to avoid heap
@@ -29,9 +35,11 @@
*/
#define TEXTBUFLEN 1024
+extern pg_locale_t create_pg_locale_icu(Oid collid, MemoryContext context);
+
+#ifdef USE_ICU
+
extern UCollator *pg_ucol_open(const char *loc_str);
-extern UCollator *make_icu_collator(const char *iculocstr,
- const char *icurules);
extern int strncoll_icu(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
@@ -49,6 +57,8 @@ extern size_t strnxfrm_prefix_icu(char *dest, size_t destsize,
*/
static UConverter *icu_converter = NULL;
+static UCollator *make_icu_collator(const char *iculocstr,
+ const char *icurules);
static int strncoll_icu_no_utf8(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
@@ -63,6 +73,85 @@ static int32_t uchar_convert(UConverter *converter,
const char *src, int32_t srclen);
static void icu_set_collation_attributes(UCollator *collator, const char *loc,
UErrorCode *status);
+#endif
+
+pg_locale_t
+create_pg_locale_icu(Oid collid, MemoryContext context)
+{
+#ifdef USE_ICU
+ bool deterministic;
+ const char *iculocstr;
+ const char *icurules = NULL;
+ UCollator *collator;
+ pg_locale_t result;
+
+ if (collid == DEFAULT_COLLATION_OID)
+ {
+ HeapTuple tp;
+ Datum datum;
+ bool isnull;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+
+ /* default database collation is always deterministic */
+ deterministic = true;
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp,
+ Anum_pg_database_datlocale);
+ iculocstr = TextDatumGetCString(datum);
+ datum = SysCacheGetAttr(DATABASEOID, tp,
+ Anum_pg_database_daticurules, &isnull);
+ if (!isnull)
+ icurules = TextDatumGetCString(datum);
+
+ ReleaseSysCache(tp);
+ }
+ else
+ {
+ Form_pg_collation collform;
+ HeapTuple tp;
+ Datum datum;
+ bool isnull;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", collid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ deterministic = collform->collisdeterministic;
+ datum = SysCacheGetAttrNotNull(COLLOID, tp,
+ Anum_pg_collation_colllocale);
+ iculocstr = TextDatumGetCString(datum);
+ datum = SysCacheGetAttr(COLLOID, tp,
+ Anum_pg_collation_collicurules, &isnull);
+ if (!isnull)
+ icurules = TextDatumGetCString(datum);
+
+ ReleaseSysCache(tp);
+ }
+
+ collator = make_icu_collator(iculocstr, icurules);
+
+ result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
+ result->info.icu.locale = MemoryContextStrdup(context, iculocstr);
+ result->info.icu.ucol = collator;
+ result->provider = COLLPROVIDER_ICU;
+ result->deterministic = deterministic;
+ result->collate_is_c = false;
+ result->ctype_is_c = false;
+
+ return result;
+#else
+ /* could get here if a collation was created by a build with ICU */
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ICU is not supported in this build")));
+
+ return NULL;
+#endif
+}
+
+#ifdef USE_ICU
/*
* Wrapper around ucol_open() to handle API differences for older ICU
@@ -160,7 +249,7 @@ pg_ucol_open(const char *loc_str)
*
* Ensure that no path leaks a UCollator.
*/
-UCollator *
+static UCollator *
make_icu_collator(const char *iculocstr, const char *icurules)
{
if (!icurules)
diff --git a/src/backend/utils/adt/pg_locale_libc.c b/src/backend/utils/adt/pg_locale_libc.c
index 83f310fc71..374ac37ba0 100644
--- a/src/backend/utils/adt/pg_locale_libc.c
+++ b/src/backend/utils/adt/pg_locale_libc.c
@@ -11,10 +11,16 @@
#include "postgres.h"
+#include "access/htup_details.h"
+#include "catalog/pg_database.h"
#include "catalog/pg_collation.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/formatting.h"
+#include "utils/memutils.h"
#include "utils/pg_locale.h"
+#include "utils/syscache.h"
/*
* Size of stack buffer to use for string transformations, used to avoid heap
@@ -24,15 +30,16 @@
*/
#define TEXTBUFLEN 1024
-extern locale_t make_libc_collator(const char *collate,
- const char *ctype);
+extern pg_locale_t create_pg_locale_libc(Oid collid, MemoryContext context);
+
extern int strncoll_libc(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
extern size_t strnxfrm_libc(char *dest, size_t destsize,
const char *src, ssize_t srclen,
pg_locale_t locale);
-
+static locale_t make_libc_collator(const char *collate,
+ const char *ctype);
static void report_newlocale_failure(const char *localename);
#ifdef WIN32
@@ -41,6 +48,65 @@ static int strncoll_libc_win32_utf8(const char *arg1, ssize_t len1,
pg_locale_t locale);
#endif
+pg_locale_t
+create_pg_locale_libc(Oid collid, MemoryContext context)
+{
+ const char *collate;
+ const char *ctype;
+ locale_t loc;
+ pg_locale_t result;
+
+ if (collid == DEFAULT_COLLATION_OID)
+ {
+ HeapTuple tp;
+ Datum datum;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp,
+ Anum_pg_database_datcollate);
+ collate = TextDatumGetCString(datum);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp,
+ Anum_pg_database_datctype);
+ ctype = TextDatumGetCString(datum);
+
+ ReleaseSysCache(tp);
+ }
+ else
+ {
+ HeapTuple tp;
+ Datum datum;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", collid);
+
+ datum = SysCacheGetAttrNotNull(COLLOID, tp,
+ Anum_pg_collation_collcollate);
+ collate = TextDatumGetCString(datum);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp,
+ Anum_pg_collation_collctype);
+ ctype = TextDatumGetCString(datum);
+
+ ReleaseSysCache(tp);
+ }
+
+
+ loc = make_libc_collator(collate, ctype);
+
+ result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
+ result->provider = COLLPROVIDER_LIBC;
+ result->deterministic = true;
+ result->collate_is_c = (strcmp(collate, "C") == 0) ||
+ (strcmp(collate, "POSIX") == 0);
+ result->ctype_is_c = (strcmp(ctype, "C") == 0) ||
+ (strcmp(ctype, "POSIX") == 0);
+ result->info.lt = loc;
+
+ return result;
+}
+
/*
* Create a locale_t with the given collation and ctype.
*
@@ -49,7 +115,7 @@ static int strncoll_libc_win32_utf8(const char *arg1, ssize_t len1,
*
* Ensure that no path leaks a locale_t.
*/
-locale_t
+static locale_t
make_libc_collator(const char *collate, const char *ctype)
{
locale_t loc = 0;
--
2.34.1
[text/x-patch] v8-0002-Control-collation-behavior-with-a-method-table.patch (18.9K, ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/3-v8-0002-Control-collation-behavior-with-a-method-table.patch)
download | inline diff:
From eefd65d1d05111cd12a93902e8acf009d2f4c39f Mon Sep 17 00:00:00 2001
From: Jeff Davis <jeff@j-davis.com>
Date: Thu, 26 Sep 2024 11:27:29 -0700
Subject: [PATCH v8 2/7] Control collation behavior with a method table.
Previously, behavior branched based on the provider.
A method table is less error prone and easier to hook.
---
src/backend/utils/adt/pg_locale.c | 123 +++------------------
src/backend/utils/adt/pg_locale_icu.c | 147 +++++++++++++++----------
src/backend/utils/adt/pg_locale_libc.c | 40 +++++--
src/include/utils/pg_locale.h | 33 ++++++
4 files changed, 167 insertions(+), 176 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index ec5f509c4e..00eca68717 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -92,27 +92,12 @@
/* pg_locale_icu.c */
#ifdef USE_ICU
extern UCollator *pg_ucol_open(const char *loc_str);
-extern int strncoll_icu(const char *arg1, ssize_t len1,
- const char *arg2, ssize_t len2,
- pg_locale_t locale);
-extern size_t strnxfrm_icu(char *dest, size_t destsize,
- const char *src, ssize_t srclen,
- pg_locale_t locale);
-extern size_t strnxfrm_prefix_icu(char *dest, size_t destsize,
- const char *src, ssize_t srclen,
- pg_locale_t locale);
#endif
/* pg_locale_libc.c */
extern pg_locale_t create_pg_locale_builtin(Oid collid, MemoryContext context);
extern pg_locale_t create_pg_locale_icu(Oid collid, MemoryContext context);
extern pg_locale_t create_pg_locale_libc(Oid collid, MemoryContext context);
-extern int strncoll_libc(const char *arg1, ssize_t len1,
- const char *arg2, ssize_t len2,
- pg_locale_t locale);
-extern size_t strnxfrm_libc(char *dest, size_t destsize,
- const char *src, ssize_t srclen,
- pg_locale_t locale);
/* GUC settings */
char *locale_messages;
@@ -1239,6 +1224,9 @@ create_pg_locale(Oid collid, MemoryContext context)
/* shouldn't happen */
PGLOCALE_SUPPORT_ERROR(collform->collprovider);
+ Assert((result->collate_is_c && result->collate == NULL) ||
+ (!result->collate_is_c && result->collate != NULL));
+
datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
&isnull);
if (!isnull)
@@ -1490,19 +1478,7 @@ get_collation_actual_version(char collprovider, const char *collcollate)
int
pg_strcoll(const char *arg1, const char *arg2, pg_locale_t locale)
{
- int result;
-
- if (locale->provider == COLLPROVIDER_LIBC)
- result = strncoll_libc(arg1, -1, arg2, -1, locale);
-#ifdef USE_ICU
- else if (locale->provider == COLLPROVIDER_ICU)
- result = strncoll_icu(arg1, -1, arg2, -1, locale);
-#endif
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(locale->provider);
-
- return result;
+ return locale->collate->strncoll(arg1, -1, arg2, -1, locale);
}
/*
@@ -1523,51 +1499,25 @@ int
pg_strncoll(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2,
pg_locale_t locale)
{
- int result;
-
- if (locale->provider == COLLPROVIDER_LIBC)
- result = strncoll_libc(arg1, len1, arg2, len2, locale);
-#ifdef USE_ICU
- else if (locale->provider == COLLPROVIDER_ICU)
- result = strncoll_icu(arg1, len1, arg2, len2, locale);
-#endif
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(locale->provider);
-
- return result;
+ return locale->collate->strncoll(arg1, len1, arg2, len2, locale);
}
/*
* Return true if the collation provider supports pg_strxfrm() and
* pg_strnxfrm(); otherwise false.
*
- * Unfortunately, it seems that strxfrm() for non-C collations is broken on
- * many common platforms; testing of multiple versions of glibc reveals that,
- * for many locales, strcoll() and strxfrm() do not return consistent
- * results. While no other libc other than Cygwin has so far been shown to
- * have a problem, we take the conservative course of action for right now and
- * disable this categorically. (Users who are certain this isn't a problem on
- * their system can define TRUST_STRXFRM.)
*
* No similar problem is known for the ICU provider.
*/
bool
pg_strxfrm_enabled(pg_locale_t locale)
{
- if (locale->provider == COLLPROVIDER_LIBC)
-#ifdef TRUST_STRXFRM
- return true;
-#else
- return false;
-#endif
- else if (locale->provider == COLLPROVIDER_ICU)
- return true;
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(locale->provider);
-
- return false; /* keep compiler quiet */
+ /*
+ * locale->collate->strnxfrm is still a required method, even if it may
+ * have the wrong behavior, because the planner uses it for estimates in
+ * some cases.
+ */
+ return locale->collate->strxfrm_is_safe;
}
/*
@@ -1578,19 +1528,7 @@ pg_strxfrm_enabled(pg_locale_t locale)
size_t
pg_strxfrm(char *dest, const char *src, size_t destsize, pg_locale_t locale)
{
- size_t result = 0; /* keep compiler quiet */
-
- if (locale->provider == COLLPROVIDER_LIBC)
- result = strnxfrm_libc(dest, destsize, src, -1, locale);
-#ifdef USE_ICU
- else if (locale->provider == COLLPROVIDER_ICU)
- result = strnxfrm_icu(dest, destsize, src, -1, locale);
-#endif
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(locale->provider);
-
- return result;
+ return locale->collate->strnxfrm(dest, destsize, src, -1, locale);
}
/*
@@ -1616,19 +1554,7 @@ size_t
pg_strnxfrm(char *dest, size_t destsize, const char *src, ssize_t srclen,
pg_locale_t locale)
{
- size_t result = 0; /* keep compiler quiet */
-
- if (locale->provider == COLLPROVIDER_LIBC)
- result = strnxfrm_libc(dest, destsize, src, srclen, locale);
-#ifdef USE_ICU
- else if (locale->provider == COLLPROVIDER_ICU)
- result = strnxfrm_icu(dest, destsize, src, srclen, locale);
-#endif
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(locale->provider);
-
- return result;
+ return locale->collate->strnxfrm(dest, destsize, src, srclen, locale);
}
/*
@@ -1638,15 +1564,7 @@ pg_strnxfrm(char *dest, size_t destsize, const char *src, ssize_t srclen,
bool
pg_strxfrm_prefix_enabled(pg_locale_t locale)
{
- if (locale->provider == COLLPROVIDER_LIBC)
- return false;
- else if (locale->provider == COLLPROVIDER_ICU)
- return true;
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(locale->provider);
-
- return false; /* keep compiler quiet */
+ return (locale->collate->strnxfrm_prefix != NULL);
}
/*
@@ -1658,7 +1576,7 @@ size_t
pg_strxfrm_prefix(char *dest, const char *src, size_t destsize,
pg_locale_t locale)
{
- return pg_strnxfrm_prefix(dest, destsize, src, -1, locale);
+ return locale->collate->strnxfrm_prefix(dest, destsize, src, -1, locale);
}
/*
@@ -1683,16 +1601,7 @@ size_t
pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
ssize_t srclen, pg_locale_t locale)
{
- size_t result = 0; /* keep compiler quiet */
-
-#ifdef USE_ICU
- if (locale->provider == COLLPROVIDER_ICU)
- result = strnxfrm_prefix_icu(dest, destsize, src, -1, locale);
- else
-#endif
- PGLOCALE_SUPPORT_ERROR(locale->provider);
-
- return result;
+ return locale->collate->strnxfrm_prefix(dest, destsize, src, srclen, locale);
}
/*
diff --git a/src/backend/utils/adt/pg_locale_icu.c b/src/backend/utils/adt/pg_locale_icu.c
index 73eb430d75..11ec9d4e4b 100644
--- a/src/backend/utils/adt/pg_locale_icu.c
+++ b/src/backend/utils/adt/pg_locale_icu.c
@@ -40,13 +40,14 @@ extern pg_locale_t create_pg_locale_icu(Oid collid, MemoryContext context);
#ifdef USE_ICU
extern UCollator *pg_ucol_open(const char *loc_str);
-extern int strncoll_icu(const char *arg1, ssize_t len1,
+
+static int strncoll_icu(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
-extern size_t strnxfrm_icu(char *dest, size_t destsize,
+static size_t strnxfrm_icu(char *dest, size_t destsize,
const char *src, ssize_t srclen,
pg_locale_t locale);
-extern size_t strnxfrm_prefix_icu(char *dest, size_t destsize,
+static size_t strnxfrm_prefix_icu(char *dest, size_t destsize,
const char *src, ssize_t srclen,
pg_locale_t locale);
@@ -59,12 +60,20 @@ static UConverter *icu_converter = NULL;
static UCollator *make_icu_collator(const char *iculocstr,
const char *icurules);
-static int strncoll_icu_no_utf8(const char *arg1, ssize_t len1,
- const char *arg2, ssize_t len2,
- pg_locale_t locale);
-static size_t strnxfrm_prefix_icu_no_utf8(char *dest, size_t destsize,
- const char *src, ssize_t srclen,
- pg_locale_t locale);
+static int strncoll_icu(const char *arg1, ssize_t len1,
+ const char *arg2, ssize_t len2,
+ pg_locale_t locale);
+static size_t strnxfrm_prefix_icu(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+#ifdef HAVE_UCOL_STRCOLLUTF8
+static int strncoll_icu_utf8(const char *arg1, ssize_t len1,
+ const char *arg2, ssize_t len2,
+ pg_locale_t locale);
+#endif
+static size_t strnxfrm_prefix_icu_utf8(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
static void init_icu_converter(void);
static size_t uchar_length(UConverter *converter,
const char *str, int32_t len);
@@ -73,6 +82,25 @@ static int32_t uchar_convert(UConverter *converter,
const char *src, int32_t srclen);
static void icu_set_collation_attributes(UCollator *collator, const char *loc,
UErrorCode *status);
+
+static const struct collate_methods collate_methods_icu = {
+ .strncoll = strncoll_icu,
+ .strnxfrm = strnxfrm_icu,
+ .strnxfrm_prefix = strnxfrm_prefix_icu,
+ .strxfrm_is_safe = true,
+};
+
+static const struct collate_methods collate_methods_icu_utf8 = {
+#ifdef HAVE_UCOL_STRCOLLUTF8
+ .strncoll = strncoll_icu_utf8,
+#else
+ .strncoll = strncoll_icu,
+#endif
+ .strnxfrm = strnxfrm_icu,
+ .strnxfrm_prefix = strnxfrm_prefix_icu_utf8,
+ .strxfrm_is_safe = true,
+};
+
#endif
pg_locale_t
@@ -139,6 +167,10 @@ create_pg_locale_icu(Oid collid, MemoryContext context)
result->deterministic = deterministic;
result->collate_is_c = false;
result->ctype_is_c = false;
+ if (GetDatabaseEncoding() == PG_UTF8)
+ result->collate = &collate_methods_icu_utf8;
+ else
+ result->collate = &collate_methods_icu;
return result;
#else
@@ -313,42 +345,36 @@ make_icu_collator(const char *iculocstr, const char *icurules)
}
/*
- * strncoll_icu
+ * strncoll_icu_utf8
*
* Call ucol_strcollUTF8() or ucol_strcoll() as appropriate for the given
* database encoding. An argument length of -1 means the string is
* NUL-terminated.
*/
+#ifdef HAVE_UCOL_STRCOLLUTF8
int
-strncoll_icu(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2,
- pg_locale_t locale)
+strncoll_icu_utf8(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2,
+ pg_locale_t locale)
{
int result;
+ UErrorCode status;
Assert(locale->provider == COLLPROVIDER_ICU);
-#ifdef HAVE_UCOL_STRCOLLUTF8
- if (GetDatabaseEncoding() == PG_UTF8)
- {
- UErrorCode status;
+ Assert(GetDatabaseEncoding() == PG_UTF8);
- status = U_ZERO_ERROR;
- result = ucol_strcollUTF8(locale->info.icu.ucol,
- arg1, len1,
- arg2, len2,
- &status);
- if (U_FAILURE(status))
- ereport(ERROR,
- (errmsg("collation failed: %s", u_errorName(status))));
- }
- else
-#endif
- {
- result = strncoll_icu_no_utf8(arg1, len1, arg2, len2, locale);
- }
+ status = U_ZERO_ERROR;
+ result = ucol_strcollUTF8(locale->info.icu.ucol,
+ arg1, len1,
+ arg2, len2,
+ &status);
+ if (U_FAILURE(status))
+ ereport(ERROR,
+ (errmsg("collation failed: %s", u_errorName(status))));
return result;
}
+#endif
/* 'srclen' of -1 means the strings are NUL-terminated */
size_t
@@ -399,37 +425,32 @@ strnxfrm_icu(char *dest, size_t destsize, const char *src, ssize_t srclen,
/* 'srclen' of -1 means the strings are NUL-terminated */
size_t
-strnxfrm_prefix_icu(char *dest, size_t destsize,
- const char *src, ssize_t srclen,
- pg_locale_t locale)
+strnxfrm_prefix_icu_utf8(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale)
{
size_t result;
+ UCharIterator iter;
+ uint32_t state[2];
+ UErrorCode status;
Assert(locale->provider == COLLPROVIDER_ICU);
- if (GetDatabaseEncoding() == PG_UTF8)
- {
- UCharIterator iter;
- uint32_t state[2];
- UErrorCode status;
+ Assert(GetDatabaseEncoding() == PG_UTF8);
- uiter_setUTF8(&iter, src, srclen);
- state[0] = state[1] = 0; /* won't need that again */
- status = U_ZERO_ERROR;
- result = ucol_nextSortKeyPart(locale->info.icu.ucol,
- &iter,
- state,
- (uint8_t *) dest,
- destsize,
- &status);
- if (U_FAILURE(status))
- ereport(ERROR,
- (errmsg("sort key generation failed: %s",
- u_errorName(status))));
- }
- else
- result = strnxfrm_prefix_icu_no_utf8(dest, destsize, src, srclen,
- locale);
+ uiter_setUTF8(&iter, src, srclen);
+ state[0] = state[1] = 0; /* won't need that again */
+ status = U_ZERO_ERROR;
+ result = ucol_nextSortKeyPart(locale->info.icu.ucol,
+ &iter,
+ state,
+ (uint8_t *) dest,
+ destsize,
+ &status);
+ if (U_FAILURE(status))
+ ereport(ERROR,
+ (errmsg("sort key generation failed: %s",
+ u_errorName(status))));
return result;
}
@@ -504,7 +525,7 @@ icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
}
/*
- * strncoll_icu_no_utf8
+ * strncoll_icu
*
* Convert the arguments from the database encoding to UChar strings, then
* call ucol_strcoll(). An argument length of -1 means that the string is
@@ -514,8 +535,8 @@ icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
* caller should call that instead.
*/
static int
-strncoll_icu_no_utf8(const char *arg1, ssize_t len1,
- const char *arg2, ssize_t len2, pg_locale_t locale)
+strncoll_icu(const char *arg1, ssize_t len1,
+ const char *arg2, ssize_t len2, pg_locale_t locale)
{
char sbuf[TEXTBUFLEN];
char *buf = sbuf;
@@ -528,6 +549,8 @@ strncoll_icu_no_utf8(const char *arg1, ssize_t len1,
int result;
Assert(locale->provider == COLLPROVIDER_ICU);
+
+ /* if encoding is UTF8, use more efficient strncoll_icu_utf8 */
#ifdef HAVE_UCOL_STRCOLLUTF8
Assert(GetDatabaseEncoding() != PG_UTF8);
#endif
@@ -561,9 +584,9 @@ strncoll_icu_no_utf8(const char *arg1, ssize_t len1,
/* 'srclen' of -1 means the strings are NUL-terminated */
static size_t
-strnxfrm_prefix_icu_no_utf8(char *dest, size_t destsize,
- const char *src, ssize_t srclen,
- pg_locale_t locale)
+strnxfrm_prefix_icu(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale)
{
char sbuf[TEXTBUFLEN];
char *buf = sbuf;
@@ -576,6 +599,8 @@ strnxfrm_prefix_icu_no_utf8(char *dest, size_t destsize,
Size result_bsize;
Assert(locale->provider == COLLPROVIDER_ICU);
+
+ /* if encoding is UTF8, use more efficient strnxfrm_prefix_icu_utf8 */
Assert(GetDatabaseEncoding() != PG_UTF8);
init_icu_converter();
diff --git a/src/backend/utils/adt/pg_locale_libc.c b/src/backend/utils/adt/pg_locale_libc.c
index 374ac37ba0..c7be6dd4f9 100644
--- a/src/backend/utils/adt/pg_locale_libc.c
+++ b/src/backend/utils/adt/pg_locale_libc.c
@@ -32,10 +32,10 @@
extern pg_locale_t create_pg_locale_libc(Oid collid, MemoryContext context);
-extern int strncoll_libc(const char *arg1, ssize_t len1,
+static int strncoll_libc(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
-extern size_t strnxfrm_libc(char *dest, size_t destsize,
+static size_t strnxfrm_libc(char *dest, size_t destsize,
const char *src, ssize_t srclen,
pg_locale_t locale);
static locale_t make_libc_collator(const char *collate,
@@ -48,6 +48,27 @@ static int strncoll_libc_win32_utf8(const char *arg1, ssize_t len1,
pg_locale_t locale);
#endif
+static const struct collate_methods collate_methods_libc = {
+ .strncoll = strncoll_libc,
+ .strnxfrm = strnxfrm_libc,
+ .strnxfrm_prefix = NULL,
+
+ /*
+ * Unfortunately, it seems that strxfrm() for non-C collations is broken
+ * on many common platforms; testing of multiple versions of glibc reveals
+ * that, for many locales, strcoll() and strxfrm() do not return
+ * consistent results. While no other libc other than Cygwin has so far
+ * been shown to have a problem, we take the conservative course of action
+ * for right now and disable this categorically. (Users who are certain
+ * this isn't a problem on their system can define TRUST_STRXFRM.)
+ */
+#ifdef TRUST_STRXFRM
+ .strxfrm_is_safe = true,
+#else
+ .strxfrm_is_safe = false,
+#endif
+};
+
pg_locale_t
create_pg_locale_libc(Oid collid, MemoryContext context)
{
@@ -103,6 +124,15 @@ create_pg_locale_libc(Oid collid, MemoryContext context)
result->ctype_is_c = (strcmp(ctype, "C") == 0) ||
(strcmp(ctype, "POSIX") == 0);
result->info.lt = loc;
+ if (!result->collate_is_c)
+ {
+#ifdef WIN32
+ if (GetDatabaseEncoding() == PG_UTF8)
+ result->collate = &collate_methods_libc_win32_utf8;
+ else
+#endif
+ result->collate = &collate_methods_libc;
+ }
return result;
}
@@ -200,12 +230,6 @@ strncoll_libc(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2,
Assert(locale->provider == COLLPROVIDER_LIBC);
-#ifdef WIN32
- /* check for this case before doing the work for nul-termination */
- if (GetDatabaseEncoding() == PG_UTF8)
- return strncoll_libc_win32_utf8(arg1, len1, arg2, len2, locale);
-#endif /* WIN32 */
-
if (bufsize1 + bufsize2 > TEXTBUFLEN)
buf = palloc(bufsize1 + bufsize2);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 37ecf95193..2f05dffcdd 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -60,6 +60,36 @@ extern struct lconv *PGLC_localeconv(void);
extern void cache_locale_time(void);
+struct pg_locale_struct;
+typedef struct pg_locale_struct *pg_locale_t;
+
+/* methods that define collation behavior */
+struct collate_methods
+{
+ /* required */
+ int (*strncoll) (const char *arg1, ssize_t len1,
+ const char *arg2, ssize_t len2,
+ pg_locale_t locale);
+
+ /* required */
+ size_t (*strnxfrm) (char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+
+ /* optional */
+ size_t (*strnxfrm_prefix) (char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+
+ /*
+ * If the strnxfrm method is not trusted to return the correct results,
+ * set strxfrm_is_safe to false. It set to false, the method will not be
+ * used in most cases, but the planner still expects it to be there for
+ * estimation purposes (where incorrect results are acceptable).
+ */
+ bool strxfrm_is_safe;
+};
+
/*
* We use a discriminated union to hold either a locale_t or an ICU collator.
* pg_locale_t is occasionally checked for truth, so make it a pointer.
@@ -82,6 +112,9 @@ struct pg_locale_struct
bool deterministic;
bool collate_is_c;
bool ctype_is_c;
+
+ const struct collate_methods *collate; /* NULL if collate_is_c */
+
union
{
struct
--
2.34.1
[text/x-patch] v8-0003-Control-ctype-behavior-internally-with-a-method-t.patch (63.2K, ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/4-v8-0003-Control-ctype-behavior-internally-with-a-method-t.patch)
download | inline diff:
From bfd26c4c54cf1a1b430dcdd9188cacb38b30b7f6 Mon Sep 17 00:00:00 2001
From: Jeff Davis <jeff@j-davis.com>
Date: Thu, 26 Sep 2024 12:12:51 -0700
Subject: [PATCH v8 3/7] Control ctype behavior internally with a method table.
Previously, pattern matching and case mapping behavior branched based
on the provider.
Refactor to use a method table, which is less error-prone and easier
to hook.
---
src/backend/regex/regc_pg_locale.c | 388 ++++--------------
src/backend/utils/adt/formatting.c | 445 +++------------------
src/backend/utils/adt/like.c | 22 +-
src/backend/utils/adt/like_support.c | 7 +-
src/backend/utils/adt/pg_locale.c | 71 ++++
src/backend/utils/adt/pg_locale_builtin.c | 129 ++++++
src/backend/utils/adt/pg_locale_icu.c | 188 ++++++++-
src/backend/utils/adt/pg_locale_libc.c | 465 ++++++++++++++++++++++
src/include/utils/pg_locale.h | 71 +++-
src/tools/pgindent/typedefs.list | 1 -
10 files changed, 1062 insertions(+), 725 deletions(-)
diff --git a/src/backend/regex/regc_pg_locale.c b/src/backend/regex/regc_pg_locale.c
index b75784b6ce..e898634fdf 100644
--- a/src/backend/regex/regc_pg_locale.c
+++ b/src/backend/regex/regc_pg_locale.c
@@ -63,33 +63,18 @@
* NB: the coding here assumes pg_wchar is an unsigned type.
*/
-typedef enum
-{
- PG_REGEX_STRATEGY_C, /* C locale (encoding independent) */
- PG_REGEX_STRATEGY_BUILTIN, /* built-in Unicode semantics */
- PG_REGEX_STRATEGY_LIBC_WIDE, /* Use locale_t <wctype.h> functions */
- PG_REGEX_STRATEGY_LIBC_1BYTE, /* Use locale_t <ctype.h> functions */
- PG_REGEX_STRATEGY_ICU, /* Use ICU uchar.h functions */
-} PG_Locale_Strategy;
-
-static PG_Locale_Strategy pg_regex_strategy;
static pg_locale_t pg_regex_locale;
static Oid pg_regex_collation;
+static struct pg_locale_struct dummy_c_locale = {
+ .collate_is_c = true,
+ .ctype_is_c = true,
+};
+
/*
* Hard-wired character properties for C locale
*/
-#define PG_ISDIGIT 0x01
-#define PG_ISALPHA 0x02
-#define PG_ISALNUM (PG_ISDIGIT | PG_ISALPHA)
-#define PG_ISUPPER 0x04
-#define PG_ISLOWER 0x08
-#define PG_ISGRAPH 0x10
-#define PG_ISPRINT 0x20
-#define PG_ISPUNCT 0x40
-#define PG_ISSPACE 0x80
-
-static const unsigned char pg_char_properties[128] = {
+static const unsigned char char_properties_tbl[128] = {
/* NUL */ 0,
/* ^A */ 0,
/* ^B */ 0,
@@ -232,7 +217,6 @@ void
pg_set_regex_collation(Oid collation)
{
pg_locale_t locale = 0;
- PG_Locale_Strategy strategy;
if (!OidIsValid(collation))
{
@@ -253,8 +237,8 @@ pg_set_regex_collation(Oid collation)
* catalog access is available, so we can't call
* pg_newlocale_from_collation().
*/
- strategy = PG_REGEX_STRATEGY_C;
collation = C_COLLATION_OID;
+ locale = &dummy_c_locale;
}
else
{
@@ -271,32 +255,11 @@ pg_set_regex_collation(Oid collation)
* C/POSIX collations use this path regardless of database
* encoding
*/
- strategy = PG_REGEX_STRATEGY_C;
- locale = 0;
+ locale = &dummy_c_locale;
collation = C_COLLATION_OID;
}
- else if (locale->provider == COLLPROVIDER_BUILTIN)
- {
- Assert(GetDatabaseEncoding() == PG_UTF8);
- strategy = PG_REGEX_STRATEGY_BUILTIN;
- }
-#ifdef USE_ICU
- else if (locale->provider == COLLPROVIDER_ICU)
- {
- strategy = PG_REGEX_STRATEGY_ICU;
- }
-#endif
- else
- {
- Assert(locale->provider == COLLPROVIDER_LIBC);
- if (GetDatabaseEncoding() == PG_UTF8)
- strategy = PG_REGEX_STRATEGY_LIBC_WIDE;
- else
- strategy = PG_REGEX_STRATEGY_LIBC_1BYTE;
- }
}
- pg_regex_strategy = strategy;
pg_regex_locale = locale;
pg_regex_collation = collation;
}
@@ -304,82 +267,31 @@ pg_set_regex_collation(Oid collation)
static int
pg_wc_isdigit(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISDIGIT));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_isdigit(c, true);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswdigit_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- isdigit_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_isdigit(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISDIGIT));
+ else
+ return char_properties(c, PG_ISDIGIT, pg_regex_locale) != 0;
}
static int
pg_wc_isalpha(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISALPHA));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_isalpha(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswalpha_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- isalpha_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_isalpha(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISALPHA));
+ else
+ return char_properties(c, PG_ISALPHA, pg_regex_locale) != 0;
}
static int
pg_wc_isalnum(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISALNUM));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_isalnum(c, true);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswalnum_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- isalnum_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_isalnum(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISALNUM));
+ else
+ return char_properties(c, PG_ISDIGIT | PG_ISALPHA, pg_regex_locale) != 0;
}
static int
@@ -394,219 +306,87 @@ pg_wc_isword(pg_wchar c)
static int
pg_wc_isupper(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISUPPER));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_isupper(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswupper_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- isupper_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_isupper(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISUPPER));
+ else
+ return char_properties(c, PG_ISUPPER, pg_regex_locale) != 0;
}
static int
pg_wc_islower(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISLOWER));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_islower(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswlower_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- islower_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_islower(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISLOWER));
+ else
+ return char_properties(c, PG_ISLOWER, pg_regex_locale) != 0;
}
static int
pg_wc_isgraph(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISGRAPH));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_isgraph(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswgraph_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- isgraph_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_isgraph(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISGRAPH));
+ else
+ return char_properties(c, PG_ISGRAPH, pg_regex_locale) != 0;
}
static int
pg_wc_isprint(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISPRINT));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_isprint(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswprint_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- isprint_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_isprint(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISPRINT));
+ else
+ return char_properties(c, PG_ISPRINT, pg_regex_locale) != 0;
}
static int
pg_wc_ispunct(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISPUNCT));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_ispunct(c, true);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswpunct_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- ispunct_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_ispunct(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISPUNCT));
+ else
+ return char_properties(c, PG_ISPUNCT, pg_regex_locale) != 0;
}
static int
pg_wc_isspace(pg_wchar c)
{
- switch (pg_regex_strategy)
- {
- case PG_REGEX_STRATEGY_C:
- return (c <= (pg_wchar) 127 &&
- (pg_char_properties[c] & PG_ISSPACE));
- case PG_REGEX_STRATEGY_BUILTIN:
- return pg_u_isspace(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return iswspace_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- return (c <= (pg_wchar) UCHAR_MAX &&
- isspace_l((unsigned char) c, pg_regex_locale->info.lt));
- break;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_isspace(c);
-#endif
- break;
- }
- return 0; /* can't get here, but keep compiler quiet */
+ if (pg_regex_locale->ctype_is_c)
+ return (c <= (pg_wchar) 127 &&
+ (char_properties_tbl[c] & PG_ISSPACE));
+ else
+ return char_properties(c, PG_ISSPACE, pg_regex_locale) != 0;
}
static pg_wchar
pg_wc_toupper(pg_wchar c)
{
- switch (pg_regex_strategy)
+ if (pg_regex_locale->ctype_is_c)
{
- case PG_REGEX_STRATEGY_C:
- if (c <= (pg_wchar) 127)
- return pg_ascii_toupper((unsigned char) c);
- return c;
- case PG_REGEX_STRATEGY_BUILTIN:
- return unicode_uppercase_simple(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return towupper_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- if (c <= (pg_wchar) UCHAR_MAX)
- return toupper_l((unsigned char) c, pg_regex_locale->info.lt);
- return c;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_toupper(c);
-#endif
- break;
+ if (c <= (pg_wchar) 127)
+ return pg_ascii_toupper((unsigned char) c);
+ return c;
}
- return 0; /* can't get here, but keep compiler quiet */
+ else
+ return pg_regex_locale->ctype->wc_toupper(c, pg_regex_locale);
}
static pg_wchar
pg_wc_tolower(pg_wchar c)
{
- switch (pg_regex_strategy)
+ if (pg_regex_locale->ctype_is_c)
{
- case PG_REGEX_STRATEGY_C:
- if (c <= (pg_wchar) 127)
- return pg_ascii_tolower((unsigned char) c);
- return c;
- case PG_REGEX_STRATEGY_BUILTIN:
- return unicode_lowercase_simple(c);
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
- return towlower_l((wint_t) c, pg_regex_locale->info.lt);
- /* FALL THRU */
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
- if (c <= (pg_wchar) UCHAR_MAX)
- return tolower_l((unsigned char) c, pg_regex_locale->info.lt);
- return c;
- case PG_REGEX_STRATEGY_ICU:
-#ifdef USE_ICU
- return u_tolower(c);
-#endif
- break;
+ if (c <= (pg_wchar) 127)
+ return pg_ascii_tolower((unsigned char) c);
+ return c;
}
- return 0; /* can't get here, but keep compiler quiet */
+ else
+ return pg_regex_locale->ctype->wc_tolower(c, pg_regex_locale);
}
@@ -732,37 +512,25 @@ pg_ctype_get_cache(pg_wc_probefunc probefunc, int cclasscode)
* would always be true for production values of MAX_SIMPLE_CHR, but it's
* useful to allow it to be small for testing purposes.)
*/
- switch (pg_regex_strategy)
+ if (pg_regex_locale->ctype_is_c)
{
- case PG_REGEX_STRATEGY_C:
#if MAX_SIMPLE_CHR >= 127
- max_chr = (pg_wchar) 127;
- pcc->cv.cclasscode = -1;
+ max_chr = (pg_wchar) 127;
+ pcc->cv.cclasscode = -1;
#else
- max_chr = (pg_wchar) MAX_SIMPLE_CHR;
+ max_chr = (pg_wchar) MAX_SIMPLE_CHR;
#endif
- break;
- case PG_REGEX_STRATEGY_BUILTIN:
- max_chr = (pg_wchar) MAX_SIMPLE_CHR;
- break;
- case PG_REGEX_STRATEGY_LIBC_WIDE:
- max_chr = (pg_wchar) MAX_SIMPLE_CHR;
- break;
- case PG_REGEX_STRATEGY_LIBC_1BYTE:
-#if MAX_SIMPLE_CHR >= UCHAR_MAX
- max_chr = (pg_wchar) UCHAR_MAX;
+ }
+ else
+ {
+ if (pg_regex_locale->ctype->max_chr != 0 &&
+ pg_regex_locale->ctype->max_chr <= MAX_SIMPLE_CHR)
+ {
+ max_chr = pg_regex_locale->ctype->max_chr;
pcc->cv.cclasscode = -1;
-#else
- max_chr = (pg_wchar) MAX_SIMPLE_CHR;
-#endif
- break;
- case PG_REGEX_STRATEGY_ICU:
+ }
+ else
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
- break;
- default:
- Assert(false);
- max_chr = 0; /* can't get here, but keep compiler quiet */
- break;
}
/*
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index 85a7dd4561..6a0571f93e 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -1570,52 +1570,6 @@ str_numth(char *dest, char *num, int type)
* upper/lower/initcap functions
*****************************************************************************/
-#ifdef USE_ICU
-
-typedef int32_t (*ICU_Convert_Func) (UChar *dest, int32_t destCapacity,
- const UChar *src, int32_t srcLength,
- const char *locale,
- UErrorCode *pErrorCode);
-
-static int32_t
-icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
- UChar **buff_dest, UChar *buff_source, int32_t len_source)
-{
- UErrorCode status;
- int32_t len_dest;
-
- len_dest = len_source; /* try first with same length */
- *buff_dest = palloc(len_dest * sizeof(**buff_dest));
- status = U_ZERO_ERROR;
- len_dest = func(*buff_dest, len_dest, buff_source, len_source,
- mylocale->info.icu.locale, &status);
- if (status == U_BUFFER_OVERFLOW_ERROR)
- {
- /* try again with adjusted length */
- pfree(*buff_dest);
- *buff_dest = palloc(len_dest * sizeof(**buff_dest));
- status = U_ZERO_ERROR;
- len_dest = func(*buff_dest, len_dest, buff_source, len_source,
- mylocale->info.icu.locale, &status);
- }
- if (U_FAILURE(status))
- ereport(ERROR,
- (errmsg("case conversion failed: %s", u_errorName(status))));
- return len_dest;
-}
-
-static int32_t
-u_strToTitle_default_BI(UChar *dest, int32_t destCapacity,
- const UChar *src, int32_t srcLength,
- const char *locale,
- UErrorCode *pErrorCode)
-{
- return u_strToTitle(dest, destCapacity, src, srcLength,
- NULL, locale, pErrorCode);
-}
-
-#endif /* USE_ICU */
-
/*
* If the system provides the needed functions for wide-character manipulation
* (which are all standardized by C99), then we implement upper/lower/initcap
@@ -1663,101 +1617,28 @@ str_tolower(const char *buff, size_t nbytes, Oid collid)
}
else
{
-#ifdef USE_ICU
- if (mylocale->provider == COLLPROVIDER_ICU)
+ const char *src = buff;
+ size_t srclen = nbytes;
+ size_t dstsize;
+ char *dst;
+ size_t needed;
+
+ /* first try buffer of equal size plus terminating NUL */
+ dstsize = srclen + 1;
+ dst = palloc(dstsize);
+
+ needed = pg_strlower(dst, dstsize, src, srclen, mylocale);
+ if (needed + 1 > dstsize)
{
- int32_t len_uchar;
- int32_t len_conv;
- UChar *buff_uchar;
- UChar *buff_conv;
-
- len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
- len_conv = icu_convert_case(u_strToLower, mylocale,
- &buff_conv, buff_uchar, len_uchar);
- icu_from_uchar(&result, buff_conv, len_conv);
- pfree(buff_uchar);
- pfree(buff_conv);
+ /* grow buffer if needed and retry */
+ dstsize = needed + 1;
+ dst = repalloc(dst, dstsize);
+ needed = pg_strlower(dst, dstsize, src, srclen, mylocale);
+ Assert(needed + 1 <= dstsize);
}
- else
-#endif
- if (mylocale->provider == COLLPROVIDER_BUILTIN)
- {
- const char *src = buff;
- size_t srclen = nbytes;
- size_t dstsize;
- char *dst;
- size_t needed;
-
- Assert(GetDatabaseEncoding() == PG_UTF8);
-
- /* first try buffer of equal size plus terminating NUL */
- dstsize = srclen + 1;
- dst = palloc(dstsize);
-
- needed = unicode_strlower(dst, dstsize, src, srclen);
- if (needed + 1 > dstsize)
- {
- /* grow buffer if needed and retry */
- dstsize = needed + 1;
- dst = repalloc(dst, dstsize);
- needed = unicode_strlower(dst, dstsize, src, srclen);
- Assert(needed + 1 == dstsize);
- }
-
- Assert(dst[needed] == '\0');
- result = dst;
- }
- else
- {
- Assert(mylocale->provider == COLLPROVIDER_LIBC);
-
- if (pg_database_encoding_max_length() > 1)
- {
- wchar_t *workspace;
- size_t curr_char;
- size_t result_size;
-
- /* Overflow paranoia */
- if ((nbytes + 1) > (INT_MAX / sizeof(wchar_t)))
- ereport(ERROR,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
-
- /* Output workspace cannot have more codes than input bytes */
- workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t));
-
- char2wchar(workspace, nbytes + 1, buff, nbytes, mylocale);
-
- for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
- workspace[curr_char] = towlower_l(workspace[curr_char], mylocale->info.lt);
- /*
- * Make result large enough; case change might change number
- * of bytes
- */
- result_size = curr_char * pg_database_encoding_max_length() + 1;
- result = palloc(result_size);
-
- wchar2char(result, workspace, result_size, mylocale);
- pfree(workspace);
- }
- else
- {
- char *p;
-
- result = pnstrdup(buff, nbytes);
-
- /*
- * Note: we assume that tolower_l() will not be so broken as
- * to need an isupper_l() guard test. When using the default
- * collation, we apply the traditional Postgres behavior that
- * forces ASCII-style treatment of I/i, but in non-default
- * collations you get exactly what the collation says.
- */
- for (p = result; *p; p++)
- *p = tolower_l((unsigned char) *p, mylocale->info.lt);
- }
- }
+ Assert(dst[needed] == '\0');
+ result = dst;
}
return result;
@@ -1800,147 +1681,33 @@ str_toupper(const char *buff, size_t nbytes, Oid collid)
}
else
{
-#ifdef USE_ICU
- if (mylocale->provider == COLLPROVIDER_ICU)
- {
- int32_t len_uchar,
- len_conv;
- UChar *buff_uchar;
- UChar *buff_conv;
-
- len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
- len_conv = icu_convert_case(u_strToUpper, mylocale,
- &buff_conv, buff_uchar, len_uchar);
- icu_from_uchar(&result, buff_conv, len_conv);
- pfree(buff_uchar);
- pfree(buff_conv);
- }
- else
-#endif
- if (mylocale->provider == COLLPROVIDER_BUILTIN)
+ const char *src = buff;
+ size_t srclen = nbytes;
+ size_t dstsize;
+ char *dst;
+ size_t needed;
+
+ /* first try buffer of equal size plus terminating NUL */
+ dstsize = srclen + 1;
+ dst = palloc(dstsize);
+
+ needed = pg_strupper(dst, dstsize, src, srclen, mylocale);
+ if (needed + 1 > dstsize)
{
- const char *src = buff;
- size_t srclen = nbytes;
- size_t dstsize;
- char *dst;
- size_t needed;
-
- Assert(GetDatabaseEncoding() == PG_UTF8);
-
- /* first try buffer of equal size plus terminating NUL */
- dstsize = srclen + 1;
- dst = palloc(dstsize);
-
- needed = unicode_strupper(dst, dstsize, src, srclen);
- if (needed + 1 > dstsize)
- {
- /* grow buffer if needed and retry */
- dstsize = needed + 1;
- dst = repalloc(dst, dstsize);
- needed = unicode_strupper(dst, dstsize, src, srclen);
- Assert(needed + 1 == dstsize);
- }
-
- Assert(dst[needed] == '\0');
- result = dst;
+ /* grow buffer if needed and retry */
+ dstsize = needed + 1;
+ dst = repalloc(dst, dstsize);
+ needed = pg_strupper(dst, dstsize, src, srclen, mylocale);
+ Assert(needed + 1 <= dstsize);
}
- else
- {
- Assert(mylocale->provider == COLLPROVIDER_LIBC);
-
- if (pg_database_encoding_max_length() > 1)
- {
- wchar_t *workspace;
- size_t curr_char;
- size_t result_size;
-
- /* Overflow paranoia */
- if ((nbytes + 1) > (INT_MAX / sizeof(wchar_t)))
- ereport(ERROR,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
-
- /* Output workspace cannot have more codes than input bytes */
- workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t));
-
- char2wchar(workspace, nbytes + 1, buff, nbytes, mylocale);
-
- for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
- workspace[curr_char] = towupper_l(workspace[curr_char], mylocale->info.lt);
- /*
- * Make result large enough; case change might change number
- * of bytes
- */
- result_size = curr_char * pg_database_encoding_max_length() + 1;
- result = palloc(result_size);
-
- wchar2char(result, workspace, result_size, mylocale);
- pfree(workspace);
- }
- else
- {
- char *p;
-
- result = pnstrdup(buff, nbytes);
-
- /*
- * Note: we assume that toupper_l() will not be so broken as
- * to need an islower_l() guard test. When using the default
- * collation, we apply the traditional Postgres behavior that
- * forces ASCII-style treatment of I/i, but in non-default
- * collations you get exactly what the collation says.
- */
- for (p = result; *p; p++)
- *p = toupper_l((unsigned char) *p, mylocale->info.lt);
- }
- }
+ Assert(dst[needed] == '\0');
+ result = dst;
}
return result;
}
-struct WordBoundaryState
-{
- const char *str;
- size_t len;
- size_t offset;
- bool init;
- bool prev_alnum;
-};
-
-/*
- * Simple word boundary iterator that draws boundaries each time the result of
- * pg_u_isalnum() changes.
- */
-static size_t
-initcap_wbnext(void *state)
-{
- struct WordBoundaryState *wbstate = (struct WordBoundaryState *) state;
-
- while (wbstate->offset < wbstate->len &&
- wbstate->str[wbstate->offset] != '\0')
- {
- pg_wchar u = utf8_to_unicode((unsigned char *) wbstate->str +
- wbstate->offset);
- bool curr_alnum = pg_u_isalnum(u, true);
-
- if (!wbstate->init || curr_alnum != wbstate->prev_alnum)
- {
- size_t prev_offset = wbstate->offset;
-
- wbstate->init = true;
- wbstate->offset += unicode_utf8len(u);
- wbstate->prev_alnum = curr_alnum;
- return prev_offset;
- }
-
- wbstate->offset += unicode_utf8len(u);
- }
-
- return wbstate->len;
-}
-
/*
* collation-aware, wide-character-aware initcap function
*
@@ -1951,7 +1718,6 @@ char *
str_initcap(const char *buff, size_t nbytes, Oid collid)
{
char *result;
- int wasalnum = false;
pg_locale_t mylocale;
if (!buff)
@@ -1979,125 +1745,28 @@ str_initcap(const char *buff, size_t nbytes, Oid collid)
}
else
{
-#ifdef USE_ICU
- if (mylocale->provider == COLLPROVIDER_ICU)
+ const char *src = buff;
+ size_t srclen = nbytes;
+ size_t dstsize;
+ char *dst;
+ size_t needed;
+
+ /* first try buffer of equal size plus terminating NUL */
+ dstsize = srclen + 1;
+ dst = palloc(dstsize);
+
+ needed = pg_strtitle(dst, dstsize, src, srclen, mylocale);
+ if (needed + 1 > dstsize)
{
- int32_t len_uchar,
- len_conv;
- UChar *buff_uchar;
- UChar *buff_conv;
-
- len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
- len_conv = icu_convert_case(u_strToTitle_default_BI, mylocale,
- &buff_conv, buff_uchar, len_uchar);
- icu_from_uchar(&result, buff_conv, len_conv);
- pfree(buff_uchar);
- pfree(buff_conv);
+ /* grow buffer if needed and retry */
+ dstsize = needed + 1;
+ dst = repalloc(dst, dstsize);
+ needed = pg_strtitle(dst, dstsize, src, srclen, mylocale);
+ Assert(needed + 1 <= dstsize);
}
- else
-#endif
- if (mylocale->provider == COLLPROVIDER_BUILTIN)
- {
- const char *src = buff;
- size_t srclen = nbytes;
- size_t dstsize;
- char *dst;
- size_t needed;
- struct WordBoundaryState wbstate = {
- .str = src,
- .len = srclen,
- .offset = 0,
- .init = false,
- .prev_alnum = false,
- };
-
- Assert(GetDatabaseEncoding() == PG_UTF8);
-
- /* first try buffer of equal size plus terminating NUL */
- dstsize = srclen + 1;
- dst = palloc(dstsize);
-
- needed = unicode_strtitle(dst, dstsize, src, srclen,
- initcap_wbnext, &wbstate);
- if (needed + 1 > dstsize)
- {
- /* reset iterator */
- wbstate.offset = 0;
- wbstate.init = false;
-
- /* grow buffer if needed and retry */
- dstsize = needed + 1;
- dst = repalloc(dst, dstsize);
- needed = unicode_strtitle(dst, dstsize, src, srclen,
- initcap_wbnext, &wbstate);
- Assert(needed + 1 == dstsize);
- }
-
- result = dst;
- }
- else
- {
- Assert(mylocale->provider == COLLPROVIDER_LIBC);
-
- if (pg_database_encoding_max_length() > 1)
- {
- wchar_t *workspace;
- size_t curr_char;
- size_t result_size;
-
- /* Overflow paranoia */
- if ((nbytes + 1) > (INT_MAX / sizeof(wchar_t)))
- ereport(ERROR,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
-
- /* Output workspace cannot have more codes than input bytes */
- workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t));
-
- char2wchar(workspace, nbytes + 1, buff, nbytes, mylocale);
-
- for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
- {
- if (wasalnum)
- workspace[curr_char] = towlower_l(workspace[curr_char], mylocale->info.lt);
- else
- workspace[curr_char] = towupper_l(workspace[curr_char], mylocale->info.lt);
- wasalnum = iswalnum_l(workspace[curr_char], mylocale->info.lt);
- }
-
- /*
- * Make result large enough; case change might change number
- * of bytes
- */
- result_size = curr_char * pg_database_encoding_max_length() + 1;
- result = palloc(result_size);
-
- wchar2char(result, workspace, result_size, mylocale);
- pfree(workspace);
- }
- else
- {
- char *p;
- result = pnstrdup(buff, nbytes);
-
- /*
- * Note: we assume that toupper_l()/tolower_l() will not be so
- * broken as to need guard tests. When using the default
- * collation, we apply the traditional Postgres behavior that
- * forces ASCII-style treatment of I/i, but in non-default
- * collations you get exactly what the collation says.
- */
- for (p = result; *p; p++)
- {
- if (wasalnum)
- *p = tolower_l((unsigned char) *p, mylocale->info.lt);
- else
- *p = toupper_l((unsigned char) *p, mylocale->info.lt);
- wasalnum = isalnum_l((unsigned char) *p, mylocale->info.lt);
- }
- }
- }
+ Assert(dst[needed] == '\0');
+ result = dst;
}
return result;
diff --git a/src/backend/utils/adt/like.c b/src/backend/utils/adt/like.c
index 0152723b2a..5b679bcad8 100644
--- a/src/backend/utils/adt/like.c
+++ b/src/backend/utils/adt/like.c
@@ -96,7 +96,7 @@ SB_lower_char(unsigned char c, pg_locale_t locale)
if (locale->ctype_is_c)
return pg_ascii_tolower(c);
else
- return tolower_l(c, locale->info.lt);
+ return char_tolower(c, locale);
}
@@ -201,7 +201,17 @@ Generic_Text_IC_like(text *str, text *pat, Oid collation)
* way.
*/
- if (pg_database_encoding_max_length() > 1 || (locale->provider == COLLPROVIDER_ICU))
+ if (locale->ctype_is_c ||
+ (char_tolower_enabled(locale) &&
+ pg_database_encoding_max_length() == 1))
+ {
+ p = VARDATA_ANY(pat);
+ plen = VARSIZE_ANY_EXHDR(pat);
+ s = VARDATA_ANY(str);
+ slen = VARSIZE_ANY_EXHDR(str);
+ return SB_IMatchText(s, slen, p, plen, locale);
+ }
+ else
{
pat = DatumGetTextPP(DirectFunctionCall1Coll(lower, collation,
PointerGetDatum(pat)));
@@ -216,14 +226,6 @@ Generic_Text_IC_like(text *str, text *pat, Oid collation)
else
return MB_MatchText(s, slen, p, plen, 0);
}
- else
- {
- p = VARDATA_ANY(pat);
- plen = VARSIZE_ANY_EXHDR(pat);
- s = VARDATA_ANY(str);
- slen = VARSIZE_ANY_EXHDR(str);
- return SB_IMatchText(s, slen, p, plen, locale);
- }
}
/*
diff --git a/src/backend/utils/adt/like_support.c b/src/backend/utils/adt/like_support.c
index 8b15509a3b..bf718f1a3d 100644
--- a/src/backend/utils/adt/like_support.c
+++ b/src/backend/utils/adt/like_support.c
@@ -1498,13 +1498,8 @@ pattern_char_isalpha(char c, bool is_multibyte,
{
if (locale->ctype_is_c)
return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z');
- else if (is_multibyte && IS_HIGHBIT_SET(c))
- return true;
- else if (locale->provider != COLLPROVIDER_LIBC)
- return IS_HIGHBIT_SET(c) ||
- (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z');
else
- return isalpha_l((unsigned char) c, locale->info.lt);
+ return char_is_cased(c, locale);
}
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 00eca68717..2ebe4c00bf 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1227,6 +1227,9 @@ create_pg_locale(Oid collid, MemoryContext context)
Assert((result->collate_is_c && result->collate == NULL) ||
(!result->collate_is_c && result->collate != NULL));
+ Assert((result->ctype_is_c && result->ctype == NULL) ||
+ (!result->ctype_is_c && result->ctype != NULL));
+
datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
&isnull);
if (!isnull)
@@ -1470,6 +1473,27 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+size_t
+pg_strlower(char *dst, size_t dstsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ return locale->ctype->strlower(dst, dstsize, src, srclen, locale);
+}
+
+size_t
+pg_strtitle(char *dst, size_t dstsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ return locale->ctype->strtitle(dst, dstsize, src, srclen, locale);
+}
+
+size_t
+pg_strupper(char *dst, size_t dstsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ return locale->ctype->strupper(dst, dstsize, src, srclen, locale);
+}
+
/*
* pg_strcoll
*
@@ -1604,6 +1628,53 @@ pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
return locale->collate->strnxfrm_prefix(dest, destsize, src, srclen, locale);
}
+/*
+ * char_properties()
+ *
+ * Out of the properties specified in the given mask, return a new mask of the
+ * properties true for the given character.
+ */
+int
+char_properties(pg_wchar wc, int mask, pg_locale_t locale)
+{
+ return locale->ctype->char_properties(wc, mask, locale);
+}
+
+/*
+ * char_is_cased()
+ *
+ * Fuzzy test of whether the given char is case-varying or not. The argument
+ * is a single byte, so in a multibyte encoding, just assume any non-ASCII
+ * char is case-varying.
+ */
+bool
+char_is_cased(char ch, pg_locale_t locale)
+{
+ return locale->ctype->char_is_cased(ch, locale);
+}
+
+/*
+ * char_tolower_enabled()
+ *
+ * Does the provider support char_tolower()?
+ */
+bool
+char_tolower_enabled(pg_locale_t locale)
+{
+ return (locale->ctype->char_tolower != NULL);
+}
+
+/*
+ * char_tolower()
+ *
+ * Convert char (single-byte encoding) to lowercase.
+ */
+char
+char_tolower(unsigned char ch, pg_locale_t locale)
+{
+ return locale->ctype->char_tolower(ch, locale);
+}
+
/*
* Return required encoding ID for the given locale, or -1 if any encoding is
* valid for the locale.
diff --git a/src/backend/utils/adt/pg_locale_builtin.c b/src/backend/utils/adt/pg_locale_builtin.c
index 4246971a4d..5f90355355 100644
--- a/src/backend/utils/adt/pg_locale_builtin.c
+++ b/src/backend/utils/adt/pg_locale_builtin.c
@@ -13,6 +13,8 @@
#include "catalog/pg_database.h"
#include "catalog/pg_collation.h"
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "utils/builtins.h"
@@ -23,6 +25,131 @@
extern pg_locale_t create_pg_locale_builtin(Oid collid,
MemoryContext context);
+struct WordBoundaryState
+{
+ const char *str;
+ size_t len;
+ size_t offset;
+ bool init;
+ bool prev_alnum;
+};
+
+/*
+ * Simple word boundary iterator that draws boundaries each time the result of
+ * pg_u_isalnum() changes.
+ */
+static size_t
+initcap_wbnext(void *state)
+{
+ struct WordBoundaryState *wbstate = (struct WordBoundaryState *) state;
+
+ while (wbstate->offset < wbstate->len &&
+ wbstate->str[wbstate->offset] != '\0')
+ {
+ pg_wchar u = utf8_to_unicode((unsigned char *) wbstate->str +
+ wbstate->offset);
+ bool curr_alnum = pg_u_isalnum(u, true);
+
+ if (!wbstate->init || curr_alnum != wbstate->prev_alnum)
+ {
+ size_t prev_offset = wbstate->offset;
+
+ wbstate->init = true;
+ wbstate->offset += unicode_utf8len(u);
+ wbstate->prev_alnum = curr_alnum;
+ return prev_offset;
+ }
+
+ wbstate->offset += unicode_utf8len(u);
+ }
+
+ return wbstate->len;
+}
+
+static size_t
+strlower_builtin(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ return unicode_strlower(dest, destsize, src, srclen);
+}
+
+static size_t
+strtitle_builtin(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ struct WordBoundaryState wbstate = {
+ .str = src,
+ .len = srclen,
+ .offset = 0,
+ .init = false,
+ .prev_alnum = false,
+ };
+
+ return unicode_strtitle(dest, destsize, src, srclen,
+ initcap_wbnext, &wbstate);
+}
+
+static size_t
+strupper_builtin(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ return unicode_strupper(dest, destsize, src, srclen);
+}
+
+static int
+char_properties_builtin(pg_wchar wc, int mask, pg_locale_t locale)
+{
+ int result = 0;
+
+ if ((mask & PG_ISDIGIT) && pg_u_isdigit(wc, true))
+ result |= PG_ISDIGIT;
+ if ((mask & PG_ISALPHA) && pg_u_isalpha(wc))
+ result |= PG_ISALPHA;
+ if ((mask & PG_ISUPPER) && pg_u_isupper(wc))
+ result |= PG_ISUPPER;
+ if ((mask & PG_ISLOWER) && pg_u_islower(wc))
+ result |= PG_ISLOWER;
+ if ((mask & PG_ISGRAPH) && pg_u_isgraph(wc))
+ result |= PG_ISGRAPH;
+ if ((mask & PG_ISPRINT) && pg_u_isprint(wc))
+ result |= PG_ISPRINT;
+ if ((mask & PG_ISPUNCT) && pg_u_ispunct(wc, true))
+ result |= PG_ISPUNCT;
+ if ((mask & PG_ISSPACE) && pg_u_isspace(wc))
+ result |= PG_ISSPACE;
+
+ return result;
+}
+
+static bool
+char_is_cased_builtin(char ch, pg_locale_t locale)
+{
+ return IS_HIGHBIT_SET(ch) ||
+ (ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z');
+}
+
+static pg_wchar
+wc_toupper_builtin(pg_wchar wc, pg_locale_t locale)
+{
+ return unicode_uppercase_simple(wc);
+}
+
+static pg_wchar
+wc_tolower_builtin(pg_wchar wc, pg_locale_t locale)
+{
+ return unicode_lowercase_simple(wc);
+}
+
+static const struct ctype_methods ctype_methods_builtin = {
+ .strlower = strlower_builtin,
+ .strtitle = strtitle_builtin,
+ .strupper = strupper_builtin,
+ .char_properties = char_properties_builtin,
+ .char_is_cased = char_is_cased_builtin,
+ .wc_tolower = wc_tolower_builtin,
+ .wc_toupper = wc_toupper_builtin,
+};
+
pg_locale_t
create_pg_locale_builtin(Oid collid, MemoryContext context)
{
@@ -65,6 +192,8 @@ create_pg_locale_builtin(Oid collid, MemoryContext context)
result->deterministic = true;
result->collate_is_c = true;
result->ctype_is_c = (strcmp(locstr, "C") == 0);
+ if (!result->ctype_is_c)
+ result->ctype = &ctype_methods_builtin;
return result;
}
diff --git a/src/backend/utils/adt/pg_locale_icu.c b/src/backend/utils/adt/pg_locale_icu.c
index 11ec9d4e4b..5251163b1b 100644
--- a/src/backend/utils/adt/pg_locale_icu.c
+++ b/src/backend/utils/adt/pg_locale_icu.c
@@ -51,6 +51,11 @@ static size_t strnxfrm_prefix_icu(char *dest, size_t destsize,
const char *src, ssize_t srclen,
pg_locale_t locale);
+typedef int32_t (*ICU_Convert_Func) (UChar *dest, int32_t destCapacity,
+ const UChar *src, int32_t srcLength,
+ const char *locale,
+ UErrorCode *pErrorCode);
+
/*
* Converter object for converting between ICU's UChar strings and C strings
* in database encoding. Since the database encoding doesn't change, we only
@@ -60,6 +65,16 @@ static UConverter *icu_converter = NULL;
static UCollator *make_icu_collator(const char *iculocstr,
const char *icurules);
+
+static size_t strlower_icu(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+static size_t strtitle_icu(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+static size_t strupper_icu(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
static int strncoll_icu(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2,
pg_locale_t locale);
@@ -80,8 +95,63 @@ static size_t uchar_length(UConverter *converter,
static int32_t uchar_convert(UConverter *converter,
UChar *dest, int32_t destlen,
const char *src, int32_t srclen);
+static int32_t icu_to_uchar(UChar **buff_uchar, const char *buff,
+ size_t nbytes);
+static size_t icu_from_uchar(char *dest, size_t destsize,
+ const UChar *buff_uchar, int32_t len_uchar);
static void icu_set_collation_attributes(UCollator *collator, const char *loc,
UErrorCode *status);
+static int32_t icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
+ UChar **buff_dest, UChar *buff_source,
+ int32_t len_source);
+static int32_t u_strToTitle_default_BI(UChar *dest, int32_t destCapacity,
+ const UChar *src, int32_t srcLength,
+ const char *locale,
+ UErrorCode *pErrorCode);
+
+static int
+char_properties_icu(pg_wchar wc, int mask, pg_locale_t locale)
+{
+ int result = 0;
+
+ if ((mask & PG_ISDIGIT) && u_isdigit(wc))
+ result |= PG_ISDIGIT;
+ if ((mask & PG_ISALPHA) && u_isalpha(wc))
+ result |= PG_ISALPHA;
+ if ((mask & PG_ISUPPER) && u_isupper(wc))
+ result |= PG_ISUPPER;
+ if ((mask & PG_ISLOWER) && u_islower(wc))
+ result |= PG_ISLOWER;
+ if ((mask & PG_ISGRAPH) && u_isgraph(wc))
+ result |= PG_ISGRAPH;
+ if ((mask & PG_ISPRINT) && u_isprint(wc))
+ result |= PG_ISPRINT;
+ if ((mask & PG_ISPUNCT) && u_ispunct(wc))
+ result |= PG_ISPUNCT;
+ if ((mask & PG_ISSPACE) && u_isspace(wc))
+ result |= PG_ISSPACE;
+
+ return result;
+}
+
+static bool
+char_is_cased_icu(char ch, pg_locale_t locale)
+{
+ return IS_HIGHBIT_SET(ch) ||
+ (ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z');
+}
+
+static pg_wchar
+toupper_icu(pg_wchar wc, pg_locale_t locale)
+{
+ return u_toupper(wc);
+}
+
+static pg_wchar
+tolower_icu(pg_wchar wc, pg_locale_t locale)
+{
+ return u_tolower(wc);
+}
static const struct collate_methods collate_methods_icu = {
.strncoll = strncoll_icu,
@@ -101,6 +171,15 @@ static const struct collate_methods collate_methods_icu_utf8 = {
.strxfrm_is_safe = true,
};
+static const struct ctype_methods ctype_methods_icu = {
+ .strlower = strlower_icu,
+ .strtitle = strtitle_icu,
+ .strupper = strupper_icu,
+ .char_properties = char_properties_icu,
+ .char_is_cased = char_is_cased_icu,
+ .wc_toupper = toupper_icu,
+ .wc_tolower = tolower_icu,
+};
#endif
pg_locale_t
@@ -171,6 +250,7 @@ create_pg_locale_icu(Oid collid, MemoryContext context)
result->collate = &collate_methods_icu_utf8;
else
result->collate = &collate_methods_icu;
+ result->ctype = &ctype_methods_icu;
return result;
#else
@@ -344,6 +424,66 @@ make_icu_collator(const char *iculocstr, const char *icurules)
}
}
+static size_t
+strlower_icu(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ int32_t len_uchar;
+ int32_t len_conv;
+ UChar *buff_uchar;
+ UChar *buff_conv;
+ size_t result_len;
+
+ len_uchar = icu_to_uchar(&buff_uchar, src, srclen);
+ len_conv = icu_convert_case(u_strToLower, locale,
+ &buff_conv, buff_uchar, len_uchar);
+ result_len = icu_from_uchar(dest, destsize, buff_conv, len_conv);
+ pfree(buff_uchar);
+ pfree(buff_conv);
+
+ return result_len;
+}
+
+static size_t
+strtitle_icu(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ int32_t len_uchar;
+ int32_t len_conv;
+ UChar *buff_uchar;
+ UChar *buff_conv;
+ size_t result_len;
+
+ len_uchar = icu_to_uchar(&buff_uchar, src, srclen);
+ len_conv = icu_convert_case(u_strToTitle_default_BI, locale,
+ &buff_conv, buff_uchar, len_uchar);
+ result_len = icu_from_uchar(dest, destsize, buff_conv, len_conv);
+ pfree(buff_uchar);
+ pfree(buff_conv);
+
+ return result_len;
+}
+
+static size_t
+strupper_icu(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ int32_t len_uchar;
+ int32_t len_conv;
+ UChar *buff_uchar;
+ UChar *buff_conv;
+ size_t result_len;
+
+ len_uchar = icu_to_uchar(&buff_uchar, src, srclen);
+ len_conv = icu_convert_case(u_strToUpper, locale,
+ &buff_conv, buff_uchar, len_uchar);
+ result_len = icu_from_uchar(dest, destsize, buff_conv, len_conv);
+ pfree(buff_uchar);
+ pfree(buff_conv);
+
+ return result_len;
+}
+
/*
* strncoll_icu_utf8
*
@@ -467,7 +607,7 @@ strnxfrm_prefix_icu_utf8(char *dest, size_t destsize,
* The result string is nul-terminated, though most callers rely on the
* result length instead.
*/
-int32_t
+static int32_t
icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes)
{
int32_t len_uchar;
@@ -494,8 +634,8 @@ icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes)
*
* The result string is nul-terminated.
*/
-int32_t
-icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
+static size_t
+icu_from_uchar(char *dest, size_t destsize, const UChar *buff_uchar, int32_t len_uchar)
{
UErrorCode status;
int32_t len_result;
@@ -510,10 +650,11 @@ icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
(errmsg("%s failed: %s", "ucnv_fromUChars",
u_errorName(status))));
- *result = palloc(len_result + 1);
+ if (len_result + 1 > destsize)
+ return len_result;
status = U_ZERO_ERROR;
- len_result = ucnv_fromUChars(icu_converter, *result, len_result + 1,
+ len_result = ucnv_fromUChars(icu_converter, dest, len_result + 1,
buff_uchar, len_uchar, &status);
if (U_FAILURE(status) ||
status == U_STRING_NOT_TERMINATED_WARNING)
@@ -524,6 +665,43 @@ icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
return len_result;
}
+static int32_t
+icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
+ UChar **buff_dest, UChar *buff_source, int32_t len_source)
+{
+ UErrorCode status;
+ int32_t len_dest;
+
+ len_dest = len_source; /* try first with same length */
+ *buff_dest = palloc(len_dest * sizeof(**buff_dest));
+ status = U_ZERO_ERROR;
+ len_dest = func(*buff_dest, len_dest, buff_source, len_source,
+ mylocale->info.icu.locale, &status);
+ if (status == U_BUFFER_OVERFLOW_ERROR)
+ {
+ /* try again with adjusted length */
+ pfree(*buff_dest);
+ *buff_dest = palloc(len_dest * sizeof(**buff_dest));
+ status = U_ZERO_ERROR;
+ len_dest = func(*buff_dest, len_dest, buff_source, len_source,
+ mylocale->info.icu.locale, &status);
+ }
+ if (U_FAILURE(status))
+ ereport(ERROR,
+ (errmsg("case conversion failed: %s", u_errorName(status))));
+ return len_dest;
+}
+
+static int32_t
+u_strToTitle_default_BI(UChar *dest, int32_t destCapacity,
+ const UChar *src, int32_t srcLength,
+ const char *locale,
+ UErrorCode *pErrorCode)
+{
+ return u_strToTitle(dest, destCapacity, src, srcLength,
+ NULL, locale, pErrorCode);
+}
+
/*
* strncoll_icu
*
diff --git a/src/backend/utils/adt/pg_locale_libc.c b/src/backend/utils/adt/pg_locale_libc.c
index c7be6dd4f9..861c4f68e4 100644
--- a/src/backend/utils/adt/pg_locale_libc.c
+++ b/src/backend/utils/adt/pg_locale_libc.c
@@ -11,6 +11,9 @@
#include "postgres.h"
+#include <limits.h>
+#include <wctype.h>
+
#include "access/htup_details.h"
#include "catalog/pg_database.h"
#include "catalog/pg_collation.h"
@@ -48,6 +51,34 @@ static int strncoll_libc_win32_utf8(const char *arg1, ssize_t len1,
pg_locale_t locale);
#endif
+static size_t strlower_libc_sb(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+static size_t strlower_libc_mb(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+static size_t strtitle_libc_sb(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+static size_t strtitle_libc_mb(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+static size_t strupper_libc_sb(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+static size_t strupper_libc_mb(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+
+static int char_properties_libc_sb(pg_wchar wc, int mask,
+ pg_locale_t locale);
+static int char_properties_libc_mb(pg_wchar wc, int mask,
+ pg_locale_t locale);
+static pg_wchar toupper_libc_sb(pg_wchar wc, pg_locale_t locale);
+static pg_wchar toupper_libc_mb(pg_wchar wc, pg_locale_t locale);
+static pg_wchar tolower_libc_sb(pg_wchar wc, pg_locale_t locale);
+static pg_wchar tolower_libc_mb(pg_wchar wc, pg_locale_t locale);
+
static const struct collate_methods collate_methods_libc = {
.strncoll = strncoll_libc,
.strnxfrm = strnxfrm_libc,
@@ -69,6 +100,324 @@ static const struct collate_methods collate_methods_libc = {
#endif
};
+#ifdef WIN32
+static const struct collate_methods collate_methods_libc_win32_utf8 = {
+ .strncoll = strncoll_libc_win32_utf8,
+ .strnxfrm = strnxfrm_libc,
+ .strnxfrm_prefix = NULL,
+#ifdef TRUST_STRXFRM
+ .strxfrm_is_safe = true,
+#else
+ .strxfrm_is_safe = false,
+#endif
+};
+#endif
+
+static bool
+char_is_cased_libc(char ch, pg_locale_t locale)
+{
+ bool is_multibyte = pg_database_encoding_max_length() > 1;
+
+ if (is_multibyte && IS_HIGHBIT_SET(ch))
+ return true;
+ else
+ return isalpha_l((unsigned char) ch, locale->info.lt);
+}
+
+static char
+char_tolower_libc(unsigned char ch, pg_locale_t locale)
+{
+ Assert(pg_database_encoding_max_length() == 1);
+ return tolower_l(ch, locale->info.lt);
+}
+
+static const struct ctype_methods ctype_methods_libc_sb = {
+ .strlower = strlower_libc_sb,
+ .strtitle = strtitle_libc_sb,
+ .strupper = strupper_libc_sb,
+ .char_properties = char_properties_libc_sb,
+ .char_is_cased = char_is_cased_libc,
+ .char_tolower = char_tolower_libc,
+ .wc_toupper = toupper_libc_sb,
+ .wc_tolower = tolower_libc_sb,
+ .max_chr = UCHAR_MAX,
+};
+
+/*
+ * Non-UTF8 multibyte encodings use multibyte semantics for case mapping, but
+ * single-byte semantics for pattern matching.
+ */
+static const struct ctype_methods ctype_methods_libc_other_mb = {
+ .strlower = strlower_libc_mb,
+ .strtitle = strtitle_libc_mb,
+ .strupper = strupper_libc_mb,
+ .char_properties = char_properties_libc_sb,
+ .char_is_cased = char_is_cased_libc,
+ .char_tolower = char_tolower_libc,
+ .wc_toupper = toupper_libc_sb,
+ .wc_tolower = tolower_libc_sb,
+ .max_chr = UCHAR_MAX,
+};
+
+static const struct ctype_methods ctype_methods_libc_utf8 = {
+ .strlower = strlower_libc_mb,
+ .strtitle = strtitle_libc_mb,
+ .strupper = strupper_libc_mb,
+ .char_properties = char_properties_libc_mb,
+ .char_is_cased = char_is_cased_libc,
+ .char_tolower = char_tolower_libc,
+ .wc_toupper = toupper_libc_mb,
+ .wc_tolower = tolower_libc_mb,
+};
+
+static size_t
+strlower_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ if (srclen < 0)
+ srclen = strlen(src);
+
+ if (srclen + 1 <= destsize)
+ {
+ locale_t loc = locale->info.lt;
+ char *p;
+
+ if (srclen + 1 > destsize)
+ return srclen;
+
+ memcpy(dest, src, srclen);
+ dest[srclen] = '\0';
+
+ /*
+ * Note: we assume that tolower_l() will not be so broken as to need
+ * an isupper_l() guard test. When using the default collation, we
+ * apply the traditional Postgres behavior that forces ASCII-style
+ * treatment of I/i, but in non-default collations you get exactly
+ * what the collation says.
+ */
+ for (p = dest; *p; p++)
+ *p = tolower_l((unsigned char) *p, loc);
+ }
+
+ return srclen;
+}
+
+static size_t
+strlower_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ locale_t loc = locale->info.lt;
+ size_t result_size;
+ wchar_t *workspace;
+ char *result;
+ size_t curr_char;
+ size_t max_size;
+
+ if (srclen < 0)
+ srclen = strlen(src);
+
+ /* Overflow paranoia */
+ if ((srclen + 1) > (INT_MAX / sizeof(wchar_t)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ /* Output workspace cannot have more codes than input bytes */
+ workspace = (wchar_t *) palloc((srclen + 1) * sizeof(wchar_t));
+
+ char2wchar(workspace, srclen + 1, src, srclen, locale);
+
+ for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
+ workspace[curr_char] = towlower_l(workspace[curr_char], loc);
+
+ /*
+ * Make result large enough; case change might change number of bytes
+ */
+ max_size = curr_char * pg_database_encoding_max_length();
+ result = palloc(max_size + 1);
+
+ result_size = wchar2char(result, workspace, max_size + 1, locale);
+
+ if (result_size + 1 > destsize)
+ return result_size;
+
+ memcpy(dest, result, result_size);
+ dest[result_size] = '\0';
+
+ pfree(workspace);
+ pfree(result);
+
+ return result_size;
+}
+
+static size_t
+strtitle_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ if (srclen < 0)
+ srclen = strlen(src);
+
+ if (srclen + 1 <= destsize)
+ {
+ locale_t loc = locale->info.lt;
+ int wasalnum = false;
+ char *p;
+
+ memcpy(dest, src, srclen);
+ dest[srclen] = '\0';
+
+ /*
+ * Note: we assume that toupper_l()/tolower_l() will not be so broken
+ * as to need guard tests. When using the default collation, we apply
+ * the traditional Postgres behavior that forces ASCII-style treatment
+ * of I/i, but in non-default collations you get exactly what the
+ * collation says.
+ */
+ for (p = dest; *p; p++)
+ {
+ if (wasalnum)
+ *p = tolower_l((unsigned char) *p, loc);
+ else
+ *p = toupper_l((unsigned char) *p, loc);
+ wasalnum = isalnum_l((unsigned char) *p, loc);
+ }
+ }
+
+ return srclen;
+}
+
+static size_t
+strtitle_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ locale_t loc = locale->info.lt;
+ int wasalnum = false;
+ size_t result_size;
+ wchar_t *workspace;
+ char *result;
+ size_t curr_char;
+ size_t max_size;
+
+ if (srclen < 0)
+ srclen = strlen(src);
+
+ /* Overflow paranoia */
+ if ((srclen + 1) > (INT_MAX / sizeof(wchar_t)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ /* Output workspace cannot have more codes than input bytes */
+ workspace = (wchar_t *) palloc((srclen + 1) * sizeof(wchar_t));
+
+ char2wchar(workspace, srclen + 1, src, srclen, locale);
+
+ for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
+ {
+ if (wasalnum)
+ workspace[curr_char] = towlower_l(workspace[curr_char], loc);
+ else
+ workspace[curr_char] = towupper_l(workspace[curr_char], loc);
+ wasalnum = iswalnum_l(workspace[curr_char], loc);
+ }
+
+ /*
+ * Make result large enough; case change might change number of bytes
+ */
+ max_size = curr_char * pg_database_encoding_max_length();
+ result = palloc(max_size + 1);
+
+ result_size = wchar2char(result, workspace, max_size + 1, locale);
+
+ if (result_size + 1 > destsize)
+ return result_size;
+
+ memcpy(dest, result, result_size);
+ dest[result_size] = '\0';
+
+ pfree(workspace);
+ pfree(result);
+
+ return result_size;
+}
+
+static size_t
+strupper_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ if (srclen < 0)
+ srclen = strlen(src);
+
+ if (srclen + 1 <= destsize)
+ {
+ locale_t loc = locale->info.lt;
+ char *p;
+
+ memcpy(dest, src, srclen);
+ dest[srclen] = '\0';
+
+ /*
+ * Note: we assume that toupper_l() will not be so broken as to need
+ * an islower_l() guard test. When using the default collation, we
+ * apply the traditional Postgres behavior that forces ASCII-style
+ * treatment of I/i, but in non-default collations you get exactly
+ * what the collation says.
+ */
+ for (p = dest; *p; p++)
+ *p = toupper_l((unsigned char) *p, loc);
+ }
+
+ return srclen;
+}
+
+static size_t
+strupper_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
+ pg_locale_t locale)
+{
+ locale_t loc = locale->info.lt;
+ size_t result_size;
+ wchar_t *workspace;
+ char *result;
+ size_t curr_char;
+ size_t max_size;
+
+ if (srclen < 0)
+ srclen = strlen(src);
+
+ /* Overflow paranoia */
+ if ((srclen + 1) > (INT_MAX / sizeof(wchar_t)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ /* Output workspace cannot have more codes than input bytes */
+ workspace = (wchar_t *) palloc((srclen + 1) * sizeof(wchar_t));
+
+ char2wchar(workspace, srclen + 1, src, srclen, locale);
+
+ for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
+ workspace[curr_char] = towupper_l(workspace[curr_char], loc);
+
+ /*
+ * Make result large enough; case change might change number of bytes
+ */
+ max_size = curr_char * pg_database_encoding_max_length();
+ result = palloc(max_size + 1);
+
+ result_size = wchar2char(result, workspace, max_size + 1, locale);
+
+ if (result_size + 1 > destsize)
+ return result_size;
+
+ memcpy(dest, result, result_size);
+ dest[result_size] = '\0';
+
+ pfree(workspace);
+ pfree(result);
+
+ return result_size;
+}
+
pg_locale_t
create_pg_locale_libc(Oid collid, MemoryContext context)
{
@@ -133,6 +482,15 @@ create_pg_locale_libc(Oid collid, MemoryContext context)
#endif
result->collate = &collate_methods_libc;
}
+ if (!result->ctype_is_c)
+ {
+ if (GetDatabaseEncoding() == PG_UTF8)
+ result->ctype = &ctype_methods_libc_utf8;
+ else if (pg_database_encoding_max_length() > 1)
+ result->ctype = &ctype_methods_libc_other_mb;
+ else
+ result->ctype = &ctype_methods_libc_sb;
+ }
return result;
}
@@ -416,6 +774,113 @@ report_newlocale_failure(const char *localename)
localename) : 0)));
}
+static int
+char_properties_libc_sb(pg_wchar wc, int mask, pg_locale_t locale)
+{
+ int result = 0;
+
+ Assert(!locale->ctype_is_c);
+ Assert(GetDatabaseEncoding() != PG_UTF8);
+
+ if (wc > (pg_wchar) UCHAR_MAX)
+ return 0;
+
+ if ((mask & PG_ISDIGIT) && isdigit_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISDIGIT;
+ if ((mask & PG_ISALPHA) && isalpha_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISALPHA;
+ if ((mask & PG_ISUPPER) && isupper_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISUPPER;
+ if ((mask & PG_ISLOWER) && islower_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISLOWER;
+ if ((mask & PG_ISGRAPH) && isgraph_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISGRAPH;
+ if ((mask & PG_ISPRINT) && isprint_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISPRINT;
+ if ((mask & PG_ISPUNCT) && ispunct_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISPUNCT;
+ if ((mask & PG_ISSPACE) && isspace_l((unsigned char) wc, locale->info.lt))
+ result |= PG_ISSPACE;
+
+ return result;
+}
+
+static int
+char_properties_libc_mb(pg_wchar wc, int mask, pg_locale_t locale)
+{
+ int result = 0;
+
+ Assert(!locale->ctype_is_c);
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* if wchar_t cannot represent the value, just return 0 */
+ if (sizeof(wchar_t) < 4 && wc > (pg_wchar) 0xFFFF)
+ return 0;
+
+ if ((mask & PG_ISDIGIT) && iswdigit_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISDIGIT;
+ if ((mask & PG_ISALPHA) && iswalpha_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISALPHA;
+ if ((mask & PG_ISUPPER) && iswupper_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISUPPER;
+ if ((mask & PG_ISLOWER) && iswlower_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISLOWER;
+ if ((mask & PG_ISGRAPH) && iswgraph_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISGRAPH;
+ if ((mask & PG_ISPRINT) && iswprint_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISPRINT;
+ if ((mask & PG_ISPUNCT) && iswpunct_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISPUNCT;
+ if ((mask & PG_ISSPACE) && iswspace_l((wint_t) wc, locale->info.lt))
+ result |= PG_ISSPACE;
+
+ return result;
+}
+
+static pg_wchar
+toupper_libc_sb(pg_wchar wc, pg_locale_t locale)
+{
+ Assert(GetDatabaseEncoding() != PG_UTF8);
+
+ if (wc <= (pg_wchar) UCHAR_MAX)
+ return toupper_l((unsigned char) wc, locale->info.lt);
+ else
+ return wc;
+}
+
+static pg_wchar
+toupper_libc_mb(pg_wchar wc, pg_locale_t locale)
+{
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ if (sizeof(wchar_t) >= 4 || wc <= (pg_wchar) 0xFFFF)
+ return towupper_l((wint_t) wc, locale->info.lt);
+ else
+ return wc;
+}
+
+static pg_wchar
+tolower_libc_sb(pg_wchar wc, pg_locale_t locale)
+{
+ Assert(GetDatabaseEncoding() != PG_UTF8);
+
+ if (wc <= (pg_wchar) UCHAR_MAX)
+ return tolower_l((unsigned char) wc, locale->info.lt);
+ else
+ return wc;
+}
+
+static pg_wchar
+tolower_libc_mb(pg_wchar wc, pg_locale_t locale)
+{
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ if (sizeof(wchar_t) >= 4 || wc <= (pg_wchar) 0xFFFF)
+ return towlower_l((wint_t) wc, locale->info.lt);
+ else
+ return wc;
+}
+
/*
* POSIX doesn't define _l-variants of these functions, but several systems
* have them. We provide our own replacements here.
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 2f05dffcdd..c71cf38020 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -12,10 +12,25 @@
#ifndef _PG_LOCALE_
#define _PG_LOCALE_
+#include "mb/pg_wchar.h"
+
#ifdef USE_ICU
#include <unicode/ucol.h>
#endif
+/*
+ * Character properties for regular expressions.
+ */
+#define PG_ISDIGIT 0x01
+#define PG_ISALPHA 0x02
+#define PG_ISALNUM (PG_ISDIGIT | PG_ISALPHA)
+#define PG_ISUPPER 0x04
+#define PG_ISLOWER 0x08
+#define PG_ISGRAPH 0x10
+#define PG_ISPRINT 0x20
+#define PG_ISPUNCT 0x40
+#define PG_ISSPACE 0x80
+
#ifdef USE_ICU
/*
* ucol_strcollUTF8() was introduced in ICU 50, but it is buggy before ICU 53.
@@ -90,6 +105,43 @@ struct collate_methods
bool strxfrm_is_safe;
};
+struct ctype_methods
+{
+ /* case mapping: LOWER()/INITCAP()/UPPER() */
+ size_t (*strlower) (char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+ size_t (*strtitle) (char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+ size_t (*strupper) (char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+
+ /* required */
+ int (*char_properties) (pg_wchar wc, int mask, pg_locale_t locale);
+
+ /* required */
+ bool (*char_is_cased) (char ch, pg_locale_t locale);
+
+ /*
+ * Optional. If defined, will only be called for single-byte encodings. If
+ * not defined, or if the encoding is multibyte, will fall back to
+ * pg_strlower().
+ */
+ char (*char_tolower) (unsigned char ch, pg_locale_t locale);
+
+ /* required */
+ pg_wchar (*wc_toupper) (pg_wchar wc, pg_locale_t locale);
+ pg_wchar (*wc_tolower) (pg_wchar wc, pg_locale_t locale);
+
+ /*
+ * For regex and pattern matching efficiency, the maximum char value
+ * supported by the above methods. If zero, limit is set by regex code.
+ */
+ pg_wchar max_chr;
+};
+
/*
* We use a discriminated union to hold either a locale_t or an ICU collator.
* pg_locale_t is occasionally checked for truth, so make it a pointer.
@@ -114,6 +166,7 @@ struct pg_locale_struct
bool ctype_is_c;
const struct collate_methods *collate; /* NULL if collate_is_c */
+ const struct ctype_methods *ctype; /* NULL if ctype_is_c */
union
{
@@ -138,6 +191,19 @@ extern void init_database_collation(void);
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern int char_properties(pg_wchar wc, int mask, pg_locale_t locale);
+extern bool char_is_cased(char ch, pg_locale_t locale);
+extern bool char_tolower_enabled(pg_locale_t locale);
+extern char char_tolower(unsigned char ch, pg_locale_t locale);
+extern size_t pg_strlower(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+extern size_t pg_strtitle(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
+extern size_t pg_strupper(char *dest, size_t destsize,
+ const char *src, ssize_t srclen,
+ pg_locale_t locale);
extern int pg_strcoll(const char *arg1, const char *arg2, pg_locale_t locale);
extern int pg_strncoll(const char *arg1, ssize_t len1,
const char *arg2, ssize_t len2, pg_locale_t locale);
@@ -157,11 +223,6 @@ extern const char *builtin_validate_locale(int encoding, const char *locale);
extern void icu_validate_locale(const char *loc_str);
extern char *icu_language_tag(const char *loc_str, int elevel);
-#ifdef USE_ICU
-extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
-extern int32_t icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar);
-#endif
-
/* These functions convert from/to libc's wchar_t, *not* pg_wchar_t */
extern size_t wchar2char(char *to, const wchar_t *from, size_t tolen,
pg_locale_t locale);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 08521d51a9..416a2cc76b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1822,7 +1822,6 @@ PGTargetServerType
PGTernaryBool
PGTransactionStatusType
PGVerbosity
-PG_Locale_Strategy
PG_Lock_Status
PG_init_t
PGcancel
--
2.34.1
[text/x-patch] v8-0004-Remove-provider-field-from-pg_locale_t.patch (4.8K, ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/5-v8-0004-Remove-provider-field-from-pg_locale_t.patch)
download | inline diff:
From 7275cd18cfba39377cdc8e22e361a135c4c2eba7 Mon Sep 17 00:00:00 2001
From: Jeff Davis <jeff@j-davis.com>
Date: Mon, 7 Oct 2024 12:51:27 -0700
Subject: [PATCH v8 4/7] Remove provider field from pg_locale_t.
The behavior of pg_locale_t is entirely specified by methods, so a
separate provider field is no longer necessary.
---
src/backend/utils/adt/pg_locale_builtin.c | 1 -
src/backend/utils/adt/pg_locale_icu.c | 11 -----------
src/backend/utils/adt/pg_locale_libc.c | 6 ------
src/include/utils/pg_locale.h | 1 -
4 files changed, 19 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale_builtin.c b/src/backend/utils/adt/pg_locale_builtin.c
index 5f90355355..b4abde1afd 100644
--- a/src/backend/utils/adt/pg_locale_builtin.c
+++ b/src/backend/utils/adt/pg_locale_builtin.c
@@ -188,7 +188,6 @@ create_pg_locale_builtin(Oid collid, MemoryContext context)
result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
result->info.builtin.locale = MemoryContextStrdup(context, locstr);
- result->provider = COLLPROVIDER_BUILTIN;
result->deterministic = true;
result->collate_is_c = true;
result->ctype_is_c = (strcmp(locstr, "C") == 0);
diff --git a/src/backend/utils/adt/pg_locale_icu.c b/src/backend/utils/adt/pg_locale_icu.c
index 5251163b1b..7a2ca8ac84 100644
--- a/src/backend/utils/adt/pg_locale_icu.c
+++ b/src/backend/utils/adt/pg_locale_icu.c
@@ -242,7 +242,6 @@ create_pg_locale_icu(Oid collid, MemoryContext context)
result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
result->info.icu.locale = MemoryContextStrdup(context, iculocstr);
result->info.icu.ucol = collator;
- result->provider = COLLPROVIDER_ICU;
result->deterministic = deterministic;
result->collate_is_c = false;
result->ctype_is_c = false;
@@ -499,8 +498,6 @@ strncoll_icu_utf8(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2
int result;
UErrorCode status;
- Assert(locale->provider == COLLPROVIDER_ICU);
-
Assert(GetDatabaseEncoding() == PG_UTF8);
status = U_ZERO_ERROR;
@@ -528,8 +525,6 @@ strnxfrm_icu(char *dest, size_t destsize, const char *src, ssize_t srclen,
size_t uchar_bsize;
Size result_bsize;
- Assert(locale->provider == COLLPROVIDER_ICU);
-
init_icu_converter();
ulen = uchar_length(icu_converter, src, srclen);
@@ -574,8 +569,6 @@ strnxfrm_prefix_icu_utf8(char *dest, size_t destsize,
uint32_t state[2];
UErrorCode status;
- Assert(locale->provider == COLLPROVIDER_ICU);
-
Assert(GetDatabaseEncoding() == PG_UTF8);
uiter_setUTF8(&iter, src, srclen);
@@ -726,8 +719,6 @@ strncoll_icu(const char *arg1, ssize_t len1,
*uchar2;
int result;
- Assert(locale->provider == COLLPROVIDER_ICU);
-
/* if encoding is UTF8, use more efficient strncoll_icu_utf8 */
#ifdef HAVE_UCOL_STRCOLLUTF8
Assert(GetDatabaseEncoding() != PG_UTF8);
@@ -776,8 +767,6 @@ strnxfrm_prefix_icu(char *dest, size_t destsize,
size_t uchar_bsize;
Size result_bsize;
- Assert(locale->provider == COLLPROVIDER_ICU);
-
/* if encoding is UTF8, use more efficient strnxfrm_prefix_icu_utf8 */
Assert(GetDatabaseEncoding() != PG_UTF8);
diff --git a/src/backend/utils/adt/pg_locale_libc.c b/src/backend/utils/adt/pg_locale_libc.c
index 861c4f68e4..7708e9de96 100644
--- a/src/backend/utils/adt/pg_locale_libc.c
+++ b/src/backend/utils/adt/pg_locale_libc.c
@@ -466,7 +466,6 @@ create_pg_locale_libc(Oid collid, MemoryContext context)
loc = make_libc_collator(collate, ctype);
result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
- result->provider = COLLPROVIDER_LIBC;
result->deterministic = true;
result->collate_is_c = (strcmp(collate, "C") == 0) ||
(strcmp(collate, "POSIX") == 0);
@@ -586,8 +585,6 @@ strncoll_libc(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2,
const char *arg2n;
int result;
- Assert(locale->provider == COLLPROVIDER_LIBC);
-
if (bufsize1 + bufsize2 > TEXTBUFLEN)
buf = palloc(bufsize1 + bufsize2);
@@ -642,8 +639,6 @@ strnxfrm_libc(char *dest, size_t destsize, const char *src, ssize_t srclen,
size_t bufsize = srclen + 1;
size_t result;
- Assert(locale->provider == COLLPROVIDER_LIBC);
-
if (srclen == -1)
return strxfrm_l(dest, src, destsize, locale->info.lt);
@@ -687,7 +682,6 @@ strncoll_libc_win32_utf8(const char *arg1, ssize_t len1, const char *arg2,
int r;
int result;
- Assert(locale->provider == COLLPROVIDER_LIBC);
Assert(GetDatabaseEncoding() == PG_UTF8);
if (len1 == -1)
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index c71cf38020..98a860d0b3 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -160,7 +160,6 @@ struct ctype_methods
*/
struct pg_locale_struct
{
- char provider;
bool deterministic;
bool collate_is_c;
bool ctype_is_c;
--
2.34.1
[text/x-patch] v8-0005-Make-provider-data-in-pg_locale_t-an-opaque-point.patch (21.5K, ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/6-v8-0005-Make-provider-data-in-pg_locale_t-an-opaque-point.patch)
download | inline diff:
From e1c6414fcabd075b33a4ee1edbd3231b657c9e65 Mon Sep 17 00:00:00 2001
From: Jeff Davis <jeff@j-davis.com>
Date: Mon, 7 Oct 2024 13:36:44 -0700
Subject: [PATCH v8 5/7] Make provider data in pg_locale_t an opaque pointer.
---
src/backend/utils/adt/pg_locale_builtin.c | 11 +-
src/backend/utils/adt/pg_locale_icu.c | 40 +++++--
src/backend/utils/adt/pg_locale_libc.c | 131 ++++++++++++++--------
src/include/utils/pg_locale.h | 16 +--
4 files changed, 127 insertions(+), 71 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale_builtin.c b/src/backend/utils/adt/pg_locale_builtin.c
index b4abde1afd..a1eb0e459c 100644
--- a/src/backend/utils/adt/pg_locale_builtin.c
+++ b/src/backend/utils/adt/pg_locale_builtin.c
@@ -25,6 +25,11 @@
extern pg_locale_t create_pg_locale_builtin(Oid collid,
MemoryContext context);
+struct builtin_provider
+{
+ const char *locale;
+};
+
struct WordBoundaryState
{
const char *str;
@@ -154,6 +159,7 @@ pg_locale_t
create_pg_locale_builtin(Oid collid, MemoryContext context)
{
const char *locstr;
+ struct builtin_provider *builtin;
pg_locale_t result;
if (collid == DEFAULT_COLLATION_OID)
@@ -187,7 +193,10 @@ create_pg_locale_builtin(Oid collid, MemoryContext context)
result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
- result->info.builtin.locale = MemoryContextStrdup(context, locstr);
+ builtin = MemoryContextAllocZero(context, sizeof(struct builtin_provider));
+ builtin->locale = MemoryContextStrdup(context, locstr);
+ result->provider_data = (void *) builtin;
+
result->deterministic = true;
result->collate_is_c = true;
result->ctype_is_c = (strcmp(locstr, "C") == 0);
diff --git a/src/backend/utils/adt/pg_locale_icu.c b/src/backend/utils/adt/pg_locale_icu.c
index 7a2ca8ac84..57f366f741 100644
--- a/src/backend/utils/adt/pg_locale_icu.c
+++ b/src/backend/utils/adt/pg_locale_icu.c
@@ -39,6 +39,12 @@ extern pg_locale_t create_pg_locale_icu(Oid collid, MemoryContext context);
#ifdef USE_ICU
+struct icu_provider
+{
+ const char *locale;
+ UCollator *ucol;
+};
+
extern UCollator *pg_ucol_open(const char *loc_str);
static int strncoll_icu(const char *arg1, ssize_t len1,
@@ -189,6 +195,7 @@ create_pg_locale_icu(Oid collid, MemoryContext context)
bool deterministic;
const char *iculocstr;
const char *icurules = NULL;
+ struct icu_provider *icu;
UCollator *collator;
pg_locale_t result;
@@ -240,8 +247,12 @@ create_pg_locale_icu(Oid collid, MemoryContext context)
collator = make_icu_collator(iculocstr, icurules);
result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
- result->info.icu.locale = MemoryContextStrdup(context, iculocstr);
- result->info.icu.ucol = collator;
+
+ icu = MemoryContextAllocZero(context, sizeof(struct icu_provider));
+ icu->locale = MemoryContextStrdup(context, iculocstr);
+ icu->ucol = collator;
+ result->provider_data = (void *) icu;
+
result->deterministic = deterministic;
result->collate_is_c = false;
result->ctype_is_c = false;
@@ -497,11 +508,12 @@ strncoll_icu_utf8(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2
{
int result;
UErrorCode status;
+ struct icu_provider *icu = (struct icu_provider *) locale->provider_data;
Assert(GetDatabaseEncoding() == PG_UTF8);
status = U_ZERO_ERROR;
- result = ucol_strcollUTF8(locale->info.icu.ucol,
+ result = ucol_strcollUTF8(icu->ucol,
arg1, len1,
arg2, len2,
&status);
@@ -525,6 +537,8 @@ strnxfrm_icu(char *dest, size_t destsize, const char *src, ssize_t srclen,
size_t uchar_bsize;
Size result_bsize;
+ struct icu_provider *icu = (struct icu_provider *) locale->provider_data;
+
init_icu_converter();
ulen = uchar_length(icu_converter, src, srclen);
@@ -538,7 +552,7 @@ strnxfrm_icu(char *dest, size_t destsize, const char *src, ssize_t srclen,
ulen = uchar_convert(icu_converter, uchar, ulen + 1, src, srclen);
- result_bsize = ucol_getSortKey(locale->info.icu.ucol,
+ result_bsize = ucol_getSortKey(icu->ucol,
uchar, ulen,
(uint8_t *) dest, destsize);
@@ -569,12 +583,14 @@ strnxfrm_prefix_icu_utf8(char *dest, size_t destsize,
uint32_t state[2];
UErrorCode status;
+ struct icu_provider *icu = (struct icu_provider *) locale->provider_data;
+
Assert(GetDatabaseEncoding() == PG_UTF8);
uiter_setUTF8(&iter, src, srclen);
state[0] = state[1] = 0; /* won't need that again */
status = U_ZERO_ERROR;
- result = ucol_nextSortKeyPart(locale->info.icu.ucol,
+ result = ucol_nextSortKeyPart(icu->ucol,
&iter,
state,
(uint8_t *) dest,
@@ -665,11 +681,13 @@ icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
UErrorCode status;
int32_t len_dest;
+ struct icu_provider *icu = (struct icu_provider *) mylocale->provider_data;
+
len_dest = len_source; /* try first with same length */
*buff_dest = palloc(len_dest * sizeof(**buff_dest));
status = U_ZERO_ERROR;
len_dest = func(*buff_dest, len_dest, buff_source, len_source,
- mylocale->info.icu.locale, &status);
+ icu->locale, &status);
if (status == U_BUFFER_OVERFLOW_ERROR)
{
/* try again with adjusted length */
@@ -677,7 +695,7 @@ icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
*buff_dest = palloc(len_dest * sizeof(**buff_dest));
status = U_ZERO_ERROR;
len_dest = func(*buff_dest, len_dest, buff_source, len_source,
- mylocale->info.icu.locale, &status);
+ icu->locale, &status);
}
if (U_FAILURE(status))
ereport(ERROR,
@@ -719,6 +737,8 @@ strncoll_icu(const char *arg1, ssize_t len1,
*uchar2;
int result;
+ struct icu_provider *icu = (struct icu_provider *) locale->provider_data;
+
/* if encoding is UTF8, use more efficient strncoll_icu_utf8 */
#ifdef HAVE_UCOL_STRCOLLUTF8
Assert(GetDatabaseEncoding() != PG_UTF8);
@@ -741,7 +761,7 @@ strncoll_icu(const char *arg1, ssize_t len1,
ulen1 = uchar_convert(icu_converter, uchar1, ulen1 + 1, arg1, len1);
ulen2 = uchar_convert(icu_converter, uchar2, ulen2 + 1, arg2, len2);
- result = ucol_strcoll(locale->info.icu.ucol,
+ result = ucol_strcoll(icu->ucol,
uchar1, ulen1,
uchar2, ulen2);
@@ -767,6 +787,8 @@ strnxfrm_prefix_icu(char *dest, size_t destsize,
size_t uchar_bsize;
Size result_bsize;
+ struct icu_provider *icu = (struct icu_provider *) locale->provider_data;
+
/* if encoding is UTF8, use more efficient strnxfrm_prefix_icu_utf8 */
Assert(GetDatabaseEncoding() != PG_UTF8);
@@ -786,7 +808,7 @@ strnxfrm_prefix_icu(char *dest, size_t destsize,
uiter_setString(&iter, uchar, ulen);
state[0] = state[1] = 0; /* won't need that again */
status = U_ZERO_ERROR;
- result_bsize = ucol_nextSortKeyPart(locale->info.icu.ucol,
+ result_bsize = ucol_nextSortKeyPart(icu->ucol,
&iter,
state,
(uint8_t *) dest,
diff --git a/src/backend/utils/adt/pg_locale_libc.c b/src/backend/utils/adt/pg_locale_libc.c
index 7708e9de96..8738a0221b 100644
--- a/src/backend/utils/adt/pg_locale_libc.c
+++ b/src/backend/utils/adt/pg_locale_libc.c
@@ -1,3 +1,4 @@
+
/*-----------------------------------------------------------------------
*
* PostgreSQL locale utilities for libc
@@ -33,6 +34,11 @@
*/
#define TEXTBUFLEN 1024
+struct libc_provider
+{
+ locale_t lt;
+};
+
extern pg_locale_t create_pg_locale_libc(Oid collid, MemoryContext context);
static int strncoll_libc(const char *arg1, ssize_t len1,
@@ -118,17 +124,21 @@ char_is_cased_libc(char ch, pg_locale_t locale)
{
bool is_multibyte = pg_database_encoding_max_length() > 1;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
if (is_multibyte && IS_HIGHBIT_SET(ch))
return true;
else
- return isalpha_l((unsigned char) ch, locale->info.lt);
+ return isalpha_l((unsigned char) ch, libc->lt);
}
static char
char_tolower_libc(unsigned char ch, pg_locale_t locale)
{
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(pg_database_encoding_max_length() == 1);
- return tolower_l(ch, locale->info.lt);
+ return tolower_l(ch, libc->lt);
}
static const struct ctype_methods ctype_methods_libc_sb = {
@@ -179,7 +189,7 @@ strlower_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
if (srclen + 1 <= destsize)
{
- locale_t loc = locale->info.lt;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
char *p;
if (srclen + 1 > destsize)
@@ -196,7 +206,7 @@ strlower_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
* what the collation says.
*/
for (p = dest; *p; p++)
- *p = tolower_l((unsigned char) *p, loc);
+ *p = tolower_l((unsigned char) *p, libc->lt);
}
return srclen;
@@ -206,7 +216,8 @@ static size_t
strlower_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
pg_locale_t locale)
{
- locale_t loc = locale->info.lt;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
size_t result_size;
wchar_t *workspace;
char *result;
@@ -228,7 +239,7 @@ strlower_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
char2wchar(workspace, srclen + 1, src, srclen, locale);
for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
- workspace[curr_char] = towlower_l(workspace[curr_char], loc);
+ workspace[curr_char] = towlower_l(workspace[curr_char], libc->lt);
/*
* Make result large enough; case change might change number of bytes
@@ -259,7 +270,7 @@ strtitle_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
if (srclen + 1 <= destsize)
{
- locale_t loc = locale->info.lt;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
int wasalnum = false;
char *p;
@@ -276,10 +287,10 @@ strtitle_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
for (p = dest; *p; p++)
{
if (wasalnum)
- *p = tolower_l((unsigned char) *p, loc);
+ *p = tolower_l((unsigned char) *p, libc->lt);
else
- *p = toupper_l((unsigned char) *p, loc);
- wasalnum = isalnum_l((unsigned char) *p, loc);
+ *p = toupper_l((unsigned char) *p, libc->lt);
+ wasalnum = isalnum_l((unsigned char) *p, libc->lt);
}
}
@@ -290,7 +301,8 @@ static size_t
strtitle_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
pg_locale_t locale)
{
- locale_t loc = locale->info.lt;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
int wasalnum = false;
size_t result_size;
wchar_t *workspace;
@@ -315,10 +327,10 @@ strtitle_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
{
if (wasalnum)
- workspace[curr_char] = towlower_l(workspace[curr_char], loc);
+ workspace[curr_char] = towlower_l(workspace[curr_char], libc->lt);
else
- workspace[curr_char] = towupper_l(workspace[curr_char], loc);
- wasalnum = iswalnum_l(workspace[curr_char], loc);
+ workspace[curr_char] = towupper_l(workspace[curr_char], libc->lt);
+ wasalnum = iswalnum_l(workspace[curr_char], libc->lt);
}
/*
@@ -350,7 +362,7 @@ strupper_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
if (srclen + 1 <= destsize)
{
- locale_t loc = locale->info.lt;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
char *p;
memcpy(dest, src, srclen);
@@ -364,7 +376,7 @@ strupper_libc_sb(char *dest, size_t destsize, const char *src, ssize_t srclen,
* what the collation says.
*/
for (p = dest; *p; p++)
- *p = toupper_l((unsigned char) *p, loc);
+ *p = toupper_l((unsigned char) *p, libc->lt);
}
return srclen;
@@ -374,7 +386,8 @@ static size_t
strupper_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
pg_locale_t locale)
{
- locale_t loc = locale->info.lt;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
size_t result_size;
wchar_t *workspace;
char *result;
@@ -396,7 +409,7 @@ strupper_libc_mb(char *dest, size_t destsize, const char *src, ssize_t srclen,
char2wchar(workspace, srclen + 1, src, srclen, locale);
for (curr_char = 0; workspace[curr_char] != 0; curr_char++)
- workspace[curr_char] = towupper_l(workspace[curr_char], loc);
+ workspace[curr_char] = towupper_l(workspace[curr_char], libc->lt);
/*
* Make result large enough; case change might change number of bytes
@@ -424,6 +437,7 @@ create_pg_locale_libc(Oid collid, MemoryContext context)
const char *collate;
const char *ctype;
locale_t loc;
+ struct libc_provider *libc;
pg_locale_t result;
if (collid == DEFAULT_COLLATION_OID)
@@ -462,16 +476,19 @@ create_pg_locale_libc(Oid collid, MemoryContext context)
ReleaseSysCache(tp);
}
-
loc = make_libc_collator(collate, ctype);
result = MemoryContextAllocZero(context, sizeof(struct pg_locale_struct));
+
+ libc = MemoryContextAllocZero(context, sizeof(struct libc_provider));
+ libc->lt = loc;
+ result->provider_data = (void *) libc;
+
result->deterministic = true;
result->collate_is_c = (strcmp(collate, "C") == 0) ||
(strcmp(collate, "POSIX") == 0);
result->ctype_is_c = (strcmp(ctype, "C") == 0) ||
(strcmp(ctype, "POSIX") == 0);
- result->info.lt = loc;
if (!result->collate_is_c)
{
#ifdef WIN32
@@ -585,6 +602,8 @@ strncoll_libc(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2,
const char *arg2n;
int result;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
if (bufsize1 + bufsize2 > TEXTBUFLEN)
buf = palloc(bufsize1 + bufsize2);
@@ -615,7 +634,7 @@ strncoll_libc(const char *arg1, ssize_t len1, const char *arg2, ssize_t len2,
arg2n = buf2;
}
- result = strcoll_l(arg1n, arg2n, locale->info.lt);
+ result = strcoll_l(arg1n, arg2n, libc->lt);
if (buf != sbuf)
pfree(buf);
@@ -639,8 +658,10 @@ strnxfrm_libc(char *dest, size_t destsize, const char *src, ssize_t srclen,
size_t bufsize = srclen + 1;
size_t result;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
if (srclen == -1)
- return strxfrm_l(dest, src, destsize, locale->info.lt);
+ return strxfrm_l(dest, src, destsize, libc->lt);
if (bufsize > TEXTBUFLEN)
buf = palloc(bufsize);
@@ -649,7 +670,7 @@ strnxfrm_libc(char *dest, size_t destsize, const char *src, ssize_t srclen,
memcpy(buf, src, srclen);
buf[srclen] = '\0';
- result = strxfrm_l(dest, buf, destsize, locale->info.lt);
+ result = strxfrm_l(dest, buf, destsize, libc->lt);
if (buf != sbuf)
pfree(buf);
@@ -682,6 +703,8 @@ strncoll_libc_win32_utf8(const char *arg1, ssize_t len1, const char *arg2,
int r;
int result;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(GetDatabaseEncoding() == PG_UTF8);
if (len1 == -1)
@@ -726,7 +749,7 @@ strncoll_libc_win32_utf8(const char *arg1, ssize_t len1, const char *arg2,
((LPWSTR) a2p)[r] = 0;
errno = 0;
- result = wcscoll_l((LPWSTR) a1p, (LPWSTR) a2p, locale->info.lt);
+ result = wcscoll_l((LPWSTR) a1p, (LPWSTR) a2p, libc->lt);
if (result == 2147483647) /* _NLSCMPERROR; missing from mingw headers */
ereport(ERROR,
(errmsg("could not compare Unicode strings: %m")));
@@ -773,27 +796,29 @@ char_properties_libc_sb(pg_wchar wc, int mask, pg_locale_t locale)
{
int result = 0;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(!locale->ctype_is_c);
Assert(GetDatabaseEncoding() != PG_UTF8);
if (wc > (pg_wchar) UCHAR_MAX)
return 0;
- if ((mask & PG_ISDIGIT) && isdigit_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISDIGIT) && isdigit_l((unsigned char) wc, libc->lt))
result |= PG_ISDIGIT;
- if ((mask & PG_ISALPHA) && isalpha_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISALPHA) && isalpha_l((unsigned char) wc, libc->lt))
result |= PG_ISALPHA;
- if ((mask & PG_ISUPPER) && isupper_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISUPPER) && isupper_l((unsigned char) wc, libc->lt))
result |= PG_ISUPPER;
- if ((mask & PG_ISLOWER) && islower_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISLOWER) && islower_l((unsigned char) wc, libc->lt))
result |= PG_ISLOWER;
- if ((mask & PG_ISGRAPH) && isgraph_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISGRAPH) && isgraph_l((unsigned char) wc, libc->lt))
result |= PG_ISGRAPH;
- if ((mask & PG_ISPRINT) && isprint_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISPRINT) && isprint_l((unsigned char) wc, libc->lt))
result |= PG_ISPRINT;
- if ((mask & PG_ISPUNCT) && ispunct_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISPUNCT) && ispunct_l((unsigned char) wc, libc->lt))
result |= PG_ISPUNCT;
- if ((mask & PG_ISSPACE) && isspace_l((unsigned char) wc, locale->info.lt))
+ if ((mask & PG_ISSPACE) && isspace_l((unsigned char) wc, libc->lt))
result |= PG_ISSPACE;
return result;
@@ -804,6 +829,8 @@ char_properties_libc_mb(pg_wchar wc, int mask, pg_locale_t locale)
{
int result = 0;
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(!locale->ctype_is_c);
Assert(GetDatabaseEncoding() == PG_UTF8);
@@ -811,21 +838,21 @@ char_properties_libc_mb(pg_wchar wc, int mask, pg_locale_t locale)
if (sizeof(wchar_t) < 4 && wc > (pg_wchar) 0xFFFF)
return 0;
- if ((mask & PG_ISDIGIT) && iswdigit_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISDIGIT) && iswdigit_l((wint_t) wc, libc->lt))
result |= PG_ISDIGIT;
- if ((mask & PG_ISALPHA) && iswalpha_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISALPHA) && iswalpha_l((wint_t) wc, libc->lt))
result |= PG_ISALPHA;
- if ((mask & PG_ISUPPER) && iswupper_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISUPPER) && iswupper_l((wint_t) wc, libc->lt))
result |= PG_ISUPPER;
- if ((mask & PG_ISLOWER) && iswlower_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISLOWER) && iswlower_l((wint_t) wc, libc->lt))
result |= PG_ISLOWER;
- if ((mask & PG_ISGRAPH) && iswgraph_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISGRAPH) && iswgraph_l((wint_t) wc, libc->lt))
result |= PG_ISGRAPH;
- if ((mask & PG_ISPRINT) && iswprint_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISPRINT) && iswprint_l((wint_t) wc, libc->lt))
result |= PG_ISPRINT;
- if ((mask & PG_ISPUNCT) && iswpunct_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISPUNCT) && iswpunct_l((wint_t) wc, libc->lt))
result |= PG_ISPUNCT;
- if ((mask & PG_ISSPACE) && iswspace_l((wint_t) wc, locale->info.lt))
+ if ((mask & PG_ISSPACE) && iswspace_l((wint_t) wc, libc->lt))
result |= PG_ISSPACE;
return result;
@@ -834,10 +861,12 @@ char_properties_libc_mb(pg_wchar wc, int mask, pg_locale_t locale)
static pg_wchar
toupper_libc_sb(pg_wchar wc, pg_locale_t locale)
{
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(GetDatabaseEncoding() != PG_UTF8);
if (wc <= (pg_wchar) UCHAR_MAX)
- return toupper_l((unsigned char) wc, locale->info.lt);
+ return toupper_l((unsigned char) wc, libc->lt);
else
return wc;
}
@@ -845,10 +874,12 @@ toupper_libc_sb(pg_wchar wc, pg_locale_t locale)
static pg_wchar
toupper_libc_mb(pg_wchar wc, pg_locale_t locale)
{
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(GetDatabaseEncoding() == PG_UTF8);
if (sizeof(wchar_t) >= 4 || wc <= (pg_wchar) 0xFFFF)
- return towupper_l((wint_t) wc, locale->info.lt);
+ return towupper_l((wint_t) wc, libc->lt);
else
return wc;
}
@@ -856,10 +887,12 @@ toupper_libc_mb(pg_wchar wc, pg_locale_t locale)
static pg_wchar
tolower_libc_sb(pg_wchar wc, pg_locale_t locale)
{
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(GetDatabaseEncoding() != PG_UTF8);
if (wc <= (pg_wchar) UCHAR_MAX)
- return tolower_l((unsigned char) wc, locale->info.lt);
+ return tolower_l((unsigned char) wc, libc->lt);
else
return wc;
}
@@ -867,10 +900,12 @@ tolower_libc_sb(pg_wchar wc, pg_locale_t locale)
static pg_wchar
tolower_libc_mb(pg_wchar wc, pg_locale_t locale)
{
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
Assert(GetDatabaseEncoding() == PG_UTF8);
if (sizeof(wchar_t) >= 4 || wc <= (pg_wchar) 0xFFFF)
- return towlower_l((wint_t) wc, locale->info.lt);
+ return towlower_l((wint_t) wc, libc->lt);
else
return wc;
}
@@ -962,8 +997,10 @@ wchar2char(char *to, const wchar_t *from, size_t tolen, pg_locale_t locale)
}
else
{
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
/* Use wcstombs_l for nondefault locales */
- result = wcstombs_l(to, from, tolen, locale->info.lt);
+ result = wcstombs_l(to, from, tolen, libc->lt);
}
return result;
@@ -1022,8 +1059,10 @@ char2wchar(wchar_t *to, size_t tolen, const char *from, size_t fromlen,
}
else
{
+ struct libc_provider *libc = (struct libc_provider *) locale->provider_data;
+
/* Use mbstowcs_l for nondefault locales */
- result = mbstowcs_l(to, str, tolen, locale->info.lt);
+ result = mbstowcs_l(to, str, tolen, libc->lt);
}
pfree(str);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 98a860d0b3..c4c19a3c82 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -167,21 +167,7 @@ struct pg_locale_struct
const struct collate_methods *collate; /* NULL if collate_is_c */
const struct ctype_methods *ctype; /* NULL if ctype_is_c */
- union
- {
- struct
- {
- const char *locale;
- } builtin;
- locale_t lt;
-#ifdef USE_ICU
- struct
- {
- const char *locale;
- UCollator *ucol;
- } icu;
-#endif
- } info;
+ void *provider_data;
};
typedef struct pg_locale_struct *pg_locale_t;
--
2.34.1
[text/x-patch] v8-0006-Don-t-include-ICU-headers-in-pg_locale.h.patch (3.7K, ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/7-v8-0006-Don-t-include-ICU-headers-in-pg_locale.h.patch)
download | inline diff:
From 85f5b5f2bb9148e514181527c03e9d96bad68a5d Mon Sep 17 00:00:00 2001
From: Jeff Davis <jeff@j-davis.com>
Date: Wed, 9 Oct 2024 10:00:58 -0700
Subject: [PATCH v8 6/7] Don't include ICU headers in pg_locale.h.
---
src/backend/commands/collationcmds.c | 4 ++++
src/backend/utils/adt/formatting.c | 4 ----
src/backend/utils/adt/pg_locale.c | 4 ++++
src/backend/utils/adt/pg_locale_icu.c | 13 +++++++++++++
src/backend/utils/adt/varlena.c | 4 ++++
src/include/utils/pg_locale.h | 17 -----------------
6 files changed, 25 insertions(+), 21 deletions(-)
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 53b6a479aa..afc2330f51 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -14,6 +14,10 @@
*/
#include "postgres.h"
+#ifdef USE_ICU
+#include <unicode/ucol.h>
+#endif
+
#include "access/htup_details.h"
#include "access/table.h"
#include "access/xact.h"
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index 6a0571f93e..387009a4a9 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -71,10 +71,6 @@
#include <limits.h>
#include <wctype.h>
-#ifdef USE_ICU
-#include <unicode/ustring.h>
-#endif
-
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/unicode_case.h"
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2ebe4c00bf..d40ecf2357 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -54,6 +54,10 @@
#include <time.h>
+#ifdef USE_ICU
+#include <unicode/ucol.h>
+#endif
+
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_database.h"
diff --git a/src/backend/utils/adt/pg_locale_icu.c b/src/backend/utils/adt/pg_locale_icu.c
index 57f366f741..b8a455d730 100644
--- a/src/backend/utils/adt/pg_locale_icu.c
+++ b/src/backend/utils/adt/pg_locale_icu.c
@@ -13,7 +13,20 @@
#ifdef USE_ICU
#include <unicode/ucnv.h>
+#include <unicode/ucol.h>
#include <unicode/ustring.h>
+
+/*
+ * ucol_strcollUTF8() was introduced in ICU 50, but it is buggy before ICU 53.
+ * (see
+ * <https://www.postgresql.org/message-id/flat/f1438ec6-22aa-4029-9a3b-26f79d330e72%40manitou-mail.org>)
+ */
+#if U_ICU_VERSION_MAJOR_NUM >= 53
+#define HAVE_UCOL_STRCOLLUTF8 1
+#else
+#undef HAVE_UCOL_STRCOLLUTF8
+#endif
+
#endif
#include "access/htup_details.h"
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 533bebc1c7..37b3506f06 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,6 +17,10 @@
#include <ctype.h>
#include <limits.h>
+#ifdef USE_ICU
+#include <unicode/uchar.h>
+#endif
+
#include "access/detoast.h"
#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index c4c19a3c82..24f7ee4b61 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -14,10 +14,6 @@
#include "mb/pg_wchar.h"
-#ifdef USE_ICU
-#include <unicode/ucol.h>
-#endif
-
/*
* Character properties for regular expressions.
*/
@@ -31,19 +27,6 @@
#define PG_ISPUNCT 0x40
#define PG_ISSPACE 0x80
-#ifdef USE_ICU
-/*
- * ucol_strcollUTF8() was introduced in ICU 50, but it is buggy before ICU 53.
- * (see
- * <https://www.postgresql.org/message-id/flat/f1438ec6-22aa-4029-9a3b-26f79d330e72%40manitou-mail.org>)
- */
-#if U_ICU_VERSION_MAJOR_NUM >= 53
-#define HAVE_UCOL_STRCOLLUTF8 1
-#else
-#undef HAVE_UCOL_STRCOLLUTF8
-#endif
-#endif
-
/* use for libc locale names */
#define LOCALE_NAME_BUFLEN 128
--
2.34.1
[text/x-patch] v8-0007-Introduce-hooks-for-creating-custom-pg_locale_t.patch (6.5K, ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/8-v8-0007-Introduce-hooks-for-creating-custom-pg_locale_t.patch)
download | inline diff:
From 0db38e3a2f29a3638ead67ebcd98a0a414c2ad1e Mon Sep 17 00:00:00 2001
From: Jeff Davis <jeff@j-davis.com>
Date: Wed, 25 Sep 2024 16:10:28 -0700
Subject: [PATCH v8 7/7] Introduce hooks for creating custom pg_locale_t.
Now that collation, case mapping, and ctype behavior is controlled
with a method table, we can hook the behavior.
The hooks can provide their own arbitrary method table, which may be
based on a different version of ICU than what Postgres was built with,
or entirely unrelated to ICU/libc.
---
src/backend/utils/adt/pg_locale.c | 68 +++++++++++++++++++++----------
src/include/utils/pg_locale.h | 24 +++++++++++
src/tools/pgindent/typedefs.list | 2 +
3 files changed, 72 insertions(+), 22 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index d40ecf2357..c3fa68ead5 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -103,6 +103,9 @@ extern pg_locale_t create_pg_locale_builtin(Oid collid, MemoryContext context);
extern pg_locale_t create_pg_locale_icu(Oid collid, MemoryContext context);
extern pg_locale_t create_pg_locale_libc(Oid collid, MemoryContext context);
+create_pg_locale_hook_type create_pg_locale_hook = NULL;
+collation_version_hook_type collation_version_hook = NULL;
+
/* GUC settings */
char *locale_messages;
char *locale_monetary;
@@ -1209,7 +1212,7 @@ create_pg_locale(Oid collid, MemoryContext context)
{
HeapTuple tp;
Form_pg_collation collform;
- pg_locale_t result;
+ pg_locale_t result = NULL;
Datum datum;
bool isnull;
@@ -1218,15 +1221,21 @@ create_pg_locale(Oid collid, MemoryContext context)
elog(ERROR, "cache lookup failed for collation %u", collid);
collform = (Form_pg_collation) GETSTRUCT(tp);
- if (collform->collprovider == COLLPROVIDER_BUILTIN)
- result = create_pg_locale_builtin(collid, context);
- else if (collform->collprovider == COLLPROVIDER_ICU)
- result = create_pg_locale_icu(collid, context);
- else if (collform->collprovider == COLLPROVIDER_LIBC)
- result = create_pg_locale_libc(collid, context);
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(collform->collprovider);
+ if (create_pg_locale_hook != NULL)
+ result = create_pg_locale_hook(collid, context);
+
+ if (result == NULL)
+ {
+ if (collform->collprovider == COLLPROVIDER_BUILTIN)
+ result = create_pg_locale_builtin(collid, context);
+ else if (collform->collprovider == COLLPROVIDER_ICU)
+ result = create_pg_locale_icu(collid, context);
+ else if (collform->collprovider == COLLPROVIDER_LIBC)
+ result = create_pg_locale_libc(collid, context);
+ else
+ /* shouldn't happen */
+ PGLOCALE_SUPPORT_ERROR(collform->collprovider);
+ }
Assert((result->collate_is_c && result->collate == NULL) ||
(!result->collate_is_c && result->collate != NULL));
@@ -1289,7 +1298,7 @@ init_database_collation(void)
{
HeapTuple tup;
Form_pg_database dbform;
- pg_locale_t result;
+ pg_locale_t result = NULL;
Assert(default_locale == NULL);
@@ -1299,18 +1308,25 @@ init_database_collation(void)
elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
dbform = (Form_pg_database) GETSTRUCT(tup);
- if (dbform->datlocprovider == COLLPROVIDER_BUILTIN)
- result = create_pg_locale_builtin(DEFAULT_COLLATION_OID,
- TopMemoryContext);
- else if (dbform->datlocprovider == COLLPROVIDER_ICU)
- result = create_pg_locale_icu(DEFAULT_COLLATION_OID,
- TopMemoryContext);
- else if (dbform->datlocprovider == COLLPROVIDER_LIBC)
- result = create_pg_locale_libc(DEFAULT_COLLATION_OID,
+ if (create_pg_locale_hook != NULL)
+ result = create_pg_locale_hook(DEFAULT_COLLATION_OID,
TopMemoryContext);
- else
- /* shouldn't happen */
- PGLOCALE_SUPPORT_ERROR(dbform->datlocprovider);
+
+ if (result == NULL)
+ {
+ if (dbform->datlocprovider == COLLPROVIDER_BUILTIN)
+ result = create_pg_locale_builtin(DEFAULT_COLLATION_OID,
+ TopMemoryContext);
+ else if (dbform->datlocprovider == COLLPROVIDER_ICU)
+ result = create_pg_locale_icu(DEFAULT_COLLATION_OID,
+ TopMemoryContext);
+ else if (dbform->datlocprovider == COLLPROVIDER_LIBC)
+ result = create_pg_locale_libc(DEFAULT_COLLATION_OID,
+ TopMemoryContext);
+ else
+ /* shouldn't happen */
+ PGLOCALE_SUPPORT_ERROR(dbform->datlocprovider);
+ }
ReleaseSysCache(tup);
@@ -1379,6 +1395,14 @@ get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
+ if (collation_version_hook != NULL)
+ {
+ char *version;
+
+ if (collation_version_hook(collprovider, collcollate, &version))
+ return version;
+ }
+
/*
* The only two supported locales (C and C.UTF-8) are both based on memcmp
* and are not expected to change, but track the version anyway.
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 24f7ee4b61..7bd00a16a6 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -155,6 +155,30 @@ struct pg_locale_struct
typedef struct pg_locale_struct *pg_locale_t;
+/*
+ * Hooks to enable custom locale providers.
+ */
+
+/*
+ * Hook create_pg_locale(). Return result (allocated in the given context) to
+ * override; or return NULL to return control to create_pg_locale(). When
+ * creating the default database collation, collid is DEFAULT_COLLATION_OID.
+ */
+typedef pg_locale_t (*create_pg_locale_hook_type) (Oid collid,
+ MemoryContext context);
+
+/*
+ * Hook get_collation_actual_version(). Set *version out parameter and return
+ * true to override; or return false to return control to
+ * get_collation_actual_version().
+ */
+typedef bool (*collation_version_hook_type) (char collprovider,
+ const char *collcollate,
+ char **version);
+
+extern PGDLLIMPORT create_pg_locale_hook_type create_pg_locale_hook;
+extern PGDLLIMPORT collation_version_hook_type collation_version_hook;
+
extern void init_database_collation(void);
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 416a2cc76b..1d4247267b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3375,6 +3375,7 @@ cmpEntriesArg
codes_t
collation_cache_entry
collation_cache_hash
+collation_version_hook_type
color
colormaprange
compare_context
@@ -3391,6 +3392,7 @@ core_yyscan_t
corrupt_items
cost_qual_eval_context
cp_hash_func
+create_pg_locale_hook_type
create_upper_paths_hook_type
createdb_failure_params
crosstab_HashEnt
--
2.34.1
view thread (24+ messages) latest in thread
Message-ID: <78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com>
Permalink: ../78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com/
Also on: postgresql.org/message-id/78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com
copy link · copy postgr.es · copy git am
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pgsql@j-davis.com, andreas@proxel.se
Subject: Re: Collation & ctype method table, and extension hooks
In-Reply-To: <78a1b434ff40510dc5aaabe986299a09f4da90cf.camel@j-davis.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox