public inbox for [email protected]
help / color / mirror / Atom feedFrom: Jeff Davis <[email protected]>
To: Daniel Verite <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Jeremy Schneider <[email protected]>
Cc: [email protected]
Subject: Re: Built-in CTYPE provider
Date: Thu, 28 Dec 2023 18:57:16 -0800
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
On Wed, 2023-12-27 at 17:26 -0800, Jeff Davis wrote:
> Attached is an implementation of a built-in provider for the "C.UTF-
> 8"
Attached a more complete version that fixes a few bugs, stabilizes the
tests, and improves the documentation. I optimized the performance, too
-- now it's beating both libc's "C.utf8" and ICU "en-US-x-icu" for both
collation and case mapping (numbers below).
It's really nice to finally be able to have platform-independent tests
that work on any UTF-8 database.
Simple character classification:
SELECT 'Á' ~ '[[:alpha:]]' COLLATE C_UTF8;
Case mapping is more interesting (note that accented characters are
being properly mapped, and it's using the titlecase variant "Dž"):
SELECT initcap('axxE áxxÉ DŽxxDŽ Džxxx džxxx' COLLATE C_UTF8);
initcap
--------------------------
Axxe Áxxé Džxxdž Džxxx Džxxx
Even more interesting -- test that non-latin characters can still be a
member of a case-insensitive range:
-- capital delta is member of lowercase range gamma to lambda
SELECT 'Δ' ~* '[γ-λ]' COLLATE C_UTF8;
-- small delta is member of uppercase range gamma to lambda
SELECT 'δ' ~* '[Γ-Λ]' COLLATE C_UTF8;
Moreover, a lot of this behavior is locked in by strong Unicode
guarantees like [1] and [2]. Behavior that can change probably won't
change very often, and in any case will be tied to a PG major version.
All of these behaviors are very close to what glibc "C.utf8" does on my
machine. The case transformations are identical (except titlecasing
because libc doesn't support it). The character classifications have
some differences, which might be worth discussing, but I didn't see
anything terribly concerning (I am following the unicode
recommendations[3] on this topic).
Performance:
Sotring 10M strings:
libc "C" 14s
builtin C_UTF8 14s
libc "C.utf8" 20s
ICU "en-US-x-icu" 31s
Running UPPER() on 10M strings:
libc "C" 03s
builtin C_UTF8 07s
libc "C.utf8" 08s
ICU "en-US-x-icu" 15s
I didn't investigate or optimize regexes / pattern matching yet, but I
can do similar optimizations if there's any gap.
Note that I implemented the "simple" case mapping (which is what glibc
does) and the "posix compatible"[3] flavor of character classification
(which is closer to what glibc does than the "standard" flavor"). I
opted to use title case mapping for initcap(), which is a difference
from libc and I may go back to just upper/lower. These seem like
reasonable choices if we're going to name the locale after C.UTF-8.
Regards,
Jeff Davis
[1] https://www.unicode.org/policies/stability_policy.html#Case_Pair
[2] https://www.unicode.org/policies/stability_policy.html#Identity
[3] http://www.unicode.org/reports/tr18/#Compatibility_Properties
Attachments:
[text/x-patch] v15-0001-Minor-cleanup-for-unicode-update-build-and-test.patch (7.4K, ../[email protected]/2-v15-0001-Minor-cleanup-for-unicode-update-build-and-test.patch)
download | inline diff:
From dc433f10296abdb6837eceb93f096ed23c7115f5 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Wed, 22 Nov 2023 10:38:46 -0800
Subject: [PATCH v15 1/5] Minor cleanup for unicode-update build and test.
---
src/common/unicode/Makefile | 6 ++--
src/common/unicode/category_test.c | 18 ++++++------
src/common/unicode/meson.build | 44 +++++++++++++++---------------
3 files changed, 34 insertions(+), 34 deletions(-)
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 30cd75cc6a..04d81dd5cb 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -21,7 +21,7 @@ CPPFLAGS += $(ICU_CFLAGS)
# By default, do nothing.
all:
-update-unicode: unicode_category_table.h unicode_norm_table.h unicode_nonspacing_table.h unicode_east_asian_fw_table.h unicode_normprops_table.h unicode_norm_hashfunc.h unicode_version.h
+update-unicode: unicode_category_table.h unicode_east_asian_fw_table.h unicode_nonspacing_table.h unicode_norm_hashfunc.h unicode_norm_table.h unicode_normprops_table.h unicode_version.h
mv $^ $(top_srcdir)/src/include/common/
$(MAKE) category-check
$(MAKE) normalization-check
@@ -29,7 +29,7 @@ update-unicode: unicode_category_table.h unicode_norm_table.h unicode_nonspacing
# These files are part of the Unicode Character Database. Download
# them on demand. The dependency on Makefile.global is for
# UNICODE_VERSION.
-UnicodeData.txt EastAsianWidth.txt DerivedNormalizationProps.txt CompositionExclusions.txt NormalizationTest.txt: $(top_builddir)/src/Makefile.global
+CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt: $(top_builddir)/src/Makefile.global
$(DOWNLOAD) https://www.unicode.org/Public/$(UNICODE_VERSION)/ucd/$(@F)
unicode_version.h: generate-unicode_version.pl
@@ -82,4 +82,4 @@ clean:
rm -f $(OBJS) category_test category_test.o norm_test norm_test.o
distclean: clean
- rm -f UnicodeData.txt EastAsianWidth.txt CompositionExclusions.txt NormalizationTest.txt norm_test_table.h unicode_norm_table.h
+ rm -f CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
diff --git a/src/common/unicode/category_test.c b/src/common/unicode/category_test.c
index ba62716d45..d9ea806eb8 100644
--- a/src/common/unicode/category_test.c
+++ b/src/common/unicode/category_test.c
@@ -54,8 +54,8 @@ main(int argc, char **argv)
int pg_skipped_codepoints = 0;
int icu_skipped_codepoints = 0;
- printf("Postgres Unicode Version:\t%s\n", PG_UNICODE_VERSION);
- printf("ICU Unicode Version:\t\t%s\n", U_UNICODE_VERSION);
+ printf("category_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
+ printf("category_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
for (UChar32 code = 0; code <= 0x10ffff; code++)
{
@@ -79,11 +79,11 @@ main(int argc, char **argv)
icu_skipped_codepoints++;
else
{
- printf("FAILURE for codepoint %06x\n", code);
- printf("Postgres category: %02d %s %s\n", pg_category,
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres category: %02d %s %s\n", pg_category,
unicode_category_abbrev(pg_category),
unicode_category_string(pg_category));
- printf("ICU category: %02d %s %s\n", icu_category,
+ printf("category_test: ICU category: %02d %s %s\n", icu_category,
unicode_category_abbrev(icu_category),
unicode_category_string(icu_category));
printf("\n");
@@ -93,16 +93,16 @@ main(int argc, char **argv)
}
if (pg_skipped_codepoints > 0)
- printf("Skipped %d codepoints unassigned in Postgres due to Unicode version mismatch.\n",
+ printf("category_test: skipped %d codepoints unassigned in Postgres due to Unicode version mismatch\n",
pg_skipped_codepoints);
if (icu_skipped_codepoints > 0)
- printf("Skipped %d codepoints unassigned in ICU due to Unicode version mismatch.\n",
+ printf("category_test: skipped %d codepoints unassigned in ICU due to Unicode version mismatch\n",
icu_skipped_codepoints);
- printf("category_test: All tests successful!\n");
+ printf("category_test: success\n");
exit(0);
#else
- printf("ICU support required for test; skipping.\n");
+ printf("category_test: ICU support required for test; skipping\n");
exit(0);
#endif
}
diff --git a/src/common/unicode/meson.build b/src/common/unicode/meson.build
index 6af46122c4..e8cfdc1df4 100644
--- a/src/common/unicode/meson.build
+++ b/src/common/unicode/meson.build
@@ -11,7 +11,7 @@ endif
# These files are part of the Unicode Character Database. Download them on
# demand.
-foreach f : ['UnicodeData.txt', 'EastAsianWidth.txt', 'DerivedNormalizationProps.txt', 'CompositionExclusions.txt', 'NormalizationTest.txt']
+foreach f : ['CompositionExclusions.txt', 'DerivedNormalizationProps.txt', 'EastAsianWidth.txt', 'NormalizationTest.txt', 'UnicodeData.txt']
url = unicode_baseurl.format(UNICODE_VERSION, f)
target = custom_target(f,
output: f,
@@ -24,15 +24,6 @@ endforeach
update_unicode_targets = []
-update_unicode_targets += \
- custom_target('unicode_version.h',
- output: ['unicode_version.h'],
- command: [
- perl, files('generate-unicode_version.pl'),
- '--outdir', '@OUTDIR@', '--version', UNICODE_VERSION],
- build_by_default: false,
- )
-
update_unicode_targets += \
custom_target('unicode_category_table.h',
input: [unicode_data['UnicodeData.txt']],
@@ -44,14 +35,12 @@ update_unicode_targets += \
)
update_unicode_targets += \
- custom_target('unicode_norm_table.h',
- input: [unicode_data['UnicodeData.txt'], unicode_data['CompositionExclusions.txt']],
- output: ['unicode_norm_table.h', 'unicode_norm_hashfunc.h'],
- depend_files: perfect_hash_pm,
- command: [
- perl, files('generate-unicode_norm_table.pl'),
- '--outdir', '@OUTDIR@', '@INPUT@'],
+ custom_target('unicode_east_asian_fw_table.h',
+ input: [unicode_data['EastAsianWidth.txt']],
+ output: ['unicode_east_asian_fw_table.h'],
+ command: [perl, files('generate-unicode_east_asian_fw_table.pl'), '@INPUT@'],
build_by_default: false,
+ capture: true,
)
update_unicode_targets += \
@@ -65,12 +54,14 @@ update_unicode_targets += \
)
update_unicode_targets += \
- custom_target('unicode_east_asian_fw_table.h',
- input: [unicode_data['EastAsianWidth.txt']],
- output: ['unicode_east_asian_fw_table.h'],
- command: [perl, files('generate-unicode_east_asian_fw_table.pl'), '@INPUT@'],
+ custom_target('unicode_norm_table.h',
+ input: [unicode_data['UnicodeData.txt'], unicode_data['CompositionExclusions.txt']],
+ output: ['unicode_norm_table.h', 'unicode_norm_hashfunc.h'],
+ depend_files: perfect_hash_pm,
+ command: [
+ perl, files('generate-unicode_norm_table.pl'),
+ '--outdir', '@OUTDIR@', '@INPUT@'],
build_by_default: false,
- capture: true,
)
update_unicode_targets += \
@@ -83,6 +74,15 @@ update_unicode_targets += \
capture: true,
)
+update_unicode_targets += \
+ custom_target('unicode_version.h',
+ output: ['unicode_version.h'],
+ command: [
+ perl, files('generate-unicode_version.pl'),
+ '--outdir', '@OUTDIR@', '--version', UNICODE_VERSION],
+ build_by_default: false,
+ )
+
norm_test_table = custom_target('norm_test_table.h',
input: [unicode_data['NormalizationTest.txt']],
output: ['norm_test_table.h'],
--
2.34.1
[text/x-patch] v15-0002-Add-Unicode-property-tables.patch (91.5K, ../[email protected]/3-v15-0002-Add-Unicode-property-tables.patch)
download | inline diff:
From 10ab7ca76bfe7dff10e903bc61c509a781e4b74b Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Sat, 18 Nov 2023 15:34:24 -0800
Subject: [PATCH v15 2/5] Add Unicode property tables.
---
src/common/unicode/Makefile | 6 +-
src/common/unicode/category_test.c | 302 +-
.../generate-unicode_category_table.pl | 203 +-
src/common/unicode/meson.build | 4 +-
src/common/unicode_category.c | 258 +-
src/include/common/unicode_category.h | 25 +-
src/include/common/unicode_category_table.h | 2532 +++++++++++++++++
7 files changed, 3264 insertions(+), 66 deletions(-)
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 04d81dd5cb..27f0408d8b 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -29,13 +29,13 @@ update-unicode: unicode_category_table.h unicode_east_asian_fw_table.h unicode_n
# These files are part of the Unicode Character Database. Download
# them on demand. The dependency on Makefile.global is for
# UNICODE_VERSION.
-CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt: $(top_builddir)/src/Makefile.global
+CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt: $(top_builddir)/src/Makefile.global
$(DOWNLOAD) https://www.unicode.org/Public/$(UNICODE_VERSION)/ucd/$(@F)
unicode_version.h: generate-unicode_version.pl
$(PERL) $< --version $(UNICODE_VERSION)
-unicode_category_table.h: generate-unicode_category_table.pl UnicodeData.txt
+unicode_category_table.h: generate-unicode_category_table.pl DerivedCoreProperties.txt PropList.txt UnicodeData.txt
$(PERL) $<
# Generation of conversion tables used for string normalization with
@@ -82,4 +82,4 @@ clean:
rm -f $(OBJS) category_test category_test.o norm_test norm_test.o
distclean: clean
- rm -f CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
+ rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
diff --git a/src/common/unicode/category_test.c b/src/common/unicode/category_test.c
index d9ea806eb8..6451988e94 100644
--- a/src/common/unicode/category_test.c
+++ b/src/common/unicode/category_test.c
@@ -1,6 +1,7 @@
/*-------------------------------------------------------------------------
* category_test.c
- * Program to test Unicode general category functions.
+ * Program to test Unicode general category and character class
+ * functions.
*
* Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
*
@@ -14,23 +15,30 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
-
#ifdef USE_ICU
#include <unicode/uchar.h>
#endif
-#include "common/unicode_category.h"
+#include <wctype.h>
+
#include "common/unicode_version.h"
+#include "common/unicode_category.h"
+
+#define LIBC_LOCALE "C.UTF-8"
+
+static int pg_unicode_version = 0;
+#ifdef USE_ICU
+static int icu_unicode_version = 0;
+#endif
/*
* Parse version into integer for easy comparison.
*/
-#ifdef USE_ICU
static int
parse_unicode_version(const char *version)
{
- int n,
- major,
- minor;
+ int major;
+ int minor;
+ int PG_USED_FOR_ASSERTS_ONLY n;
n = sscanf(version, "%d.%d", &major, &minor);
@@ -39,56 +47,160 @@ parse_unicode_version(const char *version)
return major * 100 + minor;
}
-#endif
+#ifdef USE_ICU
/*
- * Exhaustively test that the Unicode category for each codepoint matches that
- * returned by ICU.
+ * Test Postgres Unicode tables by comparing with ICU. Test the General
+ * Category, as well as the properties Alphabetic, Lowercase, Uppercase,
+ * White_Space, and Hex_Digit.
*/
-int
-main(int argc, char **argv)
+static void
+icu_test()
{
-#ifdef USE_ICU
- int pg_unicode_version = parse_unicode_version(PG_UNICODE_VERSION);
- int icu_unicode_version = parse_unicode_version(U_UNICODE_VERSION);
int pg_skipped_codepoints = 0;
int icu_skipped_codepoints = 0;
- printf("category_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
- printf("category_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
-
- for (UChar32 code = 0; code <= 0x10ffff; code++)
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
{
uint8_t pg_category = unicode_category(code);
uint8_t icu_category = u_charType(code);
+ /* Property tests */
+ bool prop_alphabetic = pg_u_prop_alphabetic(code);
+ bool prop_lowercase = pg_u_prop_lowercase(code);
+ bool prop_uppercase = pg_u_prop_uppercase(code);
+ bool prop_white_space = pg_u_prop_white_space(code);
+ bool prop_hex_digit = pg_u_prop_hex_digit(code);
+ bool prop_join_control = pg_u_prop_join_control(code);
+
+ bool icu_prop_alphabetic = u_hasBinaryProperty(
+ code, UCHAR_ALPHABETIC);
+ bool icu_prop_lowercase = u_hasBinaryProperty(
+ code, UCHAR_LOWERCASE);
+ bool icu_prop_uppercase = u_hasBinaryProperty(
+ code, UCHAR_UPPERCASE);
+ bool icu_prop_white_space = u_hasBinaryProperty(
+ code, UCHAR_WHITE_SPACE);
+ bool icu_prop_hex_digit = u_hasBinaryProperty(
+ code, UCHAR_HEX_DIGIT);
+ bool icu_prop_join_control = u_hasBinaryProperty(
+ code, UCHAR_JOIN_CONTROL);
+
+ /*
+ * Compare with ICU for TR #18 character classes using:
+ *
+ * https://unicode-org.github.io/icu-docs/apidoc/dev/icu4c/uchar_8h.html#details
+ *
+ * which describes how to use ICU to test for membership in regex
+ * character classes ("Standard", not "POSIX Compatible").
+ *
+ * NB: the document suggests testing for some properties such as
+ * UCHAR_POSIX_ALNUM, but that doesn't mean that we're testing for the
+ * "POSIX Compatible" character classes.
+ */
+ bool isalpha = pg_u_isalpha(code);
+ bool islower = pg_u_islower(code);
+ bool isupper = pg_u_isupper(code);
+ bool ispunct = pg_u_ispunct(code, false);
+ bool isdigit = pg_u_isdigit(code, false);
+ bool isxdigit = pg_u_isxdigit(code, false);
+ bool isalnum = pg_u_isalnum(code, false);
+ bool isspace = pg_u_isspace(code);
+ bool isblank = pg_u_isblank(code);
+ bool iscntrl = pg_u_iscntrl(code);
+ bool isgraph = pg_u_isgraph(code);
+ bool isprint = pg_u_isprint(code);
+
+ bool icu_isalpha = u_isUAlphabetic(code);
+ bool icu_islower = u_isULowercase(code);
+ bool icu_isupper = u_isUUppercase(code);
+ bool icu_ispunct = u_ispunct(code);
+ bool icu_isdigit = u_isdigit(code);
+ bool icu_isxdigit = u_hasBinaryProperty(code,
+ UCHAR_POSIX_XDIGIT);
+ bool icu_isalnum = u_hasBinaryProperty(code,
+ UCHAR_POSIX_ALNUM);
+ bool icu_isspace = u_isUWhiteSpace(code);
+ bool icu_isblank = u_isblank(code);
+ bool icu_iscntrl = icu_category == PG_U_CONTROL;
+ bool icu_isgraph = u_hasBinaryProperty(code,
+ UCHAR_POSIX_GRAPH);
+ bool icu_isprint = u_hasBinaryProperty(code,
+ UCHAR_POSIX_PRINT);
+
+ /*
+ * A version mismatch means that some assigned codepoints in the newer
+ * version may be unassigned in the older version. That's OK, though
+ * the test will not cover those codepoints marked unassigned in the
+ * older version (that is, it will no longer be an exhaustive test).
+ */
+ if (pg_category == PG_U_UNASSIGNED &&
+ icu_category != PG_U_UNASSIGNED &&
+ pg_unicode_version < icu_unicode_version)
+ {
+ pg_skipped_codepoints++;
+ continue;
+ }
+
+ if (icu_category == PG_U_UNASSIGNED &&
+ pg_category != PG_U_UNASSIGNED &&
+ icu_unicode_version < pg_unicode_version)
+ {
+ icu_skipped_codepoints++;
+ continue;
+ }
+
if (pg_category != icu_category)
{
- /*
- * A version mismatch means that some assigned codepoints in the
- * newer version may be unassigned in the older version. That's
- * OK, though the test will not cover those codepoints marked
- * unassigned in the older version (that is, it will no longer be
- * an exhaustive test).
- */
- if (pg_category == PG_U_UNASSIGNED &&
- pg_unicode_version < icu_unicode_version)
- pg_skipped_codepoints++;
- else if (icu_category == PG_U_UNASSIGNED &&
- icu_unicode_version < pg_unicode_version)
- icu_skipped_codepoints++;
- else
- {
- printf("category_test: FAILURE for codepoint 0x%06x\n", code);
- printf("category_test: Postgres category: %02d %s %s\n", pg_category,
- unicode_category_abbrev(pg_category),
- unicode_category_string(pg_category));
- printf("category_test: ICU category: %02d %s %s\n", icu_category,
- unicode_category_abbrev(icu_category),
- unicode_category_string(icu_category));
- printf("\n");
- exit(1);
- }
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres category: %02d %s %s\n", pg_category,
+ unicode_category_abbrev(pg_category),
+ unicode_category_string(pg_category));
+ printf("category_test: ICU category: %02d %s %s\n", icu_category,
+ unicode_category_abbrev(icu_category),
+ unicode_category_string(icu_category));
+ printf("\n");
+ exit(1);
+ }
+
+ if (prop_alphabetic != icu_prop_alphabetic ||
+ prop_lowercase != icu_prop_lowercase ||
+ prop_uppercase != icu_prop_uppercase ||
+ prop_white_space != icu_prop_white_space ||
+ prop_hex_digit != icu_prop_hex_digit ||
+ prop_join_control != icu_prop_join_control)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres property alphabetic/lowercase/uppercase/white_space/hex_digit/join_control: %d/%d/%d/%d/%d/%d\n",
+ prop_alphabetic, prop_lowercase, prop_uppercase,
+ prop_white_space, prop_hex_digit, prop_join_control);
+ printf("category_test: ICU property alphabetic/lowercase/uppercase/white_space/hex_digit/join_control: %d/%d/%d/%d/%d/%d\n",
+ icu_prop_alphabetic, icu_prop_lowercase, icu_prop_uppercase,
+ icu_prop_white_space, icu_prop_hex_digit, icu_prop_join_control);
+ printf("\n");
+ exit(1);
+ }
+
+ if (isalpha != icu_isalpha ||
+ islower != icu_islower ||
+ isupper != icu_isupper ||
+ ispunct != icu_ispunct ||
+ isdigit != icu_isdigit ||
+ isxdigit != icu_isxdigit ||
+ isalnum != icu_isalnum ||
+ isspace != icu_isspace ||
+ isblank != icu_isblank ||
+ iscntrl != icu_iscntrl ||
+ isgraph != icu_isgraph ||
+ isprint != icu_isprint)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres class alpha/lower/upper/punct/digit/xdigit/alnum/space/blank/cntrl/graph/print: %d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d\n",
+ isalpha, islower, isupper, ispunct, isdigit, isxdigit, isalnum, isspace, isblank, iscntrl, isgraph, isprint);
+ printf("category_test: ICU class alpha/lower/upper/punct/digit/xdigit/alnum/space/blank/cntrl/graph/print: %d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d\n",
+ icu_isalpha, icu_islower, icu_isupper, icu_ispunct, icu_isdigit, icu_isxdigit, icu_isalnum, icu_isspace, icu_isblank, icu_iscntrl, icu_isgraph, icu_isprint);
+ printf("\n");
+ exit(1);
}
}
@@ -99,10 +211,104 @@ main(int argc, char **argv)
printf("category_test: skipped %d codepoints unassigned in ICU due to Unicode version mismatch\n",
icu_skipped_codepoints);
- printf("category_test: success\n");
- exit(0);
+ printf("category_test: ICU test successful\n");
+}
+#endif
+
+/*
+ * For libc, test only some characters for membership in the punctuation
+ * class. We have no guarantee that all characters will obey the same rules as
+ * pg_u_ispunct_posix(), though some coverage is still useful.
+ */
+static const unsigned char test_punct[] = {
+ ',', '$', '"', 0x85, 0x00, 'b', '&', 'Z', ' ', '\t', '\n'
+};
+
+/*
+ * Test what we can for libc, which is limited but still useful to cover the
+ * _posix-variant functions.
+ */
+static void
+libc_test()
+{
+ char * libc_locale = setlocale(LC_CTYPE, LIBC_LOCALE);
+
+ if (!libc_locale)
+ {
+ printf("category_test: libc locale \"%s\" not available; skipping\n", LIBC_LOCALE);
+ return;
+ }
+
+ /* non-exhaustive test of pg_u_ispunct_posix() */
+ for (int i = 0; i < sizeof(test_punct)/sizeof(test_punct[0]); i++)
+ {
+ pg_wchar code = (pg_wchar) test_punct[i];
+ bool ispunct = pg_u_ispunct(code, true);
+ bool libc_ispunct = iswpunct(code);
+
+ if (ispunct != libc_ispunct)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres ispunct_posix: %d\n", ispunct);
+ printf("category_test: libc iswpunct: %d\n", libc_ispunct);
+ printf("\n");
+ exit(1);
+ }
+ }
+
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
+ {
+ uint8_t pg_category = unicode_category(code);
+
+ bool isalpha = pg_u_isalpha(code);
+ bool isdigit = pg_u_isdigit(code, true);
+ bool isxdigit = pg_u_isxdigit(code, true);
+ bool isalnum = pg_u_isalnum(code, true);
+
+ bool libc_isdigit = iswdigit(code);
+ bool libc_isxdigit = iswxdigit(code);
+
+ if (pg_category == PG_U_UNASSIGNED)
+ continue;
+
+ /* check that alnum is the same as isdigit OR isalpha */
+ if (((isdigit || isalpha) && !isalnum) ||
+ (!(isdigit || isalpha) && isalnum))
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: isalnum inconsistent: isalpha/isdigit/isalnum: %d/%d/%d\n",
+ isalpha, isdigit, isalnum);
+ exit(1);
+ }
+
+ if (isdigit != libc_isdigit ||
+ isxdigit != libc_isxdigit)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres class digit/xdigit: %d/%d\n",
+ isdigit, isxdigit);
+ printf("category_test: libc class digit/xdigit: %d/%d\n",
+ libc_isdigit, libc_isxdigit);
+ printf("\n");
+ exit(1);
+ }
+ }
+}
+
+int
+main(int argc, char **argv)
+{
+ pg_unicode_version = parse_unicode_version(PG_UNICODE_VERSION);
+ printf("category_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
+
+ libc_test();
+
+#ifdef USE_ICU
+ icu_unicode_version = parse_unicode_version(U_UNICODE_VERSION);
+ printf("category_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
+
+ icu_test();
#else
- printf("category_test: ICU support required for test; skipping\n");
- exit(0);
+ printf("category_test: ICU not available; skipping\n");
#endif
}
diff --git a/src/common/unicode/generate-unicode_category_table.pl b/src/common/unicode/generate-unicode_category_table.pl
index 992b877ede..9545728443 100644
--- a/src/common/unicode/generate-unicode_category_table.pl
+++ b/src/common/unicode/generate-unicode_category_table.pl
@@ -120,8 +120,6 @@ if ($range_category ne $CATEGORY_UNASSIGNED) {
category => $range_category});
}
-my $num_ranges = scalar @category_ranges;
-
# See: https://www.unicode.org/reports/tr44/#General_Category_Values
my $categories = {
Cn => 'PG_U_UNASSIGNED',
@@ -156,11 +154,98 @@ my $categories = {
Pf => 'PG_U_FINAL_PUNCTUATION'
};
-# Start writing out the output files
+# Find White_Space and Hex_Digit characters
+my @white_space = ();
+my @hex_digits = ();
+my @join_control = ();
+open($FH, '<', "$output_path/PropList.txt")
+ or die "Could not open $output_path/PropList.txt: $!.";
+while (my $line = <$FH>)
+{
+ my $pattern = qr/([0-9A-F\.]+)\s*;\s*(\w+)\s*#.*/s;
+ next unless $line =~ $pattern;
+
+ my $code = $line =~ s/$pattern/$1/rg;
+ my $property = $line =~ s/$pattern/$2/rg;
+ my $start;
+ my $end;
+
+ if ($code =~ /\.\./) {
+ # code range
+ my @sp = split /\.\./, $code;
+ $start = hex($sp[0]);
+ $end = hex($sp[1]);
+ } else {
+ # single code point
+ $start = hex($code);
+ $end = hex($code);
+ }
+
+ if ($property eq "White_Space") {
+ push @white_space, {start => $start, end => $end};
+ }
+ elsif ($property eq "Hex_Digit") {
+ push @hex_digits, {start => $start, end => $end};
+ }
+ elsif ($property eq "Join_Control") {
+ push @join_control, {start => $start, end => $end};
+ }
+}
+
+# Find Alphabetic, Lowercase, and Uppercase characters
+my @alphabetic = ();
+my @lowercase = ();
+my @uppercase = ();
+open($FH, '<', "$output_path/DerivedCoreProperties.txt")
+ or die "Could not open $output_path/DerivedCoreProperties.txt: $!.";
+while (my $line = <$FH>)
+{
+ my $pattern = qr/^([0-9A-F\.]+)\s*;\s*(\w+)\s*#.*$/s;
+ next unless $line =~ $pattern;
+
+ my $code = $line =~ s/$pattern/$1/rg;
+ my $property = $line =~ s/$pattern/$2/rg;
+ my $start;
+ my $end;
+
+ if ($code =~ /\.\./) {
+ # code range
+ my @sp = split /\.\./, $code;
+ die "line: {$line} code: {$code} sp[0] {$sp[0]} sp[1] {$sp[1]}"
+ unless $sp[0] =~ /^[0-9A-F]+$/ && $sp[1] =~ /^[0-9A-F]+$/;
+ $start = hex($sp[0]);
+ $end = hex($sp[1]);
+ } else {
+ die "line: {$line} code: {$code}" unless $code =~ /^[0-9A-F]+$/;
+ # single code point
+ $start = hex($code);
+ $end = hex($code);
+ }
+
+ if ($property eq "Alphabetic") {
+ push @alphabetic, {start => $start, end => $end};
+ }
+ elsif ($property eq "Lowercase") {
+ push @lowercase, {start => $start, end => $end};
+ }
+ elsif ($property eq "Uppercase") {
+ push @uppercase, {start => $start, end => $end};
+ }
+}
+
+my $num_category_ranges = scalar @category_ranges;
+my $num_alphabetic_ranges = scalar @alphabetic;
+my $num_lowercase_ranges = scalar @lowercase;
+my $num_uppercase_ranges = scalar @uppercase;
+my $num_white_space_ranges = scalar @white_space;
+my $num_hex_digit_ranges = scalar @hex_digits;
+my $num_join_control_ranges = scalar @join_control;
+
+# Start writing out the output file
open my $OT, '>', $output_table_file
or die "Could not open output file $output_table_file: $!\n";
-print $OT <<HEADER;
+print $OT <<"HEADER";
/*-------------------------------------------------------------------------
*
* unicode_category_table.h
@@ -188,11 +273,20 @@ typedef struct
uint8 category; /* General Category */
} pg_category_range;
-/* table of Unicode codepoint ranges and their categories */
-static const pg_category_range unicode_categories[$num_ranges] =
+typedef struct
{
+ uint32 first; /* Unicode codepoint */
+ uint32 last; /* Unicode codepoint */
+} pg_unicode_range;
+
HEADER
+print $OT <<"CATEGORY_TABLE";
+/* table of Unicode codepoint ranges and their categories */
+static const pg_category_range unicode_categories[$num_category_ranges] =
+{
+CATEGORY_TABLE
+
my $firsttime = 1;
foreach my $range (@category_ranges) {
printf $OT ",\n" unless $firsttime;
@@ -202,4 +296,101 @@ foreach my $range (@category_ranges) {
die "category missing: $range->{category}" unless $category;
printf $OT "\t{0x%06x, 0x%06x, %s}", $range->{start}, $range->{end}, $category;
}
+
+print $OT "\n};\n\n";
+
+print $OT <<"ALPHABETIC_TABLE";
+/* table of Unicode codepoint ranges of Alphabetic characters */
+static const pg_unicode_range unicode_alphabetic[$num_alphabetic_ranges] =
+{
+ALPHABETIC_TABLE
+
+$firsttime = 1;
+foreach my $range (@alphabetic) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"LOWERCASE_TABLE";
+/* table of Unicode codepoint ranges of Lowercase characters */
+static const pg_unicode_range unicode_lowercase[$num_lowercase_ranges] =
+{
+LOWERCASE_TABLE
+
+$firsttime = 1;
+foreach my $range (@lowercase) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"UPPERCASE_TABLE";
+/* table of Unicode codepoint ranges of Uppercase characters */
+static const pg_unicode_range unicode_uppercase[$num_uppercase_ranges] =
+{
+UPPERCASE_TABLE
+
+$firsttime = 1;
+foreach my $range (@uppercase) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"WHITE_SPACE_TABLE";
+/* table of Unicode codepoint ranges of White_Space characters */
+static const pg_unicode_range unicode_white_space[$num_white_space_ranges] =
+{
+WHITE_SPACE_TABLE
+
+$firsttime = 1;
+foreach my $range (@white_space) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"HEX_DIGITS_TABLE";
+/* table of Unicode codepoint ranges of Hex_Digit characters */
+static const pg_unicode_range unicode_hex_digit[$num_hex_digit_ranges] =
+{
+HEX_DIGITS_TABLE
+
+$firsttime = 1;
+foreach my $range (@hex_digits) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"JOIN_CONTROL_TABLE";
+/* table of Unicode codepoint ranges of Join_Control characters */
+static const pg_unicode_range unicode_join_control[$num_join_control_ranges] =
+{
+JOIN_CONTROL_TABLE
+
+$firsttime = 1;
+foreach my $range (@join_control) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
print $OT "\n};\n";
diff --git a/src/common/unicode/meson.build b/src/common/unicode/meson.build
index e8cfdc1df4..3526ddb846 100644
--- a/src/common/unicode/meson.build
+++ b/src/common/unicode/meson.build
@@ -11,7 +11,7 @@ endif
# These files are part of the Unicode Character Database. Download them on
# demand.
-foreach f : ['CompositionExclusions.txt', 'DerivedNormalizationProps.txt', 'EastAsianWidth.txt', 'NormalizationTest.txt', 'UnicodeData.txt']
+foreach f : ['CompositionExclusions.txt', 'DerivedCoreProperties.txt', 'DerivedNormalizationProps.txt', 'EastAsianWidth.txt', 'NormalizationTest.txt', 'PropList.txt', 'UnicodeData.txt']
url = unicode_baseurl.format(UNICODE_VERSION, f)
target = custom_target(f,
output: f,
@@ -26,7 +26,7 @@ update_unicode_targets = []
update_unicode_targets += \
custom_target('unicode_category_table.h',
- input: [unicode_data['UnicodeData.txt']],
+ input: [unicode_data['UnicodeData.txt'], unicode_data['DerivedCoreProperties.txt'], unicode_data['PropList.txt']],
output: ['unicode_category_table.h'],
command: [
perl, files('generate-unicode_category_table.pl'),
diff --git a/src/common/unicode_category.c b/src/common/unicode_category.c
index 189cd6eca3..cfa086c42b 100644
--- a/src/common/unicode_category.c
+++ b/src/common/unicode_category.c
@@ -1,6 +1,8 @@
/*-------------------------------------------------------------------------
* unicode_category.c
- * Determine general category of Unicode characters.
+ * Determine general category and character class of Unicode
+ * characters. Encoding must be UTF8, where we assume that the pg_wchar
+ * representation is a code point.
*
* Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
*
@@ -18,24 +20,78 @@
#include "common/unicode_category.h"
#include "common/unicode_category_table.h"
+/*
+ * We use a mask word for convenience when testing for multiple categories at
+ * once. The number of Unicode General Categories should never grow, so a
+ * 32-bit mask is fine.
+ */
+#define PG_U_CATEGORY_MASK(X) ((uint32)(1 << (X)))
+
+#define PG_U_LU_MASK PG_U_CATEGORY_MASK(PG_U_UPPERCASE_LETTER)
+#define PG_U_LL_MASK PG_U_CATEGORY_MASK(PG_U_LOWERCASE_LETTER)
+#define PG_U_LT_MASK PG_U_CATEGORY_MASK(PG_U_TITLECASE_LETTER)
+#define PG_U_LC_MASK (PG_U_LU_MASK|PG_U_LL_MASK|PG_U_LT_MASK)
+#define PG_U_LM_MASK PG_U_CATEGORY_MASK(PG_U_MODIFIER_LETTER)
+#define PG_U_LO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_LETTER)
+#define PG_U_L_MASK (PG_U_LU_MASK|PG_U_LL_MASK|PG_U_LT_MASK|PG_U_LM_MASK|\
+ PG_U_LO_MASK)
+#define PG_U_MN_MASK PG_U_CATEGORY_MASK(PG_U_NONSPACING_MARK)
+#define PG_U_ME_MASK PG_U_CATEGORY_MASK(PG_U_ENCLOSING_MARK)
+#define PG_U_MC_MASK PG_U_CATEGORY_MASK(PG_U_SPACING_MARK)
+#define PG_U_M_MASK (PG_U_MN_MASK|PG_U_MC_MASK|PG_U_ME_MASK)
+#define PG_U_ND_MASK PG_U_CATEGORY_MASK(PG_U_DECIMAL_NUMBER)
+#define PG_U_NL_MASK PG_U_CATEGORY_MASK(PG_U_LETTER_NUMBER)
+#define PG_U_NO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_NUMBER)
+#define PG_U_N_MASK (PG_U_ND_MASK|PG_U_NL_MASK|PG_U_NO_MASK)
+#define PG_U_PC_MASK PG_U_CATEGORY_MASK(PG_U_CONNECTOR_PUNCTUATION)
+#define PG_U_PD_MASK PG_U_CATEGORY_MASK(PG_U_DASH_PUNCTUATION)
+#define PG_U_PS_MASK PG_U_CATEGORY_MASK(PG_U_OPEN_PUNCTUATION)
+#define PG_U_PE_MASK PG_U_CATEGORY_MASK(PG_U_CLOSE_PUNCTUATION)
+#define PG_U_PI_MASK PG_U_CATEGORY_MASK(PG_U_INITIAL_PUNCTUATION)
+#define PG_U_PF_MASK PG_U_CATEGORY_MASK(PG_U_FINAL_PUNCTUATION)
+#define PG_U_PO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_PUNCTUATION)
+#define PG_U_P_MASK (PG_U_PC_MASK|PG_U_PD_MASK|PG_U_PS_MASK|PG_U_PE_MASK|\
+ PG_U_PI_MASK|PG_U_PF_MASK|PG_U_PO_MASK)
+#define PG_U_SM_MASK PG_U_CATEGORY_MASK(PG_U_MATH_SYMBOL)
+#define PG_U_SC_MASK PG_U_CATEGORY_MASK(PG_U_CURRENCY_SYMBOL)
+#define PG_U_SK_MASK PG_U_CATEGORY_MASK(PG_U_MODIFIER_SYMBOL)
+#define PG_U_SO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_SYMBOL)
+#define PG_U_S_MASK (PG_U_SM_MASK|PG_U_SC_MASK|PG_U_SK_MASK|PG_U_SO_MASK)
+#define PG_U_ZS_MASK PG_U_CATEGORY_MASK(PG_U_SPACE_SEPARATOR)
+#define PG_U_ZL_MASK PG_U_CATEGORY_MASK(PG_U_LINE_SEPARATOR)
+#define PG_U_ZP_MASK PG_U_CATEGORY_MASK(PG_U_PARAGRAPH_SEPARATOR)
+#define PG_U_Z_MASK (PG_U_ZS_MASK|PG_U_ZL_MASK|PG_U_ZP_MASK)
+#define PG_U_CC_MASK PG_U_CATEGORY_MASK(PG_U_CONTROL)
+#define PG_U_CF_MASK PG_U_CATEGORY_MASK(PG_U_FORMAT)
+#define PG_U_CS_MASK PG_U_CATEGORY_MASK(PG_U_SURROGATE)
+#define PG_U_CO_MASK PG_U_CATEGORY_MASK(PG_U_PRIVATE_USE)
+#define PG_U_CN_MASK PG_U_CATEGORY_MASK(PG_U_UNASSIGNED)
+#define PG_U_C_MASK (PG_U_CC_MASK|PG_U_CF_MASK|PG_U_CS_MASK|PG_U_CO_MASK|\
+ PG_U_CN_MASK)
+
+#define PG_U_CHARACTER_TAB 0x09
+
+static bool range_search(const pg_unicode_range * tbl, Size size,
+ pg_wchar code);
+
/*
* Unicode general category for the given codepoint.
*/
pg_unicode_category
-unicode_category(pg_wchar ucs)
+unicode_category(pg_wchar code)
{
int min = 0;
int mid;
int max = lengthof(unicode_categories) - 1;
- Assert(ucs <= 0x10ffff);
+ Assert(code <= 0x10ffff);
while (max >= min)
{
mid = (min + max) / 2;
- if (ucs > unicode_categories[mid].last)
+ if (code > unicode_categories[mid].last)
min = mid + 1;
- else if (ucs < unicode_categories[mid].first)
+ else if (code < unicode_categories[mid].first)
max = mid - 1;
else
return unicode_categories[mid].category;
@@ -44,6 +100,171 @@ unicode_category(pg_wchar ucs)
return PG_U_UNASSIGNED;
}
+bool
+pg_u_prop_alphabetic(pg_wchar code)
+{
+ return range_search(unicode_alphabetic, lengthof(unicode_alphabetic),
+ code);
+}
+
+bool
+pg_u_prop_lowercase(pg_wchar code)
+{
+ return range_search(unicode_lowercase, lengthof(unicode_lowercase), code);
+}
+
+bool
+pg_u_prop_uppercase(pg_wchar code)
+{
+ return range_search(unicode_uppercase, lengthof(unicode_uppercase), code);
+}
+
+bool
+pg_u_prop_white_space(pg_wchar code)
+{
+ return range_search(unicode_white_space, lengthof(unicode_white_space),
+ code);
+}
+
+bool
+pg_u_prop_hex_digit(pg_wchar code)
+{
+ return range_search(unicode_hex_digit, lengthof(unicode_hex_digit), code);
+}
+
+bool
+pg_u_prop_join_control(pg_wchar code)
+{
+ return range_search(unicode_join_control, lengthof(unicode_join_control),
+ code);
+}
+
+/*
+ * The following functions implement the regex character classification as
+ * described at: http://www.unicode.org/reports/tr18/#Compatibility_Properties
+ *
+ * If 'posix' is true, implements the "POSIX Compatible" variant, otherwise
+ * the "Standard" variant.
+ */
+
+bool
+pg_u_isdigit(pg_wchar code, bool posix)
+{
+ if (posix)
+ return ('0' <= code && code <= '9');
+ else
+ return unicode_category(code) == PG_U_DECIMAL_NUMBER;
+}
+
+bool
+pg_u_isalpha(pg_wchar code)
+{
+ return pg_u_prop_alphabetic(code);
+}
+
+bool
+pg_u_isalnum(pg_wchar code, bool posix)
+{
+ return pg_u_isalpha(code) || pg_u_isdigit(code, posix);
+}
+
+bool
+pg_u_isword(pg_wchar code)
+{
+ uint32 category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ return
+ category_mask & (PG_U_M_MASK|PG_U_ND_MASK|PG_U_PC_MASK) ||
+ pg_u_isalpha(code) ||
+ pg_u_prop_join_control(code);
+}
+
+bool
+pg_u_isupper(pg_wchar code)
+{
+ return pg_u_prop_uppercase(code);
+}
+
+bool
+pg_u_islower(pg_wchar code)
+{
+ return pg_u_prop_lowercase(code);
+}
+
+bool
+pg_u_isblank(pg_wchar code)
+{
+ return code == PG_U_CHARACTER_TAB ||
+ unicode_category(code) == PG_U_SPACE_SEPARATOR;
+}
+
+bool
+pg_u_iscntrl(pg_wchar code)
+{
+ return unicode_category(code) == PG_U_CONTROL;
+}
+
+bool
+pg_u_isgraph(pg_wchar code)
+{
+ uint32 category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ if (category_mask & (PG_U_CC_MASK|PG_U_CS_MASK|PG_U_CN_MASK) ||
+ pg_u_isspace(code))
+ return false;
+ return true;
+}
+
+bool
+pg_u_isprint(pg_wchar code)
+{
+ pg_unicode_category category = unicode_category(code);
+
+ if (category == PG_U_CONTROL)
+ return false;
+
+ return pg_u_isgraph(code) || pg_u_isblank(code);
+}
+
+bool
+pg_u_ispunct(pg_wchar code, bool posix)
+{
+ uint32 category_mask;
+
+ if (posix)
+ {
+ if (pg_u_isalpha(code))
+ return false;
+
+ category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+ return category_mask & (PG_U_P_MASK|PG_U_S_MASK);
+ }
+ else
+ {
+ category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ return category_mask & PG_U_P_MASK;
+ }
+}
+
+bool
+pg_u_isspace(pg_wchar code)
+{
+ return pg_u_prop_white_space(code);
+}
+
+bool
+pg_u_isxdigit(pg_wchar code, bool posix)
+{
+ if (posix)
+ return (('0' <= code && code <= '9') ||
+ ('A' <= code && code <= 'F') ||
+ ('a' <= code && code <= 'f'));
+ else
+ return unicode_category(code) == PG_U_DECIMAL_NUMBER ||
+ pg_u_prop_hex_digit(code);
+}
+
/*
* Description of Unicode general category.
*/
@@ -191,3 +412,30 @@ unicode_category_abbrev(pg_unicode_category category)
Assert(false);
return "??"; /* keep compiler quiet */
}
+
+/*
+ * Binary search to test if given codepoint exists in one of the ranges in the
+ * given table.
+ */
+static bool
+range_search(const pg_unicode_range * tbl, Size size, pg_wchar code)
+{
+ int min = 0;
+ int mid;
+ int max = size - 1;
+
+ Assert(code <= 0x10ffff);
+
+ while (max >= min)
+ {
+ mid = (min + max) / 2;
+ if (code > tbl[mid].last)
+ min = mid + 1;
+ else if (code < tbl[mid].first)
+ max = mid - 1;
+ else
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/include/common/unicode_category.h b/src/include/common/unicode_category.h
index 81d38c7411..56fdc49802 100644
--- a/src/include/common/unicode_category.h
+++ b/src/include/common/unicode_category.h
@@ -62,7 +62,28 @@ typedef enum pg_unicode_category
} pg_unicode_category;
extern pg_unicode_category unicode_category(pg_wchar ucs);
-const char *unicode_category_string(pg_unicode_category category);
-const char *unicode_category_abbrev(pg_unicode_category category);
+extern const char *unicode_category_string(pg_unicode_category category);
+extern const char *unicode_category_abbrev(pg_unicode_category category);
+
+extern bool pg_u_prop_alphabetic(pg_wchar c);
+extern bool pg_u_prop_lowercase(pg_wchar c);
+extern bool pg_u_prop_uppercase(pg_wchar c);
+extern bool pg_u_prop_white_space(pg_wchar c);
+extern bool pg_u_prop_hex_digit(pg_wchar c);
+extern bool pg_u_prop_join_control(pg_wchar c);
+
+extern bool pg_u_isdigit(pg_wchar c, bool posix);
+extern bool pg_u_isalpha(pg_wchar c);
+extern bool pg_u_isalnum(pg_wchar c, bool posix);
+extern bool pg_u_isword(pg_wchar c);
+extern bool pg_u_isupper(pg_wchar c);
+extern bool pg_u_islower(pg_wchar c);
+extern bool pg_u_isblank(pg_wchar c);
+extern bool pg_u_iscntrl(pg_wchar c);
+extern bool pg_u_isgraph(pg_wchar c);
+extern bool pg_u_isprint(pg_wchar c);
+extern bool pg_u_ispunct(pg_wchar c, bool posix);
+extern bool pg_u_isspace(pg_wchar c);
+extern bool pg_u_isxdigit(pg_wchar c, bool posix);
#endif /* UNICODE_CATEGORY_H */
diff --git a/src/include/common/unicode_category_table.h b/src/include/common/unicode_category_table.h
index 14f1ea0677..86cdc9c0ed 100644
--- a/src/include/common/unicode_category_table.h
+++ b/src/include/common/unicode_category_table.h
@@ -25,6 +25,12 @@ typedef struct
uint8 category; /* General Category */
} pg_category_range;
+typedef struct
+{
+ uint32 first; /* Unicode codepoint */
+ uint32 last; /* Unicode codepoint */
+} pg_unicode_range;
+
/* table of Unicode codepoint ranges and their categories */
static const pg_category_range unicode_categories[3302] =
{
@@ -3331,3 +3337,2529 @@ static const pg_category_range unicode_categories[3302] =
{0x0f0000, 0x0ffffd, PG_U_PRIVATE_USE},
{0x100000, 0x10fffd, PG_U_PRIVATE_USE}
};
+
+/* table of Unicode codepoint ranges of Alphabetic characters */
+static const pg_unicode_range unicode_alphabetic[1141] =
+{
+ {0x000041, 0x00005a},
+ {0x000061, 0x00007a},
+ {0x0000aa, 0x0000aa},
+ {0x0000b5, 0x0000b5},
+ {0x0000ba, 0x0000ba},
+ {0x0000c0, 0x0000d6},
+ {0x0000d8, 0x0000f6},
+ {0x0000f8, 0x0001ba},
+ {0x0001bb, 0x0001bb},
+ {0x0001bc, 0x0001bf},
+ {0x0001c0, 0x0001c3},
+ {0x0001c4, 0x000293},
+ {0x000294, 0x000294},
+ {0x000295, 0x0002af},
+ {0x0002b0, 0x0002c1},
+ {0x0002c6, 0x0002d1},
+ {0x0002e0, 0x0002e4},
+ {0x0002ec, 0x0002ec},
+ {0x0002ee, 0x0002ee},
+ {0x000345, 0x000345},
+ {0x000370, 0x000373},
+ {0x000374, 0x000374},
+ {0x000376, 0x000377},
+ {0x00037a, 0x00037a},
+ {0x00037b, 0x00037d},
+ {0x00037f, 0x00037f},
+ {0x000386, 0x000386},
+ {0x000388, 0x00038a},
+ {0x00038c, 0x00038c},
+ {0x00038e, 0x0003a1},
+ {0x0003a3, 0x0003f5},
+ {0x0003f7, 0x000481},
+ {0x00048a, 0x00052f},
+ {0x000531, 0x000556},
+ {0x000559, 0x000559},
+ {0x000560, 0x000588},
+ {0x0005b0, 0x0005bd},
+ {0x0005bf, 0x0005bf},
+ {0x0005c1, 0x0005c2},
+ {0x0005c4, 0x0005c5},
+ {0x0005c7, 0x0005c7},
+ {0x0005d0, 0x0005ea},
+ {0x0005ef, 0x0005f2},
+ {0x000610, 0x00061a},
+ {0x000620, 0x00063f},
+ {0x000640, 0x000640},
+ {0x000641, 0x00064a},
+ {0x00064b, 0x000657},
+ {0x000659, 0x00065f},
+ {0x00066e, 0x00066f},
+ {0x000670, 0x000670},
+ {0x000671, 0x0006d3},
+ {0x0006d5, 0x0006d5},
+ {0x0006d6, 0x0006dc},
+ {0x0006e1, 0x0006e4},
+ {0x0006e5, 0x0006e6},
+ {0x0006e7, 0x0006e8},
+ {0x0006ed, 0x0006ed},
+ {0x0006ee, 0x0006ef},
+ {0x0006fa, 0x0006fc},
+ {0x0006ff, 0x0006ff},
+ {0x000710, 0x000710},
+ {0x000711, 0x000711},
+ {0x000712, 0x00072f},
+ {0x000730, 0x00073f},
+ {0x00074d, 0x0007a5},
+ {0x0007a6, 0x0007b0},
+ {0x0007b1, 0x0007b1},
+ {0x0007ca, 0x0007ea},
+ {0x0007f4, 0x0007f5},
+ {0x0007fa, 0x0007fa},
+ {0x000800, 0x000815},
+ {0x000816, 0x000817},
+ {0x00081a, 0x00081a},
+ {0x00081b, 0x000823},
+ {0x000824, 0x000824},
+ {0x000825, 0x000827},
+ {0x000828, 0x000828},
+ {0x000829, 0x00082c},
+ {0x000840, 0x000858},
+ {0x000860, 0x00086a},
+ {0x000870, 0x000887},
+ {0x000889, 0x00088e},
+ {0x0008a0, 0x0008c8},
+ {0x0008c9, 0x0008c9},
+ {0x0008d4, 0x0008df},
+ {0x0008e3, 0x0008e9},
+ {0x0008f0, 0x000902},
+ {0x000903, 0x000903},
+ {0x000904, 0x000939},
+ {0x00093a, 0x00093a},
+ {0x00093b, 0x00093b},
+ {0x00093d, 0x00093d},
+ {0x00093e, 0x000940},
+ {0x000941, 0x000948},
+ {0x000949, 0x00094c},
+ {0x00094e, 0x00094f},
+ {0x000950, 0x000950},
+ {0x000955, 0x000957},
+ {0x000958, 0x000961},
+ {0x000962, 0x000963},
+ {0x000971, 0x000971},
+ {0x000972, 0x000980},
+ {0x000981, 0x000981},
+ {0x000982, 0x000983},
+ {0x000985, 0x00098c},
+ {0x00098f, 0x000990},
+ {0x000993, 0x0009a8},
+ {0x0009aa, 0x0009b0},
+ {0x0009b2, 0x0009b2},
+ {0x0009b6, 0x0009b9},
+ {0x0009bd, 0x0009bd},
+ {0x0009be, 0x0009c0},
+ {0x0009c1, 0x0009c4},
+ {0x0009c7, 0x0009c8},
+ {0x0009cb, 0x0009cc},
+ {0x0009ce, 0x0009ce},
+ {0x0009d7, 0x0009d7},
+ {0x0009dc, 0x0009dd},
+ {0x0009df, 0x0009e1},
+ {0x0009e2, 0x0009e3},
+ {0x0009f0, 0x0009f1},
+ {0x0009fc, 0x0009fc},
+ {0x000a01, 0x000a02},
+ {0x000a03, 0x000a03},
+ {0x000a05, 0x000a0a},
+ {0x000a0f, 0x000a10},
+ {0x000a13, 0x000a28},
+ {0x000a2a, 0x000a30},
+ {0x000a32, 0x000a33},
+ {0x000a35, 0x000a36},
+ {0x000a38, 0x000a39},
+ {0x000a3e, 0x000a40},
+ {0x000a41, 0x000a42},
+ {0x000a47, 0x000a48},
+ {0x000a4b, 0x000a4c},
+ {0x000a51, 0x000a51},
+ {0x000a59, 0x000a5c},
+ {0x000a5e, 0x000a5e},
+ {0x000a70, 0x000a71},
+ {0x000a72, 0x000a74},
+ {0x000a75, 0x000a75},
+ {0x000a81, 0x000a82},
+ {0x000a83, 0x000a83},
+ {0x000a85, 0x000a8d},
+ {0x000a8f, 0x000a91},
+ {0x000a93, 0x000aa8},
+ {0x000aaa, 0x000ab0},
+ {0x000ab2, 0x000ab3},
+ {0x000ab5, 0x000ab9},
+ {0x000abd, 0x000abd},
+ {0x000abe, 0x000ac0},
+ {0x000ac1, 0x000ac5},
+ {0x000ac7, 0x000ac8},
+ {0x000ac9, 0x000ac9},
+ {0x000acb, 0x000acc},
+ {0x000ad0, 0x000ad0},
+ {0x000ae0, 0x000ae1},
+ {0x000ae2, 0x000ae3},
+ {0x000af9, 0x000af9},
+ {0x000afa, 0x000afc},
+ {0x000b01, 0x000b01},
+ {0x000b02, 0x000b03},
+ {0x000b05, 0x000b0c},
+ {0x000b0f, 0x000b10},
+ {0x000b13, 0x000b28},
+ {0x000b2a, 0x000b30},
+ {0x000b32, 0x000b33},
+ {0x000b35, 0x000b39},
+ {0x000b3d, 0x000b3d},
+ {0x000b3e, 0x000b3e},
+ {0x000b3f, 0x000b3f},
+ {0x000b40, 0x000b40},
+ {0x000b41, 0x000b44},
+ {0x000b47, 0x000b48},
+ {0x000b4b, 0x000b4c},
+ {0x000b56, 0x000b56},
+ {0x000b57, 0x000b57},
+ {0x000b5c, 0x000b5d},
+ {0x000b5f, 0x000b61},
+ {0x000b62, 0x000b63},
+ {0x000b71, 0x000b71},
+ {0x000b82, 0x000b82},
+ {0x000b83, 0x000b83},
+ {0x000b85, 0x000b8a},
+ {0x000b8e, 0x000b90},
+ {0x000b92, 0x000b95},
+ {0x000b99, 0x000b9a},
+ {0x000b9c, 0x000b9c},
+ {0x000b9e, 0x000b9f},
+ {0x000ba3, 0x000ba4},
+ {0x000ba8, 0x000baa},
+ {0x000bae, 0x000bb9},
+ {0x000bbe, 0x000bbf},
+ {0x000bc0, 0x000bc0},
+ {0x000bc1, 0x000bc2},
+ {0x000bc6, 0x000bc8},
+ {0x000bca, 0x000bcc},
+ {0x000bd0, 0x000bd0},
+ {0x000bd7, 0x000bd7},
+ {0x000c00, 0x000c00},
+ {0x000c01, 0x000c03},
+ {0x000c04, 0x000c04},
+ {0x000c05, 0x000c0c},
+ {0x000c0e, 0x000c10},
+ {0x000c12, 0x000c28},
+ {0x000c2a, 0x000c39},
+ {0x000c3d, 0x000c3d},
+ {0x000c3e, 0x000c40},
+ {0x000c41, 0x000c44},
+ {0x000c46, 0x000c48},
+ {0x000c4a, 0x000c4c},
+ {0x000c55, 0x000c56},
+ {0x000c58, 0x000c5a},
+ {0x000c5d, 0x000c5d},
+ {0x000c60, 0x000c61},
+ {0x000c62, 0x000c63},
+ {0x000c80, 0x000c80},
+ {0x000c81, 0x000c81},
+ {0x000c82, 0x000c83},
+ {0x000c85, 0x000c8c},
+ {0x000c8e, 0x000c90},
+ {0x000c92, 0x000ca8},
+ {0x000caa, 0x000cb3},
+ {0x000cb5, 0x000cb9},
+ {0x000cbd, 0x000cbd},
+ {0x000cbe, 0x000cbe},
+ {0x000cbf, 0x000cbf},
+ {0x000cc0, 0x000cc4},
+ {0x000cc6, 0x000cc6},
+ {0x000cc7, 0x000cc8},
+ {0x000cca, 0x000ccb},
+ {0x000ccc, 0x000ccc},
+ {0x000cd5, 0x000cd6},
+ {0x000cdd, 0x000cde},
+ {0x000ce0, 0x000ce1},
+ {0x000ce2, 0x000ce3},
+ {0x000cf1, 0x000cf2},
+ {0x000cf3, 0x000cf3},
+ {0x000d00, 0x000d01},
+ {0x000d02, 0x000d03},
+ {0x000d04, 0x000d0c},
+ {0x000d0e, 0x000d10},
+ {0x000d12, 0x000d3a},
+ {0x000d3d, 0x000d3d},
+ {0x000d3e, 0x000d40},
+ {0x000d41, 0x000d44},
+ {0x000d46, 0x000d48},
+ {0x000d4a, 0x000d4c},
+ {0x000d4e, 0x000d4e},
+ {0x000d54, 0x000d56},
+ {0x000d57, 0x000d57},
+ {0x000d5f, 0x000d61},
+ {0x000d62, 0x000d63},
+ {0x000d7a, 0x000d7f},
+ {0x000d81, 0x000d81},
+ {0x000d82, 0x000d83},
+ {0x000d85, 0x000d96},
+ {0x000d9a, 0x000db1},
+ {0x000db3, 0x000dbb},
+ {0x000dbd, 0x000dbd},
+ {0x000dc0, 0x000dc6},
+ {0x000dcf, 0x000dd1},
+ {0x000dd2, 0x000dd4},
+ {0x000dd6, 0x000dd6},
+ {0x000dd8, 0x000ddf},
+ {0x000df2, 0x000df3},
+ {0x000e01, 0x000e30},
+ {0x000e31, 0x000e31},
+ {0x000e32, 0x000e33},
+ {0x000e34, 0x000e3a},
+ {0x000e40, 0x000e45},
+ {0x000e46, 0x000e46},
+ {0x000e4d, 0x000e4d},
+ {0x000e81, 0x000e82},
+ {0x000e84, 0x000e84},
+ {0x000e86, 0x000e8a},
+ {0x000e8c, 0x000ea3},
+ {0x000ea5, 0x000ea5},
+ {0x000ea7, 0x000eb0},
+ {0x000eb1, 0x000eb1},
+ {0x000eb2, 0x000eb3},
+ {0x000eb4, 0x000eb9},
+ {0x000ebb, 0x000ebc},
+ {0x000ebd, 0x000ebd},
+ {0x000ec0, 0x000ec4},
+ {0x000ec6, 0x000ec6},
+ {0x000ecd, 0x000ecd},
+ {0x000edc, 0x000edf},
+ {0x000f00, 0x000f00},
+ {0x000f40, 0x000f47},
+ {0x000f49, 0x000f6c},
+ {0x000f71, 0x000f7e},
+ {0x000f7f, 0x000f7f},
+ {0x000f80, 0x000f83},
+ {0x000f88, 0x000f8c},
+ {0x000f8d, 0x000f97},
+ {0x000f99, 0x000fbc},
+ {0x001000, 0x00102a},
+ {0x00102b, 0x00102c},
+ {0x00102d, 0x001030},
+ {0x001031, 0x001031},
+ {0x001032, 0x001036},
+ {0x001038, 0x001038},
+ {0x00103b, 0x00103c},
+ {0x00103d, 0x00103e},
+ {0x00103f, 0x00103f},
+ {0x001050, 0x001055},
+ {0x001056, 0x001057},
+ {0x001058, 0x001059},
+ {0x00105a, 0x00105d},
+ {0x00105e, 0x001060},
+ {0x001061, 0x001061},
+ {0x001062, 0x001064},
+ {0x001065, 0x001066},
+ {0x001067, 0x00106d},
+ {0x00106e, 0x001070},
+ {0x001071, 0x001074},
+ {0x001075, 0x001081},
+ {0x001082, 0x001082},
+ {0x001083, 0x001084},
+ {0x001085, 0x001086},
+ {0x001087, 0x00108c},
+ {0x00108d, 0x00108d},
+ {0x00108e, 0x00108e},
+ {0x00108f, 0x00108f},
+ {0x00109a, 0x00109c},
+ {0x00109d, 0x00109d},
+ {0x0010a0, 0x0010c5},
+ {0x0010c7, 0x0010c7},
+ {0x0010cd, 0x0010cd},
+ {0x0010d0, 0x0010fa},
+ {0x0010fc, 0x0010fc},
+ {0x0010fd, 0x0010ff},
+ {0x001100, 0x001248},
+ {0x00124a, 0x00124d},
+ {0x001250, 0x001256},
+ {0x001258, 0x001258},
+ {0x00125a, 0x00125d},
+ {0x001260, 0x001288},
+ {0x00128a, 0x00128d},
+ {0x001290, 0x0012b0},
+ {0x0012b2, 0x0012b5},
+ {0x0012b8, 0x0012be},
+ {0x0012c0, 0x0012c0},
+ {0x0012c2, 0x0012c5},
+ {0x0012c8, 0x0012d6},
+ {0x0012d8, 0x001310},
+ {0x001312, 0x001315},
+ {0x001318, 0x00135a},
+ {0x001380, 0x00138f},
+ {0x0013a0, 0x0013f5},
+ {0x0013f8, 0x0013fd},
+ {0x001401, 0x00166c},
+ {0x00166f, 0x00167f},
+ {0x001681, 0x00169a},
+ {0x0016a0, 0x0016ea},
+ {0x0016ee, 0x0016f0},
+ {0x0016f1, 0x0016f8},
+ {0x001700, 0x001711},
+ {0x001712, 0x001713},
+ {0x00171f, 0x001731},
+ {0x001732, 0x001733},
+ {0x001740, 0x001751},
+ {0x001752, 0x001753},
+ {0x001760, 0x00176c},
+ {0x00176e, 0x001770},
+ {0x001772, 0x001773},
+ {0x001780, 0x0017b3},
+ {0x0017b6, 0x0017b6},
+ {0x0017b7, 0x0017bd},
+ {0x0017be, 0x0017c5},
+ {0x0017c6, 0x0017c6},
+ {0x0017c7, 0x0017c8},
+ {0x0017d7, 0x0017d7},
+ {0x0017dc, 0x0017dc},
+ {0x001820, 0x001842},
+ {0x001843, 0x001843},
+ {0x001844, 0x001878},
+ {0x001880, 0x001884},
+ {0x001885, 0x001886},
+ {0x001887, 0x0018a8},
+ {0x0018a9, 0x0018a9},
+ {0x0018aa, 0x0018aa},
+ {0x0018b0, 0x0018f5},
+ {0x001900, 0x00191e},
+ {0x001920, 0x001922},
+ {0x001923, 0x001926},
+ {0x001927, 0x001928},
+ {0x001929, 0x00192b},
+ {0x001930, 0x001931},
+ {0x001932, 0x001932},
+ {0x001933, 0x001938},
+ {0x001950, 0x00196d},
+ {0x001970, 0x001974},
+ {0x001980, 0x0019ab},
+ {0x0019b0, 0x0019c9},
+ {0x001a00, 0x001a16},
+ {0x001a17, 0x001a18},
+ {0x001a19, 0x001a1a},
+ {0x001a1b, 0x001a1b},
+ {0x001a20, 0x001a54},
+ {0x001a55, 0x001a55},
+ {0x001a56, 0x001a56},
+ {0x001a57, 0x001a57},
+ {0x001a58, 0x001a5e},
+ {0x001a61, 0x001a61},
+ {0x001a62, 0x001a62},
+ {0x001a63, 0x001a64},
+ {0x001a65, 0x001a6c},
+ {0x001a6d, 0x001a72},
+ {0x001a73, 0x001a74},
+ {0x001aa7, 0x001aa7},
+ {0x001abf, 0x001ac0},
+ {0x001acc, 0x001ace},
+ {0x001b00, 0x001b03},
+ {0x001b04, 0x001b04},
+ {0x001b05, 0x001b33},
+ {0x001b35, 0x001b35},
+ {0x001b36, 0x001b3a},
+ {0x001b3b, 0x001b3b},
+ {0x001b3c, 0x001b3c},
+ {0x001b3d, 0x001b41},
+ {0x001b42, 0x001b42},
+ {0x001b43, 0x001b43},
+ {0x001b45, 0x001b4c},
+ {0x001b80, 0x001b81},
+ {0x001b82, 0x001b82},
+ {0x001b83, 0x001ba0},
+ {0x001ba1, 0x001ba1},
+ {0x001ba2, 0x001ba5},
+ {0x001ba6, 0x001ba7},
+ {0x001ba8, 0x001ba9},
+ {0x001bac, 0x001bad},
+ {0x001bae, 0x001baf},
+ {0x001bba, 0x001be5},
+ {0x001be7, 0x001be7},
+ {0x001be8, 0x001be9},
+ {0x001bea, 0x001bec},
+ {0x001bed, 0x001bed},
+ {0x001bee, 0x001bee},
+ {0x001bef, 0x001bf1},
+ {0x001c00, 0x001c23},
+ {0x001c24, 0x001c2b},
+ {0x001c2c, 0x001c33},
+ {0x001c34, 0x001c35},
+ {0x001c36, 0x001c36},
+ {0x001c4d, 0x001c4f},
+ {0x001c5a, 0x001c77},
+ {0x001c78, 0x001c7d},
+ {0x001c80, 0x001c88},
+ {0x001c90, 0x001cba},
+ {0x001cbd, 0x001cbf},
+ {0x001ce9, 0x001cec},
+ {0x001cee, 0x001cf3},
+ {0x001cf5, 0x001cf6},
+ {0x001cfa, 0x001cfa},
+ {0x001d00, 0x001d2b},
+ {0x001d2c, 0x001d6a},
+ {0x001d6b, 0x001d77},
+ {0x001d78, 0x001d78},
+ {0x001d79, 0x001d9a},
+ {0x001d9b, 0x001dbf},
+ {0x001de7, 0x001df4},
+ {0x001e00, 0x001f15},
+ {0x001f18, 0x001f1d},
+ {0x001f20, 0x001f45},
+ {0x001f48, 0x001f4d},
+ {0x001f50, 0x001f57},
+ {0x001f59, 0x001f59},
+ {0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f7d},
+ {0x001f80, 0x001fb4},
+ {0x001fb6, 0x001fbc},
+ {0x001fbe, 0x001fbe},
+ {0x001fc2, 0x001fc4},
+ {0x001fc6, 0x001fcc},
+ {0x001fd0, 0x001fd3},
+ {0x001fd6, 0x001fdb},
+ {0x001fe0, 0x001fec},
+ {0x001ff2, 0x001ff4},
+ {0x001ff6, 0x001ffc},
+ {0x002071, 0x002071},
+ {0x00207f, 0x00207f},
+ {0x002090, 0x00209c},
+ {0x002102, 0x002102},
+ {0x002107, 0x002107},
+ {0x00210a, 0x002113},
+ {0x002115, 0x002115},
+ {0x002119, 0x00211d},
+ {0x002124, 0x002124},
+ {0x002126, 0x002126},
+ {0x002128, 0x002128},
+ {0x00212a, 0x00212d},
+ {0x00212f, 0x002134},
+ {0x002135, 0x002138},
+ {0x002139, 0x002139},
+ {0x00213c, 0x00213f},
+ {0x002145, 0x002149},
+ {0x00214e, 0x00214e},
+ {0x002160, 0x002182},
+ {0x002183, 0x002184},
+ {0x002185, 0x002188},
+ {0x0024b6, 0x0024e9},
+ {0x002c00, 0x002c7b},
+ {0x002c7c, 0x002c7d},
+ {0x002c7e, 0x002ce4},
+ {0x002ceb, 0x002cee},
+ {0x002cf2, 0x002cf3},
+ {0x002d00, 0x002d25},
+ {0x002d27, 0x002d27},
+ {0x002d2d, 0x002d2d},
+ {0x002d30, 0x002d67},
+ {0x002d6f, 0x002d6f},
+ {0x002d80, 0x002d96},
+ {0x002da0, 0x002da6},
+ {0x002da8, 0x002dae},
+ {0x002db0, 0x002db6},
+ {0x002db8, 0x002dbe},
+ {0x002dc0, 0x002dc6},
+ {0x002dc8, 0x002dce},
+ {0x002dd0, 0x002dd6},
+ {0x002dd8, 0x002dde},
+ {0x002de0, 0x002dff},
+ {0x002e2f, 0x002e2f},
+ {0x003005, 0x003005},
+ {0x003006, 0x003006},
+ {0x003007, 0x003007},
+ {0x003021, 0x003029},
+ {0x003031, 0x003035},
+ {0x003038, 0x00303a},
+ {0x00303b, 0x00303b},
+ {0x00303c, 0x00303c},
+ {0x003041, 0x003096},
+ {0x00309d, 0x00309e},
+ {0x00309f, 0x00309f},
+ {0x0030a1, 0x0030fa},
+ {0x0030fc, 0x0030fe},
+ {0x0030ff, 0x0030ff},
+ {0x003105, 0x00312f},
+ {0x003131, 0x00318e},
+ {0x0031a0, 0x0031bf},
+ {0x0031f0, 0x0031ff},
+ {0x003400, 0x004dbf},
+ {0x004e00, 0x00a014},
+ {0x00a015, 0x00a015},
+ {0x00a016, 0x00a48c},
+ {0x00a4d0, 0x00a4f7},
+ {0x00a4f8, 0x00a4fd},
+ {0x00a500, 0x00a60b},
+ {0x00a60c, 0x00a60c},
+ {0x00a610, 0x00a61f},
+ {0x00a62a, 0x00a62b},
+ {0x00a640, 0x00a66d},
+ {0x00a66e, 0x00a66e},
+ {0x00a674, 0x00a67b},
+ {0x00a67f, 0x00a67f},
+ {0x00a680, 0x00a69b},
+ {0x00a69c, 0x00a69d},
+ {0x00a69e, 0x00a69f},
+ {0x00a6a0, 0x00a6e5},
+ {0x00a6e6, 0x00a6ef},
+ {0x00a717, 0x00a71f},
+ {0x00a722, 0x00a76f},
+ {0x00a770, 0x00a770},
+ {0x00a771, 0x00a787},
+ {0x00a788, 0x00a788},
+ {0x00a78b, 0x00a78e},
+ {0x00a78f, 0x00a78f},
+ {0x00a790, 0x00a7ca},
+ {0x00a7d0, 0x00a7d1},
+ {0x00a7d3, 0x00a7d3},
+ {0x00a7d5, 0x00a7d9},
+ {0x00a7f2, 0x00a7f4},
+ {0x00a7f5, 0x00a7f6},
+ {0x00a7f7, 0x00a7f7},
+ {0x00a7f8, 0x00a7f9},
+ {0x00a7fa, 0x00a7fa},
+ {0x00a7fb, 0x00a801},
+ {0x00a802, 0x00a802},
+ {0x00a803, 0x00a805},
+ {0x00a807, 0x00a80a},
+ {0x00a80b, 0x00a80b},
+ {0x00a80c, 0x00a822},
+ {0x00a823, 0x00a824},
+ {0x00a825, 0x00a826},
+ {0x00a827, 0x00a827},
+ {0x00a840, 0x00a873},
+ {0x00a880, 0x00a881},
+ {0x00a882, 0x00a8b3},
+ {0x00a8b4, 0x00a8c3},
+ {0x00a8c5, 0x00a8c5},
+ {0x00a8f2, 0x00a8f7},
+ {0x00a8fb, 0x00a8fb},
+ {0x00a8fd, 0x00a8fe},
+ {0x00a8ff, 0x00a8ff},
+ {0x00a90a, 0x00a925},
+ {0x00a926, 0x00a92a},
+ {0x00a930, 0x00a946},
+ {0x00a947, 0x00a951},
+ {0x00a952, 0x00a952},
+ {0x00a960, 0x00a97c},
+ {0x00a980, 0x00a982},
+ {0x00a983, 0x00a983},
+ {0x00a984, 0x00a9b2},
+ {0x00a9b4, 0x00a9b5},
+ {0x00a9b6, 0x00a9b9},
+ {0x00a9ba, 0x00a9bb},
+ {0x00a9bc, 0x00a9bd},
+ {0x00a9be, 0x00a9bf},
+ {0x00a9cf, 0x00a9cf},
+ {0x00a9e0, 0x00a9e4},
+ {0x00a9e5, 0x00a9e5},
+ {0x00a9e6, 0x00a9e6},
+ {0x00a9e7, 0x00a9ef},
+ {0x00a9fa, 0x00a9fe},
+ {0x00aa00, 0x00aa28},
+ {0x00aa29, 0x00aa2e},
+ {0x00aa2f, 0x00aa30},
+ {0x00aa31, 0x00aa32},
+ {0x00aa33, 0x00aa34},
+ {0x00aa35, 0x00aa36},
+ {0x00aa40, 0x00aa42},
+ {0x00aa43, 0x00aa43},
+ {0x00aa44, 0x00aa4b},
+ {0x00aa4c, 0x00aa4c},
+ {0x00aa4d, 0x00aa4d},
+ {0x00aa60, 0x00aa6f},
+ {0x00aa70, 0x00aa70},
+ {0x00aa71, 0x00aa76},
+ {0x00aa7a, 0x00aa7a},
+ {0x00aa7b, 0x00aa7b},
+ {0x00aa7c, 0x00aa7c},
+ {0x00aa7d, 0x00aa7d},
+ {0x00aa7e, 0x00aaaf},
+ {0x00aab0, 0x00aab0},
+ {0x00aab1, 0x00aab1},
+ {0x00aab2, 0x00aab4},
+ {0x00aab5, 0x00aab6},
+ {0x00aab7, 0x00aab8},
+ {0x00aab9, 0x00aabd},
+ {0x00aabe, 0x00aabe},
+ {0x00aac0, 0x00aac0},
+ {0x00aac2, 0x00aac2},
+ {0x00aadb, 0x00aadc},
+ {0x00aadd, 0x00aadd},
+ {0x00aae0, 0x00aaea},
+ {0x00aaeb, 0x00aaeb},
+ {0x00aaec, 0x00aaed},
+ {0x00aaee, 0x00aaef},
+ {0x00aaf2, 0x00aaf2},
+ {0x00aaf3, 0x00aaf4},
+ {0x00aaf5, 0x00aaf5},
+ {0x00ab01, 0x00ab06},
+ {0x00ab09, 0x00ab0e},
+ {0x00ab11, 0x00ab16},
+ {0x00ab20, 0x00ab26},
+ {0x00ab28, 0x00ab2e},
+ {0x00ab30, 0x00ab5a},
+ {0x00ab5c, 0x00ab5f},
+ {0x00ab60, 0x00ab68},
+ {0x00ab69, 0x00ab69},
+ {0x00ab70, 0x00abbf},
+ {0x00abc0, 0x00abe2},
+ {0x00abe3, 0x00abe4},
+ {0x00abe5, 0x00abe5},
+ {0x00abe6, 0x00abe7},
+ {0x00abe8, 0x00abe8},
+ {0x00abe9, 0x00abea},
+ {0x00ac00, 0x00d7a3},
+ {0x00d7b0, 0x00d7c6},
+ {0x00d7cb, 0x00d7fb},
+ {0x00f900, 0x00fa6d},
+ {0x00fa70, 0x00fad9},
+ {0x00fb00, 0x00fb06},
+ {0x00fb13, 0x00fb17},
+ {0x00fb1d, 0x00fb1d},
+ {0x00fb1e, 0x00fb1e},
+ {0x00fb1f, 0x00fb28},
+ {0x00fb2a, 0x00fb36},
+ {0x00fb38, 0x00fb3c},
+ {0x00fb3e, 0x00fb3e},
+ {0x00fb40, 0x00fb41},
+ {0x00fb43, 0x00fb44},
+ {0x00fb46, 0x00fbb1},
+ {0x00fbd3, 0x00fd3d},
+ {0x00fd50, 0x00fd8f},
+ {0x00fd92, 0x00fdc7},
+ {0x00fdf0, 0x00fdfb},
+ {0x00fe70, 0x00fe74},
+ {0x00fe76, 0x00fefc},
+ {0x00ff21, 0x00ff3a},
+ {0x00ff41, 0x00ff5a},
+ {0x00ff66, 0x00ff6f},
+ {0x00ff70, 0x00ff70},
+ {0x00ff71, 0x00ff9d},
+ {0x00ff9e, 0x00ff9f},
+ {0x00ffa0, 0x00ffbe},
+ {0x00ffc2, 0x00ffc7},
+ {0x00ffca, 0x00ffcf},
+ {0x00ffd2, 0x00ffd7},
+ {0x00ffda, 0x00ffdc},
+ {0x010000, 0x01000b},
+ {0x01000d, 0x010026},
+ {0x010028, 0x01003a},
+ {0x01003c, 0x01003d},
+ {0x01003f, 0x01004d},
+ {0x010050, 0x01005d},
+ {0x010080, 0x0100fa},
+ {0x010140, 0x010174},
+ {0x010280, 0x01029c},
+ {0x0102a0, 0x0102d0},
+ {0x010300, 0x01031f},
+ {0x01032d, 0x010340},
+ {0x010341, 0x010341},
+ {0x010342, 0x010349},
+ {0x01034a, 0x01034a},
+ {0x010350, 0x010375},
+ {0x010376, 0x01037a},
+ {0x010380, 0x01039d},
+ {0x0103a0, 0x0103c3},
+ {0x0103c8, 0x0103cf},
+ {0x0103d1, 0x0103d5},
+ {0x010400, 0x01044f},
+ {0x010450, 0x01049d},
+ {0x0104b0, 0x0104d3},
+ {0x0104d8, 0x0104fb},
+ {0x010500, 0x010527},
+ {0x010530, 0x010563},
+ {0x010570, 0x01057a},
+ {0x01057c, 0x01058a},
+ {0x01058c, 0x010592},
+ {0x010594, 0x010595},
+ {0x010597, 0x0105a1},
+ {0x0105a3, 0x0105b1},
+ {0x0105b3, 0x0105b9},
+ {0x0105bb, 0x0105bc},
+ {0x010600, 0x010736},
+ {0x010740, 0x010755},
+ {0x010760, 0x010767},
+ {0x010780, 0x010785},
+ {0x010787, 0x0107b0},
+ {0x0107b2, 0x0107ba},
+ {0x010800, 0x010805},
+ {0x010808, 0x010808},
+ {0x01080a, 0x010835},
+ {0x010837, 0x010838},
+ {0x01083c, 0x01083c},
+ {0x01083f, 0x010855},
+ {0x010860, 0x010876},
+ {0x010880, 0x01089e},
+ {0x0108e0, 0x0108f2},
+ {0x0108f4, 0x0108f5},
+ {0x010900, 0x010915},
+ {0x010920, 0x010939},
+ {0x010980, 0x0109b7},
+ {0x0109be, 0x0109bf},
+ {0x010a00, 0x010a00},
+ {0x010a01, 0x010a03},
+ {0x010a05, 0x010a06},
+ {0x010a0c, 0x010a0f},
+ {0x010a10, 0x010a13},
+ {0x010a15, 0x010a17},
+ {0x010a19, 0x010a35},
+ {0x010a60, 0x010a7c},
+ {0x010a80, 0x010a9c},
+ {0x010ac0, 0x010ac7},
+ {0x010ac9, 0x010ae4},
+ {0x010b00, 0x010b35},
+ {0x010b40, 0x010b55},
+ {0x010b60, 0x010b72},
+ {0x010b80, 0x010b91},
+ {0x010c00, 0x010c48},
+ {0x010c80, 0x010cb2},
+ {0x010cc0, 0x010cf2},
+ {0x010d00, 0x010d23},
+ {0x010d24, 0x010d27},
+ {0x010e80, 0x010ea9},
+ {0x010eab, 0x010eac},
+ {0x010eb0, 0x010eb1},
+ {0x010f00, 0x010f1c},
+ {0x010f27, 0x010f27},
+ {0x010f30, 0x010f45},
+ {0x010f70, 0x010f81},
+ {0x010fb0, 0x010fc4},
+ {0x010fe0, 0x010ff6},
+ {0x011000, 0x011000},
+ {0x011001, 0x011001},
+ {0x011002, 0x011002},
+ {0x011003, 0x011037},
+ {0x011038, 0x011045},
+ {0x011071, 0x011072},
+ {0x011073, 0x011074},
+ {0x011075, 0x011075},
+ {0x011080, 0x011081},
+ {0x011082, 0x011082},
+ {0x011083, 0x0110af},
+ {0x0110b0, 0x0110b2},
+ {0x0110b3, 0x0110b6},
+ {0x0110b7, 0x0110b8},
+ {0x0110c2, 0x0110c2},
+ {0x0110d0, 0x0110e8},
+ {0x011100, 0x011102},
+ {0x011103, 0x011126},
+ {0x011127, 0x01112b},
+ {0x01112c, 0x01112c},
+ {0x01112d, 0x011132},
+ {0x011144, 0x011144},
+ {0x011145, 0x011146},
+ {0x011147, 0x011147},
+ {0x011150, 0x011172},
+ {0x011176, 0x011176},
+ {0x011180, 0x011181},
+ {0x011182, 0x011182},
+ {0x011183, 0x0111b2},
+ {0x0111b3, 0x0111b5},
+ {0x0111b6, 0x0111be},
+ {0x0111bf, 0x0111bf},
+ {0x0111c1, 0x0111c4},
+ {0x0111ce, 0x0111ce},
+ {0x0111cf, 0x0111cf},
+ {0x0111da, 0x0111da},
+ {0x0111dc, 0x0111dc},
+ {0x011200, 0x011211},
+ {0x011213, 0x01122b},
+ {0x01122c, 0x01122e},
+ {0x01122f, 0x011231},
+ {0x011232, 0x011233},
+ {0x011234, 0x011234},
+ {0x011237, 0x011237},
+ {0x01123e, 0x01123e},
+ {0x01123f, 0x011240},
+ {0x011241, 0x011241},
+ {0x011280, 0x011286},
+ {0x011288, 0x011288},
+ {0x01128a, 0x01128d},
+ {0x01128f, 0x01129d},
+ {0x01129f, 0x0112a8},
+ {0x0112b0, 0x0112de},
+ {0x0112df, 0x0112df},
+ {0x0112e0, 0x0112e2},
+ {0x0112e3, 0x0112e8},
+ {0x011300, 0x011301},
+ {0x011302, 0x011303},
+ {0x011305, 0x01130c},
+ {0x01130f, 0x011310},
+ {0x011313, 0x011328},
+ {0x01132a, 0x011330},
+ {0x011332, 0x011333},
+ {0x011335, 0x011339},
+ {0x01133d, 0x01133d},
+ {0x01133e, 0x01133f},
+ {0x011340, 0x011340},
+ {0x011341, 0x011344},
+ {0x011347, 0x011348},
+ {0x01134b, 0x01134c},
+ {0x011350, 0x011350},
+ {0x011357, 0x011357},
+ {0x01135d, 0x011361},
+ {0x011362, 0x011363},
+ {0x011400, 0x011434},
+ {0x011435, 0x011437},
+ {0x011438, 0x01143f},
+ {0x011440, 0x011441},
+ {0x011443, 0x011444},
+ {0x011445, 0x011445},
+ {0x011447, 0x01144a},
+ {0x01145f, 0x011461},
+ {0x011480, 0x0114af},
+ {0x0114b0, 0x0114b2},
+ {0x0114b3, 0x0114b8},
+ {0x0114b9, 0x0114b9},
+ {0x0114ba, 0x0114ba},
+ {0x0114bb, 0x0114be},
+ {0x0114bf, 0x0114c0},
+ {0x0114c1, 0x0114c1},
+ {0x0114c4, 0x0114c5},
+ {0x0114c7, 0x0114c7},
+ {0x011580, 0x0115ae},
+ {0x0115af, 0x0115b1},
+ {0x0115b2, 0x0115b5},
+ {0x0115b8, 0x0115bb},
+ {0x0115bc, 0x0115bd},
+ {0x0115be, 0x0115be},
+ {0x0115d8, 0x0115db},
+ {0x0115dc, 0x0115dd},
+ {0x011600, 0x01162f},
+ {0x011630, 0x011632},
+ {0x011633, 0x01163a},
+ {0x01163b, 0x01163c},
+ {0x01163d, 0x01163d},
+ {0x01163e, 0x01163e},
+ {0x011640, 0x011640},
+ {0x011644, 0x011644},
+ {0x011680, 0x0116aa},
+ {0x0116ab, 0x0116ab},
+ {0x0116ac, 0x0116ac},
+ {0x0116ad, 0x0116ad},
+ {0x0116ae, 0x0116af},
+ {0x0116b0, 0x0116b5},
+ {0x0116b8, 0x0116b8},
+ {0x011700, 0x01171a},
+ {0x01171d, 0x01171f},
+ {0x011720, 0x011721},
+ {0x011722, 0x011725},
+ {0x011726, 0x011726},
+ {0x011727, 0x01172a},
+ {0x011740, 0x011746},
+ {0x011800, 0x01182b},
+ {0x01182c, 0x01182e},
+ {0x01182f, 0x011837},
+ {0x011838, 0x011838},
+ {0x0118a0, 0x0118df},
+ {0x0118ff, 0x011906},
+ {0x011909, 0x011909},
+ {0x01190c, 0x011913},
+ {0x011915, 0x011916},
+ {0x011918, 0x01192f},
+ {0x011930, 0x011935},
+ {0x011937, 0x011938},
+ {0x01193b, 0x01193c},
+ {0x01193f, 0x01193f},
+ {0x011940, 0x011940},
+ {0x011941, 0x011941},
+ {0x011942, 0x011942},
+ {0x0119a0, 0x0119a7},
+ {0x0119aa, 0x0119d0},
+ {0x0119d1, 0x0119d3},
+ {0x0119d4, 0x0119d7},
+ {0x0119da, 0x0119db},
+ {0x0119dc, 0x0119df},
+ {0x0119e1, 0x0119e1},
+ {0x0119e3, 0x0119e3},
+ {0x0119e4, 0x0119e4},
+ {0x011a00, 0x011a00},
+ {0x011a01, 0x011a0a},
+ {0x011a0b, 0x011a32},
+ {0x011a35, 0x011a38},
+ {0x011a39, 0x011a39},
+ {0x011a3a, 0x011a3a},
+ {0x011a3b, 0x011a3e},
+ {0x011a50, 0x011a50},
+ {0x011a51, 0x011a56},
+ {0x011a57, 0x011a58},
+ {0x011a59, 0x011a5b},
+ {0x011a5c, 0x011a89},
+ {0x011a8a, 0x011a96},
+ {0x011a97, 0x011a97},
+ {0x011a9d, 0x011a9d},
+ {0x011ab0, 0x011af8},
+ {0x011c00, 0x011c08},
+ {0x011c0a, 0x011c2e},
+ {0x011c2f, 0x011c2f},
+ {0x011c30, 0x011c36},
+ {0x011c38, 0x011c3d},
+ {0x011c3e, 0x011c3e},
+ {0x011c40, 0x011c40},
+ {0x011c72, 0x011c8f},
+ {0x011c92, 0x011ca7},
+ {0x011ca9, 0x011ca9},
+ {0x011caa, 0x011cb0},
+ {0x011cb1, 0x011cb1},
+ {0x011cb2, 0x011cb3},
+ {0x011cb4, 0x011cb4},
+ {0x011cb5, 0x011cb6},
+ {0x011d00, 0x011d06},
+ {0x011d08, 0x011d09},
+ {0x011d0b, 0x011d30},
+ {0x011d31, 0x011d36},
+ {0x011d3a, 0x011d3a},
+ {0x011d3c, 0x011d3d},
+ {0x011d3f, 0x011d41},
+ {0x011d43, 0x011d43},
+ {0x011d46, 0x011d46},
+ {0x011d47, 0x011d47},
+ {0x011d60, 0x011d65},
+ {0x011d67, 0x011d68},
+ {0x011d6a, 0x011d89},
+ {0x011d8a, 0x011d8e},
+ {0x011d90, 0x011d91},
+ {0x011d93, 0x011d94},
+ {0x011d95, 0x011d95},
+ {0x011d96, 0x011d96},
+ {0x011d98, 0x011d98},
+ {0x011ee0, 0x011ef2},
+ {0x011ef3, 0x011ef4},
+ {0x011ef5, 0x011ef6},
+ {0x011f00, 0x011f01},
+ {0x011f02, 0x011f02},
+ {0x011f03, 0x011f03},
+ {0x011f04, 0x011f10},
+ {0x011f12, 0x011f33},
+ {0x011f34, 0x011f35},
+ {0x011f36, 0x011f3a},
+ {0x011f3e, 0x011f3f},
+ {0x011f40, 0x011f40},
+ {0x011fb0, 0x011fb0},
+ {0x012000, 0x012399},
+ {0x012400, 0x01246e},
+ {0x012480, 0x012543},
+ {0x012f90, 0x012ff0},
+ {0x013000, 0x01342f},
+ {0x013441, 0x013446},
+ {0x014400, 0x014646},
+ {0x016800, 0x016a38},
+ {0x016a40, 0x016a5e},
+ {0x016a70, 0x016abe},
+ {0x016ad0, 0x016aed},
+ {0x016b00, 0x016b2f},
+ {0x016b40, 0x016b43},
+ {0x016b63, 0x016b77},
+ {0x016b7d, 0x016b8f},
+ {0x016e40, 0x016e7f},
+ {0x016f00, 0x016f4a},
+ {0x016f4f, 0x016f4f},
+ {0x016f50, 0x016f50},
+ {0x016f51, 0x016f87},
+ {0x016f8f, 0x016f92},
+ {0x016f93, 0x016f9f},
+ {0x016fe0, 0x016fe1},
+ {0x016fe3, 0x016fe3},
+ {0x016ff0, 0x016ff1},
+ {0x017000, 0x0187f7},
+ {0x018800, 0x018cd5},
+ {0x018d00, 0x018d08},
+ {0x01aff0, 0x01aff3},
+ {0x01aff5, 0x01affb},
+ {0x01affd, 0x01affe},
+ {0x01b000, 0x01b122},
+ {0x01b132, 0x01b132},
+ {0x01b150, 0x01b152},
+ {0x01b155, 0x01b155},
+ {0x01b164, 0x01b167},
+ {0x01b170, 0x01b2fb},
+ {0x01bc00, 0x01bc6a},
+ {0x01bc70, 0x01bc7c},
+ {0x01bc80, 0x01bc88},
+ {0x01bc90, 0x01bc99},
+ {0x01bc9e, 0x01bc9e},
+ {0x01d400, 0x01d454},
+ {0x01d456, 0x01d49c},
+ {0x01d49e, 0x01d49f},
+ {0x01d4a2, 0x01d4a2},
+ {0x01d4a5, 0x01d4a6},
+ {0x01d4a9, 0x01d4ac},
+ {0x01d4ae, 0x01d4b9},
+ {0x01d4bb, 0x01d4bb},
+ {0x01d4bd, 0x01d4c3},
+ {0x01d4c5, 0x01d505},
+ {0x01d507, 0x01d50a},
+ {0x01d50d, 0x01d514},
+ {0x01d516, 0x01d51c},
+ {0x01d51e, 0x01d539},
+ {0x01d53b, 0x01d53e},
+ {0x01d540, 0x01d544},
+ {0x01d546, 0x01d546},
+ {0x01d54a, 0x01d550},
+ {0x01d552, 0x01d6a5},
+ {0x01d6a8, 0x01d6c0},
+ {0x01d6c2, 0x01d6da},
+ {0x01d6dc, 0x01d6fa},
+ {0x01d6fc, 0x01d714},
+ {0x01d716, 0x01d734},
+ {0x01d736, 0x01d74e},
+ {0x01d750, 0x01d76e},
+ {0x01d770, 0x01d788},
+ {0x01d78a, 0x01d7a8},
+ {0x01d7aa, 0x01d7c2},
+ {0x01d7c4, 0x01d7cb},
+ {0x01df00, 0x01df09},
+ {0x01df0a, 0x01df0a},
+ {0x01df0b, 0x01df1e},
+ {0x01df25, 0x01df2a},
+ {0x01e000, 0x01e006},
+ {0x01e008, 0x01e018},
+ {0x01e01b, 0x01e021},
+ {0x01e023, 0x01e024},
+ {0x01e026, 0x01e02a},
+ {0x01e030, 0x01e06d},
+ {0x01e08f, 0x01e08f},
+ {0x01e100, 0x01e12c},
+ {0x01e137, 0x01e13d},
+ {0x01e14e, 0x01e14e},
+ {0x01e290, 0x01e2ad},
+ {0x01e2c0, 0x01e2eb},
+ {0x01e4d0, 0x01e4ea},
+ {0x01e4eb, 0x01e4eb},
+ {0x01e7e0, 0x01e7e6},
+ {0x01e7e8, 0x01e7eb},
+ {0x01e7ed, 0x01e7ee},
+ {0x01e7f0, 0x01e7fe},
+ {0x01e800, 0x01e8c4},
+ {0x01e900, 0x01e943},
+ {0x01e947, 0x01e947},
+ {0x01e94b, 0x01e94b},
+ {0x01ee00, 0x01ee03},
+ {0x01ee05, 0x01ee1f},
+ {0x01ee21, 0x01ee22},
+ {0x01ee24, 0x01ee24},
+ {0x01ee27, 0x01ee27},
+ {0x01ee29, 0x01ee32},
+ {0x01ee34, 0x01ee37},
+ {0x01ee39, 0x01ee39},
+ {0x01ee3b, 0x01ee3b},
+ {0x01ee42, 0x01ee42},
+ {0x01ee47, 0x01ee47},
+ {0x01ee49, 0x01ee49},
+ {0x01ee4b, 0x01ee4b},
+ {0x01ee4d, 0x01ee4f},
+ {0x01ee51, 0x01ee52},
+ {0x01ee54, 0x01ee54},
+ {0x01ee57, 0x01ee57},
+ {0x01ee59, 0x01ee59},
+ {0x01ee5b, 0x01ee5b},
+ {0x01ee5d, 0x01ee5d},
+ {0x01ee5f, 0x01ee5f},
+ {0x01ee61, 0x01ee62},
+ {0x01ee64, 0x01ee64},
+ {0x01ee67, 0x01ee6a},
+ {0x01ee6c, 0x01ee72},
+ {0x01ee74, 0x01ee77},
+ {0x01ee79, 0x01ee7c},
+ {0x01ee7e, 0x01ee7e},
+ {0x01ee80, 0x01ee89},
+ {0x01ee8b, 0x01ee9b},
+ {0x01eea1, 0x01eea3},
+ {0x01eea5, 0x01eea9},
+ {0x01eeab, 0x01eebb},
+ {0x01f130, 0x01f149},
+ {0x01f150, 0x01f169},
+ {0x01f170, 0x01f189},
+ {0x020000, 0x02a6df},
+ {0x02a700, 0x02b739},
+ {0x02b740, 0x02b81d},
+ {0x02b820, 0x02cea1},
+ {0x02ceb0, 0x02ebe0},
+ {0x02ebf0, 0x02ee5d},
+ {0x02f800, 0x02fa1d},
+ {0x030000, 0x03134a},
+ {0x031350, 0x0323af}
+};
+
+/* table of Unicode codepoint ranges of Lowercase characters */
+static const pg_unicode_range unicode_lowercase[686] =
+{
+ {0x000061, 0x00007a},
+ {0x0000aa, 0x0000aa},
+ {0x0000b5, 0x0000b5},
+ {0x0000ba, 0x0000ba},
+ {0x0000df, 0x0000f6},
+ {0x0000f8, 0x0000ff},
+ {0x000101, 0x000101},
+ {0x000103, 0x000103},
+ {0x000105, 0x000105},
+ {0x000107, 0x000107},
+ {0x000109, 0x000109},
+ {0x00010b, 0x00010b},
+ {0x00010d, 0x00010d},
+ {0x00010f, 0x00010f},
+ {0x000111, 0x000111},
+ {0x000113, 0x000113},
+ {0x000115, 0x000115},
+ {0x000117, 0x000117},
+ {0x000119, 0x000119},
+ {0x00011b, 0x00011b},
+ {0x00011d, 0x00011d},
+ {0x00011f, 0x00011f},
+ {0x000121, 0x000121},
+ {0x000123, 0x000123},
+ {0x000125, 0x000125},
+ {0x000127, 0x000127},
+ {0x000129, 0x000129},
+ {0x00012b, 0x00012b},
+ {0x00012d, 0x00012d},
+ {0x00012f, 0x00012f},
+ {0x000131, 0x000131},
+ {0x000133, 0x000133},
+ {0x000135, 0x000135},
+ {0x000137, 0x000138},
+ {0x00013a, 0x00013a},
+ {0x00013c, 0x00013c},
+ {0x00013e, 0x00013e},
+ {0x000140, 0x000140},
+ {0x000142, 0x000142},
+ {0x000144, 0x000144},
+ {0x000146, 0x000146},
+ {0x000148, 0x000149},
+ {0x00014b, 0x00014b},
+ {0x00014d, 0x00014d},
+ {0x00014f, 0x00014f},
+ {0x000151, 0x000151},
+ {0x000153, 0x000153},
+ {0x000155, 0x000155},
+ {0x000157, 0x000157},
+ {0x000159, 0x000159},
+ {0x00015b, 0x00015b},
+ {0x00015d, 0x00015d},
+ {0x00015f, 0x00015f},
+ {0x000161, 0x000161},
+ {0x000163, 0x000163},
+ {0x000165, 0x000165},
+ {0x000167, 0x000167},
+ {0x000169, 0x000169},
+ {0x00016b, 0x00016b},
+ {0x00016d, 0x00016d},
+ {0x00016f, 0x00016f},
+ {0x000171, 0x000171},
+ {0x000173, 0x000173},
+ {0x000175, 0x000175},
+ {0x000177, 0x000177},
+ {0x00017a, 0x00017a},
+ {0x00017c, 0x00017c},
+ {0x00017e, 0x000180},
+ {0x000183, 0x000183},
+ {0x000185, 0x000185},
+ {0x000188, 0x000188},
+ {0x00018c, 0x00018d},
+ {0x000192, 0x000192},
+ {0x000195, 0x000195},
+ {0x000199, 0x00019b},
+ {0x00019e, 0x00019e},
+ {0x0001a1, 0x0001a1},
+ {0x0001a3, 0x0001a3},
+ {0x0001a5, 0x0001a5},
+ {0x0001a8, 0x0001a8},
+ {0x0001aa, 0x0001ab},
+ {0x0001ad, 0x0001ad},
+ {0x0001b0, 0x0001b0},
+ {0x0001b4, 0x0001b4},
+ {0x0001b6, 0x0001b6},
+ {0x0001b9, 0x0001ba},
+ {0x0001bd, 0x0001bf},
+ {0x0001c6, 0x0001c6},
+ {0x0001c9, 0x0001c9},
+ {0x0001cc, 0x0001cc},
+ {0x0001ce, 0x0001ce},
+ {0x0001d0, 0x0001d0},
+ {0x0001d2, 0x0001d2},
+ {0x0001d4, 0x0001d4},
+ {0x0001d6, 0x0001d6},
+ {0x0001d8, 0x0001d8},
+ {0x0001da, 0x0001da},
+ {0x0001dc, 0x0001dd},
+ {0x0001df, 0x0001df},
+ {0x0001e1, 0x0001e1},
+ {0x0001e3, 0x0001e3},
+ {0x0001e5, 0x0001e5},
+ {0x0001e7, 0x0001e7},
+ {0x0001e9, 0x0001e9},
+ {0x0001eb, 0x0001eb},
+ {0x0001ed, 0x0001ed},
+ {0x0001ef, 0x0001f0},
+ {0x0001f3, 0x0001f3},
+ {0x0001f5, 0x0001f5},
+ {0x0001f9, 0x0001f9},
+ {0x0001fb, 0x0001fb},
+ {0x0001fd, 0x0001fd},
+ {0x0001ff, 0x0001ff},
+ {0x000201, 0x000201},
+ {0x000203, 0x000203},
+ {0x000205, 0x000205},
+ {0x000207, 0x000207},
+ {0x000209, 0x000209},
+ {0x00020b, 0x00020b},
+ {0x00020d, 0x00020d},
+ {0x00020f, 0x00020f},
+ {0x000211, 0x000211},
+ {0x000213, 0x000213},
+ {0x000215, 0x000215},
+ {0x000217, 0x000217},
+ {0x000219, 0x000219},
+ {0x00021b, 0x00021b},
+ {0x00021d, 0x00021d},
+ {0x00021f, 0x00021f},
+ {0x000221, 0x000221},
+ {0x000223, 0x000223},
+ {0x000225, 0x000225},
+ {0x000227, 0x000227},
+ {0x000229, 0x000229},
+ {0x00022b, 0x00022b},
+ {0x00022d, 0x00022d},
+ {0x00022f, 0x00022f},
+ {0x000231, 0x000231},
+ {0x000233, 0x000239},
+ {0x00023c, 0x00023c},
+ {0x00023f, 0x000240},
+ {0x000242, 0x000242},
+ {0x000247, 0x000247},
+ {0x000249, 0x000249},
+ {0x00024b, 0x00024b},
+ {0x00024d, 0x00024d},
+ {0x00024f, 0x000293},
+ {0x000295, 0x0002af},
+ {0x0002b0, 0x0002b8},
+ {0x0002c0, 0x0002c1},
+ {0x0002e0, 0x0002e4},
+ {0x000345, 0x000345},
+ {0x000371, 0x000371},
+ {0x000373, 0x000373},
+ {0x000377, 0x000377},
+ {0x00037a, 0x00037a},
+ {0x00037b, 0x00037d},
+ {0x000390, 0x000390},
+ {0x0003ac, 0x0003ce},
+ {0x0003d0, 0x0003d1},
+ {0x0003d5, 0x0003d7},
+ {0x0003d9, 0x0003d9},
+ {0x0003db, 0x0003db},
+ {0x0003dd, 0x0003dd},
+ {0x0003df, 0x0003df},
+ {0x0003e1, 0x0003e1},
+ {0x0003e3, 0x0003e3},
+ {0x0003e5, 0x0003e5},
+ {0x0003e7, 0x0003e7},
+ {0x0003e9, 0x0003e9},
+ {0x0003eb, 0x0003eb},
+ {0x0003ed, 0x0003ed},
+ {0x0003ef, 0x0003f3},
+ {0x0003f5, 0x0003f5},
+ {0x0003f8, 0x0003f8},
+ {0x0003fb, 0x0003fc},
+ {0x000430, 0x00045f},
+ {0x000461, 0x000461},
+ {0x000463, 0x000463},
+ {0x000465, 0x000465},
+ {0x000467, 0x000467},
+ {0x000469, 0x000469},
+ {0x00046b, 0x00046b},
+ {0x00046d, 0x00046d},
+ {0x00046f, 0x00046f},
+ {0x000471, 0x000471},
+ {0x000473, 0x000473},
+ {0x000475, 0x000475},
+ {0x000477, 0x000477},
+ {0x000479, 0x000479},
+ {0x00047b, 0x00047b},
+ {0x00047d, 0x00047d},
+ {0x00047f, 0x00047f},
+ {0x000481, 0x000481},
+ {0x00048b, 0x00048b},
+ {0x00048d, 0x00048d},
+ {0x00048f, 0x00048f},
+ {0x000491, 0x000491},
+ {0x000493, 0x000493},
+ {0x000495, 0x000495},
+ {0x000497, 0x000497},
+ {0x000499, 0x000499},
+ {0x00049b, 0x00049b},
+ {0x00049d, 0x00049d},
+ {0x00049f, 0x00049f},
+ {0x0004a1, 0x0004a1},
+ {0x0004a3, 0x0004a3},
+ {0x0004a5, 0x0004a5},
+ {0x0004a7, 0x0004a7},
+ {0x0004a9, 0x0004a9},
+ {0x0004ab, 0x0004ab},
+ {0x0004ad, 0x0004ad},
+ {0x0004af, 0x0004af},
+ {0x0004b1, 0x0004b1},
+ {0x0004b3, 0x0004b3},
+ {0x0004b5, 0x0004b5},
+ {0x0004b7, 0x0004b7},
+ {0x0004b9, 0x0004b9},
+ {0x0004bb, 0x0004bb},
+ {0x0004bd, 0x0004bd},
+ {0x0004bf, 0x0004bf},
+ {0x0004c2, 0x0004c2},
+ {0x0004c4, 0x0004c4},
+ {0x0004c6, 0x0004c6},
+ {0x0004c8, 0x0004c8},
+ {0x0004ca, 0x0004ca},
+ {0x0004cc, 0x0004cc},
+ {0x0004ce, 0x0004cf},
+ {0x0004d1, 0x0004d1},
+ {0x0004d3, 0x0004d3},
+ {0x0004d5, 0x0004d5},
+ {0x0004d7, 0x0004d7},
+ {0x0004d9, 0x0004d9},
+ {0x0004db, 0x0004db},
+ {0x0004dd, 0x0004dd},
+ {0x0004df, 0x0004df},
+ {0x0004e1, 0x0004e1},
+ {0x0004e3, 0x0004e3},
+ {0x0004e5, 0x0004e5},
+ {0x0004e7, 0x0004e7},
+ {0x0004e9, 0x0004e9},
+ {0x0004eb, 0x0004eb},
+ {0x0004ed, 0x0004ed},
+ {0x0004ef, 0x0004ef},
+ {0x0004f1, 0x0004f1},
+ {0x0004f3, 0x0004f3},
+ {0x0004f5, 0x0004f5},
+ {0x0004f7, 0x0004f7},
+ {0x0004f9, 0x0004f9},
+ {0x0004fb, 0x0004fb},
+ {0x0004fd, 0x0004fd},
+ {0x0004ff, 0x0004ff},
+ {0x000501, 0x000501},
+ {0x000503, 0x000503},
+ {0x000505, 0x000505},
+ {0x000507, 0x000507},
+ {0x000509, 0x000509},
+ {0x00050b, 0x00050b},
+ {0x00050d, 0x00050d},
+ {0x00050f, 0x00050f},
+ {0x000511, 0x000511},
+ {0x000513, 0x000513},
+ {0x000515, 0x000515},
+ {0x000517, 0x000517},
+ {0x000519, 0x000519},
+ {0x00051b, 0x00051b},
+ {0x00051d, 0x00051d},
+ {0x00051f, 0x00051f},
+ {0x000521, 0x000521},
+ {0x000523, 0x000523},
+ {0x000525, 0x000525},
+ {0x000527, 0x000527},
+ {0x000529, 0x000529},
+ {0x00052b, 0x00052b},
+ {0x00052d, 0x00052d},
+ {0x00052f, 0x00052f},
+ {0x000560, 0x000588},
+ {0x0010d0, 0x0010fa},
+ {0x0010fc, 0x0010fc},
+ {0x0010fd, 0x0010ff},
+ {0x0013f8, 0x0013fd},
+ {0x001c80, 0x001c88},
+ {0x001d00, 0x001d2b},
+ {0x001d2c, 0x001d6a},
+ {0x001d6b, 0x001d77},
+ {0x001d78, 0x001d78},
+ {0x001d79, 0x001d9a},
+ {0x001d9b, 0x001dbf},
+ {0x001e01, 0x001e01},
+ {0x001e03, 0x001e03},
+ {0x001e05, 0x001e05},
+ {0x001e07, 0x001e07},
+ {0x001e09, 0x001e09},
+ {0x001e0b, 0x001e0b},
+ {0x001e0d, 0x001e0d},
+ {0x001e0f, 0x001e0f},
+ {0x001e11, 0x001e11},
+ {0x001e13, 0x001e13},
+ {0x001e15, 0x001e15},
+ {0x001e17, 0x001e17},
+ {0x001e19, 0x001e19},
+ {0x001e1b, 0x001e1b},
+ {0x001e1d, 0x001e1d},
+ {0x001e1f, 0x001e1f},
+ {0x001e21, 0x001e21},
+ {0x001e23, 0x001e23},
+ {0x001e25, 0x001e25},
+ {0x001e27, 0x001e27},
+ {0x001e29, 0x001e29},
+ {0x001e2b, 0x001e2b},
+ {0x001e2d, 0x001e2d},
+ {0x001e2f, 0x001e2f},
+ {0x001e31, 0x001e31},
+ {0x001e33, 0x001e33},
+ {0x001e35, 0x001e35},
+ {0x001e37, 0x001e37},
+ {0x001e39, 0x001e39},
+ {0x001e3b, 0x001e3b},
+ {0x001e3d, 0x001e3d},
+ {0x001e3f, 0x001e3f},
+ {0x001e41, 0x001e41},
+ {0x001e43, 0x001e43},
+ {0x001e45, 0x001e45},
+ {0x001e47, 0x001e47},
+ {0x001e49, 0x001e49},
+ {0x001e4b, 0x001e4b},
+ {0x001e4d, 0x001e4d},
+ {0x001e4f, 0x001e4f},
+ {0x001e51, 0x001e51},
+ {0x001e53, 0x001e53},
+ {0x001e55, 0x001e55},
+ {0x001e57, 0x001e57},
+ {0x001e59, 0x001e59},
+ {0x001e5b, 0x001e5b},
+ {0x001e5d, 0x001e5d},
+ {0x001e5f, 0x001e5f},
+ {0x001e61, 0x001e61},
+ {0x001e63, 0x001e63},
+ {0x001e65, 0x001e65},
+ {0x001e67, 0x001e67},
+ {0x001e69, 0x001e69},
+ {0x001e6b, 0x001e6b},
+ {0x001e6d, 0x001e6d},
+ {0x001e6f, 0x001e6f},
+ {0x001e71, 0x001e71},
+ {0x001e73, 0x001e73},
+ {0x001e75, 0x001e75},
+ {0x001e77, 0x001e77},
+ {0x001e79, 0x001e79},
+ {0x001e7b, 0x001e7b},
+ {0x001e7d, 0x001e7d},
+ {0x001e7f, 0x001e7f},
+ {0x001e81, 0x001e81},
+ {0x001e83, 0x001e83},
+ {0x001e85, 0x001e85},
+ {0x001e87, 0x001e87},
+ {0x001e89, 0x001e89},
+ {0x001e8b, 0x001e8b},
+ {0x001e8d, 0x001e8d},
+ {0x001e8f, 0x001e8f},
+ {0x001e91, 0x001e91},
+ {0x001e93, 0x001e93},
+ {0x001e95, 0x001e9d},
+ {0x001e9f, 0x001e9f},
+ {0x001ea1, 0x001ea1},
+ {0x001ea3, 0x001ea3},
+ {0x001ea5, 0x001ea5},
+ {0x001ea7, 0x001ea7},
+ {0x001ea9, 0x001ea9},
+ {0x001eab, 0x001eab},
+ {0x001ead, 0x001ead},
+ {0x001eaf, 0x001eaf},
+ {0x001eb1, 0x001eb1},
+ {0x001eb3, 0x001eb3},
+ {0x001eb5, 0x001eb5},
+ {0x001eb7, 0x001eb7},
+ {0x001eb9, 0x001eb9},
+ {0x001ebb, 0x001ebb},
+ {0x001ebd, 0x001ebd},
+ {0x001ebf, 0x001ebf},
+ {0x001ec1, 0x001ec1},
+ {0x001ec3, 0x001ec3},
+ {0x001ec5, 0x001ec5},
+ {0x001ec7, 0x001ec7},
+ {0x001ec9, 0x001ec9},
+ {0x001ecb, 0x001ecb},
+ {0x001ecd, 0x001ecd},
+ {0x001ecf, 0x001ecf},
+ {0x001ed1, 0x001ed1},
+ {0x001ed3, 0x001ed3},
+ {0x001ed5, 0x001ed5},
+ {0x001ed7, 0x001ed7},
+ {0x001ed9, 0x001ed9},
+ {0x001edb, 0x001edb},
+ {0x001edd, 0x001edd},
+ {0x001edf, 0x001edf},
+ {0x001ee1, 0x001ee1},
+ {0x001ee3, 0x001ee3},
+ {0x001ee5, 0x001ee5},
+ {0x001ee7, 0x001ee7},
+ {0x001ee9, 0x001ee9},
+ {0x001eeb, 0x001eeb},
+ {0x001eed, 0x001eed},
+ {0x001eef, 0x001eef},
+ {0x001ef1, 0x001ef1},
+ {0x001ef3, 0x001ef3},
+ {0x001ef5, 0x001ef5},
+ {0x001ef7, 0x001ef7},
+ {0x001ef9, 0x001ef9},
+ {0x001efb, 0x001efb},
+ {0x001efd, 0x001efd},
+ {0x001eff, 0x001f07},
+ {0x001f10, 0x001f15},
+ {0x001f20, 0x001f27},
+ {0x001f30, 0x001f37},
+ {0x001f40, 0x001f45},
+ {0x001f50, 0x001f57},
+ {0x001f60, 0x001f67},
+ {0x001f70, 0x001f7d},
+ {0x001f80, 0x001f87},
+ {0x001f90, 0x001f97},
+ {0x001fa0, 0x001fa7},
+ {0x001fb0, 0x001fb4},
+ {0x001fb6, 0x001fb7},
+ {0x001fbe, 0x001fbe},
+ {0x001fc2, 0x001fc4},
+ {0x001fc6, 0x001fc7},
+ {0x001fd0, 0x001fd3},
+ {0x001fd6, 0x001fd7},
+ {0x001fe0, 0x001fe7},
+ {0x001ff2, 0x001ff4},
+ {0x001ff6, 0x001ff7},
+ {0x002071, 0x002071},
+ {0x00207f, 0x00207f},
+ {0x002090, 0x00209c},
+ {0x00210a, 0x00210a},
+ {0x00210e, 0x00210f},
+ {0x002113, 0x002113},
+ {0x00212f, 0x00212f},
+ {0x002134, 0x002134},
+ {0x002139, 0x002139},
+ {0x00213c, 0x00213d},
+ {0x002146, 0x002149},
+ {0x00214e, 0x00214e},
+ {0x002170, 0x00217f},
+ {0x002184, 0x002184},
+ {0x0024d0, 0x0024e9},
+ {0x002c30, 0x002c5f},
+ {0x002c61, 0x002c61},
+ {0x002c65, 0x002c66},
+ {0x002c68, 0x002c68},
+ {0x002c6a, 0x002c6a},
+ {0x002c6c, 0x002c6c},
+ {0x002c71, 0x002c71},
+ {0x002c73, 0x002c74},
+ {0x002c76, 0x002c7b},
+ {0x002c7c, 0x002c7d},
+ {0x002c81, 0x002c81},
+ {0x002c83, 0x002c83},
+ {0x002c85, 0x002c85},
+ {0x002c87, 0x002c87},
+ {0x002c89, 0x002c89},
+ {0x002c8b, 0x002c8b},
+ {0x002c8d, 0x002c8d},
+ {0x002c8f, 0x002c8f},
+ {0x002c91, 0x002c91},
+ {0x002c93, 0x002c93},
+ {0x002c95, 0x002c95},
+ {0x002c97, 0x002c97},
+ {0x002c99, 0x002c99},
+ {0x002c9b, 0x002c9b},
+ {0x002c9d, 0x002c9d},
+ {0x002c9f, 0x002c9f},
+ {0x002ca1, 0x002ca1},
+ {0x002ca3, 0x002ca3},
+ {0x002ca5, 0x002ca5},
+ {0x002ca7, 0x002ca7},
+ {0x002ca9, 0x002ca9},
+ {0x002cab, 0x002cab},
+ {0x002cad, 0x002cad},
+ {0x002caf, 0x002caf},
+ {0x002cb1, 0x002cb1},
+ {0x002cb3, 0x002cb3},
+ {0x002cb5, 0x002cb5},
+ {0x002cb7, 0x002cb7},
+ {0x002cb9, 0x002cb9},
+ {0x002cbb, 0x002cbb},
+ {0x002cbd, 0x002cbd},
+ {0x002cbf, 0x002cbf},
+ {0x002cc1, 0x002cc1},
+ {0x002cc3, 0x002cc3},
+ {0x002cc5, 0x002cc5},
+ {0x002cc7, 0x002cc7},
+ {0x002cc9, 0x002cc9},
+ {0x002ccb, 0x002ccb},
+ {0x002ccd, 0x002ccd},
+ {0x002ccf, 0x002ccf},
+ {0x002cd1, 0x002cd1},
+ {0x002cd3, 0x002cd3},
+ {0x002cd5, 0x002cd5},
+ {0x002cd7, 0x002cd7},
+ {0x002cd9, 0x002cd9},
+ {0x002cdb, 0x002cdb},
+ {0x002cdd, 0x002cdd},
+ {0x002cdf, 0x002cdf},
+ {0x002ce1, 0x002ce1},
+ {0x002ce3, 0x002ce4},
+ {0x002cec, 0x002cec},
+ {0x002cee, 0x002cee},
+ {0x002cf3, 0x002cf3},
+ {0x002d00, 0x002d25},
+ {0x002d27, 0x002d27},
+ {0x002d2d, 0x002d2d},
+ {0x00a641, 0x00a641},
+ {0x00a643, 0x00a643},
+ {0x00a645, 0x00a645},
+ {0x00a647, 0x00a647},
+ {0x00a649, 0x00a649},
+ {0x00a64b, 0x00a64b},
+ {0x00a64d, 0x00a64d},
+ {0x00a64f, 0x00a64f},
+ {0x00a651, 0x00a651},
+ {0x00a653, 0x00a653},
+ {0x00a655, 0x00a655},
+ {0x00a657, 0x00a657},
+ {0x00a659, 0x00a659},
+ {0x00a65b, 0x00a65b},
+ {0x00a65d, 0x00a65d},
+ {0x00a65f, 0x00a65f},
+ {0x00a661, 0x00a661},
+ {0x00a663, 0x00a663},
+ {0x00a665, 0x00a665},
+ {0x00a667, 0x00a667},
+ {0x00a669, 0x00a669},
+ {0x00a66b, 0x00a66b},
+ {0x00a66d, 0x00a66d},
+ {0x00a681, 0x00a681},
+ {0x00a683, 0x00a683},
+ {0x00a685, 0x00a685},
+ {0x00a687, 0x00a687},
+ {0x00a689, 0x00a689},
+ {0x00a68b, 0x00a68b},
+ {0x00a68d, 0x00a68d},
+ {0x00a68f, 0x00a68f},
+ {0x00a691, 0x00a691},
+ {0x00a693, 0x00a693},
+ {0x00a695, 0x00a695},
+ {0x00a697, 0x00a697},
+ {0x00a699, 0x00a699},
+ {0x00a69b, 0x00a69b},
+ {0x00a69c, 0x00a69d},
+ {0x00a723, 0x00a723},
+ {0x00a725, 0x00a725},
+ {0x00a727, 0x00a727},
+ {0x00a729, 0x00a729},
+ {0x00a72b, 0x00a72b},
+ {0x00a72d, 0x00a72d},
+ {0x00a72f, 0x00a731},
+ {0x00a733, 0x00a733},
+ {0x00a735, 0x00a735},
+ {0x00a737, 0x00a737},
+ {0x00a739, 0x00a739},
+ {0x00a73b, 0x00a73b},
+ {0x00a73d, 0x00a73d},
+ {0x00a73f, 0x00a73f},
+ {0x00a741, 0x00a741},
+ {0x00a743, 0x00a743},
+ {0x00a745, 0x00a745},
+ {0x00a747, 0x00a747},
+ {0x00a749, 0x00a749},
+ {0x00a74b, 0x00a74b},
+ {0x00a74d, 0x00a74d},
+ {0x00a74f, 0x00a74f},
+ {0x00a751, 0x00a751},
+ {0x00a753, 0x00a753},
+ {0x00a755, 0x00a755},
+ {0x00a757, 0x00a757},
+ {0x00a759, 0x00a759},
+ {0x00a75b, 0x00a75b},
+ {0x00a75d, 0x00a75d},
+ {0x00a75f, 0x00a75f},
+ {0x00a761, 0x00a761},
+ {0x00a763, 0x00a763},
+ {0x00a765, 0x00a765},
+ {0x00a767, 0x00a767},
+ {0x00a769, 0x00a769},
+ {0x00a76b, 0x00a76b},
+ {0x00a76d, 0x00a76d},
+ {0x00a76f, 0x00a76f},
+ {0x00a770, 0x00a770},
+ {0x00a771, 0x00a778},
+ {0x00a77a, 0x00a77a},
+ {0x00a77c, 0x00a77c},
+ {0x00a77f, 0x00a77f},
+ {0x00a781, 0x00a781},
+ {0x00a783, 0x00a783},
+ {0x00a785, 0x00a785},
+ {0x00a787, 0x00a787},
+ {0x00a78c, 0x00a78c},
+ {0x00a78e, 0x00a78e},
+ {0x00a791, 0x00a791},
+ {0x00a793, 0x00a795},
+ {0x00a797, 0x00a797},
+ {0x00a799, 0x00a799},
+ {0x00a79b, 0x00a79b},
+ {0x00a79d, 0x00a79d},
+ {0x00a79f, 0x00a79f},
+ {0x00a7a1, 0x00a7a1},
+ {0x00a7a3, 0x00a7a3},
+ {0x00a7a5, 0x00a7a5},
+ {0x00a7a7, 0x00a7a7},
+ {0x00a7a9, 0x00a7a9},
+ {0x00a7af, 0x00a7af},
+ {0x00a7b5, 0x00a7b5},
+ {0x00a7b7, 0x00a7b7},
+ {0x00a7b9, 0x00a7b9},
+ {0x00a7bb, 0x00a7bb},
+ {0x00a7bd, 0x00a7bd},
+ {0x00a7bf, 0x00a7bf},
+ {0x00a7c1, 0x00a7c1},
+ {0x00a7c3, 0x00a7c3},
+ {0x00a7c8, 0x00a7c8},
+ {0x00a7ca, 0x00a7ca},
+ {0x00a7d1, 0x00a7d1},
+ {0x00a7d3, 0x00a7d3},
+ {0x00a7d5, 0x00a7d5},
+ {0x00a7d7, 0x00a7d7},
+ {0x00a7d9, 0x00a7d9},
+ {0x00a7f2, 0x00a7f4},
+ {0x00a7f6, 0x00a7f6},
+ {0x00a7f8, 0x00a7f9},
+ {0x00a7fa, 0x00a7fa},
+ {0x00ab30, 0x00ab5a},
+ {0x00ab5c, 0x00ab5f},
+ {0x00ab60, 0x00ab68},
+ {0x00ab69, 0x00ab69},
+ {0x00ab70, 0x00abbf},
+ {0x00fb00, 0x00fb06},
+ {0x00fb13, 0x00fb17},
+ {0x00ff41, 0x00ff5a},
+ {0x010428, 0x01044f},
+ {0x0104d8, 0x0104fb},
+ {0x010597, 0x0105a1},
+ {0x0105a3, 0x0105b1},
+ {0x0105b3, 0x0105b9},
+ {0x0105bb, 0x0105bc},
+ {0x010780, 0x010780},
+ {0x010783, 0x010785},
+ {0x010787, 0x0107b0},
+ {0x0107b2, 0x0107ba},
+ {0x010cc0, 0x010cf2},
+ {0x0118c0, 0x0118df},
+ {0x016e60, 0x016e7f},
+ {0x01d41a, 0x01d433},
+ {0x01d44e, 0x01d454},
+ {0x01d456, 0x01d467},
+ {0x01d482, 0x01d49b},
+ {0x01d4b6, 0x01d4b9},
+ {0x01d4bb, 0x01d4bb},
+ {0x01d4bd, 0x01d4c3},
+ {0x01d4c5, 0x01d4cf},
+ {0x01d4ea, 0x01d503},
+ {0x01d51e, 0x01d537},
+ {0x01d552, 0x01d56b},
+ {0x01d586, 0x01d59f},
+ {0x01d5ba, 0x01d5d3},
+ {0x01d5ee, 0x01d607},
+ {0x01d622, 0x01d63b},
+ {0x01d656, 0x01d66f},
+ {0x01d68a, 0x01d6a5},
+ {0x01d6c2, 0x01d6da},
+ {0x01d6dc, 0x01d6e1},
+ {0x01d6fc, 0x01d714},
+ {0x01d716, 0x01d71b},
+ {0x01d736, 0x01d74e},
+ {0x01d750, 0x01d755},
+ {0x01d770, 0x01d788},
+ {0x01d78a, 0x01d78f},
+ {0x01d7aa, 0x01d7c2},
+ {0x01d7c4, 0x01d7c9},
+ {0x01d7cb, 0x01d7cb},
+ {0x01df00, 0x01df09},
+ {0x01df0b, 0x01df1e},
+ {0x01df25, 0x01df2a},
+ {0x01e030, 0x01e06d},
+ {0x01e922, 0x01e943}
+};
+
+/* table of Unicode codepoint ranges of Uppercase characters */
+static const pg_unicode_range unicode_uppercase[651] =
+{
+ {0x000041, 0x00005a},
+ {0x0000c0, 0x0000d6},
+ {0x0000d8, 0x0000de},
+ {0x000100, 0x000100},
+ {0x000102, 0x000102},
+ {0x000104, 0x000104},
+ {0x000106, 0x000106},
+ {0x000108, 0x000108},
+ {0x00010a, 0x00010a},
+ {0x00010c, 0x00010c},
+ {0x00010e, 0x00010e},
+ {0x000110, 0x000110},
+ {0x000112, 0x000112},
+ {0x000114, 0x000114},
+ {0x000116, 0x000116},
+ {0x000118, 0x000118},
+ {0x00011a, 0x00011a},
+ {0x00011c, 0x00011c},
+ {0x00011e, 0x00011e},
+ {0x000120, 0x000120},
+ {0x000122, 0x000122},
+ {0x000124, 0x000124},
+ {0x000126, 0x000126},
+ {0x000128, 0x000128},
+ {0x00012a, 0x00012a},
+ {0x00012c, 0x00012c},
+ {0x00012e, 0x00012e},
+ {0x000130, 0x000130},
+ {0x000132, 0x000132},
+ {0x000134, 0x000134},
+ {0x000136, 0x000136},
+ {0x000139, 0x000139},
+ {0x00013b, 0x00013b},
+ {0x00013d, 0x00013d},
+ {0x00013f, 0x00013f},
+ {0x000141, 0x000141},
+ {0x000143, 0x000143},
+ {0x000145, 0x000145},
+ {0x000147, 0x000147},
+ {0x00014a, 0x00014a},
+ {0x00014c, 0x00014c},
+ {0x00014e, 0x00014e},
+ {0x000150, 0x000150},
+ {0x000152, 0x000152},
+ {0x000154, 0x000154},
+ {0x000156, 0x000156},
+ {0x000158, 0x000158},
+ {0x00015a, 0x00015a},
+ {0x00015c, 0x00015c},
+ {0x00015e, 0x00015e},
+ {0x000160, 0x000160},
+ {0x000162, 0x000162},
+ {0x000164, 0x000164},
+ {0x000166, 0x000166},
+ {0x000168, 0x000168},
+ {0x00016a, 0x00016a},
+ {0x00016c, 0x00016c},
+ {0x00016e, 0x00016e},
+ {0x000170, 0x000170},
+ {0x000172, 0x000172},
+ {0x000174, 0x000174},
+ {0x000176, 0x000176},
+ {0x000178, 0x000179},
+ {0x00017b, 0x00017b},
+ {0x00017d, 0x00017d},
+ {0x000181, 0x000182},
+ {0x000184, 0x000184},
+ {0x000186, 0x000187},
+ {0x000189, 0x00018b},
+ {0x00018e, 0x000191},
+ {0x000193, 0x000194},
+ {0x000196, 0x000198},
+ {0x00019c, 0x00019d},
+ {0x00019f, 0x0001a0},
+ {0x0001a2, 0x0001a2},
+ {0x0001a4, 0x0001a4},
+ {0x0001a6, 0x0001a7},
+ {0x0001a9, 0x0001a9},
+ {0x0001ac, 0x0001ac},
+ {0x0001ae, 0x0001af},
+ {0x0001b1, 0x0001b3},
+ {0x0001b5, 0x0001b5},
+ {0x0001b7, 0x0001b8},
+ {0x0001bc, 0x0001bc},
+ {0x0001c4, 0x0001c4},
+ {0x0001c7, 0x0001c7},
+ {0x0001ca, 0x0001ca},
+ {0x0001cd, 0x0001cd},
+ {0x0001cf, 0x0001cf},
+ {0x0001d1, 0x0001d1},
+ {0x0001d3, 0x0001d3},
+ {0x0001d5, 0x0001d5},
+ {0x0001d7, 0x0001d7},
+ {0x0001d9, 0x0001d9},
+ {0x0001db, 0x0001db},
+ {0x0001de, 0x0001de},
+ {0x0001e0, 0x0001e0},
+ {0x0001e2, 0x0001e2},
+ {0x0001e4, 0x0001e4},
+ {0x0001e6, 0x0001e6},
+ {0x0001e8, 0x0001e8},
+ {0x0001ea, 0x0001ea},
+ {0x0001ec, 0x0001ec},
+ {0x0001ee, 0x0001ee},
+ {0x0001f1, 0x0001f1},
+ {0x0001f4, 0x0001f4},
+ {0x0001f6, 0x0001f8},
+ {0x0001fa, 0x0001fa},
+ {0x0001fc, 0x0001fc},
+ {0x0001fe, 0x0001fe},
+ {0x000200, 0x000200},
+ {0x000202, 0x000202},
+ {0x000204, 0x000204},
+ {0x000206, 0x000206},
+ {0x000208, 0x000208},
+ {0x00020a, 0x00020a},
+ {0x00020c, 0x00020c},
+ {0x00020e, 0x00020e},
+ {0x000210, 0x000210},
+ {0x000212, 0x000212},
+ {0x000214, 0x000214},
+ {0x000216, 0x000216},
+ {0x000218, 0x000218},
+ {0x00021a, 0x00021a},
+ {0x00021c, 0x00021c},
+ {0x00021e, 0x00021e},
+ {0x000220, 0x000220},
+ {0x000222, 0x000222},
+ {0x000224, 0x000224},
+ {0x000226, 0x000226},
+ {0x000228, 0x000228},
+ {0x00022a, 0x00022a},
+ {0x00022c, 0x00022c},
+ {0x00022e, 0x00022e},
+ {0x000230, 0x000230},
+ {0x000232, 0x000232},
+ {0x00023a, 0x00023b},
+ {0x00023d, 0x00023e},
+ {0x000241, 0x000241},
+ {0x000243, 0x000246},
+ {0x000248, 0x000248},
+ {0x00024a, 0x00024a},
+ {0x00024c, 0x00024c},
+ {0x00024e, 0x00024e},
+ {0x000370, 0x000370},
+ {0x000372, 0x000372},
+ {0x000376, 0x000376},
+ {0x00037f, 0x00037f},
+ {0x000386, 0x000386},
+ {0x000388, 0x00038a},
+ {0x00038c, 0x00038c},
+ {0x00038e, 0x00038f},
+ {0x000391, 0x0003a1},
+ {0x0003a3, 0x0003ab},
+ {0x0003cf, 0x0003cf},
+ {0x0003d2, 0x0003d4},
+ {0x0003d8, 0x0003d8},
+ {0x0003da, 0x0003da},
+ {0x0003dc, 0x0003dc},
+ {0x0003de, 0x0003de},
+ {0x0003e0, 0x0003e0},
+ {0x0003e2, 0x0003e2},
+ {0x0003e4, 0x0003e4},
+ {0x0003e6, 0x0003e6},
+ {0x0003e8, 0x0003e8},
+ {0x0003ea, 0x0003ea},
+ {0x0003ec, 0x0003ec},
+ {0x0003ee, 0x0003ee},
+ {0x0003f4, 0x0003f4},
+ {0x0003f7, 0x0003f7},
+ {0x0003f9, 0x0003fa},
+ {0x0003fd, 0x00042f},
+ {0x000460, 0x000460},
+ {0x000462, 0x000462},
+ {0x000464, 0x000464},
+ {0x000466, 0x000466},
+ {0x000468, 0x000468},
+ {0x00046a, 0x00046a},
+ {0x00046c, 0x00046c},
+ {0x00046e, 0x00046e},
+ {0x000470, 0x000470},
+ {0x000472, 0x000472},
+ {0x000474, 0x000474},
+ {0x000476, 0x000476},
+ {0x000478, 0x000478},
+ {0x00047a, 0x00047a},
+ {0x00047c, 0x00047c},
+ {0x00047e, 0x00047e},
+ {0x000480, 0x000480},
+ {0x00048a, 0x00048a},
+ {0x00048c, 0x00048c},
+ {0x00048e, 0x00048e},
+ {0x000490, 0x000490},
+ {0x000492, 0x000492},
+ {0x000494, 0x000494},
+ {0x000496, 0x000496},
+ {0x000498, 0x000498},
+ {0x00049a, 0x00049a},
+ {0x00049c, 0x00049c},
+ {0x00049e, 0x00049e},
+ {0x0004a0, 0x0004a0},
+ {0x0004a2, 0x0004a2},
+ {0x0004a4, 0x0004a4},
+ {0x0004a6, 0x0004a6},
+ {0x0004a8, 0x0004a8},
+ {0x0004aa, 0x0004aa},
+ {0x0004ac, 0x0004ac},
+ {0x0004ae, 0x0004ae},
+ {0x0004b0, 0x0004b0},
+ {0x0004b2, 0x0004b2},
+ {0x0004b4, 0x0004b4},
+ {0x0004b6, 0x0004b6},
+ {0x0004b8, 0x0004b8},
+ {0x0004ba, 0x0004ba},
+ {0x0004bc, 0x0004bc},
+ {0x0004be, 0x0004be},
+ {0x0004c0, 0x0004c1},
+ {0x0004c3, 0x0004c3},
+ {0x0004c5, 0x0004c5},
+ {0x0004c7, 0x0004c7},
+ {0x0004c9, 0x0004c9},
+ {0x0004cb, 0x0004cb},
+ {0x0004cd, 0x0004cd},
+ {0x0004d0, 0x0004d0},
+ {0x0004d2, 0x0004d2},
+ {0x0004d4, 0x0004d4},
+ {0x0004d6, 0x0004d6},
+ {0x0004d8, 0x0004d8},
+ {0x0004da, 0x0004da},
+ {0x0004dc, 0x0004dc},
+ {0x0004de, 0x0004de},
+ {0x0004e0, 0x0004e0},
+ {0x0004e2, 0x0004e2},
+ {0x0004e4, 0x0004e4},
+ {0x0004e6, 0x0004e6},
+ {0x0004e8, 0x0004e8},
+ {0x0004ea, 0x0004ea},
+ {0x0004ec, 0x0004ec},
+ {0x0004ee, 0x0004ee},
+ {0x0004f0, 0x0004f0},
+ {0x0004f2, 0x0004f2},
+ {0x0004f4, 0x0004f4},
+ {0x0004f6, 0x0004f6},
+ {0x0004f8, 0x0004f8},
+ {0x0004fa, 0x0004fa},
+ {0x0004fc, 0x0004fc},
+ {0x0004fe, 0x0004fe},
+ {0x000500, 0x000500},
+ {0x000502, 0x000502},
+ {0x000504, 0x000504},
+ {0x000506, 0x000506},
+ {0x000508, 0x000508},
+ {0x00050a, 0x00050a},
+ {0x00050c, 0x00050c},
+ {0x00050e, 0x00050e},
+ {0x000510, 0x000510},
+ {0x000512, 0x000512},
+ {0x000514, 0x000514},
+ {0x000516, 0x000516},
+ {0x000518, 0x000518},
+ {0x00051a, 0x00051a},
+ {0x00051c, 0x00051c},
+ {0x00051e, 0x00051e},
+ {0x000520, 0x000520},
+ {0x000522, 0x000522},
+ {0x000524, 0x000524},
+ {0x000526, 0x000526},
+ {0x000528, 0x000528},
+ {0x00052a, 0x00052a},
+ {0x00052c, 0x00052c},
+ {0x00052e, 0x00052e},
+ {0x000531, 0x000556},
+ {0x0010a0, 0x0010c5},
+ {0x0010c7, 0x0010c7},
+ {0x0010cd, 0x0010cd},
+ {0x0013a0, 0x0013f5},
+ {0x001c90, 0x001cba},
+ {0x001cbd, 0x001cbf},
+ {0x001e00, 0x001e00},
+ {0x001e02, 0x001e02},
+ {0x001e04, 0x001e04},
+ {0x001e06, 0x001e06},
+ {0x001e08, 0x001e08},
+ {0x001e0a, 0x001e0a},
+ {0x001e0c, 0x001e0c},
+ {0x001e0e, 0x001e0e},
+ {0x001e10, 0x001e10},
+ {0x001e12, 0x001e12},
+ {0x001e14, 0x001e14},
+ {0x001e16, 0x001e16},
+ {0x001e18, 0x001e18},
+ {0x001e1a, 0x001e1a},
+ {0x001e1c, 0x001e1c},
+ {0x001e1e, 0x001e1e},
+ {0x001e20, 0x001e20},
+ {0x001e22, 0x001e22},
+ {0x001e24, 0x001e24},
+ {0x001e26, 0x001e26},
+ {0x001e28, 0x001e28},
+ {0x001e2a, 0x001e2a},
+ {0x001e2c, 0x001e2c},
+ {0x001e2e, 0x001e2e},
+ {0x001e30, 0x001e30},
+ {0x001e32, 0x001e32},
+ {0x001e34, 0x001e34},
+ {0x001e36, 0x001e36},
+ {0x001e38, 0x001e38},
+ {0x001e3a, 0x001e3a},
+ {0x001e3c, 0x001e3c},
+ {0x001e3e, 0x001e3e},
+ {0x001e40, 0x001e40},
+ {0x001e42, 0x001e42},
+ {0x001e44, 0x001e44},
+ {0x001e46, 0x001e46},
+ {0x001e48, 0x001e48},
+ {0x001e4a, 0x001e4a},
+ {0x001e4c, 0x001e4c},
+ {0x001e4e, 0x001e4e},
+ {0x001e50, 0x001e50},
+ {0x001e52, 0x001e52},
+ {0x001e54, 0x001e54},
+ {0x001e56, 0x001e56},
+ {0x001e58, 0x001e58},
+ {0x001e5a, 0x001e5a},
+ {0x001e5c, 0x001e5c},
+ {0x001e5e, 0x001e5e},
+ {0x001e60, 0x001e60},
+ {0x001e62, 0x001e62},
+ {0x001e64, 0x001e64},
+ {0x001e66, 0x001e66},
+ {0x001e68, 0x001e68},
+ {0x001e6a, 0x001e6a},
+ {0x001e6c, 0x001e6c},
+ {0x001e6e, 0x001e6e},
+ {0x001e70, 0x001e70},
+ {0x001e72, 0x001e72},
+ {0x001e74, 0x001e74},
+ {0x001e76, 0x001e76},
+ {0x001e78, 0x001e78},
+ {0x001e7a, 0x001e7a},
+ {0x001e7c, 0x001e7c},
+ {0x001e7e, 0x001e7e},
+ {0x001e80, 0x001e80},
+ {0x001e82, 0x001e82},
+ {0x001e84, 0x001e84},
+ {0x001e86, 0x001e86},
+ {0x001e88, 0x001e88},
+ {0x001e8a, 0x001e8a},
+ {0x001e8c, 0x001e8c},
+ {0x001e8e, 0x001e8e},
+ {0x001e90, 0x001e90},
+ {0x001e92, 0x001e92},
+ {0x001e94, 0x001e94},
+ {0x001e9e, 0x001e9e},
+ {0x001ea0, 0x001ea0},
+ {0x001ea2, 0x001ea2},
+ {0x001ea4, 0x001ea4},
+ {0x001ea6, 0x001ea6},
+ {0x001ea8, 0x001ea8},
+ {0x001eaa, 0x001eaa},
+ {0x001eac, 0x001eac},
+ {0x001eae, 0x001eae},
+ {0x001eb0, 0x001eb0},
+ {0x001eb2, 0x001eb2},
+ {0x001eb4, 0x001eb4},
+ {0x001eb6, 0x001eb6},
+ {0x001eb8, 0x001eb8},
+ {0x001eba, 0x001eba},
+ {0x001ebc, 0x001ebc},
+ {0x001ebe, 0x001ebe},
+ {0x001ec0, 0x001ec0},
+ {0x001ec2, 0x001ec2},
+ {0x001ec4, 0x001ec4},
+ {0x001ec6, 0x001ec6},
+ {0x001ec8, 0x001ec8},
+ {0x001eca, 0x001eca},
+ {0x001ecc, 0x001ecc},
+ {0x001ece, 0x001ece},
+ {0x001ed0, 0x001ed0},
+ {0x001ed2, 0x001ed2},
+ {0x001ed4, 0x001ed4},
+ {0x001ed6, 0x001ed6},
+ {0x001ed8, 0x001ed8},
+ {0x001eda, 0x001eda},
+ {0x001edc, 0x001edc},
+ {0x001ede, 0x001ede},
+ {0x001ee0, 0x001ee0},
+ {0x001ee2, 0x001ee2},
+ {0x001ee4, 0x001ee4},
+ {0x001ee6, 0x001ee6},
+ {0x001ee8, 0x001ee8},
+ {0x001eea, 0x001eea},
+ {0x001eec, 0x001eec},
+ {0x001eee, 0x001eee},
+ {0x001ef0, 0x001ef0},
+ {0x001ef2, 0x001ef2},
+ {0x001ef4, 0x001ef4},
+ {0x001ef6, 0x001ef6},
+ {0x001ef8, 0x001ef8},
+ {0x001efa, 0x001efa},
+ {0x001efc, 0x001efc},
+ {0x001efe, 0x001efe},
+ {0x001f08, 0x001f0f},
+ {0x001f18, 0x001f1d},
+ {0x001f28, 0x001f2f},
+ {0x001f38, 0x001f3f},
+ {0x001f48, 0x001f4d},
+ {0x001f59, 0x001f59},
+ {0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f5f},
+ {0x001f68, 0x001f6f},
+ {0x001fb8, 0x001fbb},
+ {0x001fc8, 0x001fcb},
+ {0x001fd8, 0x001fdb},
+ {0x001fe8, 0x001fec},
+ {0x001ff8, 0x001ffb},
+ {0x002102, 0x002102},
+ {0x002107, 0x002107},
+ {0x00210b, 0x00210d},
+ {0x002110, 0x002112},
+ {0x002115, 0x002115},
+ {0x002119, 0x00211d},
+ {0x002124, 0x002124},
+ {0x002126, 0x002126},
+ {0x002128, 0x002128},
+ {0x00212a, 0x00212d},
+ {0x002130, 0x002133},
+ {0x00213e, 0x00213f},
+ {0x002145, 0x002145},
+ {0x002160, 0x00216f},
+ {0x002183, 0x002183},
+ {0x0024b6, 0x0024cf},
+ {0x002c00, 0x002c2f},
+ {0x002c60, 0x002c60},
+ {0x002c62, 0x002c64},
+ {0x002c67, 0x002c67},
+ {0x002c69, 0x002c69},
+ {0x002c6b, 0x002c6b},
+ {0x002c6d, 0x002c70},
+ {0x002c72, 0x002c72},
+ {0x002c75, 0x002c75},
+ {0x002c7e, 0x002c80},
+ {0x002c82, 0x002c82},
+ {0x002c84, 0x002c84},
+ {0x002c86, 0x002c86},
+ {0x002c88, 0x002c88},
+ {0x002c8a, 0x002c8a},
+ {0x002c8c, 0x002c8c},
+ {0x002c8e, 0x002c8e},
+ {0x002c90, 0x002c90},
+ {0x002c92, 0x002c92},
+ {0x002c94, 0x002c94},
+ {0x002c96, 0x002c96},
+ {0x002c98, 0x002c98},
+ {0x002c9a, 0x002c9a},
+ {0x002c9c, 0x002c9c},
+ {0x002c9e, 0x002c9e},
+ {0x002ca0, 0x002ca0},
+ {0x002ca2, 0x002ca2},
+ {0x002ca4, 0x002ca4},
+ {0x002ca6, 0x002ca6},
+ {0x002ca8, 0x002ca8},
+ {0x002caa, 0x002caa},
+ {0x002cac, 0x002cac},
+ {0x002cae, 0x002cae},
+ {0x002cb0, 0x002cb0},
+ {0x002cb2, 0x002cb2},
+ {0x002cb4, 0x002cb4},
+ {0x002cb6, 0x002cb6},
+ {0x002cb8, 0x002cb8},
+ {0x002cba, 0x002cba},
+ {0x002cbc, 0x002cbc},
+ {0x002cbe, 0x002cbe},
+ {0x002cc0, 0x002cc0},
+ {0x002cc2, 0x002cc2},
+ {0x002cc4, 0x002cc4},
+ {0x002cc6, 0x002cc6},
+ {0x002cc8, 0x002cc8},
+ {0x002cca, 0x002cca},
+ {0x002ccc, 0x002ccc},
+ {0x002cce, 0x002cce},
+ {0x002cd0, 0x002cd0},
+ {0x002cd2, 0x002cd2},
+ {0x002cd4, 0x002cd4},
+ {0x002cd6, 0x002cd6},
+ {0x002cd8, 0x002cd8},
+ {0x002cda, 0x002cda},
+ {0x002cdc, 0x002cdc},
+ {0x002cde, 0x002cde},
+ {0x002ce0, 0x002ce0},
+ {0x002ce2, 0x002ce2},
+ {0x002ceb, 0x002ceb},
+ {0x002ced, 0x002ced},
+ {0x002cf2, 0x002cf2},
+ {0x00a640, 0x00a640},
+ {0x00a642, 0x00a642},
+ {0x00a644, 0x00a644},
+ {0x00a646, 0x00a646},
+ {0x00a648, 0x00a648},
+ {0x00a64a, 0x00a64a},
+ {0x00a64c, 0x00a64c},
+ {0x00a64e, 0x00a64e},
+ {0x00a650, 0x00a650},
+ {0x00a652, 0x00a652},
+ {0x00a654, 0x00a654},
+ {0x00a656, 0x00a656},
+ {0x00a658, 0x00a658},
+ {0x00a65a, 0x00a65a},
+ {0x00a65c, 0x00a65c},
+ {0x00a65e, 0x00a65e},
+ {0x00a660, 0x00a660},
+ {0x00a662, 0x00a662},
+ {0x00a664, 0x00a664},
+ {0x00a666, 0x00a666},
+ {0x00a668, 0x00a668},
+ {0x00a66a, 0x00a66a},
+ {0x00a66c, 0x00a66c},
+ {0x00a680, 0x00a680},
+ {0x00a682, 0x00a682},
+ {0x00a684, 0x00a684},
+ {0x00a686, 0x00a686},
+ {0x00a688, 0x00a688},
+ {0x00a68a, 0x00a68a},
+ {0x00a68c, 0x00a68c},
+ {0x00a68e, 0x00a68e},
+ {0x00a690, 0x00a690},
+ {0x00a692, 0x00a692},
+ {0x00a694, 0x00a694},
+ {0x00a696, 0x00a696},
+ {0x00a698, 0x00a698},
+ {0x00a69a, 0x00a69a},
+ {0x00a722, 0x00a722},
+ {0x00a724, 0x00a724},
+ {0x00a726, 0x00a726},
+ {0x00a728, 0x00a728},
+ {0x00a72a, 0x00a72a},
+ {0x00a72c, 0x00a72c},
+ {0x00a72e, 0x00a72e},
+ {0x00a732, 0x00a732},
+ {0x00a734, 0x00a734},
+ {0x00a736, 0x00a736},
+ {0x00a738, 0x00a738},
+ {0x00a73a, 0x00a73a},
+ {0x00a73c, 0x00a73c},
+ {0x00a73e, 0x00a73e},
+ {0x00a740, 0x00a740},
+ {0x00a742, 0x00a742},
+ {0x00a744, 0x00a744},
+ {0x00a746, 0x00a746},
+ {0x00a748, 0x00a748},
+ {0x00a74a, 0x00a74a},
+ {0x00a74c, 0x00a74c},
+ {0x00a74e, 0x00a74e},
+ {0x00a750, 0x00a750},
+ {0x00a752, 0x00a752},
+ {0x00a754, 0x00a754},
+ {0x00a756, 0x00a756},
+ {0x00a758, 0x00a758},
+ {0x00a75a, 0x00a75a},
+ {0x00a75c, 0x00a75c},
+ {0x00a75e, 0x00a75e},
+ {0x00a760, 0x00a760},
+ {0x00a762, 0x00a762},
+ {0x00a764, 0x00a764},
+ {0x00a766, 0x00a766},
+ {0x00a768, 0x00a768},
+ {0x00a76a, 0x00a76a},
+ {0x00a76c, 0x00a76c},
+ {0x00a76e, 0x00a76e},
+ {0x00a779, 0x00a779},
+ {0x00a77b, 0x00a77b},
+ {0x00a77d, 0x00a77e},
+ {0x00a780, 0x00a780},
+ {0x00a782, 0x00a782},
+ {0x00a784, 0x00a784},
+ {0x00a786, 0x00a786},
+ {0x00a78b, 0x00a78b},
+ {0x00a78d, 0x00a78d},
+ {0x00a790, 0x00a790},
+ {0x00a792, 0x00a792},
+ {0x00a796, 0x00a796},
+ {0x00a798, 0x00a798},
+ {0x00a79a, 0x00a79a},
+ {0x00a79c, 0x00a79c},
+ {0x00a79e, 0x00a79e},
+ {0x00a7a0, 0x00a7a0},
+ {0x00a7a2, 0x00a7a2},
+ {0x00a7a4, 0x00a7a4},
+ {0x00a7a6, 0x00a7a6},
+ {0x00a7a8, 0x00a7a8},
+ {0x00a7aa, 0x00a7ae},
+ {0x00a7b0, 0x00a7b4},
+ {0x00a7b6, 0x00a7b6},
+ {0x00a7b8, 0x00a7b8},
+ {0x00a7ba, 0x00a7ba},
+ {0x00a7bc, 0x00a7bc},
+ {0x00a7be, 0x00a7be},
+ {0x00a7c0, 0x00a7c0},
+ {0x00a7c2, 0x00a7c2},
+ {0x00a7c4, 0x00a7c7},
+ {0x00a7c9, 0x00a7c9},
+ {0x00a7d0, 0x00a7d0},
+ {0x00a7d6, 0x00a7d6},
+ {0x00a7d8, 0x00a7d8},
+ {0x00a7f5, 0x00a7f5},
+ {0x00ff21, 0x00ff3a},
+ {0x010400, 0x010427},
+ {0x0104b0, 0x0104d3},
+ {0x010570, 0x01057a},
+ {0x01057c, 0x01058a},
+ {0x01058c, 0x010592},
+ {0x010594, 0x010595},
+ {0x010c80, 0x010cb2},
+ {0x0118a0, 0x0118bf},
+ {0x016e40, 0x016e5f},
+ {0x01d400, 0x01d419},
+ {0x01d434, 0x01d44d},
+ {0x01d468, 0x01d481},
+ {0x01d49c, 0x01d49c},
+ {0x01d49e, 0x01d49f},
+ {0x01d4a2, 0x01d4a2},
+ {0x01d4a5, 0x01d4a6},
+ {0x01d4a9, 0x01d4ac},
+ {0x01d4ae, 0x01d4b5},
+ {0x01d4d0, 0x01d4e9},
+ {0x01d504, 0x01d505},
+ {0x01d507, 0x01d50a},
+ {0x01d50d, 0x01d514},
+ {0x01d516, 0x01d51c},
+ {0x01d538, 0x01d539},
+ {0x01d53b, 0x01d53e},
+ {0x01d540, 0x01d544},
+ {0x01d546, 0x01d546},
+ {0x01d54a, 0x01d550},
+ {0x01d56c, 0x01d585},
+ {0x01d5a0, 0x01d5b9},
+ {0x01d5d4, 0x01d5ed},
+ {0x01d608, 0x01d621},
+ {0x01d63c, 0x01d655},
+ {0x01d670, 0x01d689},
+ {0x01d6a8, 0x01d6c0},
+ {0x01d6e2, 0x01d6fa},
+ {0x01d71c, 0x01d734},
+ {0x01d756, 0x01d76e},
+ {0x01d790, 0x01d7a8},
+ {0x01d7ca, 0x01d7ca},
+ {0x01e900, 0x01e921},
+ {0x01f130, 0x01f149},
+ {0x01f150, 0x01f169},
+ {0x01f170, 0x01f189}
+};
+
+/* table of Unicode codepoint ranges of White_Space characters */
+static const pg_unicode_range unicode_white_space[11] =
+{
+ {0x000009, 0x00000d},
+ {0x000020, 0x000020},
+ {0x000085, 0x000085},
+ {0x0000a0, 0x0000a0},
+ {0x001680, 0x001680},
+ {0x002000, 0x00200a},
+ {0x002028, 0x002028},
+ {0x002029, 0x002029},
+ {0x00202f, 0x00202f},
+ {0x00205f, 0x00205f},
+ {0x003000, 0x003000}
+};
+
+/* table of Unicode codepoint ranges of Hex_Digit characters */
+static const pg_unicode_range unicode_hex_digit[6] =
+{
+ {0x000030, 0x000039},
+ {0x000041, 0x000046},
+ {0x000061, 0x000066},
+ {0x00ff10, 0x00ff19},
+ {0x00ff21, 0x00ff26},
+ {0x00ff41, 0x00ff46}
+};
+
+/* table of Unicode codepoint ranges of Join_Control characters */
+static const pg_unicode_range unicode_join_control[1] =
+{
+ {0x00200c, 0x00200d}
+};
--
2.34.1
[text/x-patch] v15-0003-Add-unicode-case-mapping-tables-and-functions.patch (144.8K, ../[email protected]/4-v15-0003-Add-unicode-case-mapping-tables-and-functions.patch)
download | inline diff:
From 53258636a18990937d7c1ba7e60ec645762d4fab Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Mon, 30 Oct 2023 17:38:54 -0700
Subject: [PATCH v15 3/5] Add unicode case mapping tables and functions.
---
src/common/Makefile | 1 +
src/common/meson.build | 1 +
src/common/unicode/Makefile | 13 +-
src/common/unicode/case_test.c | 128 +
.../unicode/generate-unicode_case_table.pl | 134 +
src/common/unicode/meson.build | 31 +
src/common/unicode_case.c | 87 +
src/include/common/unicode_case.h | 23 +
src/include/common/unicode_case_table.h | 2994 +++++++++++++++++
9 files changed, 3410 insertions(+), 2 deletions(-)
create mode 100644 src/common/unicode/case_test.c
create mode 100644 src/common/unicode/generate-unicode_case_table.pl
create mode 100644 src/common/unicode_case.c
create mode 100644 src/include/common/unicode_case.h
create mode 100644 src/include/common/unicode_case_table.h
diff --git a/src/common/Makefile b/src/common/Makefile
index 2ba5069dca..3d83299432 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -78,6 +78,7 @@ OBJS_COMMON = \
scram-common.o \
string.o \
stringinfo.o \
+ unicode_case.o \
unicode_category.o \
unicode_norm.o \
username.o \
diff --git a/src/common/meson.build b/src/common/meson.build
index 12fd43e87f..94c3e9529e 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -32,6 +32,7 @@ common_sources = files(
'scram-common.c',
'string.c',
'stringinfo.c',
+ 'unicode_case.c',
'unicode_category.c',
'unicode_norm.c',
'username.c',
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 27f0408d8b..8223d02375 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -21,8 +21,9 @@ CPPFLAGS += $(ICU_CFLAGS)
# By default, do nothing.
all:
-update-unicode: unicode_category_table.h unicode_east_asian_fw_table.h unicode_nonspacing_table.h unicode_norm_hashfunc.h unicode_norm_table.h unicode_normprops_table.h unicode_version.h
+update-unicode: unicode_case_table.h unicode_category_table.h unicode_east_asian_fw_table.h unicode_nonspacing_table.h unicode_norm_hashfunc.h unicode_norm_table.h unicode_normprops_table.h unicode_version.h
mv $^ $(top_srcdir)/src/include/common/
+ $(MAKE) case-check
$(MAKE) category-check
$(MAKE) normalization-check
@@ -35,6 +36,9 @@ CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.tx
unicode_version.h: generate-unicode_version.pl
$(PERL) $< --version $(UNICODE_VERSION)
+unicode_case_table.h: generate-unicode_case_table.pl UnicodeData.txt
+ $(PERL) $<
+
unicode_category_table.h: generate-unicode_category_table.pl DerivedCoreProperties.txt PropList.txt UnicodeData.txt
$(PERL) $<
@@ -55,12 +59,17 @@ unicode_normprops_table.h: generate-unicode_normprops_table.pl DerivedNormalizat
$(PERL) $^ >$@
# Test suite
+case-check: case_test
+ ./case_test
+
category-check: category_test
./category_test
normalization-check: norm_test
./norm_test
+case_test: case_test.o ../unicode_case.o | submake-common
+
category_test: category_test.o ../unicode_category.o | submake-common
norm_test: norm_test.o ../unicode_norm.o | submake-common
@@ -82,4 +91,4 @@ clean:
rm -f $(OBJS) category_test category_test.o norm_test norm_test.o
distclean: clean
- rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
+ rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_case_table.h unicode_category_table.h unicode_norm_table.h
diff --git a/src/common/unicode/case_test.c b/src/common/unicode/case_test.c
new file mode 100644
index 0000000000..6aae12a2f6
--- /dev/null
+++ b/src/common/unicode/case_test.c
@@ -0,0 +1,128 @@
+/*-------------------------------------------------------------------------
+ * case_test.c
+ * Program to test Unicode case mapping functions.
+ *
+ * Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/unicode/case_test.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include <locale.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <wctype.h>
+
+#ifdef USE_ICU
+#include <unicode/uchar.h>
+#endif
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
+#include "common/unicode_version.h"
+
+/*
+ * We expect that C.UTF-8 has the same CTYPE behavior as the simple unicode
+ * mappings, but that's not guaranteed. If there are failures in the libc
+ * test, that's useful information, but does not necessarily indicate a
+ * problem.
+ */
+#define LIBC_LOCALE "C.UTF-8"
+
+#ifdef USE_ICU
+
+/* use root locale for test */
+#define ICU_LOCALE "und"
+
+static void
+icu_test_simple(pg_wchar code)
+{
+ pg_wchar lower = unicode_lowercase_simple(code);
+ pg_wchar title = unicode_titlecase_simple(code);
+ pg_wchar upper = unicode_uppercase_simple(code);
+ pg_wchar iculower = u_tolower(code);
+ pg_wchar icutitle = u_totitle(code);
+ pg_wchar icuupper = u_toupper(code);
+
+ if (lower != iculower || title != icutitle || upper != icuupper)
+ {
+ printf("case_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("case_test: Postgres lower/title/upper: 0x%06x/0x%06x/0x%06x\n",
+ lower, title, upper);
+ printf("case_test: ICU lower/title/upper: 0x%06x/0x%06x/0x%06x\n",
+ iculower, icutitle, icuupper);
+ printf("\n");
+ exit(1);
+ }
+}
+#endif
+
+static void
+libc_test_simple(pg_wchar code)
+{
+ pg_wchar lower = unicode_lowercase_simple(code);
+ pg_wchar upper = unicode_uppercase_simple(code);
+ wchar_t libclower = towlower(code);
+ wchar_t libcupper = towupper(code);
+
+ if (lower != libclower || upper != libcupper)
+ {
+ printf("case_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("case_test: Postgres lower/upper: 0x%06x/0x%06x\n",
+ lower, upper);
+ printf("case_test: libc lower/upper: 0x%06x/0x%06x\n",
+ libclower, libcupper);
+ printf("\n");
+ exit(1);
+ }
+}
+
+/*
+ * Exhaustively compare case mappings with the results from libc and ICU.
+ */
+int
+main(int argc, char **argv)
+{
+ char *libc_locale;
+
+ libc_locale = setlocale(LC_CTYPE, LIBC_LOCALE);
+
+ printf("case_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
+#ifdef USE_ICU
+ printf("case_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
+#else
+ printf("case_test: ICU not available; skipping\n");
+#endif
+
+ if (libc_locale)
+ {
+ printf("case_test: comparing with libc locale \"%s\"\n", libc_locale);
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
+ {
+ pg_unicode_category category = unicode_category(code);
+
+ if (category != PG_U_UNASSIGNED && category != PG_U_SURROGATE)
+ libc_test_simple(code);
+ }
+ printf("case_test: libc simple mapping test successful\n");
+ }
+ else
+ printf("case_test: libc locale \"%s\" not available; skipping\n",
+ LIBC_LOCALE);
+
+#ifdef USE_ICU
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
+ {
+ pg_unicode_category category = unicode_category(code);
+
+ if (category != PG_U_UNASSIGNED && category != PG_U_SURROGATE)
+ icu_test_simple(code);
+ }
+ printf("case_test: ICU simple mapping test successful\n");
+#endif
+
+ exit(0);
+}
diff --git a/src/common/unicode/generate-unicode_case_table.pl b/src/common/unicode/generate-unicode_case_table.pl
new file mode 100644
index 0000000000..9bf1deb297
--- /dev/null
+++ b/src/common/unicode/generate-unicode_case_table.pl
@@ -0,0 +1,134 @@
+#!/usr/bin/perl
+#
+# Generate Unicode character case mappings. Does not include tailoring
+# or locale-specific mappings.
+#
+# Input: UnicodeData.txt
+# Output: unicode_case_table.h
+#
+# Copyright (c) 2000-2023, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+use Getopt::Long;
+
+use FindBin;
+use lib "$FindBin::RealBin/../../tools/";
+
+my $output_path = '.';
+
+GetOptions('outdir:s' => \$output_path);
+
+my $output_table_file = "$output_path/unicode_case_table.h";
+
+# The maximum number of codepoints that can result from case mapping
+# of a single character. See Unicode section 5.18 "Case Mappings".
+my $MAX_CASE_EXPANSION = 3;
+
+my $FH;
+
+my %simple = ();
+
+open($FH, '<', "$output_path/UnicodeData.txt")
+ or die "Could not open $output_path/UnicodeData.txt: $!.";
+while (my $line = <$FH>)
+{
+ my @elts = split(';', $line);
+ my $code = hex($elts[0]);
+ my $simple_uppercase = hex($elts[12] =~ s/^\s+|\s+$//rg);
+ my $simple_lowercase = hex($elts[13] =~ s/^\s+|\s+$//rg);
+ my $simple_titlecase = hex($elts[14] =~ s/^\s+|\s+$//rg);
+
+ die "codepoint $code out of range" if $code > 0x10FFFF;
+ die "Simple_Lowercase $code out of range" if $simple_lowercase > 0x10FFFF;
+ die "Simple_Titlecase $code out of range" if $simple_titlecase > 0x10FFFF;
+ die "Simple_Uppercase $code out of range" if $simple_uppercase > 0x10FFFF;
+
+ if ($simple_lowercase || $simple_titlecase || $simple_uppercase)
+ {
+ $simple{$code} = {Simple_Lowercase => ($simple_lowercase || $code),
+ Simple_Titlecase => ($simple_titlecase || $code),
+ Simple_Uppercase => ($simple_uppercase || $code)};
+ }
+}
+close $FH;
+
+# Start writing out the output files
+open my $OT, '>', $output_table_file
+ or die "Could not open output file $output_table_file: $!\n";
+
+# determine size of array given that codepoints <= 0x80 are dense and
+# the rest of the entries are sparse
+my $num_simple = 0x80;
+foreach my $code (sort { $a <=> $b } (keys %simple))
+{
+ $num_simple++ unless $code < 0x80;
+}
+
+print $OT <<"HEADER";
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case_table.h
+ * Case mapping and information table.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/unicode_case_table.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+/*
+ * File auto-generated by src/common/unicode/generate-unicode_case_table.pl,
+ * do not edit. There is deliberately not an #ifndef PG_UNICODE_CASE_TABLE_H
+ * here.
+ */
+
+#include "mb/pg_wchar.h"
+
+typedef struct
+{
+ pg_wchar codepoint; /* Unicode codepoint */
+ pg_wchar simple_lowercase;
+ pg_wchar simple_titlecase;
+ pg_wchar simple_uppercase;
+} pg_simple_case_map;
+
+/*
+ * Case mapping table. Dense for codepoints < 0x80 (enabling fast lookup),
+ * sparse for higher codepoints (requiring scan or binary search).
+ */
+static const pg_simple_case_map simple_case_map[$num_simple] =
+{
+HEADER
+
+my $firsttime = 1;
+
+printf $OT "\t/* begin dense entries for codepoints < 0x80 */\n";
+for (my $code = 0; $code < 0x80; $code++)
+{
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ my $lc = ($simple{$code}{Simple_Lowercase} || $code);
+ my $tc = ($simple{$code}{Simple_Titlecase} || $code);
+ my $uc = ($simple{$code}{Simple_Uppercase} || $code);
+ printf $OT "\t{0x%06x, 0x%06x, 0x%06x, 0x%06x}", $code, $lc, $tc, $uc;
+}
+printf $OT ",\n\n";
+
+$firsttime = 1;
+printf $OT "\t/* begin sparse entries for codepoints >= 0x80 */\n";
+foreach my $code (sort { $a <=> $b } (keys %simple))
+{
+ next unless $code >= 0x80; # already output above
+
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ my $map = $simple{$code};
+ printf $OT "\t{0x%06x, 0x%06x, 0x%06x, 0x%06x}",
+ $code, $map->{Simple_Lowercase}, $map->{Simple_Titlecase}, $map->{Simple_Uppercase};
+}
+print $OT "\n};\n";
diff --git a/src/common/unicode/meson.build b/src/common/unicode/meson.build
index 3526ddb846..85ff40fad9 100644
--- a/src/common/unicode/meson.build
+++ b/src/common/unicode/meson.build
@@ -24,6 +24,16 @@ endforeach
update_unicode_targets = []
+update_unicode_targets += \
+ custom_target('unicode_case_table.h',
+ input: [unicode_data['UnicodeData.txt']],
+ output: ['unicode_case_table.h'],
+ command: [
+ perl, files('generate-unicode_case_table.pl'),
+ '--outdir', '@OUTDIR@', '@INPUT@'],
+ build_by_default: false,
+ )
+
update_unicode_targets += \
custom_target('unicode_category_table.h',
input: [unicode_data['UnicodeData.txt'], unicode_data['DerivedCoreProperties.txt'], unicode_data['PropList.txt']],
@@ -92,6 +102,17 @@ norm_test_table = custom_target('norm_test_table.h',
inc = include_directories('.')
+case_test = executable('case_test',
+ ['case_test.c'],
+ dependencies: [frontend_port_code, icu],
+ include_directories: inc,
+ link_with: [common_static, pgport_static],
+ build_by_default: false,
+ kwargs: default_bin_args + {
+ 'install': false,
+ }
+)
+
category_test = executable('category_test',
['category_test.c'],
dependencies: [frontend_port_code, icu],
@@ -116,6 +137,16 @@ norm_test = executable('norm_test',
update_unicode_dep = []
+if not meson.is_cross_build()
+ update_unicode_dep += custom_target('case_test.run',
+ output: 'case_test.run',
+ input: update_unicode_targets,
+ command: [case_test, UNICODE_VERSION],
+ build_by_default: false,
+ build_always_stale: true,
+ )
+endif
+
if not meson.is_cross_build()
update_unicode_dep += custom_target('category_test.run',
output: 'category_test.run',
diff --git a/src/common/unicode_case.c b/src/common/unicode_case.c
new file mode 100644
index 0000000000..714e0453e1
--- /dev/null
+++ b/src/common/unicode_case.c
@@ -0,0 +1,87 @@
+/*-------------------------------------------------------------------------
+ * unicode_case.c
+ * Conversion to upper or lower case.
+ *
+ * Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/unicode_case.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/unicode_case.h"
+#include "common/unicode_case_table.h"
+
+/* find entry in simple case map, if any */
+static inline const pg_simple_case_map *
+find_case_map(pg_wchar ucs)
+{
+ int min = 0;
+ int mid;
+ int max = lengthof(simple_case_map) - 1;
+
+ /* all chars <= 0x80 are stored in array for fast lookup */
+ Assert(max >= 0x7f);
+ if (ucs < 0x80)
+ {
+ const pg_simple_case_map *map = &simple_case_map[ucs];
+ Assert(map->codepoint == ucs);
+ return map;
+ }
+
+ /* otherwise, binary search */
+ while (max >= min)
+ {
+ mid = (min + max) / 2;
+ if (ucs > simple_case_map[mid].codepoint)
+ min = mid + 1;
+ else if (ucs < simple_case_map[mid].codepoint)
+ max = mid - 1;
+ else
+ return &simple_case_map[mid];
+ }
+
+ return NULL;
+}
+
+/*
+ * Returns simple lowercase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_lowercase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_lowercase : ucs;
+}
+
+/*
+ * Returns simple titlecase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_titlecase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_titlecase : ucs;
+}
+
+/*
+ * Returns simple uppercase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_uppercase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_uppercase : ucs;
+}
diff --git a/src/include/common/unicode_case.h b/src/include/common/unicode_case.h
new file mode 100644
index 0000000000..fe5fe6af63
--- /dev/null
+++ b/src/include/common/unicode_case.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case.h
+ * Routines for converting character case.
+ *
+ * These definitions can be used by both frontend and backend code.
+ *
+ * Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * src/include/common/unicode_case.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef UNICODE_CASE_H
+#define UNICODE_CASE_H
+
+#include "mb/pg_wchar.h"
+
+pg_wchar unicode_lowercase_simple(pg_wchar ucs);
+pg_wchar unicode_titlecase_simple(pg_wchar ucs);
+pg_wchar unicode_uppercase_simple(pg_wchar ucs);
+
+#endif /* UNICODE_CASE_H */
diff --git a/src/include/common/unicode_case_table.h b/src/include/common/unicode_case_table.h
new file mode 100644
index 0000000000..7733457bec
--- /dev/null
+++ b/src/include/common/unicode_case_table.h
@@ -0,0 +1,2994 @@
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case_table.h
+ * Case mapping and information table.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/unicode_case_table.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+/*
+ * File auto-generated by src/common/unicode/generate-unicode_case_table.pl,
+ * do not edit. There is deliberately not an #ifndef PG_UNICODE_CASE_TABLE_H
+ * here.
+ */
+
+#include "mb/pg_wchar.h"
+
+typedef struct
+{
+ pg_wchar codepoint; /* Unicode codepoint */
+ pg_wchar simple_lowercase;
+ pg_wchar simple_titlecase;
+ pg_wchar simple_uppercase;
+} pg_simple_case_map;
+
+/*
+ * Case mapping table. Dense for codepoints < 0x80 (enabling fast lookup),
+ * sparse for higher codepoints (requiring scan or binary search).
+ */
+static const pg_simple_case_map simple_case_map[2955] =
+{
+ /* begin dense entries for codepoints < 0x80 */
+ {0x000000, 0x000000, 0x000000, 0x000000},
+ {0x000001, 0x000001, 0x000001, 0x000001},
+ {0x000002, 0x000002, 0x000002, 0x000002},
+ {0x000003, 0x000003, 0x000003, 0x000003},
+ {0x000004, 0x000004, 0x000004, 0x000004},
+ {0x000005, 0x000005, 0x000005, 0x000005},
+ {0x000006, 0x000006, 0x000006, 0x000006},
+ {0x000007, 0x000007, 0x000007, 0x000007},
+ {0x000008, 0x000008, 0x000008, 0x000008},
+ {0x000009, 0x000009, 0x000009, 0x000009},
+ {0x00000a, 0x00000a, 0x00000a, 0x00000a},
+ {0x00000b, 0x00000b, 0x00000b, 0x00000b},
+ {0x00000c, 0x00000c, 0x00000c, 0x00000c},
+ {0x00000d, 0x00000d, 0x00000d, 0x00000d},
+ {0x00000e, 0x00000e, 0x00000e, 0x00000e},
+ {0x00000f, 0x00000f, 0x00000f, 0x00000f},
+ {0x000010, 0x000010, 0x000010, 0x000010},
+ {0x000011, 0x000011, 0x000011, 0x000011},
+ {0x000012, 0x000012, 0x000012, 0x000012},
+ {0x000013, 0x000013, 0x000013, 0x000013},
+ {0x000014, 0x000014, 0x000014, 0x000014},
+ {0x000015, 0x000015, 0x000015, 0x000015},
+ {0x000016, 0x000016, 0x000016, 0x000016},
+ {0x000017, 0x000017, 0x000017, 0x000017},
+ {0x000018, 0x000018, 0x000018, 0x000018},
+ {0x000019, 0x000019, 0x000019, 0x000019},
+ {0x00001a, 0x00001a, 0x00001a, 0x00001a},
+ {0x00001b, 0x00001b, 0x00001b, 0x00001b},
+ {0x00001c, 0x00001c, 0x00001c, 0x00001c},
+ {0x00001d, 0x00001d, 0x00001d, 0x00001d},
+ {0x00001e, 0x00001e, 0x00001e, 0x00001e},
+ {0x00001f, 0x00001f, 0x00001f, 0x00001f},
+ {0x000020, 0x000020, 0x000020, 0x000020},
+ {0x000021, 0x000021, 0x000021, 0x000021},
+ {0x000022, 0x000022, 0x000022, 0x000022},
+ {0x000023, 0x000023, 0x000023, 0x000023},
+ {0x000024, 0x000024, 0x000024, 0x000024},
+ {0x000025, 0x000025, 0x000025, 0x000025},
+ {0x000026, 0x000026, 0x000026, 0x000026},
+ {0x000027, 0x000027, 0x000027, 0x000027},
+ {0x000028, 0x000028, 0x000028, 0x000028},
+ {0x000029, 0x000029, 0x000029, 0x000029},
+ {0x00002a, 0x00002a, 0x00002a, 0x00002a},
+ {0x00002b, 0x00002b, 0x00002b, 0x00002b},
+ {0x00002c, 0x00002c, 0x00002c, 0x00002c},
+ {0x00002d, 0x00002d, 0x00002d, 0x00002d},
+ {0x00002e, 0x00002e, 0x00002e, 0x00002e},
+ {0x00002f, 0x00002f, 0x00002f, 0x00002f},
+ {0x000030, 0x000030, 0x000030, 0x000030},
+ {0x000031, 0x000031, 0x000031, 0x000031},
+ {0x000032, 0x000032, 0x000032, 0x000032},
+ {0x000033, 0x000033, 0x000033, 0x000033},
+ {0x000034, 0x000034, 0x000034, 0x000034},
+ {0x000035, 0x000035, 0x000035, 0x000035},
+ {0x000036, 0x000036, 0x000036, 0x000036},
+ {0x000037, 0x000037, 0x000037, 0x000037},
+ {0x000038, 0x000038, 0x000038, 0x000038},
+ {0x000039, 0x000039, 0x000039, 0x000039},
+ {0x00003a, 0x00003a, 0x00003a, 0x00003a},
+ {0x00003b, 0x00003b, 0x00003b, 0x00003b},
+ {0x00003c, 0x00003c, 0x00003c, 0x00003c},
+ {0x00003d, 0x00003d, 0x00003d, 0x00003d},
+ {0x00003e, 0x00003e, 0x00003e, 0x00003e},
+ {0x00003f, 0x00003f, 0x00003f, 0x00003f},
+ {0x000040, 0x000040, 0x000040, 0x000040},
+ {0x000041, 0x000061, 0x000041, 0x000041},
+ {0x000042, 0x000062, 0x000042, 0x000042},
+ {0x000043, 0x000063, 0x000043, 0x000043},
+ {0x000044, 0x000064, 0x000044, 0x000044},
+ {0x000045, 0x000065, 0x000045, 0x000045},
+ {0x000046, 0x000066, 0x000046, 0x000046},
+ {0x000047, 0x000067, 0x000047, 0x000047},
+ {0x000048, 0x000068, 0x000048, 0x000048},
+ {0x000049, 0x000069, 0x000049, 0x000049},
+ {0x00004a, 0x00006a, 0x00004a, 0x00004a},
+ {0x00004b, 0x00006b, 0x00004b, 0x00004b},
+ {0x00004c, 0x00006c, 0x00004c, 0x00004c},
+ {0x00004d, 0x00006d, 0x00004d, 0x00004d},
+ {0x00004e, 0x00006e, 0x00004e, 0x00004e},
+ {0x00004f, 0x00006f, 0x00004f, 0x00004f},
+ {0x000050, 0x000070, 0x000050, 0x000050},
+ {0x000051, 0x000071, 0x000051, 0x000051},
+ {0x000052, 0x000072, 0x000052, 0x000052},
+ {0x000053, 0x000073, 0x000053, 0x000053},
+ {0x000054, 0x000074, 0x000054, 0x000054},
+ {0x000055, 0x000075, 0x000055, 0x000055},
+ {0x000056, 0x000076, 0x000056, 0x000056},
+ {0x000057, 0x000077, 0x000057, 0x000057},
+ {0x000058, 0x000078, 0x000058, 0x000058},
+ {0x000059, 0x000079, 0x000059, 0x000059},
+ {0x00005a, 0x00007a, 0x00005a, 0x00005a},
+ {0x00005b, 0x00005b, 0x00005b, 0x00005b},
+ {0x00005c, 0x00005c, 0x00005c, 0x00005c},
+ {0x00005d, 0x00005d, 0x00005d, 0x00005d},
+ {0x00005e, 0x00005e, 0x00005e, 0x00005e},
+ {0x00005f, 0x00005f, 0x00005f, 0x00005f},
+ {0x000060, 0x000060, 0x000060, 0x000060},
+ {0x000061, 0x000061, 0x000041, 0x000041},
+ {0x000062, 0x000062, 0x000042, 0x000042},
+ {0x000063, 0x000063, 0x000043, 0x000043},
+ {0x000064, 0x000064, 0x000044, 0x000044},
+ {0x000065, 0x000065, 0x000045, 0x000045},
+ {0x000066, 0x000066, 0x000046, 0x000046},
+ {0x000067, 0x000067, 0x000047, 0x000047},
+ {0x000068, 0x000068, 0x000048, 0x000048},
+ {0x000069, 0x000069, 0x000049, 0x000049},
+ {0x00006a, 0x00006a, 0x00004a, 0x00004a},
+ {0x00006b, 0x00006b, 0x00004b, 0x00004b},
+ {0x00006c, 0x00006c, 0x00004c, 0x00004c},
+ {0x00006d, 0x00006d, 0x00004d, 0x00004d},
+ {0x00006e, 0x00006e, 0x00004e, 0x00004e},
+ {0x00006f, 0x00006f, 0x00004f, 0x00004f},
+ {0x000070, 0x000070, 0x000050, 0x000050},
+ {0x000071, 0x000071, 0x000051, 0x000051},
+ {0x000072, 0x000072, 0x000052, 0x000052},
+ {0x000073, 0x000073, 0x000053, 0x000053},
+ {0x000074, 0x000074, 0x000054, 0x000054},
+ {0x000075, 0x000075, 0x000055, 0x000055},
+ {0x000076, 0x000076, 0x000056, 0x000056},
+ {0x000077, 0x000077, 0x000057, 0x000057},
+ {0x000078, 0x000078, 0x000058, 0x000058},
+ {0x000079, 0x000079, 0x000059, 0x000059},
+ {0x00007a, 0x00007a, 0x00005a, 0x00005a},
+ {0x00007b, 0x00007b, 0x00007b, 0x00007b},
+ {0x00007c, 0x00007c, 0x00007c, 0x00007c},
+ {0x00007d, 0x00007d, 0x00007d, 0x00007d},
+ {0x00007e, 0x00007e, 0x00007e, 0x00007e},
+ {0x00007f, 0x00007f, 0x00007f, 0x00007f},
+
+ /* begin sparse entries for codepoints >= 0x80 */
+ {0x0000b5, 0x0000b5, 0x00039c, 0x00039c},
+ {0x0000c0, 0x0000e0, 0x0000c0, 0x0000c0},
+ {0x0000c1, 0x0000e1, 0x0000c1, 0x0000c1},
+ {0x0000c2, 0x0000e2, 0x0000c2, 0x0000c2},
+ {0x0000c3, 0x0000e3, 0x0000c3, 0x0000c3},
+ {0x0000c4, 0x0000e4, 0x0000c4, 0x0000c4},
+ {0x0000c5, 0x0000e5, 0x0000c5, 0x0000c5},
+ {0x0000c6, 0x0000e6, 0x0000c6, 0x0000c6},
+ {0x0000c7, 0x0000e7, 0x0000c7, 0x0000c7},
+ {0x0000c8, 0x0000e8, 0x0000c8, 0x0000c8},
+ {0x0000c9, 0x0000e9, 0x0000c9, 0x0000c9},
+ {0x0000ca, 0x0000ea, 0x0000ca, 0x0000ca},
+ {0x0000cb, 0x0000eb, 0x0000cb, 0x0000cb},
+ {0x0000cc, 0x0000ec, 0x0000cc, 0x0000cc},
+ {0x0000cd, 0x0000ed, 0x0000cd, 0x0000cd},
+ {0x0000ce, 0x0000ee, 0x0000ce, 0x0000ce},
+ {0x0000cf, 0x0000ef, 0x0000cf, 0x0000cf},
+ {0x0000d0, 0x0000f0, 0x0000d0, 0x0000d0},
+ {0x0000d1, 0x0000f1, 0x0000d1, 0x0000d1},
+ {0x0000d2, 0x0000f2, 0x0000d2, 0x0000d2},
+ {0x0000d3, 0x0000f3, 0x0000d3, 0x0000d3},
+ {0x0000d4, 0x0000f4, 0x0000d4, 0x0000d4},
+ {0x0000d5, 0x0000f5, 0x0000d5, 0x0000d5},
+ {0x0000d6, 0x0000f6, 0x0000d6, 0x0000d6},
+ {0x0000d8, 0x0000f8, 0x0000d8, 0x0000d8},
+ {0x0000d9, 0x0000f9, 0x0000d9, 0x0000d9},
+ {0x0000da, 0x0000fa, 0x0000da, 0x0000da},
+ {0x0000db, 0x0000fb, 0x0000db, 0x0000db},
+ {0x0000dc, 0x0000fc, 0x0000dc, 0x0000dc},
+ {0x0000dd, 0x0000fd, 0x0000dd, 0x0000dd},
+ {0x0000de, 0x0000fe, 0x0000de, 0x0000de},
+ {0x0000e0, 0x0000e0, 0x0000c0, 0x0000c0},
+ {0x0000e1, 0x0000e1, 0x0000c1, 0x0000c1},
+ {0x0000e2, 0x0000e2, 0x0000c2, 0x0000c2},
+ {0x0000e3, 0x0000e3, 0x0000c3, 0x0000c3},
+ {0x0000e4, 0x0000e4, 0x0000c4, 0x0000c4},
+ {0x0000e5, 0x0000e5, 0x0000c5, 0x0000c5},
+ {0x0000e6, 0x0000e6, 0x0000c6, 0x0000c6},
+ {0x0000e7, 0x0000e7, 0x0000c7, 0x0000c7},
+ {0x0000e8, 0x0000e8, 0x0000c8, 0x0000c8},
+ {0x0000e9, 0x0000e9, 0x0000c9, 0x0000c9},
+ {0x0000ea, 0x0000ea, 0x0000ca, 0x0000ca},
+ {0x0000eb, 0x0000eb, 0x0000cb, 0x0000cb},
+ {0x0000ec, 0x0000ec, 0x0000cc, 0x0000cc},
+ {0x0000ed, 0x0000ed, 0x0000cd, 0x0000cd},
+ {0x0000ee, 0x0000ee, 0x0000ce, 0x0000ce},
+ {0x0000ef, 0x0000ef, 0x0000cf, 0x0000cf},
+ {0x0000f0, 0x0000f0, 0x0000d0, 0x0000d0},
+ {0x0000f1, 0x0000f1, 0x0000d1, 0x0000d1},
+ {0x0000f2, 0x0000f2, 0x0000d2, 0x0000d2},
+ {0x0000f3, 0x0000f3, 0x0000d3, 0x0000d3},
+ {0x0000f4, 0x0000f4, 0x0000d4, 0x0000d4},
+ {0x0000f5, 0x0000f5, 0x0000d5, 0x0000d5},
+ {0x0000f6, 0x0000f6, 0x0000d6, 0x0000d6},
+ {0x0000f8, 0x0000f8, 0x0000d8, 0x0000d8},
+ {0x0000f9, 0x0000f9, 0x0000d9, 0x0000d9},
+ {0x0000fa, 0x0000fa, 0x0000da, 0x0000da},
+ {0x0000fb, 0x0000fb, 0x0000db, 0x0000db},
+ {0x0000fc, 0x0000fc, 0x0000dc, 0x0000dc},
+ {0x0000fd, 0x0000fd, 0x0000dd, 0x0000dd},
+ {0x0000fe, 0x0000fe, 0x0000de, 0x0000de},
+ {0x0000ff, 0x0000ff, 0x000178, 0x000178},
+ {0x000100, 0x000101, 0x000100, 0x000100},
+ {0x000101, 0x000101, 0x000100, 0x000100},
+ {0x000102, 0x000103, 0x000102, 0x000102},
+ {0x000103, 0x000103, 0x000102, 0x000102},
+ {0x000104, 0x000105, 0x000104, 0x000104},
+ {0x000105, 0x000105, 0x000104, 0x000104},
+ {0x000106, 0x000107, 0x000106, 0x000106},
+ {0x000107, 0x000107, 0x000106, 0x000106},
+ {0x000108, 0x000109, 0x000108, 0x000108},
+ {0x000109, 0x000109, 0x000108, 0x000108},
+ {0x00010a, 0x00010b, 0x00010a, 0x00010a},
+ {0x00010b, 0x00010b, 0x00010a, 0x00010a},
+ {0x00010c, 0x00010d, 0x00010c, 0x00010c},
+ {0x00010d, 0x00010d, 0x00010c, 0x00010c},
+ {0x00010e, 0x00010f, 0x00010e, 0x00010e},
+ {0x00010f, 0x00010f, 0x00010e, 0x00010e},
+ {0x000110, 0x000111, 0x000110, 0x000110},
+ {0x000111, 0x000111, 0x000110, 0x000110},
+ {0x000112, 0x000113, 0x000112, 0x000112},
+ {0x000113, 0x000113, 0x000112, 0x000112},
+ {0x000114, 0x000115, 0x000114, 0x000114},
+ {0x000115, 0x000115, 0x000114, 0x000114},
+ {0x000116, 0x000117, 0x000116, 0x000116},
+ {0x000117, 0x000117, 0x000116, 0x000116},
+ {0x000118, 0x000119, 0x000118, 0x000118},
+ {0x000119, 0x000119, 0x000118, 0x000118},
+ {0x00011a, 0x00011b, 0x00011a, 0x00011a},
+ {0x00011b, 0x00011b, 0x00011a, 0x00011a},
+ {0x00011c, 0x00011d, 0x00011c, 0x00011c},
+ {0x00011d, 0x00011d, 0x00011c, 0x00011c},
+ {0x00011e, 0x00011f, 0x00011e, 0x00011e},
+ {0x00011f, 0x00011f, 0x00011e, 0x00011e},
+ {0x000120, 0x000121, 0x000120, 0x000120},
+ {0x000121, 0x000121, 0x000120, 0x000120},
+ {0x000122, 0x000123, 0x000122, 0x000122},
+ {0x000123, 0x000123, 0x000122, 0x000122},
+ {0x000124, 0x000125, 0x000124, 0x000124},
+ {0x000125, 0x000125, 0x000124, 0x000124},
+ {0x000126, 0x000127, 0x000126, 0x000126},
+ {0x000127, 0x000127, 0x000126, 0x000126},
+ {0x000128, 0x000129, 0x000128, 0x000128},
+ {0x000129, 0x000129, 0x000128, 0x000128},
+ {0x00012a, 0x00012b, 0x00012a, 0x00012a},
+ {0x00012b, 0x00012b, 0x00012a, 0x00012a},
+ {0x00012c, 0x00012d, 0x00012c, 0x00012c},
+ {0x00012d, 0x00012d, 0x00012c, 0x00012c},
+ {0x00012e, 0x00012f, 0x00012e, 0x00012e},
+ {0x00012f, 0x00012f, 0x00012e, 0x00012e},
+ {0x000130, 0x000069, 0x000130, 0x000130},
+ {0x000131, 0x000131, 0x000049, 0x000049},
+ {0x000132, 0x000133, 0x000132, 0x000132},
+ {0x000133, 0x000133, 0x000132, 0x000132},
+ {0x000134, 0x000135, 0x000134, 0x000134},
+ {0x000135, 0x000135, 0x000134, 0x000134},
+ {0x000136, 0x000137, 0x000136, 0x000136},
+ {0x000137, 0x000137, 0x000136, 0x000136},
+ {0x000139, 0x00013a, 0x000139, 0x000139},
+ {0x00013a, 0x00013a, 0x000139, 0x000139},
+ {0x00013b, 0x00013c, 0x00013b, 0x00013b},
+ {0x00013c, 0x00013c, 0x00013b, 0x00013b},
+ {0x00013d, 0x00013e, 0x00013d, 0x00013d},
+ {0x00013e, 0x00013e, 0x00013d, 0x00013d},
+ {0x00013f, 0x000140, 0x00013f, 0x00013f},
+ {0x000140, 0x000140, 0x00013f, 0x00013f},
+ {0x000141, 0x000142, 0x000141, 0x000141},
+ {0x000142, 0x000142, 0x000141, 0x000141},
+ {0x000143, 0x000144, 0x000143, 0x000143},
+ {0x000144, 0x000144, 0x000143, 0x000143},
+ {0x000145, 0x000146, 0x000145, 0x000145},
+ {0x000146, 0x000146, 0x000145, 0x000145},
+ {0x000147, 0x000148, 0x000147, 0x000147},
+ {0x000148, 0x000148, 0x000147, 0x000147},
+ {0x00014a, 0x00014b, 0x00014a, 0x00014a},
+ {0x00014b, 0x00014b, 0x00014a, 0x00014a},
+ {0x00014c, 0x00014d, 0x00014c, 0x00014c},
+ {0x00014d, 0x00014d, 0x00014c, 0x00014c},
+ {0x00014e, 0x00014f, 0x00014e, 0x00014e},
+ {0x00014f, 0x00014f, 0x00014e, 0x00014e},
+ {0x000150, 0x000151, 0x000150, 0x000150},
+ {0x000151, 0x000151, 0x000150, 0x000150},
+ {0x000152, 0x000153, 0x000152, 0x000152},
+ {0x000153, 0x000153, 0x000152, 0x000152},
+ {0x000154, 0x000155, 0x000154, 0x000154},
+ {0x000155, 0x000155, 0x000154, 0x000154},
+ {0x000156, 0x000157, 0x000156, 0x000156},
+ {0x000157, 0x000157, 0x000156, 0x000156},
+ {0x000158, 0x000159, 0x000158, 0x000158},
+ {0x000159, 0x000159, 0x000158, 0x000158},
+ {0x00015a, 0x00015b, 0x00015a, 0x00015a},
+ {0x00015b, 0x00015b, 0x00015a, 0x00015a},
+ {0x00015c, 0x00015d, 0x00015c, 0x00015c},
+ {0x00015d, 0x00015d, 0x00015c, 0x00015c},
+ {0x00015e, 0x00015f, 0x00015e, 0x00015e},
+ {0x00015f, 0x00015f, 0x00015e, 0x00015e},
+ {0x000160, 0x000161, 0x000160, 0x000160},
+ {0x000161, 0x000161, 0x000160, 0x000160},
+ {0x000162, 0x000163, 0x000162, 0x000162},
+ {0x000163, 0x000163, 0x000162, 0x000162},
+ {0x000164, 0x000165, 0x000164, 0x000164},
+ {0x000165, 0x000165, 0x000164, 0x000164},
+ {0x000166, 0x000167, 0x000166, 0x000166},
+ {0x000167, 0x000167, 0x000166, 0x000166},
+ {0x000168, 0x000169, 0x000168, 0x000168},
+ {0x000169, 0x000169, 0x000168, 0x000168},
+ {0x00016a, 0x00016b, 0x00016a, 0x00016a},
+ {0x00016b, 0x00016b, 0x00016a, 0x00016a},
+ {0x00016c, 0x00016d, 0x00016c, 0x00016c},
+ {0x00016d, 0x00016d, 0x00016c, 0x00016c},
+ {0x00016e, 0x00016f, 0x00016e, 0x00016e},
+ {0x00016f, 0x00016f, 0x00016e, 0x00016e},
+ {0x000170, 0x000171, 0x000170, 0x000170},
+ {0x000171, 0x000171, 0x000170, 0x000170},
+ {0x000172, 0x000173, 0x000172, 0x000172},
+ {0x000173, 0x000173, 0x000172, 0x000172},
+ {0x000174, 0x000175, 0x000174, 0x000174},
+ {0x000175, 0x000175, 0x000174, 0x000174},
+ {0x000176, 0x000177, 0x000176, 0x000176},
+ {0x000177, 0x000177, 0x000176, 0x000176},
+ {0x000178, 0x0000ff, 0x000178, 0x000178},
+ {0x000179, 0x00017a, 0x000179, 0x000179},
+ {0x00017a, 0x00017a, 0x000179, 0x000179},
+ {0x00017b, 0x00017c, 0x00017b, 0x00017b},
+ {0x00017c, 0x00017c, 0x00017b, 0x00017b},
+ {0x00017d, 0x00017e, 0x00017d, 0x00017d},
+ {0x00017e, 0x00017e, 0x00017d, 0x00017d},
+ {0x00017f, 0x00017f, 0x000053, 0x000053},
+ {0x000180, 0x000180, 0x000243, 0x000243},
+ {0x000181, 0x000253, 0x000181, 0x000181},
+ {0x000182, 0x000183, 0x000182, 0x000182},
+ {0x000183, 0x000183, 0x000182, 0x000182},
+ {0x000184, 0x000185, 0x000184, 0x000184},
+ {0x000185, 0x000185, 0x000184, 0x000184},
+ {0x000186, 0x000254, 0x000186, 0x000186},
+ {0x000187, 0x000188, 0x000187, 0x000187},
+ {0x000188, 0x000188, 0x000187, 0x000187},
+ {0x000189, 0x000256, 0x000189, 0x000189},
+ {0x00018a, 0x000257, 0x00018a, 0x00018a},
+ {0x00018b, 0x00018c, 0x00018b, 0x00018b},
+ {0x00018c, 0x00018c, 0x00018b, 0x00018b},
+ {0x00018e, 0x0001dd, 0x00018e, 0x00018e},
+ {0x00018f, 0x000259, 0x00018f, 0x00018f},
+ {0x000190, 0x00025b, 0x000190, 0x000190},
+ {0x000191, 0x000192, 0x000191, 0x000191},
+ {0x000192, 0x000192, 0x000191, 0x000191},
+ {0x000193, 0x000260, 0x000193, 0x000193},
+ {0x000194, 0x000263, 0x000194, 0x000194},
+ {0x000195, 0x000195, 0x0001f6, 0x0001f6},
+ {0x000196, 0x000269, 0x000196, 0x000196},
+ {0x000197, 0x000268, 0x000197, 0x000197},
+ {0x000198, 0x000199, 0x000198, 0x000198},
+ {0x000199, 0x000199, 0x000198, 0x000198},
+ {0x00019a, 0x00019a, 0x00023d, 0x00023d},
+ {0x00019c, 0x00026f, 0x00019c, 0x00019c},
+ {0x00019d, 0x000272, 0x00019d, 0x00019d},
+ {0x00019e, 0x00019e, 0x000220, 0x000220},
+ {0x00019f, 0x000275, 0x00019f, 0x00019f},
+ {0x0001a0, 0x0001a1, 0x0001a0, 0x0001a0},
+ {0x0001a1, 0x0001a1, 0x0001a0, 0x0001a0},
+ {0x0001a2, 0x0001a3, 0x0001a2, 0x0001a2},
+ {0x0001a3, 0x0001a3, 0x0001a2, 0x0001a2},
+ {0x0001a4, 0x0001a5, 0x0001a4, 0x0001a4},
+ {0x0001a5, 0x0001a5, 0x0001a4, 0x0001a4},
+ {0x0001a6, 0x000280, 0x0001a6, 0x0001a6},
+ {0x0001a7, 0x0001a8, 0x0001a7, 0x0001a7},
+ {0x0001a8, 0x0001a8, 0x0001a7, 0x0001a7},
+ {0x0001a9, 0x000283, 0x0001a9, 0x0001a9},
+ {0x0001ac, 0x0001ad, 0x0001ac, 0x0001ac},
+ {0x0001ad, 0x0001ad, 0x0001ac, 0x0001ac},
+ {0x0001ae, 0x000288, 0x0001ae, 0x0001ae},
+ {0x0001af, 0x0001b0, 0x0001af, 0x0001af},
+ {0x0001b0, 0x0001b0, 0x0001af, 0x0001af},
+ {0x0001b1, 0x00028a, 0x0001b1, 0x0001b1},
+ {0x0001b2, 0x00028b, 0x0001b2, 0x0001b2},
+ {0x0001b3, 0x0001b4, 0x0001b3, 0x0001b3},
+ {0x0001b4, 0x0001b4, 0x0001b3, 0x0001b3},
+ {0x0001b5, 0x0001b6, 0x0001b5, 0x0001b5},
+ {0x0001b6, 0x0001b6, 0x0001b5, 0x0001b5},
+ {0x0001b7, 0x000292, 0x0001b7, 0x0001b7},
+ {0x0001b8, 0x0001b9, 0x0001b8, 0x0001b8},
+ {0x0001b9, 0x0001b9, 0x0001b8, 0x0001b8},
+ {0x0001bc, 0x0001bd, 0x0001bc, 0x0001bc},
+ {0x0001bd, 0x0001bd, 0x0001bc, 0x0001bc},
+ {0x0001bf, 0x0001bf, 0x0001f7, 0x0001f7},
+ {0x0001c4, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c5, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c6, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c7, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001c8, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001c9, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001ca, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cb, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cc, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cd, 0x0001ce, 0x0001cd, 0x0001cd},
+ {0x0001ce, 0x0001ce, 0x0001cd, 0x0001cd},
+ {0x0001cf, 0x0001d0, 0x0001cf, 0x0001cf},
+ {0x0001d0, 0x0001d0, 0x0001cf, 0x0001cf},
+ {0x0001d1, 0x0001d2, 0x0001d1, 0x0001d1},
+ {0x0001d2, 0x0001d2, 0x0001d1, 0x0001d1},
+ {0x0001d3, 0x0001d4, 0x0001d3, 0x0001d3},
+ {0x0001d4, 0x0001d4, 0x0001d3, 0x0001d3},
+ {0x0001d5, 0x0001d6, 0x0001d5, 0x0001d5},
+ {0x0001d6, 0x0001d6, 0x0001d5, 0x0001d5},
+ {0x0001d7, 0x0001d8, 0x0001d7, 0x0001d7},
+ {0x0001d8, 0x0001d8, 0x0001d7, 0x0001d7},
+ {0x0001d9, 0x0001da, 0x0001d9, 0x0001d9},
+ {0x0001da, 0x0001da, 0x0001d9, 0x0001d9},
+ {0x0001db, 0x0001dc, 0x0001db, 0x0001db},
+ {0x0001dc, 0x0001dc, 0x0001db, 0x0001db},
+ {0x0001dd, 0x0001dd, 0x00018e, 0x00018e},
+ {0x0001de, 0x0001df, 0x0001de, 0x0001de},
+ {0x0001df, 0x0001df, 0x0001de, 0x0001de},
+ {0x0001e0, 0x0001e1, 0x0001e0, 0x0001e0},
+ {0x0001e1, 0x0001e1, 0x0001e0, 0x0001e0},
+ {0x0001e2, 0x0001e3, 0x0001e2, 0x0001e2},
+ {0x0001e3, 0x0001e3, 0x0001e2, 0x0001e2},
+ {0x0001e4, 0x0001e5, 0x0001e4, 0x0001e4},
+ {0x0001e5, 0x0001e5, 0x0001e4, 0x0001e4},
+ {0x0001e6, 0x0001e7, 0x0001e6, 0x0001e6},
+ {0x0001e7, 0x0001e7, 0x0001e6, 0x0001e6},
+ {0x0001e8, 0x0001e9, 0x0001e8, 0x0001e8},
+ {0x0001e9, 0x0001e9, 0x0001e8, 0x0001e8},
+ {0x0001ea, 0x0001eb, 0x0001ea, 0x0001ea},
+ {0x0001eb, 0x0001eb, 0x0001ea, 0x0001ea},
+ {0x0001ec, 0x0001ed, 0x0001ec, 0x0001ec},
+ {0x0001ed, 0x0001ed, 0x0001ec, 0x0001ec},
+ {0x0001ee, 0x0001ef, 0x0001ee, 0x0001ee},
+ {0x0001ef, 0x0001ef, 0x0001ee, 0x0001ee},
+ {0x0001f1, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f2, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f3, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f4, 0x0001f5, 0x0001f4, 0x0001f4},
+ {0x0001f5, 0x0001f5, 0x0001f4, 0x0001f4},
+ {0x0001f6, 0x000195, 0x0001f6, 0x0001f6},
+ {0x0001f7, 0x0001bf, 0x0001f7, 0x0001f7},
+ {0x0001f8, 0x0001f9, 0x0001f8, 0x0001f8},
+ {0x0001f9, 0x0001f9, 0x0001f8, 0x0001f8},
+ {0x0001fa, 0x0001fb, 0x0001fa, 0x0001fa},
+ {0x0001fb, 0x0001fb, 0x0001fa, 0x0001fa},
+ {0x0001fc, 0x0001fd, 0x0001fc, 0x0001fc},
+ {0x0001fd, 0x0001fd, 0x0001fc, 0x0001fc},
+ {0x0001fe, 0x0001ff, 0x0001fe, 0x0001fe},
+ {0x0001ff, 0x0001ff, 0x0001fe, 0x0001fe},
+ {0x000200, 0x000201, 0x000200, 0x000200},
+ {0x000201, 0x000201, 0x000200, 0x000200},
+ {0x000202, 0x000203, 0x000202, 0x000202},
+ {0x000203, 0x000203, 0x000202, 0x000202},
+ {0x000204, 0x000205, 0x000204, 0x000204},
+ {0x000205, 0x000205, 0x000204, 0x000204},
+ {0x000206, 0x000207, 0x000206, 0x000206},
+ {0x000207, 0x000207, 0x000206, 0x000206},
+ {0x000208, 0x000209, 0x000208, 0x000208},
+ {0x000209, 0x000209, 0x000208, 0x000208},
+ {0x00020a, 0x00020b, 0x00020a, 0x00020a},
+ {0x00020b, 0x00020b, 0x00020a, 0x00020a},
+ {0x00020c, 0x00020d, 0x00020c, 0x00020c},
+ {0x00020d, 0x00020d, 0x00020c, 0x00020c},
+ {0x00020e, 0x00020f, 0x00020e, 0x00020e},
+ {0x00020f, 0x00020f, 0x00020e, 0x00020e},
+ {0x000210, 0x000211, 0x000210, 0x000210},
+ {0x000211, 0x000211, 0x000210, 0x000210},
+ {0x000212, 0x000213, 0x000212, 0x000212},
+ {0x000213, 0x000213, 0x000212, 0x000212},
+ {0x000214, 0x000215, 0x000214, 0x000214},
+ {0x000215, 0x000215, 0x000214, 0x000214},
+ {0x000216, 0x000217, 0x000216, 0x000216},
+ {0x000217, 0x000217, 0x000216, 0x000216},
+ {0x000218, 0x000219, 0x000218, 0x000218},
+ {0x000219, 0x000219, 0x000218, 0x000218},
+ {0x00021a, 0x00021b, 0x00021a, 0x00021a},
+ {0x00021b, 0x00021b, 0x00021a, 0x00021a},
+ {0x00021c, 0x00021d, 0x00021c, 0x00021c},
+ {0x00021d, 0x00021d, 0x00021c, 0x00021c},
+ {0x00021e, 0x00021f, 0x00021e, 0x00021e},
+ {0x00021f, 0x00021f, 0x00021e, 0x00021e},
+ {0x000220, 0x00019e, 0x000220, 0x000220},
+ {0x000222, 0x000223, 0x000222, 0x000222},
+ {0x000223, 0x000223, 0x000222, 0x000222},
+ {0x000224, 0x000225, 0x000224, 0x000224},
+ {0x000225, 0x000225, 0x000224, 0x000224},
+ {0x000226, 0x000227, 0x000226, 0x000226},
+ {0x000227, 0x000227, 0x000226, 0x000226},
+ {0x000228, 0x000229, 0x000228, 0x000228},
+ {0x000229, 0x000229, 0x000228, 0x000228},
+ {0x00022a, 0x00022b, 0x00022a, 0x00022a},
+ {0x00022b, 0x00022b, 0x00022a, 0x00022a},
+ {0x00022c, 0x00022d, 0x00022c, 0x00022c},
+ {0x00022d, 0x00022d, 0x00022c, 0x00022c},
+ {0x00022e, 0x00022f, 0x00022e, 0x00022e},
+ {0x00022f, 0x00022f, 0x00022e, 0x00022e},
+ {0x000230, 0x000231, 0x000230, 0x000230},
+ {0x000231, 0x000231, 0x000230, 0x000230},
+ {0x000232, 0x000233, 0x000232, 0x000232},
+ {0x000233, 0x000233, 0x000232, 0x000232},
+ {0x00023a, 0x002c65, 0x00023a, 0x00023a},
+ {0x00023b, 0x00023c, 0x00023b, 0x00023b},
+ {0x00023c, 0x00023c, 0x00023b, 0x00023b},
+ {0x00023d, 0x00019a, 0x00023d, 0x00023d},
+ {0x00023e, 0x002c66, 0x00023e, 0x00023e},
+ {0x00023f, 0x00023f, 0x002c7e, 0x002c7e},
+ {0x000240, 0x000240, 0x002c7f, 0x002c7f},
+ {0x000241, 0x000242, 0x000241, 0x000241},
+ {0x000242, 0x000242, 0x000241, 0x000241},
+ {0x000243, 0x000180, 0x000243, 0x000243},
+ {0x000244, 0x000289, 0x000244, 0x000244},
+ {0x000245, 0x00028c, 0x000245, 0x000245},
+ {0x000246, 0x000247, 0x000246, 0x000246},
+ {0x000247, 0x000247, 0x000246, 0x000246},
+ {0x000248, 0x000249, 0x000248, 0x000248},
+ {0x000249, 0x000249, 0x000248, 0x000248},
+ {0x00024a, 0x00024b, 0x00024a, 0x00024a},
+ {0x00024b, 0x00024b, 0x00024a, 0x00024a},
+ {0x00024c, 0x00024d, 0x00024c, 0x00024c},
+ {0x00024d, 0x00024d, 0x00024c, 0x00024c},
+ {0x00024e, 0x00024f, 0x00024e, 0x00024e},
+ {0x00024f, 0x00024f, 0x00024e, 0x00024e},
+ {0x000250, 0x000250, 0x002c6f, 0x002c6f},
+ {0x000251, 0x000251, 0x002c6d, 0x002c6d},
+ {0x000252, 0x000252, 0x002c70, 0x002c70},
+ {0x000253, 0x000253, 0x000181, 0x000181},
+ {0x000254, 0x000254, 0x000186, 0x000186},
+ {0x000256, 0x000256, 0x000189, 0x000189},
+ {0x000257, 0x000257, 0x00018a, 0x00018a},
+ {0x000259, 0x000259, 0x00018f, 0x00018f},
+ {0x00025b, 0x00025b, 0x000190, 0x000190},
+ {0x00025c, 0x00025c, 0x00a7ab, 0x00a7ab},
+ {0x000260, 0x000260, 0x000193, 0x000193},
+ {0x000261, 0x000261, 0x00a7ac, 0x00a7ac},
+ {0x000263, 0x000263, 0x000194, 0x000194},
+ {0x000265, 0x000265, 0x00a78d, 0x00a78d},
+ {0x000266, 0x000266, 0x00a7aa, 0x00a7aa},
+ {0x000268, 0x000268, 0x000197, 0x000197},
+ {0x000269, 0x000269, 0x000196, 0x000196},
+ {0x00026a, 0x00026a, 0x00a7ae, 0x00a7ae},
+ {0x00026b, 0x00026b, 0x002c62, 0x002c62},
+ {0x00026c, 0x00026c, 0x00a7ad, 0x00a7ad},
+ {0x00026f, 0x00026f, 0x00019c, 0x00019c},
+ {0x000271, 0x000271, 0x002c6e, 0x002c6e},
+ {0x000272, 0x000272, 0x00019d, 0x00019d},
+ {0x000275, 0x000275, 0x00019f, 0x00019f},
+ {0x00027d, 0x00027d, 0x002c64, 0x002c64},
+ {0x000280, 0x000280, 0x0001a6, 0x0001a6},
+ {0x000282, 0x000282, 0x00a7c5, 0x00a7c5},
+ {0x000283, 0x000283, 0x0001a9, 0x0001a9},
+ {0x000287, 0x000287, 0x00a7b1, 0x00a7b1},
+ {0x000288, 0x000288, 0x0001ae, 0x0001ae},
+ {0x000289, 0x000289, 0x000244, 0x000244},
+ {0x00028a, 0x00028a, 0x0001b1, 0x0001b1},
+ {0x00028b, 0x00028b, 0x0001b2, 0x0001b2},
+ {0x00028c, 0x00028c, 0x000245, 0x000245},
+ {0x000292, 0x000292, 0x0001b7, 0x0001b7},
+ {0x00029d, 0x00029d, 0x00a7b2, 0x00a7b2},
+ {0x00029e, 0x00029e, 0x00a7b0, 0x00a7b0},
+ {0x000345, 0x000345, 0x000399, 0x000399},
+ {0x000370, 0x000371, 0x000370, 0x000370},
+ {0x000371, 0x000371, 0x000370, 0x000370},
+ {0x000372, 0x000373, 0x000372, 0x000372},
+ {0x000373, 0x000373, 0x000372, 0x000372},
+ {0x000376, 0x000377, 0x000376, 0x000376},
+ {0x000377, 0x000377, 0x000376, 0x000376},
+ {0x00037b, 0x00037b, 0x0003fd, 0x0003fd},
+ {0x00037c, 0x00037c, 0x0003fe, 0x0003fe},
+ {0x00037d, 0x00037d, 0x0003ff, 0x0003ff},
+ {0x00037f, 0x0003f3, 0x00037f, 0x00037f},
+ {0x000386, 0x0003ac, 0x000386, 0x000386},
+ {0x000388, 0x0003ad, 0x000388, 0x000388},
+ {0x000389, 0x0003ae, 0x000389, 0x000389},
+ {0x00038a, 0x0003af, 0x00038a, 0x00038a},
+ {0x00038c, 0x0003cc, 0x00038c, 0x00038c},
+ {0x00038e, 0x0003cd, 0x00038e, 0x00038e},
+ {0x00038f, 0x0003ce, 0x00038f, 0x00038f},
+ {0x000391, 0x0003b1, 0x000391, 0x000391},
+ {0x000392, 0x0003b2, 0x000392, 0x000392},
+ {0x000393, 0x0003b3, 0x000393, 0x000393},
+ {0x000394, 0x0003b4, 0x000394, 0x000394},
+ {0x000395, 0x0003b5, 0x000395, 0x000395},
+ {0x000396, 0x0003b6, 0x000396, 0x000396},
+ {0x000397, 0x0003b7, 0x000397, 0x000397},
+ {0x000398, 0x0003b8, 0x000398, 0x000398},
+ {0x000399, 0x0003b9, 0x000399, 0x000399},
+ {0x00039a, 0x0003ba, 0x00039a, 0x00039a},
+ {0x00039b, 0x0003bb, 0x00039b, 0x00039b},
+ {0x00039c, 0x0003bc, 0x00039c, 0x00039c},
+ {0x00039d, 0x0003bd, 0x00039d, 0x00039d},
+ {0x00039e, 0x0003be, 0x00039e, 0x00039e},
+ {0x00039f, 0x0003bf, 0x00039f, 0x00039f},
+ {0x0003a0, 0x0003c0, 0x0003a0, 0x0003a0},
+ {0x0003a1, 0x0003c1, 0x0003a1, 0x0003a1},
+ {0x0003a3, 0x0003c3, 0x0003a3, 0x0003a3},
+ {0x0003a4, 0x0003c4, 0x0003a4, 0x0003a4},
+ {0x0003a5, 0x0003c5, 0x0003a5, 0x0003a5},
+ {0x0003a6, 0x0003c6, 0x0003a6, 0x0003a6},
+ {0x0003a7, 0x0003c7, 0x0003a7, 0x0003a7},
+ {0x0003a8, 0x0003c8, 0x0003a8, 0x0003a8},
+ {0x0003a9, 0x0003c9, 0x0003a9, 0x0003a9},
+ {0x0003aa, 0x0003ca, 0x0003aa, 0x0003aa},
+ {0x0003ab, 0x0003cb, 0x0003ab, 0x0003ab},
+ {0x0003ac, 0x0003ac, 0x000386, 0x000386},
+ {0x0003ad, 0x0003ad, 0x000388, 0x000388},
+ {0x0003ae, 0x0003ae, 0x000389, 0x000389},
+ {0x0003af, 0x0003af, 0x00038a, 0x00038a},
+ {0x0003b1, 0x0003b1, 0x000391, 0x000391},
+ {0x0003b2, 0x0003b2, 0x000392, 0x000392},
+ {0x0003b3, 0x0003b3, 0x000393, 0x000393},
+ {0x0003b4, 0x0003b4, 0x000394, 0x000394},
+ {0x0003b5, 0x0003b5, 0x000395, 0x000395},
+ {0x0003b6, 0x0003b6, 0x000396, 0x000396},
+ {0x0003b7, 0x0003b7, 0x000397, 0x000397},
+ {0x0003b8, 0x0003b8, 0x000398, 0x000398},
+ {0x0003b9, 0x0003b9, 0x000399, 0x000399},
+ {0x0003ba, 0x0003ba, 0x00039a, 0x00039a},
+ {0x0003bb, 0x0003bb, 0x00039b, 0x00039b},
+ {0x0003bc, 0x0003bc, 0x00039c, 0x00039c},
+ {0x0003bd, 0x0003bd, 0x00039d, 0x00039d},
+ {0x0003be, 0x0003be, 0x00039e, 0x00039e},
+ {0x0003bf, 0x0003bf, 0x00039f, 0x00039f},
+ {0x0003c0, 0x0003c0, 0x0003a0, 0x0003a0},
+ {0x0003c1, 0x0003c1, 0x0003a1, 0x0003a1},
+ {0x0003c2, 0x0003c2, 0x0003a3, 0x0003a3},
+ {0x0003c3, 0x0003c3, 0x0003a3, 0x0003a3},
+ {0x0003c4, 0x0003c4, 0x0003a4, 0x0003a4},
+ {0x0003c5, 0x0003c5, 0x0003a5, 0x0003a5},
+ {0x0003c6, 0x0003c6, 0x0003a6, 0x0003a6},
+ {0x0003c7, 0x0003c7, 0x0003a7, 0x0003a7},
+ {0x0003c8, 0x0003c8, 0x0003a8, 0x0003a8},
+ {0x0003c9, 0x0003c9, 0x0003a9, 0x0003a9},
+ {0x0003ca, 0x0003ca, 0x0003aa, 0x0003aa},
+ {0x0003cb, 0x0003cb, 0x0003ab, 0x0003ab},
+ {0x0003cc, 0x0003cc, 0x00038c, 0x00038c},
+ {0x0003cd, 0x0003cd, 0x00038e, 0x00038e},
+ {0x0003ce, 0x0003ce, 0x00038f, 0x00038f},
+ {0x0003cf, 0x0003d7, 0x0003cf, 0x0003cf},
+ {0x0003d0, 0x0003d0, 0x000392, 0x000392},
+ {0x0003d1, 0x0003d1, 0x000398, 0x000398},
+ {0x0003d5, 0x0003d5, 0x0003a6, 0x0003a6},
+ {0x0003d6, 0x0003d6, 0x0003a0, 0x0003a0},
+ {0x0003d7, 0x0003d7, 0x0003cf, 0x0003cf},
+ {0x0003d8, 0x0003d9, 0x0003d8, 0x0003d8},
+ {0x0003d9, 0x0003d9, 0x0003d8, 0x0003d8},
+ {0x0003da, 0x0003db, 0x0003da, 0x0003da},
+ {0x0003db, 0x0003db, 0x0003da, 0x0003da},
+ {0x0003dc, 0x0003dd, 0x0003dc, 0x0003dc},
+ {0x0003dd, 0x0003dd, 0x0003dc, 0x0003dc},
+ {0x0003de, 0x0003df, 0x0003de, 0x0003de},
+ {0x0003df, 0x0003df, 0x0003de, 0x0003de},
+ {0x0003e0, 0x0003e1, 0x0003e0, 0x0003e0},
+ {0x0003e1, 0x0003e1, 0x0003e0, 0x0003e0},
+ {0x0003e2, 0x0003e3, 0x0003e2, 0x0003e2},
+ {0x0003e3, 0x0003e3, 0x0003e2, 0x0003e2},
+ {0x0003e4, 0x0003e5, 0x0003e4, 0x0003e4},
+ {0x0003e5, 0x0003e5, 0x0003e4, 0x0003e4},
+ {0x0003e6, 0x0003e7, 0x0003e6, 0x0003e6},
+ {0x0003e7, 0x0003e7, 0x0003e6, 0x0003e6},
+ {0x0003e8, 0x0003e9, 0x0003e8, 0x0003e8},
+ {0x0003e9, 0x0003e9, 0x0003e8, 0x0003e8},
+ {0x0003ea, 0x0003eb, 0x0003ea, 0x0003ea},
+ {0x0003eb, 0x0003eb, 0x0003ea, 0x0003ea},
+ {0x0003ec, 0x0003ed, 0x0003ec, 0x0003ec},
+ {0x0003ed, 0x0003ed, 0x0003ec, 0x0003ec},
+ {0x0003ee, 0x0003ef, 0x0003ee, 0x0003ee},
+ {0x0003ef, 0x0003ef, 0x0003ee, 0x0003ee},
+ {0x0003f0, 0x0003f0, 0x00039a, 0x00039a},
+ {0x0003f1, 0x0003f1, 0x0003a1, 0x0003a1},
+ {0x0003f2, 0x0003f2, 0x0003f9, 0x0003f9},
+ {0x0003f3, 0x0003f3, 0x00037f, 0x00037f},
+ {0x0003f4, 0x0003b8, 0x0003f4, 0x0003f4},
+ {0x0003f5, 0x0003f5, 0x000395, 0x000395},
+ {0x0003f7, 0x0003f8, 0x0003f7, 0x0003f7},
+ {0x0003f8, 0x0003f8, 0x0003f7, 0x0003f7},
+ {0x0003f9, 0x0003f2, 0x0003f9, 0x0003f9},
+ {0x0003fa, 0x0003fb, 0x0003fa, 0x0003fa},
+ {0x0003fb, 0x0003fb, 0x0003fa, 0x0003fa},
+ {0x0003fd, 0x00037b, 0x0003fd, 0x0003fd},
+ {0x0003fe, 0x00037c, 0x0003fe, 0x0003fe},
+ {0x0003ff, 0x00037d, 0x0003ff, 0x0003ff},
+ {0x000400, 0x000450, 0x000400, 0x000400},
+ {0x000401, 0x000451, 0x000401, 0x000401},
+ {0x000402, 0x000452, 0x000402, 0x000402},
+ {0x000403, 0x000453, 0x000403, 0x000403},
+ {0x000404, 0x000454, 0x000404, 0x000404},
+ {0x000405, 0x000455, 0x000405, 0x000405},
+ {0x000406, 0x000456, 0x000406, 0x000406},
+ {0x000407, 0x000457, 0x000407, 0x000407},
+ {0x000408, 0x000458, 0x000408, 0x000408},
+ {0x000409, 0x000459, 0x000409, 0x000409},
+ {0x00040a, 0x00045a, 0x00040a, 0x00040a},
+ {0x00040b, 0x00045b, 0x00040b, 0x00040b},
+ {0x00040c, 0x00045c, 0x00040c, 0x00040c},
+ {0x00040d, 0x00045d, 0x00040d, 0x00040d},
+ {0x00040e, 0x00045e, 0x00040e, 0x00040e},
+ {0x00040f, 0x00045f, 0x00040f, 0x00040f},
+ {0x000410, 0x000430, 0x000410, 0x000410},
+ {0x000411, 0x000431, 0x000411, 0x000411},
+ {0x000412, 0x000432, 0x000412, 0x000412},
+ {0x000413, 0x000433, 0x000413, 0x000413},
+ {0x000414, 0x000434, 0x000414, 0x000414},
+ {0x000415, 0x000435, 0x000415, 0x000415},
+ {0x000416, 0x000436, 0x000416, 0x000416},
+ {0x000417, 0x000437, 0x000417, 0x000417},
+ {0x000418, 0x000438, 0x000418, 0x000418},
+ {0x000419, 0x000439, 0x000419, 0x000419},
+ {0x00041a, 0x00043a, 0x00041a, 0x00041a},
+ {0x00041b, 0x00043b, 0x00041b, 0x00041b},
+ {0x00041c, 0x00043c, 0x00041c, 0x00041c},
+ {0x00041d, 0x00043d, 0x00041d, 0x00041d},
+ {0x00041e, 0x00043e, 0x00041e, 0x00041e},
+ {0x00041f, 0x00043f, 0x00041f, 0x00041f},
+ {0x000420, 0x000440, 0x000420, 0x000420},
+ {0x000421, 0x000441, 0x000421, 0x000421},
+ {0x000422, 0x000442, 0x000422, 0x000422},
+ {0x000423, 0x000443, 0x000423, 0x000423},
+ {0x000424, 0x000444, 0x000424, 0x000424},
+ {0x000425, 0x000445, 0x000425, 0x000425},
+ {0x000426, 0x000446, 0x000426, 0x000426},
+ {0x000427, 0x000447, 0x000427, 0x000427},
+ {0x000428, 0x000448, 0x000428, 0x000428},
+ {0x000429, 0x000449, 0x000429, 0x000429},
+ {0x00042a, 0x00044a, 0x00042a, 0x00042a},
+ {0x00042b, 0x00044b, 0x00042b, 0x00042b},
+ {0x00042c, 0x00044c, 0x00042c, 0x00042c},
+ {0x00042d, 0x00044d, 0x00042d, 0x00042d},
+ {0x00042e, 0x00044e, 0x00042e, 0x00042e},
+ {0x00042f, 0x00044f, 0x00042f, 0x00042f},
+ {0x000430, 0x000430, 0x000410, 0x000410},
+ {0x000431, 0x000431, 0x000411, 0x000411},
+ {0x000432, 0x000432, 0x000412, 0x000412},
+ {0x000433, 0x000433, 0x000413, 0x000413},
+ {0x000434, 0x000434, 0x000414, 0x000414},
+ {0x000435, 0x000435, 0x000415, 0x000415},
+ {0x000436, 0x000436, 0x000416, 0x000416},
+ {0x000437, 0x000437, 0x000417, 0x000417},
+ {0x000438, 0x000438, 0x000418, 0x000418},
+ {0x000439, 0x000439, 0x000419, 0x000419},
+ {0x00043a, 0x00043a, 0x00041a, 0x00041a},
+ {0x00043b, 0x00043b, 0x00041b, 0x00041b},
+ {0x00043c, 0x00043c, 0x00041c, 0x00041c},
+ {0x00043d, 0x00043d, 0x00041d, 0x00041d},
+ {0x00043e, 0x00043e, 0x00041e, 0x00041e},
+ {0x00043f, 0x00043f, 0x00041f, 0x00041f},
+ {0x000440, 0x000440, 0x000420, 0x000420},
+ {0x000441, 0x000441, 0x000421, 0x000421},
+ {0x000442, 0x000442, 0x000422, 0x000422},
+ {0x000443, 0x000443, 0x000423, 0x000423},
+ {0x000444, 0x000444, 0x000424, 0x000424},
+ {0x000445, 0x000445, 0x000425, 0x000425},
+ {0x000446, 0x000446, 0x000426, 0x000426},
+ {0x000447, 0x000447, 0x000427, 0x000427},
+ {0x000448, 0x000448, 0x000428, 0x000428},
+ {0x000449, 0x000449, 0x000429, 0x000429},
+ {0x00044a, 0x00044a, 0x00042a, 0x00042a},
+ {0x00044b, 0x00044b, 0x00042b, 0x00042b},
+ {0x00044c, 0x00044c, 0x00042c, 0x00042c},
+ {0x00044d, 0x00044d, 0x00042d, 0x00042d},
+ {0x00044e, 0x00044e, 0x00042e, 0x00042e},
+ {0x00044f, 0x00044f, 0x00042f, 0x00042f},
+ {0x000450, 0x000450, 0x000400, 0x000400},
+ {0x000451, 0x000451, 0x000401, 0x000401},
+ {0x000452, 0x000452, 0x000402, 0x000402},
+ {0x000453, 0x000453, 0x000403, 0x000403},
+ {0x000454, 0x000454, 0x000404, 0x000404},
+ {0x000455, 0x000455, 0x000405, 0x000405},
+ {0x000456, 0x000456, 0x000406, 0x000406},
+ {0x000457, 0x000457, 0x000407, 0x000407},
+ {0x000458, 0x000458, 0x000408, 0x000408},
+ {0x000459, 0x000459, 0x000409, 0x000409},
+ {0x00045a, 0x00045a, 0x00040a, 0x00040a},
+ {0x00045b, 0x00045b, 0x00040b, 0x00040b},
+ {0x00045c, 0x00045c, 0x00040c, 0x00040c},
+ {0x00045d, 0x00045d, 0x00040d, 0x00040d},
+ {0x00045e, 0x00045e, 0x00040e, 0x00040e},
+ {0x00045f, 0x00045f, 0x00040f, 0x00040f},
+ {0x000460, 0x000461, 0x000460, 0x000460},
+ {0x000461, 0x000461, 0x000460, 0x000460},
+ {0x000462, 0x000463, 0x000462, 0x000462},
+ {0x000463, 0x000463, 0x000462, 0x000462},
+ {0x000464, 0x000465, 0x000464, 0x000464},
+ {0x000465, 0x000465, 0x000464, 0x000464},
+ {0x000466, 0x000467, 0x000466, 0x000466},
+ {0x000467, 0x000467, 0x000466, 0x000466},
+ {0x000468, 0x000469, 0x000468, 0x000468},
+ {0x000469, 0x000469, 0x000468, 0x000468},
+ {0x00046a, 0x00046b, 0x00046a, 0x00046a},
+ {0x00046b, 0x00046b, 0x00046a, 0x00046a},
+ {0x00046c, 0x00046d, 0x00046c, 0x00046c},
+ {0x00046d, 0x00046d, 0x00046c, 0x00046c},
+ {0x00046e, 0x00046f, 0x00046e, 0x00046e},
+ {0x00046f, 0x00046f, 0x00046e, 0x00046e},
+ {0x000470, 0x000471, 0x000470, 0x000470},
+ {0x000471, 0x000471, 0x000470, 0x000470},
+ {0x000472, 0x000473, 0x000472, 0x000472},
+ {0x000473, 0x000473, 0x000472, 0x000472},
+ {0x000474, 0x000475, 0x000474, 0x000474},
+ {0x000475, 0x000475, 0x000474, 0x000474},
+ {0x000476, 0x000477, 0x000476, 0x000476},
+ {0x000477, 0x000477, 0x000476, 0x000476},
+ {0x000478, 0x000479, 0x000478, 0x000478},
+ {0x000479, 0x000479, 0x000478, 0x000478},
+ {0x00047a, 0x00047b, 0x00047a, 0x00047a},
+ {0x00047b, 0x00047b, 0x00047a, 0x00047a},
+ {0x00047c, 0x00047d, 0x00047c, 0x00047c},
+ {0x00047d, 0x00047d, 0x00047c, 0x00047c},
+ {0x00047e, 0x00047f, 0x00047e, 0x00047e},
+ {0x00047f, 0x00047f, 0x00047e, 0x00047e},
+ {0x000480, 0x000481, 0x000480, 0x000480},
+ {0x000481, 0x000481, 0x000480, 0x000480},
+ {0x00048a, 0x00048b, 0x00048a, 0x00048a},
+ {0x00048b, 0x00048b, 0x00048a, 0x00048a},
+ {0x00048c, 0x00048d, 0x00048c, 0x00048c},
+ {0x00048d, 0x00048d, 0x00048c, 0x00048c},
+ {0x00048e, 0x00048f, 0x00048e, 0x00048e},
+ {0x00048f, 0x00048f, 0x00048e, 0x00048e},
+ {0x000490, 0x000491, 0x000490, 0x000490},
+ {0x000491, 0x000491, 0x000490, 0x000490},
+ {0x000492, 0x000493, 0x000492, 0x000492},
+ {0x000493, 0x000493, 0x000492, 0x000492},
+ {0x000494, 0x000495, 0x000494, 0x000494},
+ {0x000495, 0x000495, 0x000494, 0x000494},
+ {0x000496, 0x000497, 0x000496, 0x000496},
+ {0x000497, 0x000497, 0x000496, 0x000496},
+ {0x000498, 0x000499, 0x000498, 0x000498},
+ {0x000499, 0x000499, 0x000498, 0x000498},
+ {0x00049a, 0x00049b, 0x00049a, 0x00049a},
+ {0x00049b, 0x00049b, 0x00049a, 0x00049a},
+ {0x00049c, 0x00049d, 0x00049c, 0x00049c},
+ {0x00049d, 0x00049d, 0x00049c, 0x00049c},
+ {0x00049e, 0x00049f, 0x00049e, 0x00049e},
+ {0x00049f, 0x00049f, 0x00049e, 0x00049e},
+ {0x0004a0, 0x0004a1, 0x0004a0, 0x0004a0},
+ {0x0004a1, 0x0004a1, 0x0004a0, 0x0004a0},
+ {0x0004a2, 0x0004a3, 0x0004a2, 0x0004a2},
+ {0x0004a3, 0x0004a3, 0x0004a2, 0x0004a2},
+ {0x0004a4, 0x0004a5, 0x0004a4, 0x0004a4},
+ {0x0004a5, 0x0004a5, 0x0004a4, 0x0004a4},
+ {0x0004a6, 0x0004a7, 0x0004a6, 0x0004a6},
+ {0x0004a7, 0x0004a7, 0x0004a6, 0x0004a6},
+ {0x0004a8, 0x0004a9, 0x0004a8, 0x0004a8},
+ {0x0004a9, 0x0004a9, 0x0004a8, 0x0004a8},
+ {0x0004aa, 0x0004ab, 0x0004aa, 0x0004aa},
+ {0x0004ab, 0x0004ab, 0x0004aa, 0x0004aa},
+ {0x0004ac, 0x0004ad, 0x0004ac, 0x0004ac},
+ {0x0004ad, 0x0004ad, 0x0004ac, 0x0004ac},
+ {0x0004ae, 0x0004af, 0x0004ae, 0x0004ae},
+ {0x0004af, 0x0004af, 0x0004ae, 0x0004ae},
+ {0x0004b0, 0x0004b1, 0x0004b0, 0x0004b0},
+ {0x0004b1, 0x0004b1, 0x0004b0, 0x0004b0},
+ {0x0004b2, 0x0004b3, 0x0004b2, 0x0004b2},
+ {0x0004b3, 0x0004b3, 0x0004b2, 0x0004b2},
+ {0x0004b4, 0x0004b5, 0x0004b4, 0x0004b4},
+ {0x0004b5, 0x0004b5, 0x0004b4, 0x0004b4},
+ {0x0004b6, 0x0004b7, 0x0004b6, 0x0004b6},
+ {0x0004b7, 0x0004b7, 0x0004b6, 0x0004b6},
+ {0x0004b8, 0x0004b9, 0x0004b8, 0x0004b8},
+ {0x0004b9, 0x0004b9, 0x0004b8, 0x0004b8},
+ {0x0004ba, 0x0004bb, 0x0004ba, 0x0004ba},
+ {0x0004bb, 0x0004bb, 0x0004ba, 0x0004ba},
+ {0x0004bc, 0x0004bd, 0x0004bc, 0x0004bc},
+ {0x0004bd, 0x0004bd, 0x0004bc, 0x0004bc},
+ {0x0004be, 0x0004bf, 0x0004be, 0x0004be},
+ {0x0004bf, 0x0004bf, 0x0004be, 0x0004be},
+ {0x0004c0, 0x0004cf, 0x0004c0, 0x0004c0},
+ {0x0004c1, 0x0004c2, 0x0004c1, 0x0004c1},
+ {0x0004c2, 0x0004c2, 0x0004c1, 0x0004c1},
+ {0x0004c3, 0x0004c4, 0x0004c3, 0x0004c3},
+ {0x0004c4, 0x0004c4, 0x0004c3, 0x0004c3},
+ {0x0004c5, 0x0004c6, 0x0004c5, 0x0004c5},
+ {0x0004c6, 0x0004c6, 0x0004c5, 0x0004c5},
+ {0x0004c7, 0x0004c8, 0x0004c7, 0x0004c7},
+ {0x0004c8, 0x0004c8, 0x0004c7, 0x0004c7},
+ {0x0004c9, 0x0004ca, 0x0004c9, 0x0004c9},
+ {0x0004ca, 0x0004ca, 0x0004c9, 0x0004c9},
+ {0x0004cb, 0x0004cc, 0x0004cb, 0x0004cb},
+ {0x0004cc, 0x0004cc, 0x0004cb, 0x0004cb},
+ {0x0004cd, 0x0004ce, 0x0004cd, 0x0004cd},
+ {0x0004ce, 0x0004ce, 0x0004cd, 0x0004cd},
+ {0x0004cf, 0x0004cf, 0x0004c0, 0x0004c0},
+ {0x0004d0, 0x0004d1, 0x0004d0, 0x0004d0},
+ {0x0004d1, 0x0004d1, 0x0004d0, 0x0004d0},
+ {0x0004d2, 0x0004d3, 0x0004d2, 0x0004d2},
+ {0x0004d3, 0x0004d3, 0x0004d2, 0x0004d2},
+ {0x0004d4, 0x0004d5, 0x0004d4, 0x0004d4},
+ {0x0004d5, 0x0004d5, 0x0004d4, 0x0004d4},
+ {0x0004d6, 0x0004d7, 0x0004d6, 0x0004d6},
+ {0x0004d7, 0x0004d7, 0x0004d6, 0x0004d6},
+ {0x0004d8, 0x0004d9, 0x0004d8, 0x0004d8},
+ {0x0004d9, 0x0004d9, 0x0004d8, 0x0004d8},
+ {0x0004da, 0x0004db, 0x0004da, 0x0004da},
+ {0x0004db, 0x0004db, 0x0004da, 0x0004da},
+ {0x0004dc, 0x0004dd, 0x0004dc, 0x0004dc},
+ {0x0004dd, 0x0004dd, 0x0004dc, 0x0004dc},
+ {0x0004de, 0x0004df, 0x0004de, 0x0004de},
+ {0x0004df, 0x0004df, 0x0004de, 0x0004de},
+ {0x0004e0, 0x0004e1, 0x0004e0, 0x0004e0},
+ {0x0004e1, 0x0004e1, 0x0004e0, 0x0004e0},
+ {0x0004e2, 0x0004e3, 0x0004e2, 0x0004e2},
+ {0x0004e3, 0x0004e3, 0x0004e2, 0x0004e2},
+ {0x0004e4, 0x0004e5, 0x0004e4, 0x0004e4},
+ {0x0004e5, 0x0004e5, 0x0004e4, 0x0004e4},
+ {0x0004e6, 0x0004e7, 0x0004e6, 0x0004e6},
+ {0x0004e7, 0x0004e7, 0x0004e6, 0x0004e6},
+ {0x0004e8, 0x0004e9, 0x0004e8, 0x0004e8},
+ {0x0004e9, 0x0004e9, 0x0004e8, 0x0004e8},
+ {0x0004ea, 0x0004eb, 0x0004ea, 0x0004ea},
+ {0x0004eb, 0x0004eb, 0x0004ea, 0x0004ea},
+ {0x0004ec, 0x0004ed, 0x0004ec, 0x0004ec},
+ {0x0004ed, 0x0004ed, 0x0004ec, 0x0004ec},
+ {0x0004ee, 0x0004ef, 0x0004ee, 0x0004ee},
+ {0x0004ef, 0x0004ef, 0x0004ee, 0x0004ee},
+ {0x0004f0, 0x0004f1, 0x0004f0, 0x0004f0},
+ {0x0004f1, 0x0004f1, 0x0004f0, 0x0004f0},
+ {0x0004f2, 0x0004f3, 0x0004f2, 0x0004f2},
+ {0x0004f3, 0x0004f3, 0x0004f2, 0x0004f2},
+ {0x0004f4, 0x0004f5, 0x0004f4, 0x0004f4},
+ {0x0004f5, 0x0004f5, 0x0004f4, 0x0004f4},
+ {0x0004f6, 0x0004f7, 0x0004f6, 0x0004f6},
+ {0x0004f7, 0x0004f7, 0x0004f6, 0x0004f6},
+ {0x0004f8, 0x0004f9, 0x0004f8, 0x0004f8},
+ {0x0004f9, 0x0004f9, 0x0004f8, 0x0004f8},
+ {0x0004fa, 0x0004fb, 0x0004fa, 0x0004fa},
+ {0x0004fb, 0x0004fb, 0x0004fa, 0x0004fa},
+ {0x0004fc, 0x0004fd, 0x0004fc, 0x0004fc},
+ {0x0004fd, 0x0004fd, 0x0004fc, 0x0004fc},
+ {0x0004fe, 0x0004ff, 0x0004fe, 0x0004fe},
+ {0x0004ff, 0x0004ff, 0x0004fe, 0x0004fe},
+ {0x000500, 0x000501, 0x000500, 0x000500},
+ {0x000501, 0x000501, 0x000500, 0x000500},
+ {0x000502, 0x000503, 0x000502, 0x000502},
+ {0x000503, 0x000503, 0x000502, 0x000502},
+ {0x000504, 0x000505, 0x000504, 0x000504},
+ {0x000505, 0x000505, 0x000504, 0x000504},
+ {0x000506, 0x000507, 0x000506, 0x000506},
+ {0x000507, 0x000507, 0x000506, 0x000506},
+ {0x000508, 0x000509, 0x000508, 0x000508},
+ {0x000509, 0x000509, 0x000508, 0x000508},
+ {0x00050a, 0x00050b, 0x00050a, 0x00050a},
+ {0x00050b, 0x00050b, 0x00050a, 0x00050a},
+ {0x00050c, 0x00050d, 0x00050c, 0x00050c},
+ {0x00050d, 0x00050d, 0x00050c, 0x00050c},
+ {0x00050e, 0x00050f, 0x00050e, 0x00050e},
+ {0x00050f, 0x00050f, 0x00050e, 0x00050e},
+ {0x000510, 0x000511, 0x000510, 0x000510},
+ {0x000511, 0x000511, 0x000510, 0x000510},
+ {0x000512, 0x000513, 0x000512, 0x000512},
+ {0x000513, 0x000513, 0x000512, 0x000512},
+ {0x000514, 0x000515, 0x000514, 0x000514},
+ {0x000515, 0x000515, 0x000514, 0x000514},
+ {0x000516, 0x000517, 0x000516, 0x000516},
+ {0x000517, 0x000517, 0x000516, 0x000516},
+ {0x000518, 0x000519, 0x000518, 0x000518},
+ {0x000519, 0x000519, 0x000518, 0x000518},
+ {0x00051a, 0x00051b, 0x00051a, 0x00051a},
+ {0x00051b, 0x00051b, 0x00051a, 0x00051a},
+ {0x00051c, 0x00051d, 0x00051c, 0x00051c},
+ {0x00051d, 0x00051d, 0x00051c, 0x00051c},
+ {0x00051e, 0x00051f, 0x00051e, 0x00051e},
+ {0x00051f, 0x00051f, 0x00051e, 0x00051e},
+ {0x000520, 0x000521, 0x000520, 0x000520},
+ {0x000521, 0x000521, 0x000520, 0x000520},
+ {0x000522, 0x000523, 0x000522, 0x000522},
+ {0x000523, 0x000523, 0x000522, 0x000522},
+ {0x000524, 0x000525, 0x000524, 0x000524},
+ {0x000525, 0x000525, 0x000524, 0x000524},
+ {0x000526, 0x000527, 0x000526, 0x000526},
+ {0x000527, 0x000527, 0x000526, 0x000526},
+ {0x000528, 0x000529, 0x000528, 0x000528},
+ {0x000529, 0x000529, 0x000528, 0x000528},
+ {0x00052a, 0x00052b, 0x00052a, 0x00052a},
+ {0x00052b, 0x00052b, 0x00052a, 0x00052a},
+ {0x00052c, 0x00052d, 0x00052c, 0x00052c},
+ {0x00052d, 0x00052d, 0x00052c, 0x00052c},
+ {0x00052e, 0x00052f, 0x00052e, 0x00052e},
+ {0x00052f, 0x00052f, 0x00052e, 0x00052e},
+ {0x000531, 0x000561, 0x000531, 0x000531},
+ {0x000532, 0x000562, 0x000532, 0x000532},
+ {0x000533, 0x000563, 0x000533, 0x000533},
+ {0x000534, 0x000564, 0x000534, 0x000534},
+ {0x000535, 0x000565, 0x000535, 0x000535},
+ {0x000536, 0x000566, 0x000536, 0x000536},
+ {0x000537, 0x000567, 0x000537, 0x000537},
+ {0x000538, 0x000568, 0x000538, 0x000538},
+ {0x000539, 0x000569, 0x000539, 0x000539},
+ {0x00053a, 0x00056a, 0x00053a, 0x00053a},
+ {0x00053b, 0x00056b, 0x00053b, 0x00053b},
+ {0x00053c, 0x00056c, 0x00053c, 0x00053c},
+ {0x00053d, 0x00056d, 0x00053d, 0x00053d},
+ {0x00053e, 0x00056e, 0x00053e, 0x00053e},
+ {0x00053f, 0x00056f, 0x00053f, 0x00053f},
+ {0x000540, 0x000570, 0x000540, 0x000540},
+ {0x000541, 0x000571, 0x000541, 0x000541},
+ {0x000542, 0x000572, 0x000542, 0x000542},
+ {0x000543, 0x000573, 0x000543, 0x000543},
+ {0x000544, 0x000574, 0x000544, 0x000544},
+ {0x000545, 0x000575, 0x000545, 0x000545},
+ {0x000546, 0x000576, 0x000546, 0x000546},
+ {0x000547, 0x000577, 0x000547, 0x000547},
+ {0x000548, 0x000578, 0x000548, 0x000548},
+ {0x000549, 0x000579, 0x000549, 0x000549},
+ {0x00054a, 0x00057a, 0x00054a, 0x00054a},
+ {0x00054b, 0x00057b, 0x00054b, 0x00054b},
+ {0x00054c, 0x00057c, 0x00054c, 0x00054c},
+ {0x00054d, 0x00057d, 0x00054d, 0x00054d},
+ {0x00054e, 0x00057e, 0x00054e, 0x00054e},
+ {0x00054f, 0x00057f, 0x00054f, 0x00054f},
+ {0x000550, 0x000580, 0x000550, 0x000550},
+ {0x000551, 0x000581, 0x000551, 0x000551},
+ {0x000552, 0x000582, 0x000552, 0x000552},
+ {0x000553, 0x000583, 0x000553, 0x000553},
+ {0x000554, 0x000584, 0x000554, 0x000554},
+ {0x000555, 0x000585, 0x000555, 0x000555},
+ {0x000556, 0x000586, 0x000556, 0x000556},
+ {0x000561, 0x000561, 0x000531, 0x000531},
+ {0x000562, 0x000562, 0x000532, 0x000532},
+ {0x000563, 0x000563, 0x000533, 0x000533},
+ {0x000564, 0x000564, 0x000534, 0x000534},
+ {0x000565, 0x000565, 0x000535, 0x000535},
+ {0x000566, 0x000566, 0x000536, 0x000536},
+ {0x000567, 0x000567, 0x000537, 0x000537},
+ {0x000568, 0x000568, 0x000538, 0x000538},
+ {0x000569, 0x000569, 0x000539, 0x000539},
+ {0x00056a, 0x00056a, 0x00053a, 0x00053a},
+ {0x00056b, 0x00056b, 0x00053b, 0x00053b},
+ {0x00056c, 0x00056c, 0x00053c, 0x00053c},
+ {0x00056d, 0x00056d, 0x00053d, 0x00053d},
+ {0x00056e, 0x00056e, 0x00053e, 0x00053e},
+ {0x00056f, 0x00056f, 0x00053f, 0x00053f},
+ {0x000570, 0x000570, 0x000540, 0x000540},
+ {0x000571, 0x000571, 0x000541, 0x000541},
+ {0x000572, 0x000572, 0x000542, 0x000542},
+ {0x000573, 0x000573, 0x000543, 0x000543},
+ {0x000574, 0x000574, 0x000544, 0x000544},
+ {0x000575, 0x000575, 0x000545, 0x000545},
+ {0x000576, 0x000576, 0x000546, 0x000546},
+ {0x000577, 0x000577, 0x000547, 0x000547},
+ {0x000578, 0x000578, 0x000548, 0x000548},
+ {0x000579, 0x000579, 0x000549, 0x000549},
+ {0x00057a, 0x00057a, 0x00054a, 0x00054a},
+ {0x00057b, 0x00057b, 0x00054b, 0x00054b},
+ {0x00057c, 0x00057c, 0x00054c, 0x00054c},
+ {0x00057d, 0x00057d, 0x00054d, 0x00054d},
+ {0x00057e, 0x00057e, 0x00054e, 0x00054e},
+ {0x00057f, 0x00057f, 0x00054f, 0x00054f},
+ {0x000580, 0x000580, 0x000550, 0x000550},
+ {0x000581, 0x000581, 0x000551, 0x000551},
+ {0x000582, 0x000582, 0x000552, 0x000552},
+ {0x000583, 0x000583, 0x000553, 0x000553},
+ {0x000584, 0x000584, 0x000554, 0x000554},
+ {0x000585, 0x000585, 0x000555, 0x000555},
+ {0x000586, 0x000586, 0x000556, 0x000556},
+ {0x0010a0, 0x002d00, 0x0010a0, 0x0010a0},
+ {0x0010a1, 0x002d01, 0x0010a1, 0x0010a1},
+ {0x0010a2, 0x002d02, 0x0010a2, 0x0010a2},
+ {0x0010a3, 0x002d03, 0x0010a3, 0x0010a3},
+ {0x0010a4, 0x002d04, 0x0010a4, 0x0010a4},
+ {0x0010a5, 0x002d05, 0x0010a5, 0x0010a5},
+ {0x0010a6, 0x002d06, 0x0010a6, 0x0010a6},
+ {0x0010a7, 0x002d07, 0x0010a7, 0x0010a7},
+ {0x0010a8, 0x002d08, 0x0010a8, 0x0010a8},
+ {0x0010a9, 0x002d09, 0x0010a9, 0x0010a9},
+ {0x0010aa, 0x002d0a, 0x0010aa, 0x0010aa},
+ {0x0010ab, 0x002d0b, 0x0010ab, 0x0010ab},
+ {0x0010ac, 0x002d0c, 0x0010ac, 0x0010ac},
+ {0x0010ad, 0x002d0d, 0x0010ad, 0x0010ad},
+ {0x0010ae, 0x002d0e, 0x0010ae, 0x0010ae},
+ {0x0010af, 0x002d0f, 0x0010af, 0x0010af},
+ {0x0010b0, 0x002d10, 0x0010b0, 0x0010b0},
+ {0x0010b1, 0x002d11, 0x0010b1, 0x0010b1},
+ {0x0010b2, 0x002d12, 0x0010b2, 0x0010b2},
+ {0x0010b3, 0x002d13, 0x0010b3, 0x0010b3},
+ {0x0010b4, 0x002d14, 0x0010b4, 0x0010b4},
+ {0x0010b5, 0x002d15, 0x0010b5, 0x0010b5},
+ {0x0010b6, 0x002d16, 0x0010b6, 0x0010b6},
+ {0x0010b7, 0x002d17, 0x0010b7, 0x0010b7},
+ {0x0010b8, 0x002d18, 0x0010b8, 0x0010b8},
+ {0x0010b9, 0x002d19, 0x0010b9, 0x0010b9},
+ {0x0010ba, 0x002d1a, 0x0010ba, 0x0010ba},
+ {0x0010bb, 0x002d1b, 0x0010bb, 0x0010bb},
+ {0x0010bc, 0x002d1c, 0x0010bc, 0x0010bc},
+ {0x0010bd, 0x002d1d, 0x0010bd, 0x0010bd},
+ {0x0010be, 0x002d1e, 0x0010be, 0x0010be},
+ {0x0010bf, 0x002d1f, 0x0010bf, 0x0010bf},
+ {0x0010c0, 0x002d20, 0x0010c0, 0x0010c0},
+ {0x0010c1, 0x002d21, 0x0010c1, 0x0010c1},
+ {0x0010c2, 0x002d22, 0x0010c2, 0x0010c2},
+ {0x0010c3, 0x002d23, 0x0010c3, 0x0010c3},
+ {0x0010c4, 0x002d24, 0x0010c4, 0x0010c4},
+ {0x0010c5, 0x002d25, 0x0010c5, 0x0010c5},
+ {0x0010c7, 0x002d27, 0x0010c7, 0x0010c7},
+ {0x0010cd, 0x002d2d, 0x0010cd, 0x0010cd},
+ {0x0010d0, 0x0010d0, 0x0010d0, 0x001c90},
+ {0x0010d1, 0x0010d1, 0x0010d1, 0x001c91},
+ {0x0010d2, 0x0010d2, 0x0010d2, 0x001c92},
+ {0x0010d3, 0x0010d3, 0x0010d3, 0x001c93},
+ {0x0010d4, 0x0010d4, 0x0010d4, 0x001c94},
+ {0x0010d5, 0x0010d5, 0x0010d5, 0x001c95},
+ {0x0010d6, 0x0010d6, 0x0010d6, 0x001c96},
+ {0x0010d7, 0x0010d7, 0x0010d7, 0x001c97},
+ {0x0010d8, 0x0010d8, 0x0010d8, 0x001c98},
+ {0x0010d9, 0x0010d9, 0x0010d9, 0x001c99},
+ {0x0010da, 0x0010da, 0x0010da, 0x001c9a},
+ {0x0010db, 0x0010db, 0x0010db, 0x001c9b},
+ {0x0010dc, 0x0010dc, 0x0010dc, 0x001c9c},
+ {0x0010dd, 0x0010dd, 0x0010dd, 0x001c9d},
+ {0x0010de, 0x0010de, 0x0010de, 0x001c9e},
+ {0x0010df, 0x0010df, 0x0010df, 0x001c9f},
+ {0x0010e0, 0x0010e0, 0x0010e0, 0x001ca0},
+ {0x0010e1, 0x0010e1, 0x0010e1, 0x001ca1},
+ {0x0010e2, 0x0010e2, 0x0010e2, 0x001ca2},
+ {0x0010e3, 0x0010e3, 0x0010e3, 0x001ca3},
+ {0x0010e4, 0x0010e4, 0x0010e4, 0x001ca4},
+ {0x0010e5, 0x0010e5, 0x0010e5, 0x001ca5},
+ {0x0010e6, 0x0010e6, 0x0010e6, 0x001ca6},
+ {0x0010e7, 0x0010e7, 0x0010e7, 0x001ca7},
+ {0x0010e8, 0x0010e8, 0x0010e8, 0x001ca8},
+ {0x0010e9, 0x0010e9, 0x0010e9, 0x001ca9},
+ {0x0010ea, 0x0010ea, 0x0010ea, 0x001caa},
+ {0x0010eb, 0x0010eb, 0x0010eb, 0x001cab},
+ {0x0010ec, 0x0010ec, 0x0010ec, 0x001cac},
+ {0x0010ed, 0x0010ed, 0x0010ed, 0x001cad},
+ {0x0010ee, 0x0010ee, 0x0010ee, 0x001cae},
+ {0x0010ef, 0x0010ef, 0x0010ef, 0x001caf},
+ {0x0010f0, 0x0010f0, 0x0010f0, 0x001cb0},
+ {0x0010f1, 0x0010f1, 0x0010f1, 0x001cb1},
+ {0x0010f2, 0x0010f2, 0x0010f2, 0x001cb2},
+ {0x0010f3, 0x0010f3, 0x0010f3, 0x001cb3},
+ {0x0010f4, 0x0010f4, 0x0010f4, 0x001cb4},
+ {0x0010f5, 0x0010f5, 0x0010f5, 0x001cb5},
+ {0x0010f6, 0x0010f6, 0x0010f6, 0x001cb6},
+ {0x0010f7, 0x0010f7, 0x0010f7, 0x001cb7},
+ {0x0010f8, 0x0010f8, 0x0010f8, 0x001cb8},
+ {0x0010f9, 0x0010f9, 0x0010f9, 0x001cb9},
+ {0x0010fa, 0x0010fa, 0x0010fa, 0x001cba},
+ {0x0010fd, 0x0010fd, 0x0010fd, 0x001cbd},
+ {0x0010fe, 0x0010fe, 0x0010fe, 0x001cbe},
+ {0x0010ff, 0x0010ff, 0x0010ff, 0x001cbf},
+ {0x0013a0, 0x00ab70, 0x0013a0, 0x0013a0},
+ {0x0013a1, 0x00ab71, 0x0013a1, 0x0013a1},
+ {0x0013a2, 0x00ab72, 0x0013a2, 0x0013a2},
+ {0x0013a3, 0x00ab73, 0x0013a3, 0x0013a3},
+ {0x0013a4, 0x00ab74, 0x0013a4, 0x0013a4},
+ {0x0013a5, 0x00ab75, 0x0013a5, 0x0013a5},
+ {0x0013a6, 0x00ab76, 0x0013a6, 0x0013a6},
+ {0x0013a7, 0x00ab77, 0x0013a7, 0x0013a7},
+ {0x0013a8, 0x00ab78, 0x0013a8, 0x0013a8},
+ {0x0013a9, 0x00ab79, 0x0013a9, 0x0013a9},
+ {0x0013aa, 0x00ab7a, 0x0013aa, 0x0013aa},
+ {0x0013ab, 0x00ab7b, 0x0013ab, 0x0013ab},
+ {0x0013ac, 0x00ab7c, 0x0013ac, 0x0013ac},
+ {0x0013ad, 0x00ab7d, 0x0013ad, 0x0013ad},
+ {0x0013ae, 0x00ab7e, 0x0013ae, 0x0013ae},
+ {0x0013af, 0x00ab7f, 0x0013af, 0x0013af},
+ {0x0013b0, 0x00ab80, 0x0013b0, 0x0013b0},
+ {0x0013b1, 0x00ab81, 0x0013b1, 0x0013b1},
+ {0x0013b2, 0x00ab82, 0x0013b2, 0x0013b2},
+ {0x0013b3, 0x00ab83, 0x0013b3, 0x0013b3},
+ {0x0013b4, 0x00ab84, 0x0013b4, 0x0013b4},
+ {0x0013b5, 0x00ab85, 0x0013b5, 0x0013b5},
+ {0x0013b6, 0x00ab86, 0x0013b6, 0x0013b6},
+ {0x0013b7, 0x00ab87, 0x0013b7, 0x0013b7},
+ {0x0013b8, 0x00ab88, 0x0013b8, 0x0013b8},
+ {0x0013b9, 0x00ab89, 0x0013b9, 0x0013b9},
+ {0x0013ba, 0x00ab8a, 0x0013ba, 0x0013ba},
+ {0x0013bb, 0x00ab8b, 0x0013bb, 0x0013bb},
+ {0x0013bc, 0x00ab8c, 0x0013bc, 0x0013bc},
+ {0x0013bd, 0x00ab8d, 0x0013bd, 0x0013bd},
+ {0x0013be, 0x00ab8e, 0x0013be, 0x0013be},
+ {0x0013bf, 0x00ab8f, 0x0013bf, 0x0013bf},
+ {0x0013c0, 0x00ab90, 0x0013c0, 0x0013c0},
+ {0x0013c1, 0x00ab91, 0x0013c1, 0x0013c1},
+ {0x0013c2, 0x00ab92, 0x0013c2, 0x0013c2},
+ {0x0013c3, 0x00ab93, 0x0013c3, 0x0013c3},
+ {0x0013c4, 0x00ab94, 0x0013c4, 0x0013c4},
+ {0x0013c5, 0x00ab95, 0x0013c5, 0x0013c5},
+ {0x0013c6, 0x00ab96, 0x0013c6, 0x0013c6},
+ {0x0013c7, 0x00ab97, 0x0013c7, 0x0013c7},
+ {0x0013c8, 0x00ab98, 0x0013c8, 0x0013c8},
+ {0x0013c9, 0x00ab99, 0x0013c9, 0x0013c9},
+ {0x0013ca, 0x00ab9a, 0x0013ca, 0x0013ca},
+ {0x0013cb, 0x00ab9b, 0x0013cb, 0x0013cb},
+ {0x0013cc, 0x00ab9c, 0x0013cc, 0x0013cc},
+ {0x0013cd, 0x00ab9d, 0x0013cd, 0x0013cd},
+ {0x0013ce, 0x00ab9e, 0x0013ce, 0x0013ce},
+ {0x0013cf, 0x00ab9f, 0x0013cf, 0x0013cf},
+ {0x0013d0, 0x00aba0, 0x0013d0, 0x0013d0},
+ {0x0013d1, 0x00aba1, 0x0013d1, 0x0013d1},
+ {0x0013d2, 0x00aba2, 0x0013d2, 0x0013d2},
+ {0x0013d3, 0x00aba3, 0x0013d3, 0x0013d3},
+ {0x0013d4, 0x00aba4, 0x0013d4, 0x0013d4},
+ {0x0013d5, 0x00aba5, 0x0013d5, 0x0013d5},
+ {0x0013d6, 0x00aba6, 0x0013d6, 0x0013d6},
+ {0x0013d7, 0x00aba7, 0x0013d7, 0x0013d7},
+ {0x0013d8, 0x00aba8, 0x0013d8, 0x0013d8},
+ {0x0013d9, 0x00aba9, 0x0013d9, 0x0013d9},
+ {0x0013da, 0x00abaa, 0x0013da, 0x0013da},
+ {0x0013db, 0x00abab, 0x0013db, 0x0013db},
+ {0x0013dc, 0x00abac, 0x0013dc, 0x0013dc},
+ {0x0013dd, 0x00abad, 0x0013dd, 0x0013dd},
+ {0x0013de, 0x00abae, 0x0013de, 0x0013de},
+ {0x0013df, 0x00abaf, 0x0013df, 0x0013df},
+ {0x0013e0, 0x00abb0, 0x0013e0, 0x0013e0},
+ {0x0013e1, 0x00abb1, 0x0013e1, 0x0013e1},
+ {0x0013e2, 0x00abb2, 0x0013e2, 0x0013e2},
+ {0x0013e3, 0x00abb3, 0x0013e3, 0x0013e3},
+ {0x0013e4, 0x00abb4, 0x0013e4, 0x0013e4},
+ {0x0013e5, 0x00abb5, 0x0013e5, 0x0013e5},
+ {0x0013e6, 0x00abb6, 0x0013e6, 0x0013e6},
+ {0x0013e7, 0x00abb7, 0x0013e7, 0x0013e7},
+ {0x0013e8, 0x00abb8, 0x0013e8, 0x0013e8},
+ {0x0013e9, 0x00abb9, 0x0013e9, 0x0013e9},
+ {0x0013ea, 0x00abba, 0x0013ea, 0x0013ea},
+ {0x0013eb, 0x00abbb, 0x0013eb, 0x0013eb},
+ {0x0013ec, 0x00abbc, 0x0013ec, 0x0013ec},
+ {0x0013ed, 0x00abbd, 0x0013ed, 0x0013ed},
+ {0x0013ee, 0x00abbe, 0x0013ee, 0x0013ee},
+ {0x0013ef, 0x00abbf, 0x0013ef, 0x0013ef},
+ {0x0013f0, 0x0013f8, 0x0013f0, 0x0013f0},
+ {0x0013f1, 0x0013f9, 0x0013f1, 0x0013f1},
+ {0x0013f2, 0x0013fa, 0x0013f2, 0x0013f2},
+ {0x0013f3, 0x0013fb, 0x0013f3, 0x0013f3},
+ {0x0013f4, 0x0013fc, 0x0013f4, 0x0013f4},
+ {0x0013f5, 0x0013fd, 0x0013f5, 0x0013f5},
+ {0x0013f8, 0x0013f8, 0x0013f0, 0x0013f0},
+ {0x0013f9, 0x0013f9, 0x0013f1, 0x0013f1},
+ {0x0013fa, 0x0013fa, 0x0013f2, 0x0013f2},
+ {0x0013fb, 0x0013fb, 0x0013f3, 0x0013f3},
+ {0x0013fc, 0x0013fc, 0x0013f4, 0x0013f4},
+ {0x0013fd, 0x0013fd, 0x0013f5, 0x0013f5},
+ {0x001c80, 0x001c80, 0x000412, 0x000412},
+ {0x001c81, 0x001c81, 0x000414, 0x000414},
+ {0x001c82, 0x001c82, 0x00041e, 0x00041e},
+ {0x001c83, 0x001c83, 0x000421, 0x000421},
+ {0x001c84, 0x001c84, 0x000422, 0x000422},
+ {0x001c85, 0x001c85, 0x000422, 0x000422},
+ {0x001c86, 0x001c86, 0x00042a, 0x00042a},
+ {0x001c87, 0x001c87, 0x000462, 0x000462},
+ {0x001c88, 0x001c88, 0x00a64a, 0x00a64a},
+ {0x001c90, 0x0010d0, 0x001c90, 0x001c90},
+ {0x001c91, 0x0010d1, 0x001c91, 0x001c91},
+ {0x001c92, 0x0010d2, 0x001c92, 0x001c92},
+ {0x001c93, 0x0010d3, 0x001c93, 0x001c93},
+ {0x001c94, 0x0010d4, 0x001c94, 0x001c94},
+ {0x001c95, 0x0010d5, 0x001c95, 0x001c95},
+ {0x001c96, 0x0010d6, 0x001c96, 0x001c96},
+ {0x001c97, 0x0010d7, 0x001c97, 0x001c97},
+ {0x001c98, 0x0010d8, 0x001c98, 0x001c98},
+ {0x001c99, 0x0010d9, 0x001c99, 0x001c99},
+ {0x001c9a, 0x0010da, 0x001c9a, 0x001c9a},
+ {0x001c9b, 0x0010db, 0x001c9b, 0x001c9b},
+ {0x001c9c, 0x0010dc, 0x001c9c, 0x001c9c},
+ {0x001c9d, 0x0010dd, 0x001c9d, 0x001c9d},
+ {0x001c9e, 0x0010de, 0x001c9e, 0x001c9e},
+ {0x001c9f, 0x0010df, 0x001c9f, 0x001c9f},
+ {0x001ca0, 0x0010e0, 0x001ca0, 0x001ca0},
+ {0x001ca1, 0x0010e1, 0x001ca1, 0x001ca1},
+ {0x001ca2, 0x0010e2, 0x001ca2, 0x001ca2},
+ {0x001ca3, 0x0010e3, 0x001ca3, 0x001ca3},
+ {0x001ca4, 0x0010e4, 0x001ca4, 0x001ca4},
+ {0x001ca5, 0x0010e5, 0x001ca5, 0x001ca5},
+ {0x001ca6, 0x0010e6, 0x001ca6, 0x001ca6},
+ {0x001ca7, 0x0010e7, 0x001ca7, 0x001ca7},
+ {0x001ca8, 0x0010e8, 0x001ca8, 0x001ca8},
+ {0x001ca9, 0x0010e9, 0x001ca9, 0x001ca9},
+ {0x001caa, 0x0010ea, 0x001caa, 0x001caa},
+ {0x001cab, 0x0010eb, 0x001cab, 0x001cab},
+ {0x001cac, 0x0010ec, 0x001cac, 0x001cac},
+ {0x001cad, 0x0010ed, 0x001cad, 0x001cad},
+ {0x001cae, 0x0010ee, 0x001cae, 0x001cae},
+ {0x001caf, 0x0010ef, 0x001caf, 0x001caf},
+ {0x001cb0, 0x0010f0, 0x001cb0, 0x001cb0},
+ {0x001cb1, 0x0010f1, 0x001cb1, 0x001cb1},
+ {0x001cb2, 0x0010f2, 0x001cb2, 0x001cb2},
+ {0x001cb3, 0x0010f3, 0x001cb3, 0x001cb3},
+ {0x001cb4, 0x0010f4, 0x001cb4, 0x001cb4},
+ {0x001cb5, 0x0010f5, 0x001cb5, 0x001cb5},
+ {0x001cb6, 0x0010f6, 0x001cb6, 0x001cb6},
+ {0x001cb7, 0x0010f7, 0x001cb7, 0x001cb7},
+ {0x001cb8, 0x0010f8, 0x001cb8, 0x001cb8},
+ {0x001cb9, 0x0010f9, 0x001cb9, 0x001cb9},
+ {0x001cba, 0x0010fa, 0x001cba, 0x001cba},
+ {0x001cbd, 0x0010fd, 0x001cbd, 0x001cbd},
+ {0x001cbe, 0x0010fe, 0x001cbe, 0x001cbe},
+ {0x001cbf, 0x0010ff, 0x001cbf, 0x001cbf},
+ {0x001d79, 0x001d79, 0x00a77d, 0x00a77d},
+ {0x001d7d, 0x001d7d, 0x002c63, 0x002c63},
+ {0x001d8e, 0x001d8e, 0x00a7c6, 0x00a7c6},
+ {0x001e00, 0x001e01, 0x001e00, 0x001e00},
+ {0x001e01, 0x001e01, 0x001e00, 0x001e00},
+ {0x001e02, 0x001e03, 0x001e02, 0x001e02},
+ {0x001e03, 0x001e03, 0x001e02, 0x001e02},
+ {0x001e04, 0x001e05, 0x001e04, 0x001e04},
+ {0x001e05, 0x001e05, 0x001e04, 0x001e04},
+ {0x001e06, 0x001e07, 0x001e06, 0x001e06},
+ {0x001e07, 0x001e07, 0x001e06, 0x001e06},
+ {0x001e08, 0x001e09, 0x001e08, 0x001e08},
+ {0x001e09, 0x001e09, 0x001e08, 0x001e08},
+ {0x001e0a, 0x001e0b, 0x001e0a, 0x001e0a},
+ {0x001e0b, 0x001e0b, 0x001e0a, 0x001e0a},
+ {0x001e0c, 0x001e0d, 0x001e0c, 0x001e0c},
+ {0x001e0d, 0x001e0d, 0x001e0c, 0x001e0c},
+ {0x001e0e, 0x001e0f, 0x001e0e, 0x001e0e},
+ {0x001e0f, 0x001e0f, 0x001e0e, 0x001e0e},
+ {0x001e10, 0x001e11, 0x001e10, 0x001e10},
+ {0x001e11, 0x001e11, 0x001e10, 0x001e10},
+ {0x001e12, 0x001e13, 0x001e12, 0x001e12},
+ {0x001e13, 0x001e13, 0x001e12, 0x001e12},
+ {0x001e14, 0x001e15, 0x001e14, 0x001e14},
+ {0x001e15, 0x001e15, 0x001e14, 0x001e14},
+ {0x001e16, 0x001e17, 0x001e16, 0x001e16},
+ {0x001e17, 0x001e17, 0x001e16, 0x001e16},
+ {0x001e18, 0x001e19, 0x001e18, 0x001e18},
+ {0x001e19, 0x001e19, 0x001e18, 0x001e18},
+ {0x001e1a, 0x001e1b, 0x001e1a, 0x001e1a},
+ {0x001e1b, 0x001e1b, 0x001e1a, 0x001e1a},
+ {0x001e1c, 0x001e1d, 0x001e1c, 0x001e1c},
+ {0x001e1d, 0x001e1d, 0x001e1c, 0x001e1c},
+ {0x001e1e, 0x001e1f, 0x001e1e, 0x001e1e},
+ {0x001e1f, 0x001e1f, 0x001e1e, 0x001e1e},
+ {0x001e20, 0x001e21, 0x001e20, 0x001e20},
+ {0x001e21, 0x001e21, 0x001e20, 0x001e20},
+ {0x001e22, 0x001e23, 0x001e22, 0x001e22},
+ {0x001e23, 0x001e23, 0x001e22, 0x001e22},
+ {0x001e24, 0x001e25, 0x001e24, 0x001e24},
+ {0x001e25, 0x001e25, 0x001e24, 0x001e24},
+ {0x001e26, 0x001e27, 0x001e26, 0x001e26},
+ {0x001e27, 0x001e27, 0x001e26, 0x001e26},
+ {0x001e28, 0x001e29, 0x001e28, 0x001e28},
+ {0x001e29, 0x001e29, 0x001e28, 0x001e28},
+ {0x001e2a, 0x001e2b, 0x001e2a, 0x001e2a},
+ {0x001e2b, 0x001e2b, 0x001e2a, 0x001e2a},
+ {0x001e2c, 0x001e2d, 0x001e2c, 0x001e2c},
+ {0x001e2d, 0x001e2d, 0x001e2c, 0x001e2c},
+ {0x001e2e, 0x001e2f, 0x001e2e, 0x001e2e},
+ {0x001e2f, 0x001e2f, 0x001e2e, 0x001e2e},
+ {0x001e30, 0x001e31, 0x001e30, 0x001e30},
+ {0x001e31, 0x001e31, 0x001e30, 0x001e30},
+ {0x001e32, 0x001e33, 0x001e32, 0x001e32},
+ {0x001e33, 0x001e33, 0x001e32, 0x001e32},
+ {0x001e34, 0x001e35, 0x001e34, 0x001e34},
+ {0x001e35, 0x001e35, 0x001e34, 0x001e34},
+ {0x001e36, 0x001e37, 0x001e36, 0x001e36},
+ {0x001e37, 0x001e37, 0x001e36, 0x001e36},
+ {0x001e38, 0x001e39, 0x001e38, 0x001e38},
+ {0x001e39, 0x001e39, 0x001e38, 0x001e38},
+ {0x001e3a, 0x001e3b, 0x001e3a, 0x001e3a},
+ {0x001e3b, 0x001e3b, 0x001e3a, 0x001e3a},
+ {0x001e3c, 0x001e3d, 0x001e3c, 0x001e3c},
+ {0x001e3d, 0x001e3d, 0x001e3c, 0x001e3c},
+ {0x001e3e, 0x001e3f, 0x001e3e, 0x001e3e},
+ {0x001e3f, 0x001e3f, 0x001e3e, 0x001e3e},
+ {0x001e40, 0x001e41, 0x001e40, 0x001e40},
+ {0x001e41, 0x001e41, 0x001e40, 0x001e40},
+ {0x001e42, 0x001e43, 0x001e42, 0x001e42},
+ {0x001e43, 0x001e43, 0x001e42, 0x001e42},
+ {0x001e44, 0x001e45, 0x001e44, 0x001e44},
+ {0x001e45, 0x001e45, 0x001e44, 0x001e44},
+ {0x001e46, 0x001e47, 0x001e46, 0x001e46},
+ {0x001e47, 0x001e47, 0x001e46, 0x001e46},
+ {0x001e48, 0x001e49, 0x001e48, 0x001e48},
+ {0x001e49, 0x001e49, 0x001e48, 0x001e48},
+ {0x001e4a, 0x001e4b, 0x001e4a, 0x001e4a},
+ {0x001e4b, 0x001e4b, 0x001e4a, 0x001e4a},
+ {0x001e4c, 0x001e4d, 0x001e4c, 0x001e4c},
+ {0x001e4d, 0x001e4d, 0x001e4c, 0x001e4c},
+ {0x001e4e, 0x001e4f, 0x001e4e, 0x001e4e},
+ {0x001e4f, 0x001e4f, 0x001e4e, 0x001e4e},
+ {0x001e50, 0x001e51, 0x001e50, 0x001e50},
+ {0x001e51, 0x001e51, 0x001e50, 0x001e50},
+ {0x001e52, 0x001e53, 0x001e52, 0x001e52},
+ {0x001e53, 0x001e53, 0x001e52, 0x001e52},
+ {0x001e54, 0x001e55, 0x001e54, 0x001e54},
+ {0x001e55, 0x001e55, 0x001e54, 0x001e54},
+ {0x001e56, 0x001e57, 0x001e56, 0x001e56},
+ {0x001e57, 0x001e57, 0x001e56, 0x001e56},
+ {0x001e58, 0x001e59, 0x001e58, 0x001e58},
+ {0x001e59, 0x001e59, 0x001e58, 0x001e58},
+ {0x001e5a, 0x001e5b, 0x001e5a, 0x001e5a},
+ {0x001e5b, 0x001e5b, 0x001e5a, 0x001e5a},
+ {0x001e5c, 0x001e5d, 0x001e5c, 0x001e5c},
+ {0x001e5d, 0x001e5d, 0x001e5c, 0x001e5c},
+ {0x001e5e, 0x001e5f, 0x001e5e, 0x001e5e},
+ {0x001e5f, 0x001e5f, 0x001e5e, 0x001e5e},
+ {0x001e60, 0x001e61, 0x001e60, 0x001e60},
+ {0x001e61, 0x001e61, 0x001e60, 0x001e60},
+ {0x001e62, 0x001e63, 0x001e62, 0x001e62},
+ {0x001e63, 0x001e63, 0x001e62, 0x001e62},
+ {0x001e64, 0x001e65, 0x001e64, 0x001e64},
+ {0x001e65, 0x001e65, 0x001e64, 0x001e64},
+ {0x001e66, 0x001e67, 0x001e66, 0x001e66},
+ {0x001e67, 0x001e67, 0x001e66, 0x001e66},
+ {0x001e68, 0x001e69, 0x001e68, 0x001e68},
+ {0x001e69, 0x001e69, 0x001e68, 0x001e68},
+ {0x001e6a, 0x001e6b, 0x001e6a, 0x001e6a},
+ {0x001e6b, 0x001e6b, 0x001e6a, 0x001e6a},
+ {0x001e6c, 0x001e6d, 0x001e6c, 0x001e6c},
+ {0x001e6d, 0x001e6d, 0x001e6c, 0x001e6c},
+ {0x001e6e, 0x001e6f, 0x001e6e, 0x001e6e},
+ {0x001e6f, 0x001e6f, 0x001e6e, 0x001e6e},
+ {0x001e70, 0x001e71, 0x001e70, 0x001e70},
+ {0x001e71, 0x001e71, 0x001e70, 0x001e70},
+ {0x001e72, 0x001e73, 0x001e72, 0x001e72},
+ {0x001e73, 0x001e73, 0x001e72, 0x001e72},
+ {0x001e74, 0x001e75, 0x001e74, 0x001e74},
+ {0x001e75, 0x001e75, 0x001e74, 0x001e74},
+ {0x001e76, 0x001e77, 0x001e76, 0x001e76},
+ {0x001e77, 0x001e77, 0x001e76, 0x001e76},
+ {0x001e78, 0x001e79, 0x001e78, 0x001e78},
+ {0x001e79, 0x001e79, 0x001e78, 0x001e78},
+ {0x001e7a, 0x001e7b, 0x001e7a, 0x001e7a},
+ {0x001e7b, 0x001e7b, 0x001e7a, 0x001e7a},
+ {0x001e7c, 0x001e7d, 0x001e7c, 0x001e7c},
+ {0x001e7d, 0x001e7d, 0x001e7c, 0x001e7c},
+ {0x001e7e, 0x001e7f, 0x001e7e, 0x001e7e},
+ {0x001e7f, 0x001e7f, 0x001e7e, 0x001e7e},
+ {0x001e80, 0x001e81, 0x001e80, 0x001e80},
+ {0x001e81, 0x001e81, 0x001e80, 0x001e80},
+ {0x001e82, 0x001e83, 0x001e82, 0x001e82},
+ {0x001e83, 0x001e83, 0x001e82, 0x001e82},
+ {0x001e84, 0x001e85, 0x001e84, 0x001e84},
+ {0x001e85, 0x001e85, 0x001e84, 0x001e84},
+ {0x001e86, 0x001e87, 0x001e86, 0x001e86},
+ {0x001e87, 0x001e87, 0x001e86, 0x001e86},
+ {0x001e88, 0x001e89, 0x001e88, 0x001e88},
+ {0x001e89, 0x001e89, 0x001e88, 0x001e88},
+ {0x001e8a, 0x001e8b, 0x001e8a, 0x001e8a},
+ {0x001e8b, 0x001e8b, 0x001e8a, 0x001e8a},
+ {0x001e8c, 0x001e8d, 0x001e8c, 0x001e8c},
+ {0x001e8d, 0x001e8d, 0x001e8c, 0x001e8c},
+ {0x001e8e, 0x001e8f, 0x001e8e, 0x001e8e},
+ {0x001e8f, 0x001e8f, 0x001e8e, 0x001e8e},
+ {0x001e90, 0x001e91, 0x001e90, 0x001e90},
+ {0x001e91, 0x001e91, 0x001e90, 0x001e90},
+ {0x001e92, 0x001e93, 0x001e92, 0x001e92},
+ {0x001e93, 0x001e93, 0x001e92, 0x001e92},
+ {0x001e94, 0x001e95, 0x001e94, 0x001e94},
+ {0x001e95, 0x001e95, 0x001e94, 0x001e94},
+ {0x001e9b, 0x001e9b, 0x001e60, 0x001e60},
+ {0x001e9e, 0x0000df, 0x001e9e, 0x001e9e},
+ {0x001ea0, 0x001ea1, 0x001ea0, 0x001ea0},
+ {0x001ea1, 0x001ea1, 0x001ea0, 0x001ea0},
+ {0x001ea2, 0x001ea3, 0x001ea2, 0x001ea2},
+ {0x001ea3, 0x001ea3, 0x001ea2, 0x001ea2},
+ {0x001ea4, 0x001ea5, 0x001ea4, 0x001ea4},
+ {0x001ea5, 0x001ea5, 0x001ea4, 0x001ea4},
+ {0x001ea6, 0x001ea7, 0x001ea6, 0x001ea6},
+ {0x001ea7, 0x001ea7, 0x001ea6, 0x001ea6},
+ {0x001ea8, 0x001ea9, 0x001ea8, 0x001ea8},
+ {0x001ea9, 0x001ea9, 0x001ea8, 0x001ea8},
+ {0x001eaa, 0x001eab, 0x001eaa, 0x001eaa},
+ {0x001eab, 0x001eab, 0x001eaa, 0x001eaa},
+ {0x001eac, 0x001ead, 0x001eac, 0x001eac},
+ {0x001ead, 0x001ead, 0x001eac, 0x001eac},
+ {0x001eae, 0x001eaf, 0x001eae, 0x001eae},
+ {0x001eaf, 0x001eaf, 0x001eae, 0x001eae},
+ {0x001eb0, 0x001eb1, 0x001eb0, 0x001eb0},
+ {0x001eb1, 0x001eb1, 0x001eb0, 0x001eb0},
+ {0x001eb2, 0x001eb3, 0x001eb2, 0x001eb2},
+ {0x001eb3, 0x001eb3, 0x001eb2, 0x001eb2},
+ {0x001eb4, 0x001eb5, 0x001eb4, 0x001eb4},
+ {0x001eb5, 0x001eb5, 0x001eb4, 0x001eb4},
+ {0x001eb6, 0x001eb7, 0x001eb6, 0x001eb6},
+ {0x001eb7, 0x001eb7, 0x001eb6, 0x001eb6},
+ {0x001eb8, 0x001eb9, 0x001eb8, 0x001eb8},
+ {0x001eb9, 0x001eb9, 0x001eb8, 0x001eb8},
+ {0x001eba, 0x001ebb, 0x001eba, 0x001eba},
+ {0x001ebb, 0x001ebb, 0x001eba, 0x001eba},
+ {0x001ebc, 0x001ebd, 0x001ebc, 0x001ebc},
+ {0x001ebd, 0x001ebd, 0x001ebc, 0x001ebc},
+ {0x001ebe, 0x001ebf, 0x001ebe, 0x001ebe},
+ {0x001ebf, 0x001ebf, 0x001ebe, 0x001ebe},
+ {0x001ec0, 0x001ec1, 0x001ec0, 0x001ec0},
+ {0x001ec1, 0x001ec1, 0x001ec0, 0x001ec0},
+ {0x001ec2, 0x001ec3, 0x001ec2, 0x001ec2},
+ {0x001ec3, 0x001ec3, 0x001ec2, 0x001ec2},
+ {0x001ec4, 0x001ec5, 0x001ec4, 0x001ec4},
+ {0x001ec5, 0x001ec5, 0x001ec4, 0x001ec4},
+ {0x001ec6, 0x001ec7, 0x001ec6, 0x001ec6},
+ {0x001ec7, 0x001ec7, 0x001ec6, 0x001ec6},
+ {0x001ec8, 0x001ec9, 0x001ec8, 0x001ec8},
+ {0x001ec9, 0x001ec9, 0x001ec8, 0x001ec8},
+ {0x001eca, 0x001ecb, 0x001eca, 0x001eca},
+ {0x001ecb, 0x001ecb, 0x001eca, 0x001eca},
+ {0x001ecc, 0x001ecd, 0x001ecc, 0x001ecc},
+ {0x001ecd, 0x001ecd, 0x001ecc, 0x001ecc},
+ {0x001ece, 0x001ecf, 0x001ece, 0x001ece},
+ {0x001ecf, 0x001ecf, 0x001ece, 0x001ece},
+ {0x001ed0, 0x001ed1, 0x001ed0, 0x001ed0},
+ {0x001ed1, 0x001ed1, 0x001ed0, 0x001ed0},
+ {0x001ed2, 0x001ed3, 0x001ed2, 0x001ed2},
+ {0x001ed3, 0x001ed3, 0x001ed2, 0x001ed2},
+ {0x001ed4, 0x001ed5, 0x001ed4, 0x001ed4},
+ {0x001ed5, 0x001ed5, 0x001ed4, 0x001ed4},
+ {0x001ed6, 0x001ed7, 0x001ed6, 0x001ed6},
+ {0x001ed7, 0x001ed7, 0x001ed6, 0x001ed6},
+ {0x001ed8, 0x001ed9, 0x001ed8, 0x001ed8},
+ {0x001ed9, 0x001ed9, 0x001ed8, 0x001ed8},
+ {0x001eda, 0x001edb, 0x001eda, 0x001eda},
+ {0x001edb, 0x001edb, 0x001eda, 0x001eda},
+ {0x001edc, 0x001edd, 0x001edc, 0x001edc},
+ {0x001edd, 0x001edd, 0x001edc, 0x001edc},
+ {0x001ede, 0x001edf, 0x001ede, 0x001ede},
+ {0x001edf, 0x001edf, 0x001ede, 0x001ede},
+ {0x001ee0, 0x001ee1, 0x001ee0, 0x001ee0},
+ {0x001ee1, 0x001ee1, 0x001ee0, 0x001ee0},
+ {0x001ee2, 0x001ee3, 0x001ee2, 0x001ee2},
+ {0x001ee3, 0x001ee3, 0x001ee2, 0x001ee2},
+ {0x001ee4, 0x001ee5, 0x001ee4, 0x001ee4},
+ {0x001ee5, 0x001ee5, 0x001ee4, 0x001ee4},
+ {0x001ee6, 0x001ee7, 0x001ee6, 0x001ee6},
+ {0x001ee7, 0x001ee7, 0x001ee6, 0x001ee6},
+ {0x001ee8, 0x001ee9, 0x001ee8, 0x001ee8},
+ {0x001ee9, 0x001ee9, 0x001ee8, 0x001ee8},
+ {0x001eea, 0x001eeb, 0x001eea, 0x001eea},
+ {0x001eeb, 0x001eeb, 0x001eea, 0x001eea},
+ {0x001eec, 0x001eed, 0x001eec, 0x001eec},
+ {0x001eed, 0x001eed, 0x001eec, 0x001eec},
+ {0x001eee, 0x001eef, 0x001eee, 0x001eee},
+ {0x001eef, 0x001eef, 0x001eee, 0x001eee},
+ {0x001ef0, 0x001ef1, 0x001ef0, 0x001ef0},
+ {0x001ef1, 0x001ef1, 0x001ef0, 0x001ef0},
+ {0x001ef2, 0x001ef3, 0x001ef2, 0x001ef2},
+ {0x001ef3, 0x001ef3, 0x001ef2, 0x001ef2},
+ {0x001ef4, 0x001ef5, 0x001ef4, 0x001ef4},
+ {0x001ef5, 0x001ef5, 0x001ef4, 0x001ef4},
+ {0x001ef6, 0x001ef7, 0x001ef6, 0x001ef6},
+ {0x001ef7, 0x001ef7, 0x001ef6, 0x001ef6},
+ {0x001ef8, 0x001ef9, 0x001ef8, 0x001ef8},
+ {0x001ef9, 0x001ef9, 0x001ef8, 0x001ef8},
+ {0x001efa, 0x001efb, 0x001efa, 0x001efa},
+ {0x001efb, 0x001efb, 0x001efa, 0x001efa},
+ {0x001efc, 0x001efd, 0x001efc, 0x001efc},
+ {0x001efd, 0x001efd, 0x001efc, 0x001efc},
+ {0x001efe, 0x001eff, 0x001efe, 0x001efe},
+ {0x001eff, 0x001eff, 0x001efe, 0x001efe},
+ {0x001f00, 0x001f00, 0x001f08, 0x001f08},
+ {0x001f01, 0x001f01, 0x001f09, 0x001f09},
+ {0x001f02, 0x001f02, 0x001f0a, 0x001f0a},
+ {0x001f03, 0x001f03, 0x001f0b, 0x001f0b},
+ {0x001f04, 0x001f04, 0x001f0c, 0x001f0c},
+ {0x001f05, 0x001f05, 0x001f0d, 0x001f0d},
+ {0x001f06, 0x001f06, 0x001f0e, 0x001f0e},
+ {0x001f07, 0x001f07, 0x001f0f, 0x001f0f},
+ {0x001f08, 0x001f00, 0x001f08, 0x001f08},
+ {0x001f09, 0x001f01, 0x001f09, 0x001f09},
+ {0x001f0a, 0x001f02, 0x001f0a, 0x001f0a},
+ {0x001f0b, 0x001f03, 0x001f0b, 0x001f0b},
+ {0x001f0c, 0x001f04, 0x001f0c, 0x001f0c},
+ {0x001f0d, 0x001f05, 0x001f0d, 0x001f0d},
+ {0x001f0e, 0x001f06, 0x001f0e, 0x001f0e},
+ {0x001f0f, 0x001f07, 0x001f0f, 0x001f0f},
+ {0x001f10, 0x001f10, 0x001f18, 0x001f18},
+ {0x001f11, 0x001f11, 0x001f19, 0x001f19},
+ {0x001f12, 0x001f12, 0x001f1a, 0x001f1a},
+ {0x001f13, 0x001f13, 0x001f1b, 0x001f1b},
+ {0x001f14, 0x001f14, 0x001f1c, 0x001f1c},
+ {0x001f15, 0x001f15, 0x001f1d, 0x001f1d},
+ {0x001f18, 0x001f10, 0x001f18, 0x001f18},
+ {0x001f19, 0x001f11, 0x001f19, 0x001f19},
+ {0x001f1a, 0x001f12, 0x001f1a, 0x001f1a},
+ {0x001f1b, 0x001f13, 0x001f1b, 0x001f1b},
+ {0x001f1c, 0x001f14, 0x001f1c, 0x001f1c},
+ {0x001f1d, 0x001f15, 0x001f1d, 0x001f1d},
+ {0x001f20, 0x001f20, 0x001f28, 0x001f28},
+ {0x001f21, 0x001f21, 0x001f29, 0x001f29},
+ {0x001f22, 0x001f22, 0x001f2a, 0x001f2a},
+ {0x001f23, 0x001f23, 0x001f2b, 0x001f2b},
+ {0x001f24, 0x001f24, 0x001f2c, 0x001f2c},
+ {0x001f25, 0x001f25, 0x001f2d, 0x001f2d},
+ {0x001f26, 0x001f26, 0x001f2e, 0x001f2e},
+ {0x001f27, 0x001f27, 0x001f2f, 0x001f2f},
+ {0x001f28, 0x001f20, 0x001f28, 0x001f28},
+ {0x001f29, 0x001f21, 0x001f29, 0x001f29},
+ {0x001f2a, 0x001f22, 0x001f2a, 0x001f2a},
+ {0x001f2b, 0x001f23, 0x001f2b, 0x001f2b},
+ {0x001f2c, 0x001f24, 0x001f2c, 0x001f2c},
+ {0x001f2d, 0x001f25, 0x001f2d, 0x001f2d},
+ {0x001f2e, 0x001f26, 0x001f2e, 0x001f2e},
+ {0x001f2f, 0x001f27, 0x001f2f, 0x001f2f},
+ {0x001f30, 0x001f30, 0x001f38, 0x001f38},
+ {0x001f31, 0x001f31, 0x001f39, 0x001f39},
+ {0x001f32, 0x001f32, 0x001f3a, 0x001f3a},
+ {0x001f33, 0x001f33, 0x001f3b, 0x001f3b},
+ {0x001f34, 0x001f34, 0x001f3c, 0x001f3c},
+ {0x001f35, 0x001f35, 0x001f3d, 0x001f3d},
+ {0x001f36, 0x001f36, 0x001f3e, 0x001f3e},
+ {0x001f37, 0x001f37, 0x001f3f, 0x001f3f},
+ {0x001f38, 0x001f30, 0x001f38, 0x001f38},
+ {0x001f39, 0x001f31, 0x001f39, 0x001f39},
+ {0x001f3a, 0x001f32, 0x001f3a, 0x001f3a},
+ {0x001f3b, 0x001f33, 0x001f3b, 0x001f3b},
+ {0x001f3c, 0x001f34, 0x001f3c, 0x001f3c},
+ {0x001f3d, 0x001f35, 0x001f3d, 0x001f3d},
+ {0x001f3e, 0x001f36, 0x001f3e, 0x001f3e},
+ {0x001f3f, 0x001f37, 0x001f3f, 0x001f3f},
+ {0x001f40, 0x001f40, 0x001f48, 0x001f48},
+ {0x001f41, 0x001f41, 0x001f49, 0x001f49},
+ {0x001f42, 0x001f42, 0x001f4a, 0x001f4a},
+ {0x001f43, 0x001f43, 0x001f4b, 0x001f4b},
+ {0x001f44, 0x001f44, 0x001f4c, 0x001f4c},
+ {0x001f45, 0x001f45, 0x001f4d, 0x001f4d},
+ {0x001f48, 0x001f40, 0x001f48, 0x001f48},
+ {0x001f49, 0x001f41, 0x001f49, 0x001f49},
+ {0x001f4a, 0x001f42, 0x001f4a, 0x001f4a},
+ {0x001f4b, 0x001f43, 0x001f4b, 0x001f4b},
+ {0x001f4c, 0x001f44, 0x001f4c, 0x001f4c},
+ {0x001f4d, 0x001f45, 0x001f4d, 0x001f4d},
+ {0x001f51, 0x001f51, 0x001f59, 0x001f59},
+ {0x001f53, 0x001f53, 0x001f5b, 0x001f5b},
+ {0x001f55, 0x001f55, 0x001f5d, 0x001f5d},
+ {0x001f57, 0x001f57, 0x001f5f, 0x001f5f},
+ {0x001f59, 0x001f51, 0x001f59, 0x001f59},
+ {0x001f5b, 0x001f53, 0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f55, 0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f57, 0x001f5f, 0x001f5f},
+ {0x001f60, 0x001f60, 0x001f68, 0x001f68},
+ {0x001f61, 0x001f61, 0x001f69, 0x001f69},
+ {0x001f62, 0x001f62, 0x001f6a, 0x001f6a},
+ {0x001f63, 0x001f63, 0x001f6b, 0x001f6b},
+ {0x001f64, 0x001f64, 0x001f6c, 0x001f6c},
+ {0x001f65, 0x001f65, 0x001f6d, 0x001f6d},
+ {0x001f66, 0x001f66, 0x001f6e, 0x001f6e},
+ {0x001f67, 0x001f67, 0x001f6f, 0x001f6f},
+ {0x001f68, 0x001f60, 0x001f68, 0x001f68},
+ {0x001f69, 0x001f61, 0x001f69, 0x001f69},
+ {0x001f6a, 0x001f62, 0x001f6a, 0x001f6a},
+ {0x001f6b, 0x001f63, 0x001f6b, 0x001f6b},
+ {0x001f6c, 0x001f64, 0x001f6c, 0x001f6c},
+ {0x001f6d, 0x001f65, 0x001f6d, 0x001f6d},
+ {0x001f6e, 0x001f66, 0x001f6e, 0x001f6e},
+ {0x001f6f, 0x001f67, 0x001f6f, 0x001f6f},
+ {0x001f70, 0x001f70, 0x001fba, 0x001fba},
+ {0x001f71, 0x001f71, 0x001fbb, 0x001fbb},
+ {0x001f72, 0x001f72, 0x001fc8, 0x001fc8},
+ {0x001f73, 0x001f73, 0x001fc9, 0x001fc9},
+ {0x001f74, 0x001f74, 0x001fca, 0x001fca},
+ {0x001f75, 0x001f75, 0x001fcb, 0x001fcb},
+ {0x001f76, 0x001f76, 0x001fda, 0x001fda},
+ {0x001f77, 0x001f77, 0x001fdb, 0x001fdb},
+ {0x001f78, 0x001f78, 0x001ff8, 0x001ff8},
+ {0x001f79, 0x001f79, 0x001ff9, 0x001ff9},
+ {0x001f7a, 0x001f7a, 0x001fea, 0x001fea},
+ {0x001f7b, 0x001f7b, 0x001feb, 0x001feb},
+ {0x001f7c, 0x001f7c, 0x001ffa, 0x001ffa},
+ {0x001f7d, 0x001f7d, 0x001ffb, 0x001ffb},
+ {0x001f80, 0x001f80, 0x001f88, 0x001f88},
+ {0x001f81, 0x001f81, 0x001f89, 0x001f89},
+ {0x001f82, 0x001f82, 0x001f8a, 0x001f8a},
+ {0x001f83, 0x001f83, 0x001f8b, 0x001f8b},
+ {0x001f84, 0x001f84, 0x001f8c, 0x001f8c},
+ {0x001f85, 0x001f85, 0x001f8d, 0x001f8d},
+ {0x001f86, 0x001f86, 0x001f8e, 0x001f8e},
+ {0x001f87, 0x001f87, 0x001f8f, 0x001f8f},
+ {0x001f88, 0x001f80, 0x001f88, 0x001f88},
+ {0x001f89, 0x001f81, 0x001f89, 0x001f89},
+ {0x001f8a, 0x001f82, 0x001f8a, 0x001f8a},
+ {0x001f8b, 0x001f83, 0x001f8b, 0x001f8b},
+ {0x001f8c, 0x001f84, 0x001f8c, 0x001f8c},
+ {0x001f8d, 0x001f85, 0x001f8d, 0x001f8d},
+ {0x001f8e, 0x001f86, 0x001f8e, 0x001f8e},
+ {0x001f8f, 0x001f87, 0x001f8f, 0x001f8f},
+ {0x001f90, 0x001f90, 0x001f98, 0x001f98},
+ {0x001f91, 0x001f91, 0x001f99, 0x001f99},
+ {0x001f92, 0x001f92, 0x001f9a, 0x001f9a},
+ {0x001f93, 0x001f93, 0x001f9b, 0x001f9b},
+ {0x001f94, 0x001f94, 0x001f9c, 0x001f9c},
+ {0x001f95, 0x001f95, 0x001f9d, 0x001f9d},
+ {0x001f96, 0x001f96, 0x001f9e, 0x001f9e},
+ {0x001f97, 0x001f97, 0x001f9f, 0x001f9f},
+ {0x001f98, 0x001f90, 0x001f98, 0x001f98},
+ {0x001f99, 0x001f91, 0x001f99, 0x001f99},
+ {0x001f9a, 0x001f92, 0x001f9a, 0x001f9a},
+ {0x001f9b, 0x001f93, 0x001f9b, 0x001f9b},
+ {0x001f9c, 0x001f94, 0x001f9c, 0x001f9c},
+ {0x001f9d, 0x001f95, 0x001f9d, 0x001f9d},
+ {0x001f9e, 0x001f96, 0x001f9e, 0x001f9e},
+ {0x001f9f, 0x001f97, 0x001f9f, 0x001f9f},
+ {0x001fa0, 0x001fa0, 0x001fa8, 0x001fa8},
+ {0x001fa1, 0x001fa1, 0x001fa9, 0x001fa9},
+ {0x001fa2, 0x001fa2, 0x001faa, 0x001faa},
+ {0x001fa3, 0x001fa3, 0x001fab, 0x001fab},
+ {0x001fa4, 0x001fa4, 0x001fac, 0x001fac},
+ {0x001fa5, 0x001fa5, 0x001fad, 0x001fad},
+ {0x001fa6, 0x001fa6, 0x001fae, 0x001fae},
+ {0x001fa7, 0x001fa7, 0x001faf, 0x001faf},
+ {0x001fa8, 0x001fa0, 0x001fa8, 0x001fa8},
+ {0x001fa9, 0x001fa1, 0x001fa9, 0x001fa9},
+ {0x001faa, 0x001fa2, 0x001faa, 0x001faa},
+ {0x001fab, 0x001fa3, 0x001fab, 0x001fab},
+ {0x001fac, 0x001fa4, 0x001fac, 0x001fac},
+ {0x001fad, 0x001fa5, 0x001fad, 0x001fad},
+ {0x001fae, 0x001fa6, 0x001fae, 0x001fae},
+ {0x001faf, 0x001fa7, 0x001faf, 0x001faf},
+ {0x001fb0, 0x001fb0, 0x001fb8, 0x001fb8},
+ {0x001fb1, 0x001fb1, 0x001fb9, 0x001fb9},
+ {0x001fb3, 0x001fb3, 0x001fbc, 0x001fbc},
+ {0x001fb8, 0x001fb0, 0x001fb8, 0x001fb8},
+ {0x001fb9, 0x001fb1, 0x001fb9, 0x001fb9},
+ {0x001fba, 0x001f70, 0x001fba, 0x001fba},
+ {0x001fbb, 0x001f71, 0x001fbb, 0x001fbb},
+ {0x001fbc, 0x001fb3, 0x001fbc, 0x001fbc},
+ {0x001fbe, 0x001fbe, 0x000399, 0x000399},
+ {0x001fc3, 0x001fc3, 0x001fcc, 0x001fcc},
+ {0x001fc8, 0x001f72, 0x001fc8, 0x001fc8},
+ {0x001fc9, 0x001f73, 0x001fc9, 0x001fc9},
+ {0x001fca, 0x001f74, 0x001fca, 0x001fca},
+ {0x001fcb, 0x001f75, 0x001fcb, 0x001fcb},
+ {0x001fcc, 0x001fc3, 0x001fcc, 0x001fcc},
+ {0x001fd0, 0x001fd0, 0x001fd8, 0x001fd8},
+ {0x001fd1, 0x001fd1, 0x001fd9, 0x001fd9},
+ {0x001fd8, 0x001fd0, 0x001fd8, 0x001fd8},
+ {0x001fd9, 0x001fd1, 0x001fd9, 0x001fd9},
+ {0x001fda, 0x001f76, 0x001fda, 0x001fda},
+ {0x001fdb, 0x001f77, 0x001fdb, 0x001fdb},
+ {0x001fe0, 0x001fe0, 0x001fe8, 0x001fe8},
+ {0x001fe1, 0x001fe1, 0x001fe9, 0x001fe9},
+ {0x001fe5, 0x001fe5, 0x001fec, 0x001fec},
+ {0x001fe8, 0x001fe0, 0x001fe8, 0x001fe8},
+ {0x001fe9, 0x001fe1, 0x001fe9, 0x001fe9},
+ {0x001fea, 0x001f7a, 0x001fea, 0x001fea},
+ {0x001feb, 0x001f7b, 0x001feb, 0x001feb},
+ {0x001fec, 0x001fe5, 0x001fec, 0x001fec},
+ {0x001ff3, 0x001ff3, 0x001ffc, 0x001ffc},
+ {0x001ff8, 0x001f78, 0x001ff8, 0x001ff8},
+ {0x001ff9, 0x001f79, 0x001ff9, 0x001ff9},
+ {0x001ffa, 0x001f7c, 0x001ffa, 0x001ffa},
+ {0x001ffb, 0x001f7d, 0x001ffb, 0x001ffb},
+ {0x001ffc, 0x001ff3, 0x001ffc, 0x001ffc},
+ {0x002126, 0x0003c9, 0x002126, 0x002126},
+ {0x00212a, 0x00006b, 0x00212a, 0x00212a},
+ {0x00212b, 0x0000e5, 0x00212b, 0x00212b},
+ {0x002132, 0x00214e, 0x002132, 0x002132},
+ {0x00214e, 0x00214e, 0x002132, 0x002132},
+ {0x002160, 0x002170, 0x002160, 0x002160},
+ {0x002161, 0x002171, 0x002161, 0x002161},
+ {0x002162, 0x002172, 0x002162, 0x002162},
+ {0x002163, 0x002173, 0x002163, 0x002163},
+ {0x002164, 0x002174, 0x002164, 0x002164},
+ {0x002165, 0x002175, 0x002165, 0x002165},
+ {0x002166, 0x002176, 0x002166, 0x002166},
+ {0x002167, 0x002177, 0x002167, 0x002167},
+ {0x002168, 0x002178, 0x002168, 0x002168},
+ {0x002169, 0x002179, 0x002169, 0x002169},
+ {0x00216a, 0x00217a, 0x00216a, 0x00216a},
+ {0x00216b, 0x00217b, 0x00216b, 0x00216b},
+ {0x00216c, 0x00217c, 0x00216c, 0x00216c},
+ {0x00216d, 0x00217d, 0x00216d, 0x00216d},
+ {0x00216e, 0x00217e, 0x00216e, 0x00216e},
+ {0x00216f, 0x00217f, 0x00216f, 0x00216f},
+ {0x002170, 0x002170, 0x002160, 0x002160},
+ {0x002171, 0x002171, 0x002161, 0x002161},
+ {0x002172, 0x002172, 0x002162, 0x002162},
+ {0x002173, 0x002173, 0x002163, 0x002163},
+ {0x002174, 0x002174, 0x002164, 0x002164},
+ {0x002175, 0x002175, 0x002165, 0x002165},
+ {0x002176, 0x002176, 0x002166, 0x002166},
+ {0x002177, 0x002177, 0x002167, 0x002167},
+ {0x002178, 0x002178, 0x002168, 0x002168},
+ {0x002179, 0x002179, 0x002169, 0x002169},
+ {0x00217a, 0x00217a, 0x00216a, 0x00216a},
+ {0x00217b, 0x00217b, 0x00216b, 0x00216b},
+ {0x00217c, 0x00217c, 0x00216c, 0x00216c},
+ {0x00217d, 0x00217d, 0x00216d, 0x00216d},
+ {0x00217e, 0x00217e, 0x00216e, 0x00216e},
+ {0x00217f, 0x00217f, 0x00216f, 0x00216f},
+ {0x002183, 0x002184, 0x002183, 0x002183},
+ {0x002184, 0x002184, 0x002183, 0x002183},
+ {0x0024b6, 0x0024d0, 0x0024b6, 0x0024b6},
+ {0x0024b7, 0x0024d1, 0x0024b7, 0x0024b7},
+ {0x0024b8, 0x0024d2, 0x0024b8, 0x0024b8},
+ {0x0024b9, 0x0024d3, 0x0024b9, 0x0024b9},
+ {0x0024ba, 0x0024d4, 0x0024ba, 0x0024ba},
+ {0x0024bb, 0x0024d5, 0x0024bb, 0x0024bb},
+ {0x0024bc, 0x0024d6, 0x0024bc, 0x0024bc},
+ {0x0024bd, 0x0024d7, 0x0024bd, 0x0024bd},
+ {0x0024be, 0x0024d8, 0x0024be, 0x0024be},
+ {0x0024bf, 0x0024d9, 0x0024bf, 0x0024bf},
+ {0x0024c0, 0x0024da, 0x0024c0, 0x0024c0},
+ {0x0024c1, 0x0024db, 0x0024c1, 0x0024c1},
+ {0x0024c2, 0x0024dc, 0x0024c2, 0x0024c2},
+ {0x0024c3, 0x0024dd, 0x0024c3, 0x0024c3},
+ {0x0024c4, 0x0024de, 0x0024c4, 0x0024c4},
+ {0x0024c5, 0x0024df, 0x0024c5, 0x0024c5},
+ {0x0024c6, 0x0024e0, 0x0024c6, 0x0024c6},
+ {0x0024c7, 0x0024e1, 0x0024c7, 0x0024c7},
+ {0x0024c8, 0x0024e2, 0x0024c8, 0x0024c8},
+ {0x0024c9, 0x0024e3, 0x0024c9, 0x0024c9},
+ {0x0024ca, 0x0024e4, 0x0024ca, 0x0024ca},
+ {0x0024cb, 0x0024e5, 0x0024cb, 0x0024cb},
+ {0x0024cc, 0x0024e6, 0x0024cc, 0x0024cc},
+ {0x0024cd, 0x0024e7, 0x0024cd, 0x0024cd},
+ {0x0024ce, 0x0024e8, 0x0024ce, 0x0024ce},
+ {0x0024cf, 0x0024e9, 0x0024cf, 0x0024cf},
+ {0x0024d0, 0x0024d0, 0x0024b6, 0x0024b6},
+ {0x0024d1, 0x0024d1, 0x0024b7, 0x0024b7},
+ {0x0024d2, 0x0024d2, 0x0024b8, 0x0024b8},
+ {0x0024d3, 0x0024d3, 0x0024b9, 0x0024b9},
+ {0x0024d4, 0x0024d4, 0x0024ba, 0x0024ba},
+ {0x0024d5, 0x0024d5, 0x0024bb, 0x0024bb},
+ {0x0024d6, 0x0024d6, 0x0024bc, 0x0024bc},
+ {0x0024d7, 0x0024d7, 0x0024bd, 0x0024bd},
+ {0x0024d8, 0x0024d8, 0x0024be, 0x0024be},
+ {0x0024d9, 0x0024d9, 0x0024bf, 0x0024bf},
+ {0x0024da, 0x0024da, 0x0024c0, 0x0024c0},
+ {0x0024db, 0x0024db, 0x0024c1, 0x0024c1},
+ {0x0024dc, 0x0024dc, 0x0024c2, 0x0024c2},
+ {0x0024dd, 0x0024dd, 0x0024c3, 0x0024c3},
+ {0x0024de, 0x0024de, 0x0024c4, 0x0024c4},
+ {0x0024df, 0x0024df, 0x0024c5, 0x0024c5},
+ {0x0024e0, 0x0024e0, 0x0024c6, 0x0024c6},
+ {0x0024e1, 0x0024e1, 0x0024c7, 0x0024c7},
+ {0x0024e2, 0x0024e2, 0x0024c8, 0x0024c8},
+ {0x0024e3, 0x0024e3, 0x0024c9, 0x0024c9},
+ {0x0024e4, 0x0024e4, 0x0024ca, 0x0024ca},
+ {0x0024e5, 0x0024e5, 0x0024cb, 0x0024cb},
+ {0x0024e6, 0x0024e6, 0x0024cc, 0x0024cc},
+ {0x0024e7, 0x0024e7, 0x0024cd, 0x0024cd},
+ {0x0024e8, 0x0024e8, 0x0024ce, 0x0024ce},
+ {0x0024e9, 0x0024e9, 0x0024cf, 0x0024cf},
+ {0x002c00, 0x002c30, 0x002c00, 0x002c00},
+ {0x002c01, 0x002c31, 0x002c01, 0x002c01},
+ {0x002c02, 0x002c32, 0x002c02, 0x002c02},
+ {0x002c03, 0x002c33, 0x002c03, 0x002c03},
+ {0x002c04, 0x002c34, 0x002c04, 0x002c04},
+ {0x002c05, 0x002c35, 0x002c05, 0x002c05},
+ {0x002c06, 0x002c36, 0x002c06, 0x002c06},
+ {0x002c07, 0x002c37, 0x002c07, 0x002c07},
+ {0x002c08, 0x002c38, 0x002c08, 0x002c08},
+ {0x002c09, 0x002c39, 0x002c09, 0x002c09},
+ {0x002c0a, 0x002c3a, 0x002c0a, 0x002c0a},
+ {0x002c0b, 0x002c3b, 0x002c0b, 0x002c0b},
+ {0x002c0c, 0x002c3c, 0x002c0c, 0x002c0c},
+ {0x002c0d, 0x002c3d, 0x002c0d, 0x002c0d},
+ {0x002c0e, 0x002c3e, 0x002c0e, 0x002c0e},
+ {0x002c0f, 0x002c3f, 0x002c0f, 0x002c0f},
+ {0x002c10, 0x002c40, 0x002c10, 0x002c10},
+ {0x002c11, 0x002c41, 0x002c11, 0x002c11},
+ {0x002c12, 0x002c42, 0x002c12, 0x002c12},
+ {0x002c13, 0x002c43, 0x002c13, 0x002c13},
+ {0x002c14, 0x002c44, 0x002c14, 0x002c14},
+ {0x002c15, 0x002c45, 0x002c15, 0x002c15},
+ {0x002c16, 0x002c46, 0x002c16, 0x002c16},
+ {0x002c17, 0x002c47, 0x002c17, 0x002c17},
+ {0x002c18, 0x002c48, 0x002c18, 0x002c18},
+ {0x002c19, 0x002c49, 0x002c19, 0x002c19},
+ {0x002c1a, 0x002c4a, 0x002c1a, 0x002c1a},
+ {0x002c1b, 0x002c4b, 0x002c1b, 0x002c1b},
+ {0x002c1c, 0x002c4c, 0x002c1c, 0x002c1c},
+ {0x002c1d, 0x002c4d, 0x002c1d, 0x002c1d},
+ {0x002c1e, 0x002c4e, 0x002c1e, 0x002c1e},
+ {0x002c1f, 0x002c4f, 0x002c1f, 0x002c1f},
+ {0x002c20, 0x002c50, 0x002c20, 0x002c20},
+ {0x002c21, 0x002c51, 0x002c21, 0x002c21},
+ {0x002c22, 0x002c52, 0x002c22, 0x002c22},
+ {0x002c23, 0x002c53, 0x002c23, 0x002c23},
+ {0x002c24, 0x002c54, 0x002c24, 0x002c24},
+ {0x002c25, 0x002c55, 0x002c25, 0x002c25},
+ {0x002c26, 0x002c56, 0x002c26, 0x002c26},
+ {0x002c27, 0x002c57, 0x002c27, 0x002c27},
+ {0x002c28, 0x002c58, 0x002c28, 0x002c28},
+ {0x002c29, 0x002c59, 0x002c29, 0x002c29},
+ {0x002c2a, 0x002c5a, 0x002c2a, 0x002c2a},
+ {0x002c2b, 0x002c5b, 0x002c2b, 0x002c2b},
+ {0x002c2c, 0x002c5c, 0x002c2c, 0x002c2c},
+ {0x002c2d, 0x002c5d, 0x002c2d, 0x002c2d},
+ {0x002c2e, 0x002c5e, 0x002c2e, 0x002c2e},
+ {0x002c2f, 0x002c5f, 0x002c2f, 0x002c2f},
+ {0x002c30, 0x002c30, 0x002c00, 0x002c00},
+ {0x002c31, 0x002c31, 0x002c01, 0x002c01},
+ {0x002c32, 0x002c32, 0x002c02, 0x002c02},
+ {0x002c33, 0x002c33, 0x002c03, 0x002c03},
+ {0x002c34, 0x002c34, 0x002c04, 0x002c04},
+ {0x002c35, 0x002c35, 0x002c05, 0x002c05},
+ {0x002c36, 0x002c36, 0x002c06, 0x002c06},
+ {0x002c37, 0x002c37, 0x002c07, 0x002c07},
+ {0x002c38, 0x002c38, 0x002c08, 0x002c08},
+ {0x002c39, 0x002c39, 0x002c09, 0x002c09},
+ {0x002c3a, 0x002c3a, 0x002c0a, 0x002c0a},
+ {0x002c3b, 0x002c3b, 0x002c0b, 0x002c0b},
+ {0x002c3c, 0x002c3c, 0x002c0c, 0x002c0c},
+ {0x002c3d, 0x002c3d, 0x002c0d, 0x002c0d},
+ {0x002c3e, 0x002c3e, 0x002c0e, 0x002c0e},
+ {0x002c3f, 0x002c3f, 0x002c0f, 0x002c0f},
+ {0x002c40, 0x002c40, 0x002c10, 0x002c10},
+ {0x002c41, 0x002c41, 0x002c11, 0x002c11},
+ {0x002c42, 0x002c42, 0x002c12, 0x002c12},
+ {0x002c43, 0x002c43, 0x002c13, 0x002c13},
+ {0x002c44, 0x002c44, 0x002c14, 0x002c14},
+ {0x002c45, 0x002c45, 0x002c15, 0x002c15},
+ {0x002c46, 0x002c46, 0x002c16, 0x002c16},
+ {0x002c47, 0x002c47, 0x002c17, 0x002c17},
+ {0x002c48, 0x002c48, 0x002c18, 0x002c18},
+ {0x002c49, 0x002c49, 0x002c19, 0x002c19},
+ {0x002c4a, 0x002c4a, 0x002c1a, 0x002c1a},
+ {0x002c4b, 0x002c4b, 0x002c1b, 0x002c1b},
+ {0x002c4c, 0x002c4c, 0x002c1c, 0x002c1c},
+ {0x002c4d, 0x002c4d, 0x002c1d, 0x002c1d},
+ {0x002c4e, 0x002c4e, 0x002c1e, 0x002c1e},
+ {0x002c4f, 0x002c4f, 0x002c1f, 0x002c1f},
+ {0x002c50, 0x002c50, 0x002c20, 0x002c20},
+ {0x002c51, 0x002c51, 0x002c21, 0x002c21},
+ {0x002c52, 0x002c52, 0x002c22, 0x002c22},
+ {0x002c53, 0x002c53, 0x002c23, 0x002c23},
+ {0x002c54, 0x002c54, 0x002c24, 0x002c24},
+ {0x002c55, 0x002c55, 0x002c25, 0x002c25},
+ {0x002c56, 0x002c56, 0x002c26, 0x002c26},
+ {0x002c57, 0x002c57, 0x002c27, 0x002c27},
+ {0x002c58, 0x002c58, 0x002c28, 0x002c28},
+ {0x002c59, 0x002c59, 0x002c29, 0x002c29},
+ {0x002c5a, 0x002c5a, 0x002c2a, 0x002c2a},
+ {0x002c5b, 0x002c5b, 0x002c2b, 0x002c2b},
+ {0x002c5c, 0x002c5c, 0x002c2c, 0x002c2c},
+ {0x002c5d, 0x002c5d, 0x002c2d, 0x002c2d},
+ {0x002c5e, 0x002c5e, 0x002c2e, 0x002c2e},
+ {0x002c5f, 0x002c5f, 0x002c2f, 0x002c2f},
+ {0x002c60, 0x002c61, 0x002c60, 0x002c60},
+ {0x002c61, 0x002c61, 0x002c60, 0x002c60},
+ {0x002c62, 0x00026b, 0x002c62, 0x002c62},
+ {0x002c63, 0x001d7d, 0x002c63, 0x002c63},
+ {0x002c64, 0x00027d, 0x002c64, 0x002c64},
+ {0x002c65, 0x002c65, 0x00023a, 0x00023a},
+ {0x002c66, 0x002c66, 0x00023e, 0x00023e},
+ {0x002c67, 0x002c68, 0x002c67, 0x002c67},
+ {0x002c68, 0x002c68, 0x002c67, 0x002c67},
+ {0x002c69, 0x002c6a, 0x002c69, 0x002c69},
+ {0x002c6a, 0x002c6a, 0x002c69, 0x002c69},
+ {0x002c6b, 0x002c6c, 0x002c6b, 0x002c6b},
+ {0x002c6c, 0x002c6c, 0x002c6b, 0x002c6b},
+ {0x002c6d, 0x000251, 0x002c6d, 0x002c6d},
+ {0x002c6e, 0x000271, 0x002c6e, 0x002c6e},
+ {0x002c6f, 0x000250, 0x002c6f, 0x002c6f},
+ {0x002c70, 0x000252, 0x002c70, 0x002c70},
+ {0x002c72, 0x002c73, 0x002c72, 0x002c72},
+ {0x002c73, 0x002c73, 0x002c72, 0x002c72},
+ {0x002c75, 0x002c76, 0x002c75, 0x002c75},
+ {0x002c76, 0x002c76, 0x002c75, 0x002c75},
+ {0x002c7e, 0x00023f, 0x002c7e, 0x002c7e},
+ {0x002c7f, 0x000240, 0x002c7f, 0x002c7f},
+ {0x002c80, 0x002c81, 0x002c80, 0x002c80},
+ {0x002c81, 0x002c81, 0x002c80, 0x002c80},
+ {0x002c82, 0x002c83, 0x002c82, 0x002c82},
+ {0x002c83, 0x002c83, 0x002c82, 0x002c82},
+ {0x002c84, 0x002c85, 0x002c84, 0x002c84},
+ {0x002c85, 0x002c85, 0x002c84, 0x002c84},
+ {0x002c86, 0x002c87, 0x002c86, 0x002c86},
+ {0x002c87, 0x002c87, 0x002c86, 0x002c86},
+ {0x002c88, 0x002c89, 0x002c88, 0x002c88},
+ {0x002c89, 0x002c89, 0x002c88, 0x002c88},
+ {0x002c8a, 0x002c8b, 0x002c8a, 0x002c8a},
+ {0x002c8b, 0x002c8b, 0x002c8a, 0x002c8a},
+ {0x002c8c, 0x002c8d, 0x002c8c, 0x002c8c},
+ {0x002c8d, 0x002c8d, 0x002c8c, 0x002c8c},
+ {0x002c8e, 0x002c8f, 0x002c8e, 0x002c8e},
+ {0x002c8f, 0x002c8f, 0x002c8e, 0x002c8e},
+ {0x002c90, 0x002c91, 0x002c90, 0x002c90},
+ {0x002c91, 0x002c91, 0x002c90, 0x002c90},
+ {0x002c92, 0x002c93, 0x002c92, 0x002c92},
+ {0x002c93, 0x002c93, 0x002c92, 0x002c92},
+ {0x002c94, 0x002c95, 0x002c94, 0x002c94},
+ {0x002c95, 0x002c95, 0x002c94, 0x002c94},
+ {0x002c96, 0x002c97, 0x002c96, 0x002c96},
+ {0x002c97, 0x002c97, 0x002c96, 0x002c96},
+ {0x002c98, 0x002c99, 0x002c98, 0x002c98},
+ {0x002c99, 0x002c99, 0x002c98, 0x002c98},
+ {0x002c9a, 0x002c9b, 0x002c9a, 0x002c9a},
+ {0x002c9b, 0x002c9b, 0x002c9a, 0x002c9a},
+ {0x002c9c, 0x002c9d, 0x002c9c, 0x002c9c},
+ {0x002c9d, 0x002c9d, 0x002c9c, 0x002c9c},
+ {0x002c9e, 0x002c9f, 0x002c9e, 0x002c9e},
+ {0x002c9f, 0x002c9f, 0x002c9e, 0x002c9e},
+ {0x002ca0, 0x002ca1, 0x002ca0, 0x002ca0},
+ {0x002ca1, 0x002ca1, 0x002ca0, 0x002ca0},
+ {0x002ca2, 0x002ca3, 0x002ca2, 0x002ca2},
+ {0x002ca3, 0x002ca3, 0x002ca2, 0x002ca2},
+ {0x002ca4, 0x002ca5, 0x002ca4, 0x002ca4},
+ {0x002ca5, 0x002ca5, 0x002ca4, 0x002ca4},
+ {0x002ca6, 0x002ca7, 0x002ca6, 0x002ca6},
+ {0x002ca7, 0x002ca7, 0x002ca6, 0x002ca6},
+ {0x002ca8, 0x002ca9, 0x002ca8, 0x002ca8},
+ {0x002ca9, 0x002ca9, 0x002ca8, 0x002ca8},
+ {0x002caa, 0x002cab, 0x002caa, 0x002caa},
+ {0x002cab, 0x002cab, 0x002caa, 0x002caa},
+ {0x002cac, 0x002cad, 0x002cac, 0x002cac},
+ {0x002cad, 0x002cad, 0x002cac, 0x002cac},
+ {0x002cae, 0x002caf, 0x002cae, 0x002cae},
+ {0x002caf, 0x002caf, 0x002cae, 0x002cae},
+ {0x002cb0, 0x002cb1, 0x002cb0, 0x002cb0},
+ {0x002cb1, 0x002cb1, 0x002cb0, 0x002cb0},
+ {0x002cb2, 0x002cb3, 0x002cb2, 0x002cb2},
+ {0x002cb3, 0x002cb3, 0x002cb2, 0x002cb2},
+ {0x002cb4, 0x002cb5, 0x002cb4, 0x002cb4},
+ {0x002cb5, 0x002cb5, 0x002cb4, 0x002cb4},
+ {0x002cb6, 0x002cb7, 0x002cb6, 0x002cb6},
+ {0x002cb7, 0x002cb7, 0x002cb6, 0x002cb6},
+ {0x002cb8, 0x002cb9, 0x002cb8, 0x002cb8},
+ {0x002cb9, 0x002cb9, 0x002cb8, 0x002cb8},
+ {0x002cba, 0x002cbb, 0x002cba, 0x002cba},
+ {0x002cbb, 0x002cbb, 0x002cba, 0x002cba},
+ {0x002cbc, 0x002cbd, 0x002cbc, 0x002cbc},
+ {0x002cbd, 0x002cbd, 0x002cbc, 0x002cbc},
+ {0x002cbe, 0x002cbf, 0x002cbe, 0x002cbe},
+ {0x002cbf, 0x002cbf, 0x002cbe, 0x002cbe},
+ {0x002cc0, 0x002cc1, 0x002cc0, 0x002cc0},
+ {0x002cc1, 0x002cc1, 0x002cc0, 0x002cc0},
+ {0x002cc2, 0x002cc3, 0x002cc2, 0x002cc2},
+ {0x002cc3, 0x002cc3, 0x002cc2, 0x002cc2},
+ {0x002cc4, 0x002cc5, 0x002cc4, 0x002cc4},
+ {0x002cc5, 0x002cc5, 0x002cc4, 0x002cc4},
+ {0x002cc6, 0x002cc7, 0x002cc6, 0x002cc6},
+ {0x002cc7, 0x002cc7, 0x002cc6, 0x002cc6},
+ {0x002cc8, 0x002cc9, 0x002cc8, 0x002cc8},
+ {0x002cc9, 0x002cc9, 0x002cc8, 0x002cc8},
+ {0x002cca, 0x002ccb, 0x002cca, 0x002cca},
+ {0x002ccb, 0x002ccb, 0x002cca, 0x002cca},
+ {0x002ccc, 0x002ccd, 0x002ccc, 0x002ccc},
+ {0x002ccd, 0x002ccd, 0x002ccc, 0x002ccc},
+ {0x002cce, 0x002ccf, 0x002cce, 0x002cce},
+ {0x002ccf, 0x002ccf, 0x002cce, 0x002cce},
+ {0x002cd0, 0x002cd1, 0x002cd0, 0x002cd0},
+ {0x002cd1, 0x002cd1, 0x002cd0, 0x002cd0},
+ {0x002cd2, 0x002cd3, 0x002cd2, 0x002cd2},
+ {0x002cd3, 0x002cd3, 0x002cd2, 0x002cd2},
+ {0x002cd4, 0x002cd5, 0x002cd4, 0x002cd4},
+ {0x002cd5, 0x002cd5, 0x002cd4, 0x002cd4},
+ {0x002cd6, 0x002cd7, 0x002cd6, 0x002cd6},
+ {0x002cd7, 0x002cd7, 0x002cd6, 0x002cd6},
+ {0x002cd8, 0x002cd9, 0x002cd8, 0x002cd8},
+ {0x002cd9, 0x002cd9, 0x002cd8, 0x002cd8},
+ {0x002cda, 0x002cdb, 0x002cda, 0x002cda},
+ {0x002cdb, 0x002cdb, 0x002cda, 0x002cda},
+ {0x002cdc, 0x002cdd, 0x002cdc, 0x002cdc},
+ {0x002cdd, 0x002cdd, 0x002cdc, 0x002cdc},
+ {0x002cde, 0x002cdf, 0x002cde, 0x002cde},
+ {0x002cdf, 0x002cdf, 0x002cde, 0x002cde},
+ {0x002ce0, 0x002ce1, 0x002ce0, 0x002ce0},
+ {0x002ce1, 0x002ce1, 0x002ce0, 0x002ce0},
+ {0x002ce2, 0x002ce3, 0x002ce2, 0x002ce2},
+ {0x002ce3, 0x002ce3, 0x002ce2, 0x002ce2},
+ {0x002ceb, 0x002cec, 0x002ceb, 0x002ceb},
+ {0x002cec, 0x002cec, 0x002ceb, 0x002ceb},
+ {0x002ced, 0x002cee, 0x002ced, 0x002ced},
+ {0x002cee, 0x002cee, 0x002ced, 0x002ced},
+ {0x002cf2, 0x002cf3, 0x002cf2, 0x002cf2},
+ {0x002cf3, 0x002cf3, 0x002cf2, 0x002cf2},
+ {0x002d00, 0x002d00, 0x0010a0, 0x0010a0},
+ {0x002d01, 0x002d01, 0x0010a1, 0x0010a1},
+ {0x002d02, 0x002d02, 0x0010a2, 0x0010a2},
+ {0x002d03, 0x002d03, 0x0010a3, 0x0010a3},
+ {0x002d04, 0x002d04, 0x0010a4, 0x0010a4},
+ {0x002d05, 0x002d05, 0x0010a5, 0x0010a5},
+ {0x002d06, 0x002d06, 0x0010a6, 0x0010a6},
+ {0x002d07, 0x002d07, 0x0010a7, 0x0010a7},
+ {0x002d08, 0x002d08, 0x0010a8, 0x0010a8},
+ {0x002d09, 0x002d09, 0x0010a9, 0x0010a9},
+ {0x002d0a, 0x002d0a, 0x0010aa, 0x0010aa},
+ {0x002d0b, 0x002d0b, 0x0010ab, 0x0010ab},
+ {0x002d0c, 0x002d0c, 0x0010ac, 0x0010ac},
+ {0x002d0d, 0x002d0d, 0x0010ad, 0x0010ad},
+ {0x002d0e, 0x002d0e, 0x0010ae, 0x0010ae},
+ {0x002d0f, 0x002d0f, 0x0010af, 0x0010af},
+ {0x002d10, 0x002d10, 0x0010b0, 0x0010b0},
+ {0x002d11, 0x002d11, 0x0010b1, 0x0010b1},
+ {0x002d12, 0x002d12, 0x0010b2, 0x0010b2},
+ {0x002d13, 0x002d13, 0x0010b3, 0x0010b3},
+ {0x002d14, 0x002d14, 0x0010b4, 0x0010b4},
+ {0x002d15, 0x002d15, 0x0010b5, 0x0010b5},
+ {0x002d16, 0x002d16, 0x0010b6, 0x0010b6},
+ {0x002d17, 0x002d17, 0x0010b7, 0x0010b7},
+ {0x002d18, 0x002d18, 0x0010b8, 0x0010b8},
+ {0x002d19, 0x002d19, 0x0010b9, 0x0010b9},
+ {0x002d1a, 0x002d1a, 0x0010ba, 0x0010ba},
+ {0x002d1b, 0x002d1b, 0x0010bb, 0x0010bb},
+ {0x002d1c, 0x002d1c, 0x0010bc, 0x0010bc},
+ {0x002d1d, 0x002d1d, 0x0010bd, 0x0010bd},
+ {0x002d1e, 0x002d1e, 0x0010be, 0x0010be},
+ {0x002d1f, 0x002d1f, 0x0010bf, 0x0010bf},
+ {0x002d20, 0x002d20, 0x0010c0, 0x0010c0},
+ {0x002d21, 0x002d21, 0x0010c1, 0x0010c1},
+ {0x002d22, 0x002d22, 0x0010c2, 0x0010c2},
+ {0x002d23, 0x002d23, 0x0010c3, 0x0010c3},
+ {0x002d24, 0x002d24, 0x0010c4, 0x0010c4},
+ {0x002d25, 0x002d25, 0x0010c5, 0x0010c5},
+ {0x002d27, 0x002d27, 0x0010c7, 0x0010c7},
+ {0x002d2d, 0x002d2d, 0x0010cd, 0x0010cd},
+ {0x00a640, 0x00a641, 0x00a640, 0x00a640},
+ {0x00a641, 0x00a641, 0x00a640, 0x00a640},
+ {0x00a642, 0x00a643, 0x00a642, 0x00a642},
+ {0x00a643, 0x00a643, 0x00a642, 0x00a642},
+ {0x00a644, 0x00a645, 0x00a644, 0x00a644},
+ {0x00a645, 0x00a645, 0x00a644, 0x00a644},
+ {0x00a646, 0x00a647, 0x00a646, 0x00a646},
+ {0x00a647, 0x00a647, 0x00a646, 0x00a646},
+ {0x00a648, 0x00a649, 0x00a648, 0x00a648},
+ {0x00a649, 0x00a649, 0x00a648, 0x00a648},
+ {0x00a64a, 0x00a64b, 0x00a64a, 0x00a64a},
+ {0x00a64b, 0x00a64b, 0x00a64a, 0x00a64a},
+ {0x00a64c, 0x00a64d, 0x00a64c, 0x00a64c},
+ {0x00a64d, 0x00a64d, 0x00a64c, 0x00a64c},
+ {0x00a64e, 0x00a64f, 0x00a64e, 0x00a64e},
+ {0x00a64f, 0x00a64f, 0x00a64e, 0x00a64e},
+ {0x00a650, 0x00a651, 0x00a650, 0x00a650},
+ {0x00a651, 0x00a651, 0x00a650, 0x00a650},
+ {0x00a652, 0x00a653, 0x00a652, 0x00a652},
+ {0x00a653, 0x00a653, 0x00a652, 0x00a652},
+ {0x00a654, 0x00a655, 0x00a654, 0x00a654},
+ {0x00a655, 0x00a655, 0x00a654, 0x00a654},
+ {0x00a656, 0x00a657, 0x00a656, 0x00a656},
+ {0x00a657, 0x00a657, 0x00a656, 0x00a656},
+ {0x00a658, 0x00a659, 0x00a658, 0x00a658},
+ {0x00a659, 0x00a659, 0x00a658, 0x00a658},
+ {0x00a65a, 0x00a65b, 0x00a65a, 0x00a65a},
+ {0x00a65b, 0x00a65b, 0x00a65a, 0x00a65a},
+ {0x00a65c, 0x00a65d, 0x00a65c, 0x00a65c},
+ {0x00a65d, 0x00a65d, 0x00a65c, 0x00a65c},
+ {0x00a65e, 0x00a65f, 0x00a65e, 0x00a65e},
+ {0x00a65f, 0x00a65f, 0x00a65e, 0x00a65e},
+ {0x00a660, 0x00a661, 0x00a660, 0x00a660},
+ {0x00a661, 0x00a661, 0x00a660, 0x00a660},
+ {0x00a662, 0x00a663, 0x00a662, 0x00a662},
+ {0x00a663, 0x00a663, 0x00a662, 0x00a662},
+ {0x00a664, 0x00a665, 0x00a664, 0x00a664},
+ {0x00a665, 0x00a665, 0x00a664, 0x00a664},
+ {0x00a666, 0x00a667, 0x00a666, 0x00a666},
+ {0x00a667, 0x00a667, 0x00a666, 0x00a666},
+ {0x00a668, 0x00a669, 0x00a668, 0x00a668},
+ {0x00a669, 0x00a669, 0x00a668, 0x00a668},
+ {0x00a66a, 0x00a66b, 0x00a66a, 0x00a66a},
+ {0x00a66b, 0x00a66b, 0x00a66a, 0x00a66a},
+ {0x00a66c, 0x00a66d, 0x00a66c, 0x00a66c},
+ {0x00a66d, 0x00a66d, 0x00a66c, 0x00a66c},
+ {0x00a680, 0x00a681, 0x00a680, 0x00a680},
+ {0x00a681, 0x00a681, 0x00a680, 0x00a680},
+ {0x00a682, 0x00a683, 0x00a682, 0x00a682},
+ {0x00a683, 0x00a683, 0x00a682, 0x00a682},
+ {0x00a684, 0x00a685, 0x00a684, 0x00a684},
+ {0x00a685, 0x00a685, 0x00a684, 0x00a684},
+ {0x00a686, 0x00a687, 0x00a686, 0x00a686},
+ {0x00a687, 0x00a687, 0x00a686, 0x00a686},
+ {0x00a688, 0x00a689, 0x00a688, 0x00a688},
+ {0x00a689, 0x00a689, 0x00a688, 0x00a688},
+ {0x00a68a, 0x00a68b, 0x00a68a, 0x00a68a},
+ {0x00a68b, 0x00a68b, 0x00a68a, 0x00a68a},
+ {0x00a68c, 0x00a68d, 0x00a68c, 0x00a68c},
+ {0x00a68d, 0x00a68d, 0x00a68c, 0x00a68c},
+ {0x00a68e, 0x00a68f, 0x00a68e, 0x00a68e},
+ {0x00a68f, 0x00a68f, 0x00a68e, 0x00a68e},
+ {0x00a690, 0x00a691, 0x00a690, 0x00a690},
+ {0x00a691, 0x00a691, 0x00a690, 0x00a690},
+ {0x00a692, 0x00a693, 0x00a692, 0x00a692},
+ {0x00a693, 0x00a693, 0x00a692, 0x00a692},
+ {0x00a694, 0x00a695, 0x00a694, 0x00a694},
+ {0x00a695, 0x00a695, 0x00a694, 0x00a694},
+ {0x00a696, 0x00a697, 0x00a696, 0x00a696},
+ {0x00a697, 0x00a697, 0x00a696, 0x00a696},
+ {0x00a698, 0x00a699, 0x00a698, 0x00a698},
+ {0x00a699, 0x00a699, 0x00a698, 0x00a698},
+ {0x00a69a, 0x00a69b, 0x00a69a, 0x00a69a},
+ {0x00a69b, 0x00a69b, 0x00a69a, 0x00a69a},
+ {0x00a722, 0x00a723, 0x00a722, 0x00a722},
+ {0x00a723, 0x00a723, 0x00a722, 0x00a722},
+ {0x00a724, 0x00a725, 0x00a724, 0x00a724},
+ {0x00a725, 0x00a725, 0x00a724, 0x00a724},
+ {0x00a726, 0x00a727, 0x00a726, 0x00a726},
+ {0x00a727, 0x00a727, 0x00a726, 0x00a726},
+ {0x00a728, 0x00a729, 0x00a728, 0x00a728},
+ {0x00a729, 0x00a729, 0x00a728, 0x00a728},
+ {0x00a72a, 0x00a72b, 0x00a72a, 0x00a72a},
+ {0x00a72b, 0x00a72b, 0x00a72a, 0x00a72a},
+ {0x00a72c, 0x00a72d, 0x00a72c, 0x00a72c},
+ {0x00a72d, 0x00a72d, 0x00a72c, 0x00a72c},
+ {0x00a72e, 0x00a72f, 0x00a72e, 0x00a72e},
+ {0x00a72f, 0x00a72f, 0x00a72e, 0x00a72e},
+ {0x00a732, 0x00a733, 0x00a732, 0x00a732},
+ {0x00a733, 0x00a733, 0x00a732, 0x00a732},
+ {0x00a734, 0x00a735, 0x00a734, 0x00a734},
+ {0x00a735, 0x00a735, 0x00a734, 0x00a734},
+ {0x00a736, 0x00a737, 0x00a736, 0x00a736},
+ {0x00a737, 0x00a737, 0x00a736, 0x00a736},
+ {0x00a738, 0x00a739, 0x00a738, 0x00a738},
+ {0x00a739, 0x00a739, 0x00a738, 0x00a738},
+ {0x00a73a, 0x00a73b, 0x00a73a, 0x00a73a},
+ {0x00a73b, 0x00a73b, 0x00a73a, 0x00a73a},
+ {0x00a73c, 0x00a73d, 0x00a73c, 0x00a73c},
+ {0x00a73d, 0x00a73d, 0x00a73c, 0x00a73c},
+ {0x00a73e, 0x00a73f, 0x00a73e, 0x00a73e},
+ {0x00a73f, 0x00a73f, 0x00a73e, 0x00a73e},
+ {0x00a740, 0x00a741, 0x00a740, 0x00a740},
+ {0x00a741, 0x00a741, 0x00a740, 0x00a740},
+ {0x00a742, 0x00a743, 0x00a742, 0x00a742},
+ {0x00a743, 0x00a743, 0x00a742, 0x00a742},
+ {0x00a744, 0x00a745, 0x00a744, 0x00a744},
+ {0x00a745, 0x00a745, 0x00a744, 0x00a744},
+ {0x00a746, 0x00a747, 0x00a746, 0x00a746},
+ {0x00a747, 0x00a747, 0x00a746, 0x00a746},
+ {0x00a748, 0x00a749, 0x00a748, 0x00a748},
+ {0x00a749, 0x00a749, 0x00a748, 0x00a748},
+ {0x00a74a, 0x00a74b, 0x00a74a, 0x00a74a},
+ {0x00a74b, 0x00a74b, 0x00a74a, 0x00a74a},
+ {0x00a74c, 0x00a74d, 0x00a74c, 0x00a74c},
+ {0x00a74d, 0x00a74d, 0x00a74c, 0x00a74c},
+ {0x00a74e, 0x00a74f, 0x00a74e, 0x00a74e},
+ {0x00a74f, 0x00a74f, 0x00a74e, 0x00a74e},
+ {0x00a750, 0x00a751, 0x00a750, 0x00a750},
+ {0x00a751, 0x00a751, 0x00a750, 0x00a750},
+ {0x00a752, 0x00a753, 0x00a752, 0x00a752},
+ {0x00a753, 0x00a753, 0x00a752, 0x00a752},
+ {0x00a754, 0x00a755, 0x00a754, 0x00a754},
+ {0x00a755, 0x00a755, 0x00a754, 0x00a754},
+ {0x00a756, 0x00a757, 0x00a756, 0x00a756},
+ {0x00a757, 0x00a757, 0x00a756, 0x00a756},
+ {0x00a758, 0x00a759, 0x00a758, 0x00a758},
+ {0x00a759, 0x00a759, 0x00a758, 0x00a758},
+ {0x00a75a, 0x00a75b, 0x00a75a, 0x00a75a},
+ {0x00a75b, 0x00a75b, 0x00a75a, 0x00a75a},
+ {0x00a75c, 0x00a75d, 0x00a75c, 0x00a75c},
+ {0x00a75d, 0x00a75d, 0x00a75c, 0x00a75c},
+ {0x00a75e, 0x00a75f, 0x00a75e, 0x00a75e},
+ {0x00a75f, 0x00a75f, 0x00a75e, 0x00a75e},
+ {0x00a760, 0x00a761, 0x00a760, 0x00a760},
+ {0x00a761, 0x00a761, 0x00a760, 0x00a760},
+ {0x00a762, 0x00a763, 0x00a762, 0x00a762},
+ {0x00a763, 0x00a763, 0x00a762, 0x00a762},
+ {0x00a764, 0x00a765, 0x00a764, 0x00a764},
+ {0x00a765, 0x00a765, 0x00a764, 0x00a764},
+ {0x00a766, 0x00a767, 0x00a766, 0x00a766},
+ {0x00a767, 0x00a767, 0x00a766, 0x00a766},
+ {0x00a768, 0x00a769, 0x00a768, 0x00a768},
+ {0x00a769, 0x00a769, 0x00a768, 0x00a768},
+ {0x00a76a, 0x00a76b, 0x00a76a, 0x00a76a},
+ {0x00a76b, 0x00a76b, 0x00a76a, 0x00a76a},
+ {0x00a76c, 0x00a76d, 0x00a76c, 0x00a76c},
+ {0x00a76d, 0x00a76d, 0x00a76c, 0x00a76c},
+ {0x00a76e, 0x00a76f, 0x00a76e, 0x00a76e},
+ {0x00a76f, 0x00a76f, 0x00a76e, 0x00a76e},
+ {0x00a779, 0x00a77a, 0x00a779, 0x00a779},
+ {0x00a77a, 0x00a77a, 0x00a779, 0x00a779},
+ {0x00a77b, 0x00a77c, 0x00a77b, 0x00a77b},
+ {0x00a77c, 0x00a77c, 0x00a77b, 0x00a77b},
+ {0x00a77d, 0x001d79, 0x00a77d, 0x00a77d},
+ {0x00a77e, 0x00a77f, 0x00a77e, 0x00a77e},
+ {0x00a77f, 0x00a77f, 0x00a77e, 0x00a77e},
+ {0x00a780, 0x00a781, 0x00a780, 0x00a780},
+ {0x00a781, 0x00a781, 0x00a780, 0x00a780},
+ {0x00a782, 0x00a783, 0x00a782, 0x00a782},
+ {0x00a783, 0x00a783, 0x00a782, 0x00a782},
+ {0x00a784, 0x00a785, 0x00a784, 0x00a784},
+ {0x00a785, 0x00a785, 0x00a784, 0x00a784},
+ {0x00a786, 0x00a787, 0x00a786, 0x00a786},
+ {0x00a787, 0x00a787, 0x00a786, 0x00a786},
+ {0x00a78b, 0x00a78c, 0x00a78b, 0x00a78b},
+ {0x00a78c, 0x00a78c, 0x00a78b, 0x00a78b},
+ {0x00a78d, 0x000265, 0x00a78d, 0x00a78d},
+ {0x00a790, 0x00a791, 0x00a790, 0x00a790},
+ {0x00a791, 0x00a791, 0x00a790, 0x00a790},
+ {0x00a792, 0x00a793, 0x00a792, 0x00a792},
+ {0x00a793, 0x00a793, 0x00a792, 0x00a792},
+ {0x00a794, 0x00a794, 0x00a7c4, 0x00a7c4},
+ {0x00a796, 0x00a797, 0x00a796, 0x00a796},
+ {0x00a797, 0x00a797, 0x00a796, 0x00a796},
+ {0x00a798, 0x00a799, 0x00a798, 0x00a798},
+ {0x00a799, 0x00a799, 0x00a798, 0x00a798},
+ {0x00a79a, 0x00a79b, 0x00a79a, 0x00a79a},
+ {0x00a79b, 0x00a79b, 0x00a79a, 0x00a79a},
+ {0x00a79c, 0x00a79d, 0x00a79c, 0x00a79c},
+ {0x00a79d, 0x00a79d, 0x00a79c, 0x00a79c},
+ {0x00a79e, 0x00a79f, 0x00a79e, 0x00a79e},
+ {0x00a79f, 0x00a79f, 0x00a79e, 0x00a79e},
+ {0x00a7a0, 0x00a7a1, 0x00a7a0, 0x00a7a0},
+ {0x00a7a1, 0x00a7a1, 0x00a7a0, 0x00a7a0},
+ {0x00a7a2, 0x00a7a3, 0x00a7a2, 0x00a7a2},
+ {0x00a7a3, 0x00a7a3, 0x00a7a2, 0x00a7a2},
+ {0x00a7a4, 0x00a7a5, 0x00a7a4, 0x00a7a4},
+ {0x00a7a5, 0x00a7a5, 0x00a7a4, 0x00a7a4},
+ {0x00a7a6, 0x00a7a7, 0x00a7a6, 0x00a7a6},
+ {0x00a7a7, 0x00a7a7, 0x00a7a6, 0x00a7a6},
+ {0x00a7a8, 0x00a7a9, 0x00a7a8, 0x00a7a8},
+ {0x00a7a9, 0x00a7a9, 0x00a7a8, 0x00a7a8},
+ {0x00a7aa, 0x000266, 0x00a7aa, 0x00a7aa},
+ {0x00a7ab, 0x00025c, 0x00a7ab, 0x00a7ab},
+ {0x00a7ac, 0x000261, 0x00a7ac, 0x00a7ac},
+ {0x00a7ad, 0x00026c, 0x00a7ad, 0x00a7ad},
+ {0x00a7ae, 0x00026a, 0x00a7ae, 0x00a7ae},
+ {0x00a7b0, 0x00029e, 0x00a7b0, 0x00a7b0},
+ {0x00a7b1, 0x000287, 0x00a7b1, 0x00a7b1},
+ {0x00a7b2, 0x00029d, 0x00a7b2, 0x00a7b2},
+ {0x00a7b3, 0x00ab53, 0x00a7b3, 0x00a7b3},
+ {0x00a7b4, 0x00a7b5, 0x00a7b4, 0x00a7b4},
+ {0x00a7b5, 0x00a7b5, 0x00a7b4, 0x00a7b4},
+ {0x00a7b6, 0x00a7b7, 0x00a7b6, 0x00a7b6},
+ {0x00a7b7, 0x00a7b7, 0x00a7b6, 0x00a7b6},
+ {0x00a7b8, 0x00a7b9, 0x00a7b8, 0x00a7b8},
+ {0x00a7b9, 0x00a7b9, 0x00a7b8, 0x00a7b8},
+ {0x00a7ba, 0x00a7bb, 0x00a7ba, 0x00a7ba},
+ {0x00a7bb, 0x00a7bb, 0x00a7ba, 0x00a7ba},
+ {0x00a7bc, 0x00a7bd, 0x00a7bc, 0x00a7bc},
+ {0x00a7bd, 0x00a7bd, 0x00a7bc, 0x00a7bc},
+ {0x00a7be, 0x00a7bf, 0x00a7be, 0x00a7be},
+ {0x00a7bf, 0x00a7bf, 0x00a7be, 0x00a7be},
+ {0x00a7c0, 0x00a7c1, 0x00a7c0, 0x00a7c0},
+ {0x00a7c1, 0x00a7c1, 0x00a7c0, 0x00a7c0},
+ {0x00a7c2, 0x00a7c3, 0x00a7c2, 0x00a7c2},
+ {0x00a7c3, 0x00a7c3, 0x00a7c2, 0x00a7c2},
+ {0x00a7c4, 0x00a794, 0x00a7c4, 0x00a7c4},
+ {0x00a7c5, 0x000282, 0x00a7c5, 0x00a7c5},
+ {0x00a7c6, 0x001d8e, 0x00a7c6, 0x00a7c6},
+ {0x00a7c7, 0x00a7c8, 0x00a7c7, 0x00a7c7},
+ {0x00a7c8, 0x00a7c8, 0x00a7c7, 0x00a7c7},
+ {0x00a7c9, 0x00a7ca, 0x00a7c9, 0x00a7c9},
+ {0x00a7ca, 0x00a7ca, 0x00a7c9, 0x00a7c9},
+ {0x00a7d0, 0x00a7d1, 0x00a7d0, 0x00a7d0},
+ {0x00a7d1, 0x00a7d1, 0x00a7d0, 0x00a7d0},
+ {0x00a7d6, 0x00a7d7, 0x00a7d6, 0x00a7d6},
+ {0x00a7d7, 0x00a7d7, 0x00a7d6, 0x00a7d6},
+ {0x00a7d8, 0x00a7d9, 0x00a7d8, 0x00a7d8},
+ {0x00a7d9, 0x00a7d9, 0x00a7d8, 0x00a7d8},
+ {0x00a7f5, 0x00a7f6, 0x00a7f5, 0x00a7f5},
+ {0x00a7f6, 0x00a7f6, 0x00a7f5, 0x00a7f5},
+ {0x00ab53, 0x00ab53, 0x00a7b3, 0x00a7b3},
+ {0x00ab70, 0x00ab70, 0x0013a0, 0x0013a0},
+ {0x00ab71, 0x00ab71, 0x0013a1, 0x0013a1},
+ {0x00ab72, 0x00ab72, 0x0013a2, 0x0013a2},
+ {0x00ab73, 0x00ab73, 0x0013a3, 0x0013a3},
+ {0x00ab74, 0x00ab74, 0x0013a4, 0x0013a4},
+ {0x00ab75, 0x00ab75, 0x0013a5, 0x0013a5},
+ {0x00ab76, 0x00ab76, 0x0013a6, 0x0013a6},
+ {0x00ab77, 0x00ab77, 0x0013a7, 0x0013a7},
+ {0x00ab78, 0x00ab78, 0x0013a8, 0x0013a8},
+ {0x00ab79, 0x00ab79, 0x0013a9, 0x0013a9},
+ {0x00ab7a, 0x00ab7a, 0x0013aa, 0x0013aa},
+ {0x00ab7b, 0x00ab7b, 0x0013ab, 0x0013ab},
+ {0x00ab7c, 0x00ab7c, 0x0013ac, 0x0013ac},
+ {0x00ab7d, 0x00ab7d, 0x0013ad, 0x0013ad},
+ {0x00ab7e, 0x00ab7e, 0x0013ae, 0x0013ae},
+ {0x00ab7f, 0x00ab7f, 0x0013af, 0x0013af},
+ {0x00ab80, 0x00ab80, 0x0013b0, 0x0013b0},
+ {0x00ab81, 0x00ab81, 0x0013b1, 0x0013b1},
+ {0x00ab82, 0x00ab82, 0x0013b2, 0x0013b2},
+ {0x00ab83, 0x00ab83, 0x0013b3, 0x0013b3},
+ {0x00ab84, 0x00ab84, 0x0013b4, 0x0013b4},
+ {0x00ab85, 0x00ab85, 0x0013b5, 0x0013b5},
+ {0x00ab86, 0x00ab86, 0x0013b6, 0x0013b6},
+ {0x00ab87, 0x00ab87, 0x0013b7, 0x0013b7},
+ {0x00ab88, 0x00ab88, 0x0013b8, 0x0013b8},
+ {0x00ab89, 0x00ab89, 0x0013b9, 0x0013b9},
+ {0x00ab8a, 0x00ab8a, 0x0013ba, 0x0013ba},
+ {0x00ab8b, 0x00ab8b, 0x0013bb, 0x0013bb},
+ {0x00ab8c, 0x00ab8c, 0x0013bc, 0x0013bc},
+ {0x00ab8d, 0x00ab8d, 0x0013bd, 0x0013bd},
+ {0x00ab8e, 0x00ab8e, 0x0013be, 0x0013be},
+ {0x00ab8f, 0x00ab8f, 0x0013bf, 0x0013bf},
+ {0x00ab90, 0x00ab90, 0x0013c0, 0x0013c0},
+ {0x00ab91, 0x00ab91, 0x0013c1, 0x0013c1},
+ {0x00ab92, 0x00ab92, 0x0013c2, 0x0013c2},
+ {0x00ab93, 0x00ab93, 0x0013c3, 0x0013c3},
+ {0x00ab94, 0x00ab94, 0x0013c4, 0x0013c4},
+ {0x00ab95, 0x00ab95, 0x0013c5, 0x0013c5},
+ {0x00ab96, 0x00ab96, 0x0013c6, 0x0013c6},
+ {0x00ab97, 0x00ab97, 0x0013c7, 0x0013c7},
+ {0x00ab98, 0x00ab98, 0x0013c8, 0x0013c8},
+ {0x00ab99, 0x00ab99, 0x0013c9, 0x0013c9},
+ {0x00ab9a, 0x00ab9a, 0x0013ca, 0x0013ca},
+ {0x00ab9b, 0x00ab9b, 0x0013cb, 0x0013cb},
+ {0x00ab9c, 0x00ab9c, 0x0013cc, 0x0013cc},
+ {0x00ab9d, 0x00ab9d, 0x0013cd, 0x0013cd},
+ {0x00ab9e, 0x00ab9e, 0x0013ce, 0x0013ce},
+ {0x00ab9f, 0x00ab9f, 0x0013cf, 0x0013cf},
+ {0x00aba0, 0x00aba0, 0x0013d0, 0x0013d0},
+ {0x00aba1, 0x00aba1, 0x0013d1, 0x0013d1},
+ {0x00aba2, 0x00aba2, 0x0013d2, 0x0013d2},
+ {0x00aba3, 0x00aba3, 0x0013d3, 0x0013d3},
+ {0x00aba4, 0x00aba4, 0x0013d4, 0x0013d4},
+ {0x00aba5, 0x00aba5, 0x0013d5, 0x0013d5},
+ {0x00aba6, 0x00aba6, 0x0013d6, 0x0013d6},
+ {0x00aba7, 0x00aba7, 0x0013d7, 0x0013d7},
+ {0x00aba8, 0x00aba8, 0x0013d8, 0x0013d8},
+ {0x00aba9, 0x00aba9, 0x0013d9, 0x0013d9},
+ {0x00abaa, 0x00abaa, 0x0013da, 0x0013da},
+ {0x00abab, 0x00abab, 0x0013db, 0x0013db},
+ {0x00abac, 0x00abac, 0x0013dc, 0x0013dc},
+ {0x00abad, 0x00abad, 0x0013dd, 0x0013dd},
+ {0x00abae, 0x00abae, 0x0013de, 0x0013de},
+ {0x00abaf, 0x00abaf, 0x0013df, 0x0013df},
+ {0x00abb0, 0x00abb0, 0x0013e0, 0x0013e0},
+ {0x00abb1, 0x00abb1, 0x0013e1, 0x0013e1},
+ {0x00abb2, 0x00abb2, 0x0013e2, 0x0013e2},
+ {0x00abb3, 0x00abb3, 0x0013e3, 0x0013e3},
+ {0x00abb4, 0x00abb4, 0x0013e4, 0x0013e4},
+ {0x00abb5, 0x00abb5, 0x0013e5, 0x0013e5},
+ {0x00abb6, 0x00abb6, 0x0013e6, 0x0013e6},
+ {0x00abb7, 0x00abb7, 0x0013e7, 0x0013e7},
+ {0x00abb8, 0x00abb8, 0x0013e8, 0x0013e8},
+ {0x00abb9, 0x00abb9, 0x0013e9, 0x0013e9},
+ {0x00abba, 0x00abba, 0x0013ea, 0x0013ea},
+ {0x00abbb, 0x00abbb, 0x0013eb, 0x0013eb},
+ {0x00abbc, 0x00abbc, 0x0013ec, 0x0013ec},
+ {0x00abbd, 0x00abbd, 0x0013ed, 0x0013ed},
+ {0x00abbe, 0x00abbe, 0x0013ee, 0x0013ee},
+ {0x00abbf, 0x00abbf, 0x0013ef, 0x0013ef},
+ {0x00ff21, 0x00ff41, 0x00ff21, 0x00ff21},
+ {0x00ff22, 0x00ff42, 0x00ff22, 0x00ff22},
+ {0x00ff23, 0x00ff43, 0x00ff23, 0x00ff23},
+ {0x00ff24, 0x00ff44, 0x00ff24, 0x00ff24},
+ {0x00ff25, 0x00ff45, 0x00ff25, 0x00ff25},
+ {0x00ff26, 0x00ff46, 0x00ff26, 0x00ff26},
+ {0x00ff27, 0x00ff47, 0x00ff27, 0x00ff27},
+ {0x00ff28, 0x00ff48, 0x00ff28, 0x00ff28},
+ {0x00ff29, 0x00ff49, 0x00ff29, 0x00ff29},
+ {0x00ff2a, 0x00ff4a, 0x00ff2a, 0x00ff2a},
+ {0x00ff2b, 0x00ff4b, 0x00ff2b, 0x00ff2b},
+ {0x00ff2c, 0x00ff4c, 0x00ff2c, 0x00ff2c},
+ {0x00ff2d, 0x00ff4d, 0x00ff2d, 0x00ff2d},
+ {0x00ff2e, 0x00ff4e, 0x00ff2e, 0x00ff2e},
+ {0x00ff2f, 0x00ff4f, 0x00ff2f, 0x00ff2f},
+ {0x00ff30, 0x00ff50, 0x00ff30, 0x00ff30},
+ {0x00ff31, 0x00ff51, 0x00ff31, 0x00ff31},
+ {0x00ff32, 0x00ff52, 0x00ff32, 0x00ff32},
+ {0x00ff33, 0x00ff53, 0x00ff33, 0x00ff33},
+ {0x00ff34, 0x00ff54, 0x00ff34, 0x00ff34},
+ {0x00ff35, 0x00ff55, 0x00ff35, 0x00ff35},
+ {0x00ff36, 0x00ff56, 0x00ff36, 0x00ff36},
+ {0x00ff37, 0x00ff57, 0x00ff37, 0x00ff37},
+ {0x00ff38, 0x00ff58, 0x00ff38, 0x00ff38},
+ {0x00ff39, 0x00ff59, 0x00ff39, 0x00ff39},
+ {0x00ff3a, 0x00ff5a, 0x00ff3a, 0x00ff3a},
+ {0x00ff41, 0x00ff41, 0x00ff21, 0x00ff21},
+ {0x00ff42, 0x00ff42, 0x00ff22, 0x00ff22},
+ {0x00ff43, 0x00ff43, 0x00ff23, 0x00ff23},
+ {0x00ff44, 0x00ff44, 0x00ff24, 0x00ff24},
+ {0x00ff45, 0x00ff45, 0x00ff25, 0x00ff25},
+ {0x00ff46, 0x00ff46, 0x00ff26, 0x00ff26},
+ {0x00ff47, 0x00ff47, 0x00ff27, 0x00ff27},
+ {0x00ff48, 0x00ff48, 0x00ff28, 0x00ff28},
+ {0x00ff49, 0x00ff49, 0x00ff29, 0x00ff29},
+ {0x00ff4a, 0x00ff4a, 0x00ff2a, 0x00ff2a},
+ {0x00ff4b, 0x00ff4b, 0x00ff2b, 0x00ff2b},
+ {0x00ff4c, 0x00ff4c, 0x00ff2c, 0x00ff2c},
+ {0x00ff4d, 0x00ff4d, 0x00ff2d, 0x00ff2d},
+ {0x00ff4e, 0x00ff4e, 0x00ff2e, 0x00ff2e},
+ {0x00ff4f, 0x00ff4f, 0x00ff2f, 0x00ff2f},
+ {0x00ff50, 0x00ff50, 0x00ff30, 0x00ff30},
+ {0x00ff51, 0x00ff51, 0x00ff31, 0x00ff31},
+ {0x00ff52, 0x00ff52, 0x00ff32, 0x00ff32},
+ {0x00ff53, 0x00ff53, 0x00ff33, 0x00ff33},
+ {0x00ff54, 0x00ff54, 0x00ff34, 0x00ff34},
+ {0x00ff55, 0x00ff55, 0x00ff35, 0x00ff35},
+ {0x00ff56, 0x00ff56, 0x00ff36, 0x00ff36},
+ {0x00ff57, 0x00ff57, 0x00ff37, 0x00ff37},
+ {0x00ff58, 0x00ff58, 0x00ff38, 0x00ff38},
+ {0x00ff59, 0x00ff59, 0x00ff39, 0x00ff39},
+ {0x00ff5a, 0x00ff5a, 0x00ff3a, 0x00ff3a},
+ {0x010400, 0x010428, 0x010400, 0x010400},
+ {0x010401, 0x010429, 0x010401, 0x010401},
+ {0x010402, 0x01042a, 0x010402, 0x010402},
+ {0x010403, 0x01042b, 0x010403, 0x010403},
+ {0x010404, 0x01042c, 0x010404, 0x010404},
+ {0x010405, 0x01042d, 0x010405, 0x010405},
+ {0x010406, 0x01042e, 0x010406, 0x010406},
+ {0x010407, 0x01042f, 0x010407, 0x010407},
+ {0x010408, 0x010430, 0x010408, 0x010408},
+ {0x010409, 0x010431, 0x010409, 0x010409},
+ {0x01040a, 0x010432, 0x01040a, 0x01040a},
+ {0x01040b, 0x010433, 0x01040b, 0x01040b},
+ {0x01040c, 0x010434, 0x01040c, 0x01040c},
+ {0x01040d, 0x010435, 0x01040d, 0x01040d},
+ {0x01040e, 0x010436, 0x01040e, 0x01040e},
+ {0x01040f, 0x010437, 0x01040f, 0x01040f},
+ {0x010410, 0x010438, 0x010410, 0x010410},
+ {0x010411, 0x010439, 0x010411, 0x010411},
+ {0x010412, 0x01043a, 0x010412, 0x010412},
+ {0x010413, 0x01043b, 0x010413, 0x010413},
+ {0x010414, 0x01043c, 0x010414, 0x010414},
+ {0x010415, 0x01043d, 0x010415, 0x010415},
+ {0x010416, 0x01043e, 0x010416, 0x010416},
+ {0x010417, 0x01043f, 0x010417, 0x010417},
+ {0x010418, 0x010440, 0x010418, 0x010418},
+ {0x010419, 0x010441, 0x010419, 0x010419},
+ {0x01041a, 0x010442, 0x01041a, 0x01041a},
+ {0x01041b, 0x010443, 0x01041b, 0x01041b},
+ {0x01041c, 0x010444, 0x01041c, 0x01041c},
+ {0x01041d, 0x010445, 0x01041d, 0x01041d},
+ {0x01041e, 0x010446, 0x01041e, 0x01041e},
+ {0x01041f, 0x010447, 0x01041f, 0x01041f},
+ {0x010420, 0x010448, 0x010420, 0x010420},
+ {0x010421, 0x010449, 0x010421, 0x010421},
+ {0x010422, 0x01044a, 0x010422, 0x010422},
+ {0x010423, 0x01044b, 0x010423, 0x010423},
+ {0x010424, 0x01044c, 0x010424, 0x010424},
+ {0x010425, 0x01044d, 0x010425, 0x010425},
+ {0x010426, 0x01044e, 0x010426, 0x010426},
+ {0x010427, 0x01044f, 0x010427, 0x010427},
+ {0x010428, 0x010428, 0x010400, 0x010400},
+ {0x010429, 0x010429, 0x010401, 0x010401},
+ {0x01042a, 0x01042a, 0x010402, 0x010402},
+ {0x01042b, 0x01042b, 0x010403, 0x010403},
+ {0x01042c, 0x01042c, 0x010404, 0x010404},
+ {0x01042d, 0x01042d, 0x010405, 0x010405},
+ {0x01042e, 0x01042e, 0x010406, 0x010406},
+ {0x01042f, 0x01042f, 0x010407, 0x010407},
+ {0x010430, 0x010430, 0x010408, 0x010408},
+ {0x010431, 0x010431, 0x010409, 0x010409},
+ {0x010432, 0x010432, 0x01040a, 0x01040a},
+ {0x010433, 0x010433, 0x01040b, 0x01040b},
+ {0x010434, 0x010434, 0x01040c, 0x01040c},
+ {0x010435, 0x010435, 0x01040d, 0x01040d},
+ {0x010436, 0x010436, 0x01040e, 0x01040e},
+ {0x010437, 0x010437, 0x01040f, 0x01040f},
+ {0x010438, 0x010438, 0x010410, 0x010410},
+ {0x010439, 0x010439, 0x010411, 0x010411},
+ {0x01043a, 0x01043a, 0x010412, 0x010412},
+ {0x01043b, 0x01043b, 0x010413, 0x010413},
+ {0x01043c, 0x01043c, 0x010414, 0x010414},
+ {0x01043d, 0x01043d, 0x010415, 0x010415},
+ {0x01043e, 0x01043e, 0x010416, 0x010416},
+ {0x01043f, 0x01043f, 0x010417, 0x010417},
+ {0x010440, 0x010440, 0x010418, 0x010418},
+ {0x010441, 0x010441, 0x010419, 0x010419},
+ {0x010442, 0x010442, 0x01041a, 0x01041a},
+ {0x010443, 0x010443, 0x01041b, 0x01041b},
+ {0x010444, 0x010444, 0x01041c, 0x01041c},
+ {0x010445, 0x010445, 0x01041d, 0x01041d},
+ {0x010446, 0x010446, 0x01041e, 0x01041e},
+ {0x010447, 0x010447, 0x01041f, 0x01041f},
+ {0x010448, 0x010448, 0x010420, 0x010420},
+ {0x010449, 0x010449, 0x010421, 0x010421},
+ {0x01044a, 0x01044a, 0x010422, 0x010422},
+ {0x01044b, 0x01044b, 0x010423, 0x010423},
+ {0x01044c, 0x01044c, 0x010424, 0x010424},
+ {0x01044d, 0x01044d, 0x010425, 0x010425},
+ {0x01044e, 0x01044e, 0x010426, 0x010426},
+ {0x01044f, 0x01044f, 0x010427, 0x010427},
+ {0x0104b0, 0x0104d8, 0x0104b0, 0x0104b0},
+ {0x0104b1, 0x0104d9, 0x0104b1, 0x0104b1},
+ {0x0104b2, 0x0104da, 0x0104b2, 0x0104b2},
+ {0x0104b3, 0x0104db, 0x0104b3, 0x0104b3},
+ {0x0104b4, 0x0104dc, 0x0104b4, 0x0104b4},
+ {0x0104b5, 0x0104dd, 0x0104b5, 0x0104b5},
+ {0x0104b6, 0x0104de, 0x0104b6, 0x0104b6},
+ {0x0104b7, 0x0104df, 0x0104b7, 0x0104b7},
+ {0x0104b8, 0x0104e0, 0x0104b8, 0x0104b8},
+ {0x0104b9, 0x0104e1, 0x0104b9, 0x0104b9},
+ {0x0104ba, 0x0104e2, 0x0104ba, 0x0104ba},
+ {0x0104bb, 0x0104e3, 0x0104bb, 0x0104bb},
+ {0x0104bc, 0x0104e4, 0x0104bc, 0x0104bc},
+ {0x0104bd, 0x0104e5, 0x0104bd, 0x0104bd},
+ {0x0104be, 0x0104e6, 0x0104be, 0x0104be},
+ {0x0104bf, 0x0104e7, 0x0104bf, 0x0104bf},
+ {0x0104c0, 0x0104e8, 0x0104c0, 0x0104c0},
+ {0x0104c1, 0x0104e9, 0x0104c1, 0x0104c1},
+ {0x0104c2, 0x0104ea, 0x0104c2, 0x0104c2},
+ {0x0104c3, 0x0104eb, 0x0104c3, 0x0104c3},
+ {0x0104c4, 0x0104ec, 0x0104c4, 0x0104c4},
+ {0x0104c5, 0x0104ed, 0x0104c5, 0x0104c5},
+ {0x0104c6, 0x0104ee, 0x0104c6, 0x0104c6},
+ {0x0104c7, 0x0104ef, 0x0104c7, 0x0104c7},
+ {0x0104c8, 0x0104f0, 0x0104c8, 0x0104c8},
+ {0x0104c9, 0x0104f1, 0x0104c9, 0x0104c9},
+ {0x0104ca, 0x0104f2, 0x0104ca, 0x0104ca},
+ {0x0104cb, 0x0104f3, 0x0104cb, 0x0104cb},
+ {0x0104cc, 0x0104f4, 0x0104cc, 0x0104cc},
+ {0x0104cd, 0x0104f5, 0x0104cd, 0x0104cd},
+ {0x0104ce, 0x0104f6, 0x0104ce, 0x0104ce},
+ {0x0104cf, 0x0104f7, 0x0104cf, 0x0104cf},
+ {0x0104d0, 0x0104f8, 0x0104d0, 0x0104d0},
+ {0x0104d1, 0x0104f9, 0x0104d1, 0x0104d1},
+ {0x0104d2, 0x0104fa, 0x0104d2, 0x0104d2},
+ {0x0104d3, 0x0104fb, 0x0104d3, 0x0104d3},
+ {0x0104d8, 0x0104d8, 0x0104b0, 0x0104b0},
+ {0x0104d9, 0x0104d9, 0x0104b1, 0x0104b1},
+ {0x0104da, 0x0104da, 0x0104b2, 0x0104b2},
+ {0x0104db, 0x0104db, 0x0104b3, 0x0104b3},
+ {0x0104dc, 0x0104dc, 0x0104b4, 0x0104b4},
+ {0x0104dd, 0x0104dd, 0x0104b5, 0x0104b5},
+ {0x0104de, 0x0104de, 0x0104b6, 0x0104b6},
+ {0x0104df, 0x0104df, 0x0104b7, 0x0104b7},
+ {0x0104e0, 0x0104e0, 0x0104b8, 0x0104b8},
+ {0x0104e1, 0x0104e1, 0x0104b9, 0x0104b9},
+ {0x0104e2, 0x0104e2, 0x0104ba, 0x0104ba},
+ {0x0104e3, 0x0104e3, 0x0104bb, 0x0104bb},
+ {0x0104e4, 0x0104e4, 0x0104bc, 0x0104bc},
+ {0x0104e5, 0x0104e5, 0x0104bd, 0x0104bd},
+ {0x0104e6, 0x0104e6, 0x0104be, 0x0104be},
+ {0x0104e7, 0x0104e7, 0x0104bf, 0x0104bf},
+ {0x0104e8, 0x0104e8, 0x0104c0, 0x0104c0},
+ {0x0104e9, 0x0104e9, 0x0104c1, 0x0104c1},
+ {0x0104ea, 0x0104ea, 0x0104c2, 0x0104c2},
+ {0x0104eb, 0x0104eb, 0x0104c3, 0x0104c3},
+ {0x0104ec, 0x0104ec, 0x0104c4, 0x0104c4},
+ {0x0104ed, 0x0104ed, 0x0104c5, 0x0104c5},
+ {0x0104ee, 0x0104ee, 0x0104c6, 0x0104c6},
+ {0x0104ef, 0x0104ef, 0x0104c7, 0x0104c7},
+ {0x0104f0, 0x0104f0, 0x0104c8, 0x0104c8},
+ {0x0104f1, 0x0104f1, 0x0104c9, 0x0104c9},
+ {0x0104f2, 0x0104f2, 0x0104ca, 0x0104ca},
+ {0x0104f3, 0x0104f3, 0x0104cb, 0x0104cb},
+ {0x0104f4, 0x0104f4, 0x0104cc, 0x0104cc},
+ {0x0104f5, 0x0104f5, 0x0104cd, 0x0104cd},
+ {0x0104f6, 0x0104f6, 0x0104ce, 0x0104ce},
+ {0x0104f7, 0x0104f7, 0x0104cf, 0x0104cf},
+ {0x0104f8, 0x0104f8, 0x0104d0, 0x0104d0},
+ {0x0104f9, 0x0104f9, 0x0104d1, 0x0104d1},
+ {0x0104fa, 0x0104fa, 0x0104d2, 0x0104d2},
+ {0x0104fb, 0x0104fb, 0x0104d3, 0x0104d3},
+ {0x010570, 0x010597, 0x010570, 0x010570},
+ {0x010571, 0x010598, 0x010571, 0x010571},
+ {0x010572, 0x010599, 0x010572, 0x010572},
+ {0x010573, 0x01059a, 0x010573, 0x010573},
+ {0x010574, 0x01059b, 0x010574, 0x010574},
+ {0x010575, 0x01059c, 0x010575, 0x010575},
+ {0x010576, 0x01059d, 0x010576, 0x010576},
+ {0x010577, 0x01059e, 0x010577, 0x010577},
+ {0x010578, 0x01059f, 0x010578, 0x010578},
+ {0x010579, 0x0105a0, 0x010579, 0x010579},
+ {0x01057a, 0x0105a1, 0x01057a, 0x01057a},
+ {0x01057c, 0x0105a3, 0x01057c, 0x01057c},
+ {0x01057d, 0x0105a4, 0x01057d, 0x01057d},
+ {0x01057e, 0x0105a5, 0x01057e, 0x01057e},
+ {0x01057f, 0x0105a6, 0x01057f, 0x01057f},
+ {0x010580, 0x0105a7, 0x010580, 0x010580},
+ {0x010581, 0x0105a8, 0x010581, 0x010581},
+ {0x010582, 0x0105a9, 0x010582, 0x010582},
+ {0x010583, 0x0105aa, 0x010583, 0x010583},
+ {0x010584, 0x0105ab, 0x010584, 0x010584},
+ {0x010585, 0x0105ac, 0x010585, 0x010585},
+ {0x010586, 0x0105ad, 0x010586, 0x010586},
+ {0x010587, 0x0105ae, 0x010587, 0x010587},
+ {0x010588, 0x0105af, 0x010588, 0x010588},
+ {0x010589, 0x0105b0, 0x010589, 0x010589},
+ {0x01058a, 0x0105b1, 0x01058a, 0x01058a},
+ {0x01058c, 0x0105b3, 0x01058c, 0x01058c},
+ {0x01058d, 0x0105b4, 0x01058d, 0x01058d},
+ {0x01058e, 0x0105b5, 0x01058e, 0x01058e},
+ {0x01058f, 0x0105b6, 0x01058f, 0x01058f},
+ {0x010590, 0x0105b7, 0x010590, 0x010590},
+ {0x010591, 0x0105b8, 0x010591, 0x010591},
+ {0x010592, 0x0105b9, 0x010592, 0x010592},
+ {0x010594, 0x0105bb, 0x010594, 0x010594},
+ {0x010595, 0x0105bc, 0x010595, 0x010595},
+ {0x010597, 0x010597, 0x010570, 0x010570},
+ {0x010598, 0x010598, 0x010571, 0x010571},
+ {0x010599, 0x010599, 0x010572, 0x010572},
+ {0x01059a, 0x01059a, 0x010573, 0x010573},
+ {0x01059b, 0x01059b, 0x010574, 0x010574},
+ {0x01059c, 0x01059c, 0x010575, 0x010575},
+ {0x01059d, 0x01059d, 0x010576, 0x010576},
+ {0x01059e, 0x01059e, 0x010577, 0x010577},
+ {0x01059f, 0x01059f, 0x010578, 0x010578},
+ {0x0105a0, 0x0105a0, 0x010579, 0x010579},
+ {0x0105a1, 0x0105a1, 0x01057a, 0x01057a},
+ {0x0105a3, 0x0105a3, 0x01057c, 0x01057c},
+ {0x0105a4, 0x0105a4, 0x01057d, 0x01057d},
+ {0x0105a5, 0x0105a5, 0x01057e, 0x01057e},
+ {0x0105a6, 0x0105a6, 0x01057f, 0x01057f},
+ {0x0105a7, 0x0105a7, 0x010580, 0x010580},
+ {0x0105a8, 0x0105a8, 0x010581, 0x010581},
+ {0x0105a9, 0x0105a9, 0x010582, 0x010582},
+ {0x0105aa, 0x0105aa, 0x010583, 0x010583},
+ {0x0105ab, 0x0105ab, 0x010584, 0x010584},
+ {0x0105ac, 0x0105ac, 0x010585, 0x010585},
+ {0x0105ad, 0x0105ad, 0x010586, 0x010586},
+ {0x0105ae, 0x0105ae, 0x010587, 0x010587},
+ {0x0105af, 0x0105af, 0x010588, 0x010588},
+ {0x0105b0, 0x0105b0, 0x010589, 0x010589},
+ {0x0105b1, 0x0105b1, 0x01058a, 0x01058a},
+ {0x0105b3, 0x0105b3, 0x01058c, 0x01058c},
+ {0x0105b4, 0x0105b4, 0x01058d, 0x01058d},
+ {0x0105b5, 0x0105b5, 0x01058e, 0x01058e},
+ {0x0105b6, 0x0105b6, 0x01058f, 0x01058f},
+ {0x0105b7, 0x0105b7, 0x010590, 0x010590},
+ {0x0105b8, 0x0105b8, 0x010591, 0x010591},
+ {0x0105b9, 0x0105b9, 0x010592, 0x010592},
+ {0x0105bb, 0x0105bb, 0x010594, 0x010594},
+ {0x0105bc, 0x0105bc, 0x010595, 0x010595},
+ {0x010c80, 0x010cc0, 0x010c80, 0x010c80},
+ {0x010c81, 0x010cc1, 0x010c81, 0x010c81},
+ {0x010c82, 0x010cc2, 0x010c82, 0x010c82},
+ {0x010c83, 0x010cc3, 0x010c83, 0x010c83},
+ {0x010c84, 0x010cc4, 0x010c84, 0x010c84},
+ {0x010c85, 0x010cc5, 0x010c85, 0x010c85},
+ {0x010c86, 0x010cc6, 0x010c86, 0x010c86},
+ {0x010c87, 0x010cc7, 0x010c87, 0x010c87},
+ {0x010c88, 0x010cc8, 0x010c88, 0x010c88},
+ {0x010c89, 0x010cc9, 0x010c89, 0x010c89},
+ {0x010c8a, 0x010cca, 0x010c8a, 0x010c8a},
+ {0x010c8b, 0x010ccb, 0x010c8b, 0x010c8b},
+ {0x010c8c, 0x010ccc, 0x010c8c, 0x010c8c},
+ {0x010c8d, 0x010ccd, 0x010c8d, 0x010c8d},
+ {0x010c8e, 0x010cce, 0x010c8e, 0x010c8e},
+ {0x010c8f, 0x010ccf, 0x010c8f, 0x010c8f},
+ {0x010c90, 0x010cd0, 0x010c90, 0x010c90},
+ {0x010c91, 0x010cd1, 0x010c91, 0x010c91},
+ {0x010c92, 0x010cd2, 0x010c92, 0x010c92},
+ {0x010c93, 0x010cd3, 0x010c93, 0x010c93},
+ {0x010c94, 0x010cd4, 0x010c94, 0x010c94},
+ {0x010c95, 0x010cd5, 0x010c95, 0x010c95},
+ {0x010c96, 0x010cd6, 0x010c96, 0x010c96},
+ {0x010c97, 0x010cd7, 0x010c97, 0x010c97},
+ {0x010c98, 0x010cd8, 0x010c98, 0x010c98},
+ {0x010c99, 0x010cd9, 0x010c99, 0x010c99},
+ {0x010c9a, 0x010cda, 0x010c9a, 0x010c9a},
+ {0x010c9b, 0x010cdb, 0x010c9b, 0x010c9b},
+ {0x010c9c, 0x010cdc, 0x010c9c, 0x010c9c},
+ {0x010c9d, 0x010cdd, 0x010c9d, 0x010c9d},
+ {0x010c9e, 0x010cde, 0x010c9e, 0x010c9e},
+ {0x010c9f, 0x010cdf, 0x010c9f, 0x010c9f},
+ {0x010ca0, 0x010ce0, 0x010ca0, 0x010ca0},
+ {0x010ca1, 0x010ce1, 0x010ca1, 0x010ca1},
+ {0x010ca2, 0x010ce2, 0x010ca2, 0x010ca2},
+ {0x010ca3, 0x010ce3, 0x010ca3, 0x010ca3},
+ {0x010ca4, 0x010ce4, 0x010ca4, 0x010ca4},
+ {0x010ca5, 0x010ce5, 0x010ca5, 0x010ca5},
+ {0x010ca6, 0x010ce6, 0x010ca6, 0x010ca6},
+ {0x010ca7, 0x010ce7, 0x010ca7, 0x010ca7},
+ {0x010ca8, 0x010ce8, 0x010ca8, 0x010ca8},
+ {0x010ca9, 0x010ce9, 0x010ca9, 0x010ca9},
+ {0x010caa, 0x010cea, 0x010caa, 0x010caa},
+ {0x010cab, 0x010ceb, 0x010cab, 0x010cab},
+ {0x010cac, 0x010cec, 0x010cac, 0x010cac},
+ {0x010cad, 0x010ced, 0x010cad, 0x010cad},
+ {0x010cae, 0x010cee, 0x010cae, 0x010cae},
+ {0x010caf, 0x010cef, 0x010caf, 0x010caf},
+ {0x010cb0, 0x010cf0, 0x010cb0, 0x010cb0},
+ {0x010cb1, 0x010cf1, 0x010cb1, 0x010cb1},
+ {0x010cb2, 0x010cf2, 0x010cb2, 0x010cb2},
+ {0x010cc0, 0x010cc0, 0x010c80, 0x010c80},
+ {0x010cc1, 0x010cc1, 0x010c81, 0x010c81},
+ {0x010cc2, 0x010cc2, 0x010c82, 0x010c82},
+ {0x010cc3, 0x010cc3, 0x010c83, 0x010c83},
+ {0x010cc4, 0x010cc4, 0x010c84, 0x010c84},
+ {0x010cc5, 0x010cc5, 0x010c85, 0x010c85},
+ {0x010cc6, 0x010cc6, 0x010c86, 0x010c86},
+ {0x010cc7, 0x010cc7, 0x010c87, 0x010c87},
+ {0x010cc8, 0x010cc8, 0x010c88, 0x010c88},
+ {0x010cc9, 0x010cc9, 0x010c89, 0x010c89},
+ {0x010cca, 0x010cca, 0x010c8a, 0x010c8a},
+ {0x010ccb, 0x010ccb, 0x010c8b, 0x010c8b},
+ {0x010ccc, 0x010ccc, 0x010c8c, 0x010c8c},
+ {0x010ccd, 0x010ccd, 0x010c8d, 0x010c8d},
+ {0x010cce, 0x010cce, 0x010c8e, 0x010c8e},
+ {0x010ccf, 0x010ccf, 0x010c8f, 0x010c8f},
+ {0x010cd0, 0x010cd0, 0x010c90, 0x010c90},
+ {0x010cd1, 0x010cd1, 0x010c91, 0x010c91},
+ {0x010cd2, 0x010cd2, 0x010c92, 0x010c92},
+ {0x010cd3, 0x010cd3, 0x010c93, 0x010c93},
+ {0x010cd4, 0x010cd4, 0x010c94, 0x010c94},
+ {0x010cd5, 0x010cd5, 0x010c95, 0x010c95},
+ {0x010cd6, 0x010cd6, 0x010c96, 0x010c96},
+ {0x010cd7, 0x010cd7, 0x010c97, 0x010c97},
+ {0x010cd8, 0x010cd8, 0x010c98, 0x010c98},
+ {0x010cd9, 0x010cd9, 0x010c99, 0x010c99},
+ {0x010cda, 0x010cda, 0x010c9a, 0x010c9a},
+ {0x010cdb, 0x010cdb, 0x010c9b, 0x010c9b},
+ {0x010cdc, 0x010cdc, 0x010c9c, 0x010c9c},
+ {0x010cdd, 0x010cdd, 0x010c9d, 0x010c9d},
+ {0x010cde, 0x010cde, 0x010c9e, 0x010c9e},
+ {0x010cdf, 0x010cdf, 0x010c9f, 0x010c9f},
+ {0x010ce0, 0x010ce0, 0x010ca0, 0x010ca0},
+ {0x010ce1, 0x010ce1, 0x010ca1, 0x010ca1},
+ {0x010ce2, 0x010ce2, 0x010ca2, 0x010ca2},
+ {0x010ce3, 0x010ce3, 0x010ca3, 0x010ca3},
+ {0x010ce4, 0x010ce4, 0x010ca4, 0x010ca4},
+ {0x010ce5, 0x010ce5, 0x010ca5, 0x010ca5},
+ {0x010ce6, 0x010ce6, 0x010ca6, 0x010ca6},
+ {0x010ce7, 0x010ce7, 0x010ca7, 0x010ca7},
+ {0x010ce8, 0x010ce8, 0x010ca8, 0x010ca8},
+ {0x010ce9, 0x010ce9, 0x010ca9, 0x010ca9},
+ {0x010cea, 0x010cea, 0x010caa, 0x010caa},
+ {0x010ceb, 0x010ceb, 0x010cab, 0x010cab},
+ {0x010cec, 0x010cec, 0x010cac, 0x010cac},
+ {0x010ced, 0x010ced, 0x010cad, 0x010cad},
+ {0x010cee, 0x010cee, 0x010cae, 0x010cae},
+ {0x010cef, 0x010cef, 0x010caf, 0x010caf},
+ {0x010cf0, 0x010cf0, 0x010cb0, 0x010cb0},
+ {0x010cf1, 0x010cf1, 0x010cb1, 0x010cb1},
+ {0x010cf2, 0x010cf2, 0x010cb2, 0x010cb2},
+ {0x0118a0, 0x0118c0, 0x0118a0, 0x0118a0},
+ {0x0118a1, 0x0118c1, 0x0118a1, 0x0118a1},
+ {0x0118a2, 0x0118c2, 0x0118a2, 0x0118a2},
+ {0x0118a3, 0x0118c3, 0x0118a3, 0x0118a3},
+ {0x0118a4, 0x0118c4, 0x0118a4, 0x0118a4},
+ {0x0118a5, 0x0118c5, 0x0118a5, 0x0118a5},
+ {0x0118a6, 0x0118c6, 0x0118a6, 0x0118a6},
+ {0x0118a7, 0x0118c7, 0x0118a7, 0x0118a7},
+ {0x0118a8, 0x0118c8, 0x0118a8, 0x0118a8},
+ {0x0118a9, 0x0118c9, 0x0118a9, 0x0118a9},
+ {0x0118aa, 0x0118ca, 0x0118aa, 0x0118aa},
+ {0x0118ab, 0x0118cb, 0x0118ab, 0x0118ab},
+ {0x0118ac, 0x0118cc, 0x0118ac, 0x0118ac},
+ {0x0118ad, 0x0118cd, 0x0118ad, 0x0118ad},
+ {0x0118ae, 0x0118ce, 0x0118ae, 0x0118ae},
+ {0x0118af, 0x0118cf, 0x0118af, 0x0118af},
+ {0x0118b0, 0x0118d0, 0x0118b0, 0x0118b0},
+ {0x0118b1, 0x0118d1, 0x0118b1, 0x0118b1},
+ {0x0118b2, 0x0118d2, 0x0118b2, 0x0118b2},
+ {0x0118b3, 0x0118d3, 0x0118b3, 0x0118b3},
+ {0x0118b4, 0x0118d4, 0x0118b4, 0x0118b4},
+ {0x0118b5, 0x0118d5, 0x0118b5, 0x0118b5},
+ {0x0118b6, 0x0118d6, 0x0118b6, 0x0118b6},
+ {0x0118b7, 0x0118d7, 0x0118b7, 0x0118b7},
+ {0x0118b8, 0x0118d8, 0x0118b8, 0x0118b8},
+ {0x0118b9, 0x0118d9, 0x0118b9, 0x0118b9},
+ {0x0118ba, 0x0118da, 0x0118ba, 0x0118ba},
+ {0x0118bb, 0x0118db, 0x0118bb, 0x0118bb},
+ {0x0118bc, 0x0118dc, 0x0118bc, 0x0118bc},
+ {0x0118bd, 0x0118dd, 0x0118bd, 0x0118bd},
+ {0x0118be, 0x0118de, 0x0118be, 0x0118be},
+ {0x0118bf, 0x0118df, 0x0118bf, 0x0118bf},
+ {0x0118c0, 0x0118c0, 0x0118a0, 0x0118a0},
+ {0x0118c1, 0x0118c1, 0x0118a1, 0x0118a1},
+ {0x0118c2, 0x0118c2, 0x0118a2, 0x0118a2},
+ {0x0118c3, 0x0118c3, 0x0118a3, 0x0118a3},
+ {0x0118c4, 0x0118c4, 0x0118a4, 0x0118a4},
+ {0x0118c5, 0x0118c5, 0x0118a5, 0x0118a5},
+ {0x0118c6, 0x0118c6, 0x0118a6, 0x0118a6},
+ {0x0118c7, 0x0118c7, 0x0118a7, 0x0118a7},
+ {0x0118c8, 0x0118c8, 0x0118a8, 0x0118a8},
+ {0x0118c9, 0x0118c9, 0x0118a9, 0x0118a9},
+ {0x0118ca, 0x0118ca, 0x0118aa, 0x0118aa},
+ {0x0118cb, 0x0118cb, 0x0118ab, 0x0118ab},
+ {0x0118cc, 0x0118cc, 0x0118ac, 0x0118ac},
+ {0x0118cd, 0x0118cd, 0x0118ad, 0x0118ad},
+ {0x0118ce, 0x0118ce, 0x0118ae, 0x0118ae},
+ {0x0118cf, 0x0118cf, 0x0118af, 0x0118af},
+ {0x0118d0, 0x0118d0, 0x0118b0, 0x0118b0},
+ {0x0118d1, 0x0118d1, 0x0118b1, 0x0118b1},
+ {0x0118d2, 0x0118d2, 0x0118b2, 0x0118b2},
+ {0x0118d3, 0x0118d3, 0x0118b3, 0x0118b3},
+ {0x0118d4, 0x0118d4, 0x0118b4, 0x0118b4},
+ {0x0118d5, 0x0118d5, 0x0118b5, 0x0118b5},
+ {0x0118d6, 0x0118d6, 0x0118b6, 0x0118b6},
+ {0x0118d7, 0x0118d7, 0x0118b7, 0x0118b7},
+ {0x0118d8, 0x0118d8, 0x0118b8, 0x0118b8},
+ {0x0118d9, 0x0118d9, 0x0118b9, 0x0118b9},
+ {0x0118da, 0x0118da, 0x0118ba, 0x0118ba},
+ {0x0118db, 0x0118db, 0x0118bb, 0x0118bb},
+ {0x0118dc, 0x0118dc, 0x0118bc, 0x0118bc},
+ {0x0118dd, 0x0118dd, 0x0118bd, 0x0118bd},
+ {0x0118de, 0x0118de, 0x0118be, 0x0118be},
+ {0x0118df, 0x0118df, 0x0118bf, 0x0118bf},
+ {0x016e40, 0x016e60, 0x016e40, 0x016e40},
+ {0x016e41, 0x016e61, 0x016e41, 0x016e41},
+ {0x016e42, 0x016e62, 0x016e42, 0x016e42},
+ {0x016e43, 0x016e63, 0x016e43, 0x016e43},
+ {0x016e44, 0x016e64, 0x016e44, 0x016e44},
+ {0x016e45, 0x016e65, 0x016e45, 0x016e45},
+ {0x016e46, 0x016e66, 0x016e46, 0x016e46},
+ {0x016e47, 0x016e67, 0x016e47, 0x016e47},
+ {0x016e48, 0x016e68, 0x016e48, 0x016e48},
+ {0x016e49, 0x016e69, 0x016e49, 0x016e49},
+ {0x016e4a, 0x016e6a, 0x016e4a, 0x016e4a},
+ {0x016e4b, 0x016e6b, 0x016e4b, 0x016e4b},
+ {0x016e4c, 0x016e6c, 0x016e4c, 0x016e4c},
+ {0x016e4d, 0x016e6d, 0x016e4d, 0x016e4d},
+ {0x016e4e, 0x016e6e, 0x016e4e, 0x016e4e},
+ {0x016e4f, 0x016e6f, 0x016e4f, 0x016e4f},
+ {0x016e50, 0x016e70, 0x016e50, 0x016e50},
+ {0x016e51, 0x016e71, 0x016e51, 0x016e51},
+ {0x016e52, 0x016e72, 0x016e52, 0x016e52},
+ {0x016e53, 0x016e73, 0x016e53, 0x016e53},
+ {0x016e54, 0x016e74, 0x016e54, 0x016e54},
+ {0x016e55, 0x016e75, 0x016e55, 0x016e55},
+ {0x016e56, 0x016e76, 0x016e56, 0x016e56},
+ {0x016e57, 0x016e77, 0x016e57, 0x016e57},
+ {0x016e58, 0x016e78, 0x016e58, 0x016e58},
+ {0x016e59, 0x016e79, 0x016e59, 0x016e59},
+ {0x016e5a, 0x016e7a, 0x016e5a, 0x016e5a},
+ {0x016e5b, 0x016e7b, 0x016e5b, 0x016e5b},
+ {0x016e5c, 0x016e7c, 0x016e5c, 0x016e5c},
+ {0x016e5d, 0x016e7d, 0x016e5d, 0x016e5d},
+ {0x016e5e, 0x016e7e, 0x016e5e, 0x016e5e},
+ {0x016e5f, 0x016e7f, 0x016e5f, 0x016e5f},
+ {0x016e60, 0x016e60, 0x016e40, 0x016e40},
+ {0x016e61, 0x016e61, 0x016e41, 0x016e41},
+ {0x016e62, 0x016e62, 0x016e42, 0x016e42},
+ {0x016e63, 0x016e63, 0x016e43, 0x016e43},
+ {0x016e64, 0x016e64, 0x016e44, 0x016e44},
+ {0x016e65, 0x016e65, 0x016e45, 0x016e45},
+ {0x016e66, 0x016e66, 0x016e46, 0x016e46},
+ {0x016e67, 0x016e67, 0x016e47, 0x016e47},
+ {0x016e68, 0x016e68, 0x016e48, 0x016e48},
+ {0x016e69, 0x016e69, 0x016e49, 0x016e49},
+ {0x016e6a, 0x016e6a, 0x016e4a, 0x016e4a},
+ {0x016e6b, 0x016e6b, 0x016e4b, 0x016e4b},
+ {0x016e6c, 0x016e6c, 0x016e4c, 0x016e4c},
+ {0x016e6d, 0x016e6d, 0x016e4d, 0x016e4d},
+ {0x016e6e, 0x016e6e, 0x016e4e, 0x016e4e},
+ {0x016e6f, 0x016e6f, 0x016e4f, 0x016e4f},
+ {0x016e70, 0x016e70, 0x016e50, 0x016e50},
+ {0x016e71, 0x016e71, 0x016e51, 0x016e51},
+ {0x016e72, 0x016e72, 0x016e52, 0x016e52},
+ {0x016e73, 0x016e73, 0x016e53, 0x016e53},
+ {0x016e74, 0x016e74, 0x016e54, 0x016e54},
+ {0x016e75, 0x016e75, 0x016e55, 0x016e55},
+ {0x016e76, 0x016e76, 0x016e56, 0x016e56},
+ {0x016e77, 0x016e77, 0x016e57, 0x016e57},
+ {0x016e78, 0x016e78, 0x016e58, 0x016e58},
+ {0x016e79, 0x016e79, 0x016e59, 0x016e59},
+ {0x016e7a, 0x016e7a, 0x016e5a, 0x016e5a},
+ {0x016e7b, 0x016e7b, 0x016e5b, 0x016e5b},
+ {0x016e7c, 0x016e7c, 0x016e5c, 0x016e5c},
+ {0x016e7d, 0x016e7d, 0x016e5d, 0x016e5d},
+ {0x016e7e, 0x016e7e, 0x016e5e, 0x016e5e},
+ {0x016e7f, 0x016e7f, 0x016e5f, 0x016e5f},
+ {0x01e900, 0x01e922, 0x01e900, 0x01e900},
+ {0x01e901, 0x01e923, 0x01e901, 0x01e901},
+ {0x01e902, 0x01e924, 0x01e902, 0x01e902},
+ {0x01e903, 0x01e925, 0x01e903, 0x01e903},
+ {0x01e904, 0x01e926, 0x01e904, 0x01e904},
+ {0x01e905, 0x01e927, 0x01e905, 0x01e905},
+ {0x01e906, 0x01e928, 0x01e906, 0x01e906},
+ {0x01e907, 0x01e929, 0x01e907, 0x01e907},
+ {0x01e908, 0x01e92a, 0x01e908, 0x01e908},
+ {0x01e909, 0x01e92b, 0x01e909, 0x01e909},
+ {0x01e90a, 0x01e92c, 0x01e90a, 0x01e90a},
+ {0x01e90b, 0x01e92d, 0x01e90b, 0x01e90b},
+ {0x01e90c, 0x01e92e, 0x01e90c, 0x01e90c},
+ {0x01e90d, 0x01e92f, 0x01e90d, 0x01e90d},
+ {0x01e90e, 0x01e930, 0x01e90e, 0x01e90e},
+ {0x01e90f, 0x01e931, 0x01e90f, 0x01e90f},
+ {0x01e910, 0x01e932, 0x01e910, 0x01e910},
+ {0x01e911, 0x01e933, 0x01e911, 0x01e911},
+ {0x01e912, 0x01e934, 0x01e912, 0x01e912},
+ {0x01e913, 0x01e935, 0x01e913, 0x01e913},
+ {0x01e914, 0x01e936, 0x01e914, 0x01e914},
+ {0x01e915, 0x01e937, 0x01e915, 0x01e915},
+ {0x01e916, 0x01e938, 0x01e916, 0x01e916},
+ {0x01e917, 0x01e939, 0x01e917, 0x01e917},
+ {0x01e918, 0x01e93a, 0x01e918, 0x01e918},
+ {0x01e919, 0x01e93b, 0x01e919, 0x01e919},
+ {0x01e91a, 0x01e93c, 0x01e91a, 0x01e91a},
+ {0x01e91b, 0x01e93d, 0x01e91b, 0x01e91b},
+ {0x01e91c, 0x01e93e, 0x01e91c, 0x01e91c},
+ {0x01e91d, 0x01e93f, 0x01e91d, 0x01e91d},
+ {0x01e91e, 0x01e940, 0x01e91e, 0x01e91e},
+ {0x01e91f, 0x01e941, 0x01e91f, 0x01e91f},
+ {0x01e920, 0x01e942, 0x01e920, 0x01e920},
+ {0x01e921, 0x01e943, 0x01e921, 0x01e921},
+ {0x01e922, 0x01e922, 0x01e900, 0x01e900},
+ {0x01e923, 0x01e923, 0x01e901, 0x01e901},
+ {0x01e924, 0x01e924, 0x01e902, 0x01e902},
+ {0x01e925, 0x01e925, 0x01e903, 0x01e903},
+ {0x01e926, 0x01e926, 0x01e904, 0x01e904},
+ {0x01e927, 0x01e927, 0x01e905, 0x01e905},
+ {0x01e928, 0x01e928, 0x01e906, 0x01e906},
+ {0x01e929, 0x01e929, 0x01e907, 0x01e907},
+ {0x01e92a, 0x01e92a, 0x01e908, 0x01e908},
+ {0x01e92b, 0x01e92b, 0x01e909, 0x01e909},
+ {0x01e92c, 0x01e92c, 0x01e90a, 0x01e90a},
+ {0x01e92d, 0x01e92d, 0x01e90b, 0x01e90b},
+ {0x01e92e, 0x01e92e, 0x01e90c, 0x01e90c},
+ {0x01e92f, 0x01e92f, 0x01e90d, 0x01e90d},
+ {0x01e930, 0x01e930, 0x01e90e, 0x01e90e},
+ {0x01e931, 0x01e931, 0x01e90f, 0x01e90f},
+ {0x01e932, 0x01e932, 0x01e910, 0x01e910},
+ {0x01e933, 0x01e933, 0x01e911, 0x01e911},
+ {0x01e934, 0x01e934, 0x01e912, 0x01e912},
+ {0x01e935, 0x01e935, 0x01e913, 0x01e913},
+ {0x01e936, 0x01e936, 0x01e914, 0x01e914},
+ {0x01e937, 0x01e937, 0x01e915, 0x01e915},
+ {0x01e938, 0x01e938, 0x01e916, 0x01e916},
+ {0x01e939, 0x01e939, 0x01e917, 0x01e917},
+ {0x01e93a, 0x01e93a, 0x01e918, 0x01e918},
+ {0x01e93b, 0x01e93b, 0x01e919, 0x01e919},
+ {0x01e93c, 0x01e93c, 0x01e91a, 0x01e91a},
+ {0x01e93d, 0x01e93d, 0x01e91b, 0x01e91b},
+ {0x01e93e, 0x01e93e, 0x01e91c, 0x01e91c},
+ {0x01e93f, 0x01e93f, 0x01e91d, 0x01e91d},
+ {0x01e940, 0x01e940, 0x01e91e, 0x01e91e},
+ {0x01e941, 0x01e941, 0x01e91f, 0x01e91f},
+ {0x01e942, 0x01e942, 0x01e920, 0x01e920},
+ {0x01e943, 0x01e943, 0x01e921, 0x01e921}
+};
--
2.34.1
[text/x-patch] v15-0004-Catalog-changes-preparing-for-builtin-collation-.patch (46.2K, ../[email protected]/5-v15-0004-Catalog-changes-preparing-for-builtin-collation-.patch)
download | inline diff:
From 15feab3483f0f31a65bf3621f553af8ae47de55d Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Tue, 26 Dec 2023 13:32:48 -0800
Subject: [PATCH v15 4/5] Catalog changes preparing for builtin collation
provider.
daticulocale -> datlocale, colliculocale -> colllocale.
---
src/backend/catalog/pg_collation.c | 10 +--
src/backend/commands/collationcmds.c | 34 +++----
src/backend/commands/dbcommands.c | 68 +++++++-------
src/backend/utils/adt/pg_locale.c | 4 +-
src/backend/utils/init/postinit.c | 2 +-
src/bin/initdb/initdb.c | 41 ++++-----
src/bin/initdb/t/001_initdb.pl | 4 +-
src/bin/pg_dump/pg_dump.c | 90 +++++++++++--------
src/bin/pg_upgrade/info.c | 31 ++++---
src/bin/pg_upgrade/pg_upgrade.c | 34 ++++---
src/bin/pg_upgrade/pg_upgrade.h | 2 +-
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 13 ++-
src/bin/psql/describe.c | 20 +++--
src/include/catalog/pg_collation.dat | 2 +-
src/include/catalog/pg_collation.h | 4 +-
src/include/catalog/pg_database.dat | 2 +-
src/include/catalog/pg_database.h | 2 +-
.../regress/expected/collate.icu.utf8.out | 4 +-
src/test/regress/expected/psql.out | 18 ++--
src/test/regress/sql/collate.icu.utf8.sql | 4 +-
20 files changed, 220 insertions(+), 169 deletions(-)
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index fd022e6fc2..b615d2fc7a 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,7 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *colliculocale,
+ const char *colllocale,
const char *collicurules,
const char *collversion,
bool if_not_exists,
@@ -68,7 +68,7 @@ CollationCreate(const char *collname, Oid collnamespace,
Assert(collname);
Assert(collnamespace);
Assert(collowner);
- Assert((collcollate && collctype) || colliculocale);
+ Assert((collcollate && collctype) || colllocale);
/*
* Make sure there is no existing collation of same name & encoding.
@@ -191,10 +191,10 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collctype - 1] = CStringGetTextDatum(collctype);
else
nulls[Anum_pg_collation_collctype - 1] = true;
- if (colliculocale)
- values[Anum_pg_collation_colliculocale - 1] = CStringGetTextDatum(colliculocale);
+ if (colllocale)
+ values[Anum_pg_collation_colllocale - 1] = CStringGetTextDatum(colllocale);
else
- nulls[Anum_pg_collation_colliculocale - 1] = true;
+ nulls[Anum_pg_collation_colllocale - 1] = true;
if (collicurules)
values[Anum_pg_collation_collicurules - 1] = CStringGetTextDatum(collicurules);
else
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index a4532bf81f..19e61b3e5b 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *versionEl = NULL;
char *collcollate;
char *collctype;
- char *colliculocale;
+ char *colllocale;
char *collicurules;
bool collisdeterministic;
int collencoding;
@@ -159,11 +159,11 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
else
collctype = NULL;
- datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_colliculocale, &isnull);
+ datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_colllocale, &isnull);
if (!isnull)
- colliculocale = TextDatumGetCString(datum);
+ colllocale = TextDatumGetCString(datum);
else
- colliculocale = NULL;
+ colllocale = NULL;
/*
* When the ICU locale comes from an existing collation, do not
@@ -196,7 +196,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collcollate = NULL;
collctype = NULL;
- colliculocale = NULL;
+ colllocale = NULL;
collicurules = NULL;
if (providerEl)
@@ -236,7 +236,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collctype = defGetString(localeEl);
}
else
- colliculocale = defGetString(localeEl);
+ colllocale = defGetString(localeEl);
}
if (lccollateEl)
@@ -261,7 +261,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
else if (collprovider == COLLPROVIDER_ICU)
{
- if (!colliculocale)
+ if (!colllocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("parameter \"%s\" must be specified",
@@ -273,20 +273,20 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
*/
if (!IsBinaryUpgrade)
{
- char *langtag = icu_language_tag(colliculocale,
+ char *langtag = icu_language_tag(colllocale,
icu_validation_level);
- if (langtag && strcmp(colliculocale, langtag) != 0)
+ if (langtag && strcmp(colllocale, langtag) != 0)
{
ereport(NOTICE,
(errmsg("using standard form \"%s\" for ICU locale \"%s\"",
- langtag, colliculocale)));
+ langtag, colllocale)));
- colliculocale = langtag;
+ colllocale = langtag;
}
}
- icu_validate_locale(colliculocale);
+ icu_validate_locale(colllocale);
}
/*
@@ -334,7 +334,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
if (!collversion)
- collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colliculocale : collcollate);
+ collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colllocale : collcollate);
newoid = CollationCreate(collName,
collNamespace,
@@ -344,7 +344,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- colliculocale,
+ colllocale,
collicurules,
collversion,
if_not_exists,
@@ -435,7 +435,7 @@ AlterCollation(AlterCollationStmt *stmt)
datum = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion, &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
newversion = get_collation_actual_version(collForm->collprovider, TextDatumGetCString(datum));
/* cannot change from NULL to non-NULL or vice versa */
@@ -502,7 +502,7 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup,
provider == COLLPROVIDER_ICU ?
- Anum_pg_database_daticulocale : Anum_pg_database_datcollate);
+ Anum_pg_database_datlocale : Anum_pg_database_datcollate);
locale = TextDatumGetCString(datum);
@@ -523,7 +523,7 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
Assert(provider != COLLPROVIDER_DEFAULT);
datum = SysCacheGetAttrNotNull(COLLOID, colltp,
provider == COLLPROVIDER_ICU ?
- Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
locale = TextDatumGetCString(datum);
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index ae38f83024..c8732be067 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -118,7 +118,7 @@ static bool get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, bool *dbHasLoginEvtP,
TransactionId *dbFrozenXidP, MultiXactId *dbMinMultiP,
- Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbIculocale,
+ Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbLocale,
char **dbIcurules,
char *dbLocProvider,
char **dbCollversion);
@@ -675,7 +675,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
int src_encoding = -1;
char *src_collate = NULL;
char *src_ctype = NULL;
- char *src_iculocale = NULL;
+ char *src_locale = NULL;
char *src_icurules = NULL;
char src_locprovider = '\0';
char *src_collversion = NULL;
@@ -713,7 +713,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
const char *dbtemplate = NULL;
char *dbcollate = NULL;
char *dbctype = NULL;
- char *dbiculocale = NULL;
+ char *dblocale = NULL;
char *dbicurules = NULL;
char dblocprovider = '\0';
char *canonname;
@@ -903,7 +903,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
if (dctype && dctype->arg)
dbctype = defGetString(dctype);
if (diculocale && diculocale->arg)
- dbiculocale = defGetString(diculocale);
+ dblocale = defGetString(diculocale);
if (dicurules && dicurules->arg)
dbicurules = defGetString(dicurules);
if (dlocprovider && dlocprovider->arg)
@@ -971,7 +971,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
&src_dboid, &src_owner, &src_encoding,
&src_istemplate, &src_allowconn, &src_hasloginevt,
&src_frozenxid, &src_minmxid, &src_deftablespace,
- &src_collate, &src_ctype, &src_iculocale, &src_icurules, &src_locprovider,
+ &src_collate, &src_ctype, &src_locale, &src_icurules, &src_locprovider,
&src_collversion))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_DATABASE),
@@ -1027,12 +1027,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
dbctype = src_ctype;
if (dblocprovider == '\0')
dblocprovider = src_locprovider;
- if (dbiculocale == NULL && dblocprovider == COLLPROVIDER_ICU)
+ if (dblocale == NULL && dblocprovider == COLLPROVIDER_ICU)
{
if (dlocale && dlocale->arg)
- dbiculocale = defGetString(dlocale);
+ dblocale = defGetString(dlocale);
else
- dbiculocale = src_iculocale;
+ dblocale = src_locale;
}
if (dbicurules == NULL && dblocprovider == COLLPROVIDER_ICU)
dbicurules = src_icurules;
@@ -1071,7 +1071,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* This would happen if template0 uses the libc provider but the new
* database uses icu.
*/
- if (!dbiculocale)
+ if (!dblocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("LOCALE or ICU_LOCALE must be specified")));
@@ -1081,26 +1081,26 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* database, preserve locale string. Otherwise, canonicalize to a
* language tag.
*/
- if (!IsBinaryUpgrade && dbiculocale != src_iculocale)
+ if (!IsBinaryUpgrade && dblocale != src_locale)
{
- char *langtag = icu_language_tag(dbiculocale,
+ char *langtag = icu_language_tag(dblocale,
icu_validation_level);
- if (langtag && strcmp(dbiculocale, langtag) != 0)
+ if (langtag && strcmp(dblocale, langtag) != 0)
{
ereport(NOTICE,
(errmsg("using standard form \"%s\" for ICU locale \"%s\"",
- langtag, dbiculocale)));
+ langtag, dblocale)));
- dbiculocale = langtag;
+ dblocale = langtag;
}
}
- icu_validate_locale(dbiculocale);
+ icu_validate_locale(dblocale);
}
else
{
- if (dbiculocale)
+ if (dblocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU locale cannot be specified unless locale provider is ICU")));
@@ -1157,13 +1157,13 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
char *val1;
char *val2;
- Assert(dbiculocale);
- Assert(src_iculocale);
- if (strcmp(dbiculocale, src_iculocale) != 0)
+ Assert(dblocale);
+ Assert(src_locale);
+ if (strcmp(dblocale, src_locale) != 0)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("new ICU locale (%s) is incompatible with the ICU locale of the template database (%s)",
- dbiculocale, src_iculocale),
+ dblocale, src_locale),
errhint("Use the same ICU locale as in the template database, or use template0 as template.")));
val1 = dbicurules;
@@ -1197,7 +1197,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
char *actual_versionstr;
- actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dbiculocale : dbcollate);
+ actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
if (!actual_versionstr)
ereport(ERROR,
(errmsg("template database \"%s\" has a collation version, but no actual collation version could be determined",
@@ -1225,7 +1225,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* collation version, which is normally only the case for template0.
*/
if (dbcollversion == NULL)
- dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dbiculocale : dbcollate);
+ dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
/* Resolve default tablespace for new database */
if (dtablespacename && dtablespacename->arg)
@@ -1364,8 +1364,8 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* block on the unique index, and fail after we commit).
*/
- Assert((dblocprovider == COLLPROVIDER_ICU && dbiculocale) ||
- (dblocprovider != COLLPROVIDER_ICU && !dbiculocale));
+ Assert((dblocprovider == COLLPROVIDER_ICU && dblocale) ||
+ (dblocprovider != COLLPROVIDER_ICU && !dblocale));
/* Form tuple */
new_record[Anum_pg_database_oid - 1] = ObjectIdGetDatum(dboid);
@@ -1383,10 +1383,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
new_record[Anum_pg_database_dattablespace - 1] = ObjectIdGetDatum(dst_deftablespace);
new_record[Anum_pg_database_datcollate - 1] = CStringGetTextDatum(dbcollate);
new_record[Anum_pg_database_datctype - 1] = CStringGetTextDatum(dbctype);
- if (dbiculocale)
- new_record[Anum_pg_database_daticulocale - 1] = CStringGetTextDatum(dbiculocale);
+ if (dblocale)
+ new_record[Anum_pg_database_datlocale - 1] = CStringGetTextDatum(dblocale);
else
- new_record_nulls[Anum_pg_database_daticulocale - 1] = true;
+ new_record_nulls[Anum_pg_database_datlocale - 1] = true;
if (dbicurules)
new_record[Anum_pg_database_daticurules - 1] = CStringGetTextDatum(dbicurules);
else
@@ -2472,7 +2472,7 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
datum = heap_getattr(tuple, Anum_pg_database_datcollversion, RelationGetDescr(rel), &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_daticulocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
+ datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
if (isnull)
elog(ERROR, "unexpected null in pg_database");
newversion = get_collation_actual_version(datForm->datlocprovider, TextDatumGetCString(datum));
@@ -2670,7 +2670,7 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
datlocprovider = ((Form_pg_database) GETSTRUCT(tp))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_daticulocale : Anum_pg_database_datcollate);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate);
version = get_collation_actual_version(datlocprovider, TextDatumGetCString(datum));
ReleaseSysCache(tp);
@@ -2697,7 +2697,7 @@ get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, bool *dbHasLoginEvtP,
TransactionId *dbFrozenXidP, MultiXactId *dbMinMultiP,
- Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbIculocale,
+ Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbLocale,
char **dbIcurules,
char *dbLocProvider,
char **dbCollversion)
@@ -2808,13 +2808,13 @@ get_db_info(const char *name, LOCKMODE lockmode,
datum = SysCacheGetAttrNotNull(DATABASEOID, tuple, Anum_pg_database_datctype);
*dbCtype = TextDatumGetCString(datum);
}
- if (dbIculocale)
+ if (dbLocale)
{
- datum = SysCacheGetAttr(DATABASEOID, tuple, Anum_pg_database_daticulocale, &isnull);
+ datum = SysCacheGetAttr(DATABASEOID, tuple, Anum_pg_database_datlocale, &isnull);
if (isnull)
- *dbIculocale = NULL;
+ *dbLocale = NULL;
else
- *dbIculocale = TextDatumGetCString(datum);
+ *dbLocale = TextDatumGetCString(datum);
}
if (dbIcurules)
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 1dee4622d6..9d8634ec25 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1606,7 +1606,7 @@ pg_newlocale_from_collation(Oid collid)
const char *iculocstr;
const char *icurules;
- datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colliculocale);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
iculocstr = TextDatumGetCString(datum);
datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collicurules, &isnull);
@@ -1627,7 +1627,7 @@ pg_newlocale_from_collation(Oid collid)
collversionstr = TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
actual_versionstr = get_collation_actual_version(collform->collprovider,
TextDatumGetCString(datum));
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 552cf9d950..702bad6d8a 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -427,7 +427,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
{
char *icurules;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_daticulocale);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
iculocale = TextDatumGetCString(datum);
datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_daticurules, &isnull);
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index e68b40d2b5..eea40b930f 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -145,7 +145,8 @@ static char *lc_numeric = NULL;
static char *lc_time = NULL;
static char *lc_messages = NULL;
static char locale_provider = COLLPROVIDER_LIBC;
-static char *icu_locale = NULL;
+static char *datlocale = NULL;
+static bool icu_locale_specified = false;
static char *icu_rules = NULL;
static const char *default_text_search_config = NULL;
static char *username = NULL;
@@ -1523,8 +1524,8 @@ bootstrap_template1(void)
bki_lines = replace_token(bki_lines, "LC_CTYPE",
escape_quotes_bki(lc_ctype));
- bki_lines = replace_token(bki_lines, "ICU_LOCALE",
- icu_locale ? escape_quotes_bki(icu_locale) : "_null_");
+ bki_lines = replace_token(bki_lines, "DATLOCALE",
+ datlocale ? escape_quotes_bki(datlocale) : "_null_");
bki_lines = replace_token(bki_lines, "ICU_RULES",
icu_rules ? escape_quotes_bki(icu_rules) : "_null_");
@@ -2371,8 +2372,8 @@ setlocales(void)
lc_monetary = locale;
if (!lc_messages)
lc_messages = locale;
- if (!icu_locale && locale_provider == COLLPROVIDER_ICU)
- icu_locale = locale;
+ if (!datlocale && locale_provider != COLLPROVIDER_LIBC)
+ datlocale = locale;
}
/*
@@ -2398,22 +2399,21 @@ setlocales(void)
lc_messages = canonname;
#endif
+ if (locale_provider != COLLPROVIDER_LIBC && datlocale == NULL)
+ pg_fatal("locale must be specified unless provider is libc");
+
if (locale_provider == COLLPROVIDER_ICU)
{
char *langtag;
- /* acquire default locale from the environment, if not specified */
- if (icu_locale == NULL)
- pg_fatal("ICU locale must be specified");
-
/* canonicalize to a language tag */
- langtag = icu_language_tag(icu_locale);
+ langtag = icu_language_tag(datlocale);
printf(_("Using language tag \"%s\" for ICU locale \"%s\".\n"),
- langtag, icu_locale);
- pg_free(icu_locale);
- icu_locale = langtag;
+ langtag, datlocale);
+ pg_free(datlocale);
+ datlocale = langtag;
- icu_validate_locale(icu_locale);
+ icu_validate_locale(datlocale);
/*
* In supported builds, the ICU locale ID will be opened during
@@ -2607,14 +2607,14 @@ setup_locale_encoding(void)
strcmp(lc_ctype, lc_numeric) == 0 &&
strcmp(lc_ctype, lc_monetary) == 0 &&
strcmp(lc_ctype, lc_messages) == 0 &&
- (!icu_locale || strcmp(lc_ctype, icu_locale) == 0))
+ (!datlocale || strcmp(lc_ctype, datlocale) == 0))
printf(_("The database cluster will be initialized with locale \"%s\".\n"), lc_ctype);
else
{
printf(_("The database cluster will be initialized with this locale configuration:\n"));
- printf(_(" provider: %s\n"), collprovider_name(locale_provider));
- if (icu_locale)
- printf(_(" ICU locale: %s\n"), icu_locale);
+ printf(_(" default collation provider: %s\n"), collprovider_name(locale_provider));
+ if (locale_provider != COLLPROVIDER_LIBC)
+ printf(_(" default collation locale: %s\n"), datlocale);
printf(_(" LC_COLLATE: %s\n"
" LC_CTYPE: %s\n"
" LC_MESSAGES: %s\n"
@@ -3285,7 +3285,8 @@ main(int argc, char *argv[])
pg_fatal("unrecognized locale provider: %s", optarg);
break;
case 16:
- icu_locale = pg_strdup(optarg);
+ datlocale = pg_strdup(optarg);
+ icu_locale_specified = true;
break;
case 17:
icu_rules = pg_strdup(optarg);
@@ -3320,7 +3321,7 @@ main(int argc, char *argv[])
exit(1);
}
- if (icu_locale && locale_provider != COLLPROVIDER_ICU)
+ if (icu_locale_specified && locale_provider != COLLPROVIDER_ICU)
pg_fatal("%s cannot be specified unless locale provider \"%s\" is chosen",
"--icu-locale", "icu");
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index 45f96cd8bb..7606db1987 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -117,7 +117,7 @@ if ($ENV{with_icu} eq 'yes')
{
command_fails_like(
[ 'initdb', '--no-sync', '--locale-provider=icu', "$tempdir/data2" ],
- qr/initdb: error: ICU locale must be specified/,
+ qr/initdb: error: locale must be specified unless provider is libc/,
'locale provider ICU requires --icu-locale');
command_ok(
@@ -138,7 +138,7 @@ if ($ENV{with_icu} eq 'yes')
'--lc-monetary=C', '--lc-time=C',
"$tempdir/data4"
],
- qr/^\s+ICU locale:\s+und\n/ms,
+ qr/^\s+default collation locale:\s+und\n/ms,
'options --locale-provider=icu --locale=und --lc-*=C');
command_fails_like(
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 8c0b5486b9..3d829c99d7 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -2961,7 +2961,7 @@ dumpDatabase(Archive *fout)
i_datlocprovider,
i_collate,
i_ctype,
- i_daticulocale,
+ i_datlocale,
i_daticurules,
i_frozenxid,
i_minmxid,
@@ -2980,7 +2980,7 @@ dumpDatabase(Archive *fout)
*datlocprovider,
*collate,
*ctype,
- *iculocale,
+ *locale,
*icurules,
*datistemplate,
*datconnlimit,
@@ -3004,10 +3004,12 @@ dumpDatabase(Archive *fout)
appendPQExpBufferStr(dbQry, "datminmxid, ");
else
appendPQExpBufferStr(dbQry, "0 AS datminmxid, ");
- if (fout->remoteVersion >= 150000)
- appendPQExpBufferStr(dbQry, "datlocprovider, daticulocale, datcollversion, ");
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(dbQry, "datlocprovider, datlocale, datcollversion, ");
+ else if (fout->remoteVersion >= 150000)
+ appendPQExpBufferStr(dbQry, "datlocprovider, daticulocale AS datlocale, datcollversion, ");
else
- appendPQExpBufferStr(dbQry, "'c' AS datlocprovider, NULL AS daticulocale, NULL AS datcollversion, ");
+ appendPQExpBufferStr(dbQry, "'c' AS datlocprovider, NULL AS datlocale, NULL AS datcollversion, ");
if (fout->remoteVersion >= 160000)
appendPQExpBufferStr(dbQry, "daticurules, ");
else
@@ -3028,7 +3030,7 @@ dumpDatabase(Archive *fout)
i_datlocprovider = PQfnumber(res, "datlocprovider");
i_collate = PQfnumber(res, "datcollate");
i_ctype = PQfnumber(res, "datctype");
- i_daticulocale = PQfnumber(res, "daticulocale");
+ i_datlocale = PQfnumber(res, "datlocale");
i_daticurules = PQfnumber(res, "daticurules");
i_frozenxid = PQfnumber(res, "datfrozenxid");
i_minmxid = PQfnumber(res, "datminmxid");
@@ -3047,10 +3049,10 @@ dumpDatabase(Archive *fout)
datlocprovider = PQgetvalue(res, 0, i_datlocprovider);
collate = PQgetvalue(res, 0, i_collate);
ctype = PQgetvalue(res, 0, i_ctype);
- if (!PQgetisnull(res, 0, i_daticulocale))
- iculocale = PQgetvalue(res, 0, i_daticulocale);
+ if (!PQgetisnull(res, 0, i_datlocale))
+ locale = PQgetvalue(res, 0, i_datlocale);
else
- iculocale = NULL;
+ locale = NULL;
if (!PQgetisnull(res, 0, i_daticurules))
icurules = PQgetvalue(res, 0, i_daticurules);
else
@@ -3097,29 +3099,36 @@ dumpDatabase(Archive *fout)
pg_fatal("unrecognized locale provider: %s",
datlocprovider);
- if (strlen(collate) > 0 && strcmp(collate, ctype) == 0)
+ if (!locale && datlocprovider[0] != 'c')
+ pg_log_warning("database '%s' with provider '%s' missing datlocale",
+ datname, datlocprovider);
+
+ if (locale && datlocprovider[0] == 'c')
+ pg_log_warning("database '%s' with provider 'c' has non-NULL locale '%s'",
+ datname, locale);
+
+ /* if collate and ctype are equal, and locale is NULL, use LOCALE */
+ if (!locale && strlen(collate) > 0 && strcmp(collate, ctype) == 0)
+ locale = collate;
+
+ /* output LC_COLLATE and LC_CTYPE if different from LOCALE */
+ if (strlen(collate) > 0 && (!locale || strcmp(collate, locale) != 0))
{
- appendPQExpBufferStr(creaQry, " LOCALE = ");
+ appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
appendStringLiteralAH(creaQry, collate, fout);
}
- else
+ if (strlen(ctype) > 0 && (!locale || strcmp(ctype, locale) != 0))
{
- if (strlen(collate) > 0)
- {
- appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
- appendStringLiteralAH(creaQry, collate, fout);
- }
- if (strlen(ctype) > 0)
- {
- appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
- appendStringLiteralAH(creaQry, ctype, fout);
- }
+ appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
+ appendStringLiteralAH(creaQry, ctype, fout);
}
- if (iculocale)
+
+ if (locale)
{
- appendPQExpBufferStr(creaQry, " ICU_LOCALE = ");
- appendStringLiteralAH(creaQry, iculocale, fout);
+ appendPQExpBufferStr(creaQry, " LOCALE = ");
+ appendStringLiteralAH(creaQry, locale, fout);
}
+
if (icurules)
{
appendPQExpBufferStr(creaQry, " ICU_RULES = ");
@@ -13552,12 +13561,12 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
int i_collisdeterministic;
int i_collcollate;
int i_collctype;
- int i_colliculocale;
+ int i_colllocale;
int i_collicurules;
const char *collprovider;
const char *collcollate;
const char *collctype;
- const char *colliculocale;
+ const char *colllocale;
const char *collicurules;
/* Do nothing in data-only dump */
@@ -13589,12 +13598,15 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
appendPQExpBufferStr(query,
"true AS collisdeterministic, ");
- if (fout->remoteVersion >= 150000)
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(query,
+ "colllocale, ");
+ else if (fout->remoteVersion >= 150000)
appendPQExpBufferStr(query,
- "colliculocale, ");
+ "colliculocale AS colllocale, ");
else
appendPQExpBufferStr(query,
- "NULL AS colliculocale, ");
+ "NULL AS colllocale, ");
if (fout->remoteVersion >= 160000)
appendPQExpBufferStr(query,
@@ -13616,7 +13628,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
i_collisdeterministic = PQfnumber(res, "collisdeterministic");
i_collcollate = PQfnumber(res, "collcollate");
i_collctype = PQfnumber(res, "collctype");
- i_colliculocale = PQfnumber(res, "colliculocale");
+ i_colllocale = PQfnumber(res, "colllocale");
i_collicurules = PQfnumber(res, "collicurules");
collprovider = PQgetvalue(res, 0, i_collprovider);
@@ -13643,10 +13655,10 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
collctype = NULL;
}
- if (!PQgetisnull(res, 0, i_colliculocale))
- colliculocale = PQgetvalue(res, 0, i_colliculocale);
+ if (!PQgetisnull(res, 0, i_colllocale))
+ colllocale = PQgetvalue(res, 0, i_colllocale);
else
- colliculocale = NULL;
+ colllocale = NULL;
if (!PQgetisnull(res, 0, i_collicurules))
collicurules = PQgetvalue(res, 0, i_collicurules);
@@ -13676,7 +13688,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
if (collprovider[0] == 'd')
{
- if (collcollate || collctype || colliculocale || collicurules)
+ if (collcollate || collctype || colllocale || collicurules)
pg_log_warning("invalid collation \"%s\"", qcollname);
/* no locale -- the default collation cannot be reloaded anyway */
@@ -13685,16 +13697,16 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
{
if (fout->remoteVersion >= 150000)
{
- if (collcollate || collctype || !colliculocale)
+ if (collcollate || collctype || !colllocale)
pg_log_warning("invalid collation \"%s\"", qcollname);
appendPQExpBufferStr(q, ", locale = ");
- appendStringLiteralAH(q, colliculocale ? colliculocale : "",
+ appendStringLiteralAH(q, colllocale ? colllocale : "",
fout);
}
else
{
- if (!collcollate || !collctype || colliculocale ||
+ if (!collcollate || !collctype || colllocale ||
strcmp(collcollate, collctype) != 0)
pg_log_warning("invalid collation \"%s\"", qcollname);
@@ -13710,7 +13722,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
}
else if (collprovider[0] == 'c')
{
- if (colliculocale || collicurules || !collcollate || !collctype)
+ if (colllocale || collicurules || !collcollate || !collctype)
pg_log_warning("invalid collation \"%s\"", qcollname);
if (collcollate && collctype && strcmp(collcollate, collctype) == 0)
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 4878aa22bf..38d52f5c64 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -323,18 +323,24 @@ get_template0_info(ClusterInfo *cluster)
int i_datlocprovider;
int i_datcollate;
int i_datctype;
- int i_daticulocale;
+ int i_datlocale;
- if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
+ if (GET_MAJOR_VERSION(cluster->major_version) >= 1700)
dbres = executeQueryOrDie(conn,
"SELECT encoding, datlocprovider, "
- " datcollate, datctype, daticulocale "
+ " datcollate, datctype, datlocale "
+ "FROM pg_catalog.pg_database "
+ "WHERE datname='template0'");
+ else if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
+ dbres = executeQueryOrDie(conn,
+ "SELECT encoding, datlocprovider, "
+ " datcollate, datctype, daticulocale AS datlocale"
"FROM pg_catalog.pg_database "
"WHERE datname='template0'");
else
dbres = executeQueryOrDie(conn,
"SELECT encoding, 'c' AS datlocprovider, "
- " datcollate, datctype, NULL AS daticulocale "
+ " datcollate, datctype, NULL AS datlocale "
"FROM pg_catalog.pg_database "
"WHERE datname='template0'");
@@ -348,16 +354,16 @@ get_template0_info(ClusterInfo *cluster)
i_datlocprovider = PQfnumber(dbres, "datlocprovider");
i_datcollate = PQfnumber(dbres, "datcollate");
i_datctype = PQfnumber(dbres, "datctype");
- i_daticulocale = PQfnumber(dbres, "daticulocale");
+ i_datlocale = PQfnumber(dbres, "datlocale");
locale->db_encoding = atoi(PQgetvalue(dbres, 0, i_datencoding));
locale->db_collprovider = PQgetvalue(dbres, 0, i_datlocprovider)[0];
locale->db_collate = pg_strdup(PQgetvalue(dbres, 0, i_datcollate));
locale->db_ctype = pg_strdup(PQgetvalue(dbres, 0, i_datctype));
- if (PQgetisnull(dbres, 0, i_daticulocale))
- locale->db_iculocale = NULL;
+ if (PQgetisnull(dbres, 0, i_datlocale))
+ locale->db_locale = NULL;
else
- locale->db_iculocale = pg_strdup(PQgetvalue(dbres, 0, i_daticulocale));
+ locale->db_locale = pg_strdup(PQgetvalue(dbres, 0, i_datlocale));
cluster->template0 = locale;
@@ -387,12 +393,15 @@ get_db_infos(ClusterInfo *cluster)
snprintf(query, sizeof(query),
"SELECT d.oid, d.datname, d.encoding, d.datcollate, d.datctype, ");
- if (GET_MAJOR_VERSION(cluster->major_version) < 1500)
+ if (GET_MAJOR_VERSION(cluster->major_version) >= 1700)
+ snprintf(query + strlen(query), sizeof(query) - strlen(query),
+ "datlocprovider, datlocale, ");
+ else if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
snprintf(query + strlen(query), sizeof(query) - strlen(query),
- "'c' AS datlocprovider, NULL AS daticulocale, ");
+ "datlocprovider, daticulocale AS datlocale, ");
else
snprintf(query + strlen(query), sizeof(query) - strlen(query),
- "datlocprovider, daticulocale, ");
+ "'c' AS datlocprovider, NULL AS datlocale, ");
snprintf(query + strlen(query), sizeof(query) - strlen(query),
"pg_catalog.pg_tablespace_location(t.oid) AS spclocation "
"FROM pg_catalog.pg_database d "
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 3960af4036..a7013d200d 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -391,7 +391,7 @@ setup(char *argv0, bool *live_check)
* Copy locale and encoding information into the new cluster's template0.
*
* We need to copy the encoding, datlocprovider, datcollate, datctype, and
- * daticulocale. We don't need datcollversion because that's never set for
+ * datlocale. We don't need datcollversion because that's never set for
* template0.
*/
static void
@@ -400,7 +400,7 @@ set_locale_and_encoding(void)
PGconn *conn_new_template1;
char *datcollate_literal;
char *datctype_literal;
- char *daticulocale_literal = NULL;
+ char *datlocale_literal = NULL;
DbLocaleInfo *locale = old_cluster.template0;
prep_status("Setting locale and encoding for new cluster");
@@ -414,15 +414,29 @@ set_locale_and_encoding(void)
datctype_literal = PQescapeLiteral(conn_new_template1,
locale->db_ctype,
strlen(locale->db_ctype));
- if (locale->db_iculocale)
- daticulocale_literal = PQescapeLiteral(conn_new_template1,
- locale->db_iculocale,
- strlen(locale->db_iculocale));
+ if (locale->db_locale)
+ datlocale_literal = PQescapeLiteral(conn_new_template1,
+ locale->db_locale,
+ strlen(locale->db_locale));
else
- daticulocale_literal = pg_strdup("NULL");
+ datlocale_literal = pg_strdup("NULL");
/* update template0 in new cluster */
- if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1500)
+ if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1700)
+ PQclear(executeQueryOrDie(conn_new_template1,
+ "UPDATE pg_catalog.pg_database "
+ " SET encoding = %d, "
+ " datlocprovider = '%c', "
+ " datcollate = %s, "
+ " datctype = %s, "
+ " datlocale = %s "
+ " WHERE datname = 'template0' ",
+ locale->db_encoding,
+ locale->db_collprovider,
+ datcollate_literal,
+ datctype_literal,
+ datlocale_literal));
+ else if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1500)
PQclear(executeQueryOrDie(conn_new_template1,
"UPDATE pg_catalog.pg_database "
" SET encoding = %d, "
@@ -435,7 +449,7 @@ set_locale_and_encoding(void)
locale->db_collprovider,
datcollate_literal,
datctype_literal,
- daticulocale_literal));
+ datlocale_literal));
else
PQclear(executeQueryOrDie(conn_new_template1,
"UPDATE pg_catalog.pg_database "
@@ -449,7 +463,7 @@ set_locale_and_encoding(void)
PQfreemem(datcollate_literal);
PQfreemem(datctype_literal);
- PQfreemem(daticulocale_literal);
+ PQfreemem(datlocale_literal);
PQfinish(conn_new_template1);
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index a710f325de..81eb60cb2e 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -205,7 +205,7 @@ typedef struct
char *db_collate;
char *db_ctype;
char db_collprovider;
- char *db_iculocale;
+ char *db_locale;
int db_encoding;
} DbLocaleInfo;
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index e5f57e550a..6f359d72ce 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -113,11 +113,18 @@ my $original_provider = "c";
my $original_locale = "C";
my $original_iculocale = "";
my $provider_field = "'c' AS datlocprovider";
-my $iculocale_field = "NULL AS daticulocale";
-if ($oldnode->pg_version >= 15 && $ENV{with_icu} eq 'yes')
+my $iculocale_field = "NULL AS datlocale";
+if (int($oldnode->pg_version) >= 15 && $ENV{with_icu} eq 'yes')
{
$provider_field = "datlocprovider";
- $iculocale_field = "daticulocale";
+ if (int($oldnode->pg_version) >= 17)
+ {
+ $iculocale_field = "datlocale";
+ }
+ else
+ {
+ $iculocale_field = "daticulocale AS datlocale";
+ }
$original_provider = "i";
$original_iculocale = "fr-CA";
}
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 5077e7b358..1122843715 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -937,14 +937,18 @@ listAllDbs(const char *pattern, bool verbose)
" d.datctype as \"%s\",\n",
gettext_noop("Collate"),
gettext_noop("Ctype"));
- if (pset.sversion >= 150000)
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ " d.datlocale as \"%s\",\n",
+ gettext_noop("Locale"));
+ else if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
" d.daticulocale as \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
else
appendPQExpBuffer(&buf,
" NULL as \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
" d.daticurules as \"%s\",\n",
@@ -4975,14 +4979,18 @@ listCollations(const char *pattern, bool verbose, bool showSystem)
gettext_noop("Collate"),
gettext_noop("Ctype"));
- if (pset.sversion >= 150000)
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ " c.colllocale AS \"%s\",\n",
+ gettext_noop("Locale"));
+ else if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
" c.colliculocale AS \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
else
appendPQExpBuffer(&buf,
" c.collcollate AS \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index b6a69d1d42..2c112cd6bc 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -29,6 +29,6 @@
{ oid => '963',
descr => 'sorts using the Unicode Collation Algorithm with default settings',
collname => 'unicode', collprovider => 'i', collencoding => '-1',
- colliculocale => 'und' },
+ colllocale => 'und' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 2d7dcde183..d357c89ae6 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -42,7 +42,7 @@ CATALOG(pg_collation,3456,CollationRelationId)
#ifdef CATALOG_VARLEN /* variable-length fields start here */
text collcollate BKI_DEFAULT(_null_); /* LC_COLLATE setting */
text collctype BKI_DEFAULT(_null_); /* LC_CTYPE setting */
- text colliculocale BKI_DEFAULT(_null_); /* ICU locale ID */
+ text colllocale BKI_DEFAULT(_null_); /* locale ID */
text collicurules BKI_DEFAULT(_null_); /* ICU collation rules */
text collversion BKI_DEFAULT(_null_); /* provider-dependent
* version of collation
@@ -91,7 +91,7 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *colliculocale,
+ const char *colllocale,
const char *collicurules,
const char *collversion,
bool if_not_exists,
diff --git a/src/include/catalog/pg_database.dat b/src/include/catalog/pg_database.dat
index 8d91e3bf8d..0b54609d66 100644
--- a/src/include/catalog/pg_database.dat
+++ b/src/include/catalog/pg_database.dat
@@ -18,7 +18,7 @@
datlocprovider => 'LOCALE_PROVIDER', datistemplate => 't',
datallowconn => 't', dathasloginevt => 'f', datconnlimit => '-1', datfrozenxid => '0',
datminmxid => '1', dattablespace => 'pg_default', datcollate => 'LC_COLLATE',
- datctype => 'LC_CTYPE', daticulocale => 'ICU_LOCALE',
+ datctype => 'LC_CTYPE', datlocale => 'DATLOCALE',
daticurules => 'ICU_RULES', datacl => '_null_' },
]
diff --git a/src/include/catalog/pg_database.h b/src/include/catalog/pg_database.h
index 3e50a57004..c659b5165a 100644
--- a/src/include/catalog/pg_database.h
+++ b/src/include/catalog/pg_database.h
@@ -75,7 +75,7 @@ CATALOG(pg_database,1262,DatabaseRelationId) BKI_SHARED_RELATION BKI_ROWTYPE_OID
text datctype BKI_FORCE_NOT_NULL;
/* ICU locale ID */
- text daticulocale;
+ text datlocale;
/* ICU collation rules */
text daticurules;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 7a05c75967..8ca93f4dea 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1024,7 +1024,7 @@ SET icu_validation_level = disabled;
do $$
BEGIN
EXECUTE 'CREATE COLLATION test0 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
CREATE COLLATION test0 FROM "C"; -- fail, duplicate name
@@ -1032,7 +1032,7 @@ ERROR: collation "test0" already exists
do $$
BEGIN
EXECUTE 'CREATE COLLATION test1 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
RESET icu_validation_level;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 5d61e4c7bb..53d492c750 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6220,9 +6220,9 @@ List of schemas
(0 rows)
\dO "no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp "no.such.access.privilege"
@@ -6409,9 +6409,9 @@ cross-database references are not implemented: "no.such.schema"."no.such.languag
(0 rows)
\dO "no.such.schema"."no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp "no.such.schema"."no.such.access.privilege"
@@ -6552,9 +6552,9 @@ List of text search templates
(0 rows)
\dO regression."no.such.schema"."no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp regression."no.such.schema"."no.such.access.privilege"
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 3db9e25913..03837de846 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -363,14 +363,14 @@ SET icu_validation_level = disabled;
do $$
BEGIN
EXECUTE 'CREATE COLLATION test0 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
CREATE COLLATION test0 FROM "C"; -- fail, duplicate name
do $$
BEGIN
EXECUTE 'CREATE COLLATION test1 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
--
2.34.1
[text/x-patch] v15-0005-Introduce-collation-provider-builtin-for-C-and-C.patch (73.0K, ../[email protected]/6-v15-0005-Introduce-collation-provider-builtin-for-C-and-C.patch)
download | inline diff:
From 5ffd8d5d3a327b86c8170a8125344e9b1aeb028d Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Mon, 1 May 2023 15:38:29 -0700
Subject: [PATCH v15 5/5] Introduce collation provider "builtin" for "C" and
"C.UTF-8".
The builtin "C" locale is equal (in semantics and implementation) to
the libc "C" locale.
The builtin "C.UTF-8" locale is especially useful. It provides a fast
memcmp-based collation (like "C") that supports abbrevated keys, while
also providing richer ctype semantics (upper/lower and regexes). The
semantics are derived from Unicode by building in lookup tables in the
same way as for text normalization. By using built-in semantics, the
behavior is stabilized within a Postgres major version, and also
matches the behavior of other built-in Unicode functionality, such as
normalization.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
---
doc/src/sgml/charset.sgml | 88 +++++++--
doc/src/sgml/ref/create_collation.sgml | 11 +-
doc/src/sgml/ref/create_database.sgml | 8 +-
doc/src/sgml/ref/createdb.sgml | 2 +-
doc/src/sgml/ref/initdb.sgml | 7 +-
src/backend/catalog/pg_collation.c | 5 +-
src/backend/commands/collationcmds.c | 93 +++++++--
src/backend/commands/dbcommands.c | 123 +++++++++---
src/backend/regex/regc_pg_locale.c | 41 +++-
src/backend/utils/adt/formatting.c | 193 +++++++++++++++++++
src/backend/utils/adt/pg_locale.c | 130 +++++++++++--
src/backend/utils/init/postinit.c | 27 ++-
src/bin/initdb/initdb.c | 30 ++-
src/bin/initdb/t/001_initdb.pl | 55 ++++++
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 74 +++++--
src/bin/psql/describe.c | 4 +-
src/bin/scripts/createdb.c | 18 +-
src/bin/scripts/t/020_createdb.pl | 72 +++++++
src/include/catalog/pg_collation.dat | 9 +-
src/include/catalog/pg_collation.h | 3 +
src/include/utils/pg_locale.h | 7 +-
src/test/icu/t/010_database.pl | 18 +-
src/test/regress/expected/collate.out | 24 ++-
src/test/regress/expected/collate.utf8.out | 97 ++++++++++
src/test/regress/expected/collate.utf8_1.out | 8 +
src/test/regress/parallel_schedule | 4 +-
src/test/regress/sql/collate.sql | 10 +
src/test/regress/sql/collate.utf8.sql | 45 +++++
29 files changed, 1082 insertions(+), 139 deletions(-)
create mode 100644 src/test/regress/expected/collate.utf8.out
create mode 100644 src/test/regress/expected/collate.utf8_1.out
create mode 100644 src/test/regress/sql/collate.utf8.sql
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 74783d148f..1553deea20 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -342,22 +342,14 @@ initdb --locale=sv_SE
<title>Locale Providers</title>
<para>
- <productname>PostgreSQL</productname> supports multiple <firstterm>locale
- providers</firstterm>. This specifies which library supplies the locale
- data. One standard provider name is <literal>libc</literal>, which uses
- the locales provided by the operating system C library. These are the
- locales used by most tools provided by the operating system. Another
- provider is <literal>icu</literal>, which uses the external
- ICU<indexterm><primary>ICU</primary></indexterm> library. ICU locales can
- only be used if support for ICU was configured when PostgreSQL was built.
+ A locale provider specifies which library defines the locale behavior for
+ collations and character classifications.
</para>
<para>
The commands and tools that select the locale settings, as described
- above, each have an option to select the locale provider. The examples
- shown earlier all use the <literal>libc</literal> provider, which is the
- default. Here is an example to initialize a database cluster using the
- ICU provider:
+ above, each have an option to select the locale provider. Here is an
+ example to initialize a database cluster using the ICU provider:
<programlisting>
initdb --locale-provider=icu --icu-locale=en
</programlisting>
@@ -370,12 +362,74 @@ initdb --locale-provider=icu --icu-locale=en
</para>
<para>
- Which locale provider to use depends on individual requirements. For most
- basic uses, either provider will give adequate results. For the libc
- provider, it depends on what the operating system offers; some operating
- systems are better than others. For advanced uses, ICU offers more locale
- variants and customization options.
+ Regardless of the locale provider, the operating system is still used to
+ provide some locale-aware behavior, such as messages (see <xref
+ linkend="guc-lc-messages"/>).
</para>
+
+ <para>
+ The available locale providers are listed below.
+ </para>
+
+ <sect3 id="locale-provider-builtin">
+ <title>Builtin</title>
+ <para>
+ The <literal>builtin</literal> provider uses built-in operations. Only
+ the <literal>C</literal> and <literal>C.UTF-8</literal> locales are
+ supported for this provider.
+ </para>
+ <para>
+ The <literal>C</literal> locale behavior is identical to the
+ <literal>C</literal> locale in the libc provider. When using this locale,
+ the behavior may depend on the database encoding.
+ </para>
+ <para>
+ The <literal>C.UTF-8</literal> locale is available only for when the
+ database encoding is <literal>UTF-8</literal>, and the behavior is based
+ on Unicode. The collation uses the code point values only. The regular
+ expression character classes are based on the "POSIX Compatible"
+ semantics, and the case mapping is the "simple" variant.
+ </para>
+ </sect3>
+ <sect3 id="locale-provider-icu">
+ <title>ICU</title>
+ <para>
+ The <literal>icu</literal> provider uses the external
+ ICU<indexterm><primary>ICU</primary></indexterm>
+ library. <productname>PostgreSQL</productname> must have been configured
+ with support.
+ </para>
+ <para>
+ ICU provides collation and character classification behavior that is
+ independent of the operating system and database encoding, which is
+ preferable if you expect to transition to other platforms without any
+ change in results. <literal>LC_COLLATE</literal> and
+ <literal>LC_CTYPE</literal> can be set independently of the ICU locale.
+ </para>
+ <note>
+ <para>
+ For the ICU provider, results may depend on the version of the ICU
+ library used, as it is updated to reflect changes in natural language
+ over time.
+ </para>
+ </note>
+ </sect3>
+ <sect3 id="locale-provider-libc">
+ <title>libc</title>
+ <para>
+ The <literal>libc</literal> provider uses the operating system's C
+ library. The collation and character classification behavior is
+ controlled by the settings <literal>LC_COLLATE</literal> and
+ <literal>LC_CTYPE</literal>, so they cannot be set independently.
+ </para>
+ <note>
+ <para>
+ The same locale name may have different behavior on different platforms
+ when using the libc provider.
+ </para>
+ </note>
+ </sect3>
+
</sect2>
<sect2 id="icu-locales">
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 5cf9777764..85f18cbbe5 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -96,6 +96,11 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
<replaceable>locale</replaceable>, you cannot specify either of those
parameters.
</para>
+ <para>
+ If <replaceable>provider</replaceable> is <literal>builtin</literal>,
+ then <replaceable>locale</replaceable> must be specified and set to
+ either <literal>C</literal> or <literal>C.UTF-8</literal>.
+ </para>
</listitem>
</varlistentry>
@@ -129,9 +134,9 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
<listitem>
<para>
Specifies the provider to use for locale services associated with this
- collation. Possible values are
- <literal>icu</literal><indexterm><primary>ICU</primary></indexterm>
- (if the server was built with ICU support) or <literal>libc</literal>.
+ collation. Possible values are <literal>builtin</literal>,
+ <literal>icu</literal><indexterm><primary>ICU</primary></indexterm> (if
+ the server was built with ICU support) or <literal>libc</literal>.
<literal>libc</literal> is the default. See <xref
linkend="locale-providers"/> for details.
</para>
diff --git a/doc/src/sgml/ref/create_database.sgml b/doc/src/sgml/ref/create_database.sgml
index ce7317f81b..6dc3348d1b 100644
--- a/doc/src/sgml/ref/create_database.sgml
+++ b/doc/src/sgml/ref/create_database.sgml
@@ -162,6 +162,12 @@ CREATE DATABASE <replaceable class="parameter">name</replaceable>
linkend="create-database-lc-ctype"/>, or <xref
linkend="create-database-icu-locale"/> individually.
</para>
+ <para>
+ If <xref linkend="create-database-locale-provider"/> is
+ <literal>builtin</literal>, then <replaceable>locale</replaceable>
+ must be specified and set to either <literal>C</literal> or
+ <literal>C.UTF-8</literal>.
+ </para>
<tip>
<para>
The other locale settings <xref linkend="guc-lc-messages"/>, <xref
@@ -243,7 +249,7 @@ CREATE DATABASE <replaceable class="parameter">name</replaceable>
<listitem>
<para>
Specifies the provider to use for the default collation in this
- database. Possible values are
+ database. Possible values are <literal>builtin</literal>,
<literal>icu</literal><indexterm><primary>ICU</primary></indexterm>
(if the server was built with ICU support) or <literal>libc</literal>.
By default, the provider is the same as that of the <xref
diff --git a/doc/src/sgml/ref/createdb.sgml b/doc/src/sgml/ref/createdb.sgml
index e4647d5ce7..d3e815f659 100644
--- a/doc/src/sgml/ref/createdb.sgml
+++ b/doc/src/sgml/ref/createdb.sgml
@@ -171,7 +171,7 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
- <term><option>--locale-provider={<literal>libc</literal>|<literal>icu</literal>}</option></term>
+ <term><option>--locale-provider={<literal>builtin</literal>|<literal>libc</literal>|<literal>icu</literal>}</option></term>
<listitem>
<para>
Specifies the locale provider for the database's default collation.
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
index d43c91575c..2192d017b3 100644
--- a/doc/src/sgml/ref/initdb.sgml
+++ b/doc/src/sgml/ref/initdb.sgml
@@ -286,6 +286,11 @@ PostgreSQL documentation
environment that <command>initdb</command> runs in. Locale
support is described in <xref linkend="locale"/>.
</para>
+ <para>
+ If <option>--locale-provider</option> is <literal>builtin</literal>,
+ <option>--locale</option> must be specified and set to
+ <literal>C</literal> or <literal>C.UTF-8</literal>.
+ </para>
</listitem>
</varlistentry>
@@ -315,7 +320,7 @@ PostgreSQL documentation
</varlistentry>
<varlistentry id="app-initdb-option-locale-provider">
- <term><option>--locale-provider={<literal>libc</literal>|<literal>icu</literal>}</option></term>
+ <term><option>--locale-provider={<literal>builtin</literal>|<literal>libc</literal>|<literal>icu</literal>}</option></term>
<listitem>
<para>
This option sets the locale provider for databases created in the new
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index b615d2fc7a..2144e11fe8 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -68,7 +68,10 @@ CollationCreate(const char *collname, Oid collnamespace,
Assert(collname);
Assert(collnamespace);
Assert(collowner);
- Assert((collcollate && collctype) || colllocale);
+ Assert((collprovider == COLLPROVIDER_LIBC &&
+ collcollate && collctype && !colllocale) ||
+ (collprovider != COLLPROVIDER_LIBC &&
+ !collcollate && !collctype && colllocale));
/*
* Make sure there is no existing collation of same name & encoding.
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 19e61b3e5b..505b8ae86d 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *versionEl = NULL;
char *collcollate;
char *collctype;
- char *colllocale;
+ const char *colllocale;
char *collicurules;
bool collisdeterministic;
int collencoding;
@@ -215,7 +215,9 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (collproviderstr)
{
- if (pg_strcasecmp(collproviderstr, "icu") == 0)
+ if (pg_strcasecmp(collproviderstr, "builtin") == 0)
+ collprovider = COLLPROVIDER_BUILTIN;
+ else if (pg_strcasecmp(collproviderstr, "icu") == 0)
collprovider = COLLPROVIDER_ICU;
else if (pg_strcasecmp(collproviderstr, "libc") == 0)
collprovider = COLLPROVIDER_LIBC;
@@ -245,7 +247,17 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (lcctypeEl)
collctype = defGetString(lcctypeEl);
- if (collprovider == COLLPROVIDER_LIBC)
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ {
+ if (!colllocale)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("parameter \"locale\" must be specified")));
+
+ colllocale = builtin_validate_locale(GetDatabaseEncoding(),
+ colllocale);
+ }
+ else if (collprovider == COLLPROVIDER_LIBC)
{
if (!collcollate)
ereport(ERROR,
@@ -305,7 +317,17 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU rules cannot be specified unless locale provider is ICU")));
- if (collprovider == COLLPROVIDER_ICU)
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ {
+ /*
+ * Behavior may be different in different encodings, so set
+ * collencoding to the current database encoding. No validation is
+ * required, because the "builtin" provider is compatible with any
+ * encoding.
+ */
+ collencoding = GetDatabaseEncoding();
+ }
+ else if (collprovider == COLLPROVIDER_ICU)
{
#ifdef USE_ICU
/*
@@ -334,7 +356,18 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
if (!collversion)
- collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colllocale : collcollate);
+ {
+ const char *locale;
+
+ if (collprovider == COLLPROVIDER_ICU)
+ locale = colllocale;
+ else if (collprovider == COLLPROVIDER_LIBC)
+ locale = collcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ collversion = get_collation_actual_version(collprovider, locale);
+ }
newoid = CollationCreate(collName,
collNamespace,
@@ -409,6 +442,7 @@ AlterCollation(AlterCollationStmt *stmt)
Form_pg_collation collForm;
Datum datum;
bool isnull;
+ char *locale;
char *oldversion;
char *newversion;
ObjectAddress address;
@@ -435,8 +469,20 @@ AlterCollation(AlterCollationStmt *stmt)
datum = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion, &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
- newversion = get_collation_actual_version(collForm->collprovider, TextDatumGetCString(datum));
+ if (collForm->collprovider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, Anum_pg_collation_colllocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (collForm->collprovider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, Anum_pg_collation_collcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ newversion = get_collation_actual_version(collForm->collprovider, locale);
/* cannot change from NULL to non-NULL or vice versa */
if ((!oldversion && newversion) || (oldversion && !newversion))
@@ -500,11 +546,18 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
provider = ((Form_pg_database) GETSTRUCT(dbtup))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup,
- provider == COLLPROVIDER_ICU ?
- Anum_pg_database_datlocale : Anum_pg_database_datcollate);
-
- locale = TextDatumGetCString(datum);
+ if (provider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup, Anum_pg_database_datlocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (provider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup, Anum_pg_database_datcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
ReleaseSysCache(dbtup);
}
@@ -521,11 +574,19 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
provider = ((Form_pg_collation) GETSTRUCT(colltp))->collprovider;
Assert(provider != COLLPROVIDER_DEFAULT);
- datum = SysCacheGetAttrNotNull(COLLOID, colltp,
- provider == COLLPROVIDER_ICU ?
- Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
- locale = TextDatumGetCString(datum);
+ if (provider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, colltp, Anum_pg_collation_colllocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (provider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, colltp, Anum_pg_collation_collcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
ReleaseSysCache(colltp);
}
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index c8732be067..b04ff25562 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -698,6 +698,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
DefElem *dtemplate = NULL;
DefElem *dencoding = NULL;
DefElem *dlocale = NULL;
+ DefElem *dbuiltinlocale = NULL;
DefElem *dcollate = NULL;
DefElem *dctype = NULL;
DefElem *diculocale = NULL;
@@ -713,7 +714,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
const char *dbtemplate = NULL;
char *dbcollate = NULL;
char *dbctype = NULL;
- char *dblocale = NULL;
+ const char *dblocale = NULL;
char *dbicurules = NULL;
char dblocprovider = '\0';
char *canonname;
@@ -762,6 +763,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
errorConflictingDefElem(defel, pstate);
dlocale = defel;
}
+ else if (strcmp(defel->defname, "builtin_locale") == 0)
+ {
+ if (dbuiltinlocale)
+ errorConflictingDefElem(defel, pstate);
+ dbuiltinlocale = defel;
+ }
else if (strcmp(defel->defname, "lc_collate") == 0)
{
if (dcollate)
@@ -897,7 +904,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
dbcollate = defGetString(dlocale);
dbctype = defGetString(dlocale);
+ dblocale = defGetString(dlocale);
}
+ if (dbuiltinlocale && dbuiltinlocale->arg)
+ dblocale = defGetString(dbuiltinlocale);
if (dcollate && dcollate->arg)
dbcollate = defGetString(dcollate);
if (dctype && dctype->arg)
@@ -910,7 +920,9 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
char *locproviderstr = defGetString(dlocprovider);
- if (pg_strcasecmp(locproviderstr, "icu") == 0)
+ if (pg_strcasecmp(locproviderstr, "builtin") == 0)
+ dblocprovider = COLLPROVIDER_BUILTIN;
+ else if (pg_strcasecmp(locproviderstr, "icu") == 0)
dblocprovider = COLLPROVIDER_ICU;
else if (pg_strcasecmp(locproviderstr, "libc") == 0)
dblocprovider = COLLPROVIDER_LIBC;
@@ -1027,14 +1039,9 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
dbctype = src_ctype;
if (dblocprovider == '\0')
dblocprovider = src_locprovider;
- if (dblocale == NULL && dblocprovider == COLLPROVIDER_ICU)
- {
- if (dlocale && dlocale->arg)
- dblocale = defGetString(dlocale);
- else
- dblocale = src_locale;
- }
- if (dbicurules == NULL && dblocprovider == COLLPROVIDER_ICU)
+ if (dblocale == NULL)
+ dblocale = src_locale;
+ if (dbicurules == NULL)
dbicurules = src_icurules;
/* Some encodings are client only */
@@ -1059,6 +1066,27 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
check_encoding_locale_matches(encoding, dbcollate, dbctype);
+ if (dblocprovider == COLLPROVIDER_BUILTIN)
+ {
+ /*
+ * This would happen if template0 uses the libc provider but the new
+ * database uses builtin.
+ */
+ if (!dblocale)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("LOCALE must be specified for the builtin provider")));
+
+ dblocale = builtin_validate_locale(encoding, dblocale);
+ }
+ else
+ {
+ if (dbuiltinlocale && dbuiltinlocale->arg)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("BUILTIN_LOCALE cannot be specified unless locale provider is builtin")));
+ }
+
if (dblocprovider == COLLPROVIDER_ICU)
{
if (!(is_encoding_supported_by_icu(encoding)))
@@ -1100,7 +1128,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
}
else
{
- if (dblocale)
+ if (diculocale && diculocale->arg)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU locale cannot be specified unless locale provider is ICU")));
@@ -1111,6 +1139,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
errmsg("ICU rules cannot be specified unless locale provider is ICU")));
}
+ /* for libc, locale comes from datcollate and datctype */
+ if (dblocprovider == COLLPROVIDER_LIBC)
+ dblocale = NULL;
+
/*
* Check that the new encoding and locale settings match the source
* database. We insist on this because we simply copy the source data ---
@@ -1195,9 +1227,17 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
*/
if (src_collversion && !dcollversion)
{
- char *actual_versionstr;
+ char *actual_versionstr;
+ const char *locale;
- actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
+ if (dblocprovider == COLLPROVIDER_ICU)
+ locale = dblocale;
+ else if (dblocprovider == COLLPROVIDER_LIBC)
+ locale = dbcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ actual_versionstr = get_collation_actual_version(dblocprovider, locale);
if (!actual_versionstr)
ereport(ERROR,
(errmsg("template database \"%s\" has a collation version, but no actual collation version could be determined",
@@ -1225,7 +1265,18 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* collation version, which is normally only the case for template0.
*/
if (dbcollversion == NULL)
- dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
+ {
+ const char *locale;
+
+ if (dblocprovider == COLLPROVIDER_ICU)
+ locale = dblocale;
+ else if (dblocprovider == COLLPROVIDER_LIBC)
+ locale = dbcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ dbcollversion = get_collation_actual_version(dblocprovider, locale);
+ }
/* Resolve default tablespace for new database */
if (dtablespacename && dtablespacename->arg)
@@ -1364,8 +1415,8 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* block on the unique index, and fail after we commit).
*/
- Assert((dblocprovider == COLLPROVIDER_ICU && dblocale) ||
- (dblocprovider != COLLPROVIDER_ICU && !dblocale));
+ Assert((dblocprovider != COLLPROVIDER_LIBC && dblocale) ||
+ (dblocprovider == COLLPROVIDER_LIBC && !dblocale));
/* Form tuple */
new_record[Anum_pg_database_oid - 1] = ObjectIdGetDatum(dboid);
@@ -2446,6 +2497,7 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
ObjectAddress address;
Datum datum;
bool isnull;
+ char *locale;
char *oldversion;
char *newversion;
@@ -2472,10 +2524,24 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
datum = heap_getattr(tuple, Anum_pg_database_datcollversion, RelationGetDescr(rel), &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
- if (isnull)
- elog(ERROR, "unexpected null in pg_database");
- newversion = get_collation_actual_version(datForm->datlocprovider, TextDatumGetCString(datum));
+ if (datForm->datlocprovider == COLLPROVIDER_ICU)
+ {
+ datum = heap_getattr(tuple, Anum_pg_database_datlocale, RelationGetDescr(rel), &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected null in pg_database");
+ locale = TextDatumGetCString(datum);
+ }
+ else if (datForm->datlocprovider == COLLPROVIDER_LIBC)
+ {
+ datum = heap_getattr(tuple, Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected null in pg_database");
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ newversion = get_collation_actual_version(datForm->datlocprovider, locale);
/* cannot change from NULL to non-NULL or vice versa */
if ((!oldversion && newversion) || (oldversion && !newversion))
@@ -2660,6 +2726,7 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
HeapTuple tp;
char datlocprovider;
Datum datum;
+ char *locale;
char *version;
tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(dbid));
@@ -2670,8 +2737,20 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
datlocprovider = ((Form_pg_database) GETSTRUCT(tp))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate);
- version = get_collation_actual_version(datlocprovider, TextDatumGetCString(datum));
+ if (datlocprovider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, Anum_pg_database_datlocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (datlocprovider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, Anum_pg_database_datcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ version = get_collation_actual_version(datlocprovider, locale);
ReleaseSysCache(tp);
diff --git a/src/backend/regex/regc_pg_locale.c b/src/backend/regex/regc_pg_locale.c
index 42d15b6303..0278f45adc 100644
--- a/src/backend/regex/regc_pg_locale.c
+++ b/src/backend/regex/regc_pg_locale.c
@@ -16,6 +16,8 @@
*/
#include "catalog/pg_collation.h"
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
#include "utils/pg_locale.h"
/*
@@ -64,6 +66,7 @@
typedef enum
{
PG_REGEX_LOCALE_C, /* C locale (encoding independent) */
+ PG_REGEX_BUILTIN, /* built-in Unicode semantics */
PG_REGEX_LOCALE_WIDE, /* Use <wctype.h> functions */
PG_REGEX_LOCALE_1BYTE, /* Use <ctype.h> functions */
PG_REGEX_LOCALE_WIDE_L, /* Use locale_t <wctype.h> functions */
@@ -75,6 +78,8 @@ static PG_Locale_Strategy pg_regex_strategy;
static pg_locale_t pg_regex_locale;
static Oid pg_regex_collation;
+static bool regex_builtin_cclass_posix = false;
+
/*
* Hard-wired character properties for C locale
*/
@@ -266,7 +271,15 @@ pg_set_regex_collation(Oid collation)
if (GetDatabaseEncoding() == PG_UTF8)
{
if (pg_regex_locale)
- pg_regex_strategy = PG_REGEX_LOCALE_WIDE_L;
+ {
+ if (pg_regex_locale->provider == COLLPROVIDER_BUILTIN)
+ {
+ pg_regex_strategy = PG_REGEX_BUILTIN;
+ regex_builtin_cclass_posix = pg_regex_locale->info.builtin.cclass_posix;
+ }
+ else
+ pg_regex_strategy = PG_REGEX_LOCALE_WIDE_L;
+ }
else
pg_regex_strategy = PG_REGEX_LOCALE_WIDE;
}
@@ -290,6 +303,8 @@ pg_wc_isdigit(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISDIGIT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isdigit(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswdigit((wint_t) c);
@@ -322,6 +337,8 @@ pg_wc_isalpha(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISALPHA));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isalpha(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswalpha((wint_t) c);
@@ -354,6 +371,8 @@ pg_wc_isalnum(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISALNUM));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isalnum(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswalnum((wint_t) c);
@@ -395,6 +414,8 @@ pg_wc_isupper(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISUPPER));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isupper(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswupper((wint_t) c);
@@ -427,6 +448,8 @@ pg_wc_islower(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISLOWER));
+ case PG_REGEX_BUILTIN:
+ return pg_u_islower(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswlower((wint_t) c);
@@ -459,6 +482,8 @@ pg_wc_isgraph(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISGRAPH));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isgraph(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswgraph((wint_t) c);
@@ -491,6 +516,8 @@ pg_wc_isprint(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISPRINT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isprint(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswprint((wint_t) c);
@@ -523,6 +550,8 @@ pg_wc_ispunct(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISPUNCT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_ispunct(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswpunct((wint_t) c);
@@ -555,6 +584,8 @@ pg_wc_isspace(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISSPACE));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isspace(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswspace((wint_t) c);
@@ -588,6 +619,8 @@ pg_wc_toupper(pg_wchar c)
if (c <= (pg_wchar) 127)
return pg_ascii_toupper((unsigned char) c);
return c;
+ case PG_REGEX_BUILTIN:
+ return unicode_uppercase_simple(c);
case PG_REGEX_LOCALE_WIDE:
/* force C behavior for ASCII characters, per comments above */
if (c <= (pg_wchar) 127)
@@ -628,6 +661,8 @@ pg_wc_tolower(pg_wchar c)
if (c <= (pg_wchar) 127)
return pg_ascii_tolower((unsigned char) c);
return c;
+ case PG_REGEX_BUILTIN:
+ return unicode_lowercase_simple(c);
case PG_REGEX_LOCALE_WIDE:
/* force C behavior for ASCII characters, per comments above */
if (c <= (pg_wchar) 127)
@@ -792,6 +827,9 @@ pg_ctype_get_cache(pg_wc_probefunc probefunc, int cclasscode)
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
#endif
break;
+ case PG_REGEX_BUILTIN:
+ max_chr = (pg_wchar) MAX_SIMPLE_CHR;
+ break;
case PG_REGEX_LOCALE_WIDE:
case PG_REGEX_LOCALE_WIDE_L:
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
@@ -809,6 +847,7 @@ pg_ctype_get_cache(pg_wc_probefunc probefunc, int cclasscode)
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
break;
default:
+ Assert(false);
max_chr = 0; /* can't get here, but keep compiler quiet */
break;
}
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index d176723d95..c1f04eb03a 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -77,6 +77,8 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
#include "mb/pg_wchar.h"
#include "nodes/miscnodes.h"
#include "parser/scansup.h"
@@ -1670,6 +1672,67 @@ str_tolower(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *orig = (unsigned char *) buff;
+ unsigned char *workspace;
+ size_t workspace_size = nbytes + 1;
+ const unsigned char *sp;
+ unsigned char *rp;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ workspace = (unsigned char *)palloc(workspace_size);
+
+ sp = orig;
+ rp = workspace;
+ while (sp - orig < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(sp);
+ pg_wchar u2 = unicode_lowercase_simple(u1);
+
+ /*
+ * If we can't fit 4 more bytes, and the next character to
+ * write is multibyte, reallocate buffer to maximum size we
+ * will need.
+ */
+ if (rp - workspace > workspace_size - 4 && u2 >= 0x80)
+ {
+ int written = rp - workspace;
+
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ workspace_size = (nbytes + 1) * sizeof(pg_wchar);
+ workspace = repalloc(workspace, workspace_size);
+ rp = workspace + written;
+ }
+
+ unicode_to_utf8(u2, rp);
+ sp += pg_utf_mblen(sp);
+ rp += pg_utf_mblen(rp);
+ }
+
+ *rp = '\0';
+ rp++;
+
+ if (workspace_size == rp - workspace)
+ {
+ result = (char *) workspace;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(rp - workspace);
+ memcpy(result, workspace, rp - workspace);
+ pfree(workspace);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
@@ -1788,6 +1851,67 @@ str_toupper(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *orig = (unsigned char *) buff;
+ unsigned char *workspace;
+ size_t workspace_size = nbytes + 1;
+ const unsigned char *sp;
+ unsigned char *rp;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ workspace = (unsigned char *)palloc(workspace_size);
+
+ sp = orig;
+ rp = workspace;
+ while (sp - orig < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(sp);
+ pg_wchar u2 = unicode_uppercase_simple(u1);
+
+ /*
+ * If we can't fit 4 more bytes, and the next character to
+ * write is multibyte, reallocate buffer to maximum size we
+ * will need.
+ */
+ if (rp - workspace > workspace_size - 4 && u2 >= 0x80)
+ {
+ int written = rp - workspace;
+
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ workspace_size = (nbytes + 1) * sizeof(pg_wchar);
+ workspace = repalloc(workspace, workspace_size);
+ rp = workspace + written;
+ }
+
+ unicode_to_utf8(u2, rp);
+ sp += pg_utf_mblen(sp);
+ rp += pg_utf_mblen(rp);
+ }
+
+ *rp = '\0';
+ rp++;
+
+ if (workspace_size == rp - workspace)
+ {
+ result = (char *) workspace;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(rp - workspace);
+ memcpy(result, workspace, rp - workspace);
+ pfree(workspace);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
@@ -1907,6 +2031,75 @@ str_initcap(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *orig = (unsigned char *) buff;
+ unsigned char *workspace;
+ size_t workspace_size = nbytes + 1;
+ const unsigned char *sp;
+ unsigned char *rp;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ workspace = (unsigned char *)palloc(workspace_size);
+
+ sp = orig;
+ rp = workspace;
+ while (sp - orig < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(sp);
+ pg_wchar u2;
+
+ if (wasalnum)
+ u2 = unicode_lowercase_simple(u1);
+ else
+ u2 = unicode_titlecase_simple(u1);
+
+ /*
+ * If we can't fit 4 more bytes, and the next character to
+ * write is multibyte, reallocate buffer to maximum size we
+ * will need.
+ */
+ if (rp - workspace > workspace_size - 4 && u2 >= 0x80)
+ {
+ int written = rp - workspace;
+
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ workspace_size = (nbytes + 1) * sizeof(pg_wchar);
+ workspace = repalloc(workspace, workspace_size);
+ rp = workspace + written;
+ }
+
+ unicode_to_utf8(u2, rp);
+
+ wasalnum = pg_u_isalnum(u2, mylocale->info.builtin.cclass_posix);
+
+ sp += pg_utf_mblen(sp);
+ rp += pg_utf_mblen(rp);
+ }
+
+ *rp = '\0';
+ rp++;
+
+ if (workspace_size == rp - workspace)
+ {
+ result = (char *) workspace;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(rp - workspace);
+ memcpy(result, workspace, rp - workspace);
+ pfree(workspace);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 9d8634ec25..7067e29c2d 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1269,7 +1269,19 @@ lookup_collation_cache(Oid collation, bool set_flags)
elog(ERROR, "cache lookup failed for collation %u", collation);
collform = (Form_pg_collation) GETSTRUCT(tp);
- if (collform->collprovider == COLLPROVIDER_LIBC)
+ if (collform->collprovider == COLLPROVIDER_BUILTIN)
+ {
+ Datum datum;
+ const char *colllocale;
+
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
+ colllocale = TextDatumGetCString(datum);
+
+ cache_entry->collate_is_c = true;
+ cache_entry->ctype_is_c = ((strcmp(colllocale, "C") == 0) ||
+ (strcmp(colllocale, "POSIX") == 0));
+ }
+ else if (collform->collprovider == COLLPROVIDER_LIBC)
{
Datum datum;
const char *collcollate;
@@ -1320,16 +1332,30 @@ lc_collate_is_c(Oid collation)
if (collation == DEFAULT_COLLATION_OID)
{
static int result = -1;
- char *localeptr;
-
- if (default_locale.provider == COLLPROVIDER_ICU)
- return false;
+ const char *localeptr;
if (result >= 0)
return (bool) result;
- localeptr = setlocale(LC_COLLATE, NULL);
- if (!localeptr)
- elog(ERROR, "invalid LC_COLLATE setting");
+
+ if (default_locale.provider == COLLPROVIDER_BUILTIN)
+ {
+ result = true;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_ICU)
+ {
+ result = false;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_LIBC)
+ {
+ localeptr = setlocale(LC_CTYPE, NULL);
+ if (!localeptr)
+ elog(ERROR, "invalid LC_CTYPE setting");
+ }
+ else
+ elog(ERROR, "unexpected collation provider '%c'",
+ default_locale.provider);
if (strcmp(localeptr, "C") == 0)
result = true;
@@ -1373,16 +1399,29 @@ lc_ctype_is_c(Oid collation)
if (collation == DEFAULT_COLLATION_OID)
{
static int result = -1;
- char *localeptr;
-
- if (default_locale.provider == COLLPROVIDER_ICU)
- return false;
+ const char *localeptr;
if (result >= 0)
return (bool) result;
- localeptr = setlocale(LC_CTYPE, NULL);
- if (!localeptr)
- elog(ERROR, "invalid LC_CTYPE setting");
+
+ if (default_locale.provider == COLLPROVIDER_BUILTIN)
+ {
+ localeptr = default_locale.info.builtin.locale;
+ }
+ else if (default_locale.provider == COLLPROVIDER_ICU)
+ {
+ result = false;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_LIBC)
+ {
+ localeptr = setlocale(LC_CTYPE, NULL);
+ if (!localeptr)
+ elog(ERROR, "invalid LC_CTYPE setting");
+ }
+ else
+ elog(ERROR, "unexpected collation provider '%c'",
+ default_locale.provider);
if (strcmp(localeptr, "C") == 0)
result = true;
@@ -1390,6 +1429,7 @@ lc_ctype_is_c(Oid collation)
result = true;
else
result = false;
+
return (bool) result;
}
@@ -1520,10 +1560,10 @@ pg_newlocale_from_collation(Oid collid)
if (collid == DEFAULT_COLLATION_OID)
{
- if (default_locale.provider == COLLPROVIDER_ICU)
- return &default_locale;
- else
+ if (default_locale.provider == COLLPROVIDER_LIBC)
return (pg_locale_t) 0;
+ else
+ return &default_locale;
}
cache_entry = lookup_collation_cache(collid, false);
@@ -1548,7 +1588,18 @@ pg_newlocale_from_collation(Oid collid)
result.provider = collform->collprovider;
result.deterministic = collform->collisdeterministic;
- if (collform->collprovider == COLLPROVIDER_LIBC)
+ if (collform->collprovider == COLLPROVIDER_BUILTIN)
+ {
+ const char *locstr;
+
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
+ locstr = TextDatumGetCString(datum);
+
+ result.info.builtin.locale = MemoryContextStrdup(TopMemoryContext,
+ locstr);
+ result.info.builtin.cclass_posix = true;
+ }
+ else if (collform->collprovider == COLLPROVIDER_LIBC)
{
const char *collcollate;
const char *collctype pg_attribute_unused();
@@ -1627,6 +1678,7 @@ pg_newlocale_from_collation(Oid collid)
collversionstr = TextDatumGetCString(datum);
+ Assert(collform->collprovider != COLLPROVIDER_BUILTIN);
datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
actual_versionstr = get_collation_actual_version(collform->collprovider,
@@ -1678,6 +1730,14 @@ get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
+ /*
+ * The only two supported locales (C and C.UTF-8) are both based on memcmp
+ * and do not change. (The ctype behavior can change, but the versioning
+ * does not track that.)
+ */
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ return NULL;
+
#ifdef USE_ICU
if (collprovider == COLLPROVIDER_ICU)
{
@@ -2444,6 +2504,38 @@ pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
return result;
}
+const char *
+builtin_validate_locale(int encoding, const char *locale)
+{
+ const char *canonical_name = NULL;
+ int required_encoding = -1;
+
+ if (strcmp(locale, "C") == 0 || strcmp(locale, "POSIX") == 0)
+ {
+ canonical_name = "C";
+ }
+ else if (strcmp(locale, "C.UTF-8") == 0 || strcmp(locale, "C.UTF8") == 0)
+ {
+ required_encoding = PG_UTF8;
+ canonical_name = "C.UTF-8";
+ }
+
+ if (!canonical_name)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("invalid locale name \"%s\" for builtin provider",
+ locale)));
+
+ if (required_encoding >= 0 && encoding != required_encoding)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("encoding \"%s\" does not match locale \"%s\"",
+ pg_encoding_to_char(encoding), locale)));
+
+ return canonical_name;
+}
+
+
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 702bad6d8a..f6dc3afbfe 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -318,7 +318,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
bool isnull;
char *collate;
char *ctype;
- char *iculocale;
+ char *datlocale;
/* Fetch our pg_database row normally, via syscache */
tup = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
@@ -423,12 +423,21 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
strcmp(ctype, "POSIX") == 0)
database_ctype_is_c = true;
- if (dbform->datlocprovider == COLLPROVIDER_ICU)
+ if (dbform->datlocprovider == COLLPROVIDER_BUILTIN)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
+ datlocale = TextDatumGetCString(datum);
+
+ default_locale.info.builtin.locale = MemoryContextStrdup(
+ TopMemoryContext, datlocale);
+ default_locale.info.builtin.cclass_posix = true;
+ }
+ else if (dbform->datlocprovider == COLLPROVIDER_ICU)
{
char *icurules;
datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
- iculocale = TextDatumGetCString(datum);
+ datlocale = TextDatumGetCString(datum);
datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_daticurules, &isnull);
if (!isnull)
@@ -436,10 +445,10 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
else
icurules = NULL;
- make_icu_collator(iculocale, icurules, &default_locale);
+ make_icu_collator(datlocale, icurules, &default_locale);
}
else
- iculocale = NULL;
+ datlocale = NULL;
default_locale.provider = dbform->datlocprovider;
@@ -461,10 +470,16 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
{
char *actual_versionstr;
char *collversionstr;
+ char *locale;
collversionstr = TextDatumGetCString(datum);
- actual_versionstr = get_collation_actual_version(dbform->datlocprovider, dbform->datlocprovider == COLLPROVIDER_ICU ? iculocale : collate);
+ if (dbform->datlocprovider == COLLPROVIDER_LIBC)
+ locale = collate;
+ else
+ locale = datlocale;
+
+ actual_versionstr = get_collation_actual_version(dbform->datlocprovider, locale);
if (!actual_versionstr)
/* should not happen */
elog(WARNING,
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index eea40b930f..fab9b4c131 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -2450,7 +2450,7 @@ usage(const char *progname)
" set default locale in the respective category for\n"
" new databases (default taken from environment)\n"));
printf(_(" --no-locale equivalent to --locale=C\n"));
- printf(_(" --locale-provider={libc|icu}\n"
+ printf(_(" --locale-provider={builtin|libc|icu}\n"
" set default locale provider for new databases\n"));
printf(_(" --pwfile=FILE read password for the new superuser from file\n"));
printf(_(" -T, --text-search-config=CFG\n"
@@ -2601,7 +2601,15 @@ setup_locale_encoding(void)
{
setlocales();
- if (locale_provider == COLLPROVIDER_LIBC &&
+ if (locale_provider == COLLPROVIDER_BUILTIN &&
+ strcmp(lc_ctype, "C") == 0 &&
+ strcmp(lc_collate, "C") == 0 &&
+ strcmp(lc_time, "C") == 0 &&
+ strcmp(lc_numeric, "C") == 0 &&
+ strcmp(lc_monetary, "C") == 0 &&
+ strcmp(lc_messages, "C") == 0)
+ printf(_("The database cluster will be initialized with no locale.\n"));
+ else if (locale_provider == COLLPROVIDER_LIBC &&
strcmp(lc_ctype, lc_collate) == 0 &&
strcmp(lc_ctype, lc_time) == 0 &&
strcmp(lc_ctype, lc_numeric) == 0 &&
@@ -3107,9 +3115,10 @@ main(int argc, char *argv[])
{"allow-group-access", no_argument, NULL, 'g'},
{"discard-caches", no_argument, NULL, 14},
{"locale-provider", required_argument, NULL, 15},
- {"icu-locale", required_argument, NULL, 16},
- {"icu-rules", required_argument, NULL, 17},
- {"sync-method", required_argument, NULL, 18},
+ {"builtin-locale", required_argument, NULL, 16},
+ {"icu-locale", required_argument, NULL, 17},
+ {"icu-rules", required_argument, NULL, 18},
+ {"sync-method", required_argument, NULL, 19},
{NULL, 0, NULL, 0}
};
@@ -3277,7 +3286,9 @@ main(int argc, char *argv[])
"-c debug_discard_caches=1");
break;
case 15:
- if (strcmp(optarg, "icu") == 0)
+ if (strcmp(optarg, "builtin") == 0)
+ locale_provider = COLLPROVIDER_BUILTIN;
+ else if (strcmp(optarg, "icu") == 0)
locale_provider = COLLPROVIDER_ICU;
else if (strcmp(optarg, "libc") == 0)
locale_provider = COLLPROVIDER_LIBC;
@@ -3286,12 +3297,15 @@ main(int argc, char *argv[])
break;
case 16:
datlocale = pg_strdup(optarg);
- icu_locale_specified = true;
break;
case 17:
- icu_rules = pg_strdup(optarg);
+ datlocale = pg_strdup(optarg);
+ icu_locale_specified = true;
break;
case 18:
+ icu_rules = pg_strdup(optarg);
+ break;
+ case 19:
if (!parse_sync_method(optarg, &sync_method))
exit(1);
break;
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index 7606db1987..1701506abd 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -184,6 +184,61 @@ else
'locale provider ICU fails since no ICU support');
}
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', "$tempdir/data6"
+ ],
+ 'locale provider builtin fails without --locale'
+);
+
+command_ok(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--locale=C',
+ "$tempdir/data7"
+ ],
+ 'locale provider builtin with --locale'
+);
+
+command_ok(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '-E UTF-8',
+ '--builtin-locale=C.UTF-8', "$tempdir/data8"
+ ],
+ 'locale provider builtin with -E UTF-8 --builtin-locale=C.UTF-8'
+);
+
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '-E SQL_ASCII',
+ '--builtin-locale=C.UTF-8', "$tempdir/data9"
+ ],
+ 'locale provider builtin with --builtin-locale=C.UTF-8 fails for SQL_ASCII'
+);
+
+command_ok(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--lc-ctype=C',
+ '--locale=C', "$tempdir/data10"
+ ],
+ 'locale provider builtin with --lc-ctype'
+);
+
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--icu-locale=en',
+ "$tempdir/dataX"
+ ],
+ 'fails for locale provider builtin with ICU locale'
+);
+
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--icu-rules=""',
+ "$tempdir/dataX"
+ ],
+ 'fails for locale provider builtin with ICU rules'
+);
+
command_fails(
[ 'initdb', '--no-sync', '--locale-provider=xyz', "$tempdir/dataX" ],
'fails for invalid locale provider');
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3d829c99d7..b953db8bea 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3091,7 +3091,9 @@ dumpDatabase(Archive *fout)
}
appendPQExpBufferStr(creaQry, " LOCALE_PROVIDER = ");
- if (datlocprovider[0] == 'c')
+ if (datlocprovider[0] == 'b')
+ appendPQExpBufferStr(creaQry, "builtin");
+ else if (datlocprovider[0] == 'c')
appendPQExpBufferStr(creaQry, "libc");
else if (datlocprovider[0] == 'i')
appendPQExpBufferStr(creaQry, "icu");
@@ -13672,7 +13674,9 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
fmtQualifiedDumpable(collinfo));
appendPQExpBufferStr(q, "provider = ");
- if (collprovider[0] == 'c')
+ if (collprovider[0] == 'b')
+ appendPQExpBufferStr(q, "builtin");
+ else if (collprovider[0] == 'c')
appendPQExpBufferStr(q, "libc");
else if (collprovider[0] == 'i')
appendPQExpBufferStr(q, "icu");
@@ -13693,6 +13697,13 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
/* no locale -- the default collation cannot be reloaded anyway */
}
+ else if (collprovider[0] == 'b')
+ {
+ if (collcollate || collctype || colllocale || collicurules)
+ pg_log_warning("invalid collation \"%s\"", qcollname);
+
+ appendPQExpBufferStr(q, ", locale = 'C'");
+ }
else if (collprovider[0] == 'i')
{
if (fout->remoteVersion >= 150000)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 6f359d72ce..37a4e9d334 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -108,35 +108,71 @@ if ($oldnode->pg_version >= 11)
# can test that pg_upgrade copies the locale settings of template0
# from the old to the new cluster.
-my $original_encoding = "6"; # UTF-8
-my $original_provider = "c";
-my $original_locale = "C";
-my $original_iculocale = "";
-my $provider_field = "'c' AS datlocprovider";
-my $iculocale_field = "NULL AS datlocale";
-if (int($oldnode->pg_version) >= 15 && $ENV{with_icu} eq 'yes')
+my %encoding_number = ('UTF-8' => 6, 'SQL_ASCII' => 0);
+my $provider_field;
+my $datlocale_field;
+my $original_encoding;
+my $original_provider;
+my $original_datcollate = "C";
+my $original_datctype = "C";
+my $original_datlocale;
+
+if (int($oldnode->pg_version) >= 15)
{
$provider_field = "datlocprovider";
if (int($oldnode->pg_version) >= 17)
{
- $iculocale_field = "datlocale";
+ $datlocale_field = "datlocale";
}
else
{
- $iculocale_field = "daticulocale AS datlocale";
+ $datlocale_field = "daticulocale AS datlocale";
}
+}
+else
+{
+ $provider_field = "'c' AS datlocprovider";
+ $datlocale_field = "NULL AS datlocale";
+}
+
+if (int($oldnode->pg_version) >= 17)
+{
+ $original_encoding = "UTF-8";
+ $original_provider = "b";
+ $original_datlocale = "C.UTF-8";
+}
+elsif (int($oldnode->pg_version) >= 15 && $ENV{with_icu} eq 'yes')
+{
+ $original_encoding = "UTF-8";
$original_provider = "i";
- $original_iculocale = "fr-CA";
+ $original_datlocale = "fr-CA";
+}
+else
+{
+ my $original_encoding = "SQL_ASCII";
+ my $original_provider = "c";
+ my $original_datlocale = "";
}
my @initdb_params = @custom_opts;
-push @initdb_params, ('--encoding', 'UTF-8');
-push @initdb_params, ('--locale', $original_locale);
-if ($original_provider eq "i")
+push @initdb_params, ('--encoding', $original_encoding);
+push @initdb_params, ('--lc-collate', $original_datcollate);
+push @initdb_params, ('--lc-ctype', $original_datctype);
+
+# add --locale-provider, if supported
+my %provider_name = ('b' => 'builtin', 'i' => 'icu', 'c' => 'libc');
+if ($oldnode->pg_version >= 15)
{
- push @initdb_params, ('--locale-provider', 'icu');
- push @initdb_params, ('--icu-locale', 'fr-CA');
+ push @initdb_params, ('--locale-provider', $provider_name{$original_provider});
+ if ($original_provider eq 'b')
+ {
+ push @initdb_params, ('--builtin-locale', $original_datlocale);
+ }
+ elsif ($original_provider eq 'i')
+ {
+ push @initdb_params, ('--icu-locale', $original_datlocale);
+ }
}
$node_params{extra} = \@initdb_params;
@@ -146,10 +182,10 @@ $oldnode->start;
my $result;
$result = $oldnode->safe_psql(
'postgres',
- "SELECT encoding, $provider_field, datcollate, datctype, $iculocale_field
+ "SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_iculocale",
+ "$encoding_number{$original_encoding}|$original_provider|$original_datcollate|$original_datctype|$original_datlocale",
"check locales in original cluster");
# The default location of the source code is the root of this directory.
@@ -429,10 +465,10 @@ if (-d $log_path)
# Test that upgraded cluster has original locale settings.
$result = $newnode->safe_psql(
'postgres',
- "SELECT encoding, $provider_field, datcollate, datctype, $iculocale_field
+ "SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_iculocale",
+ "$encoding_number{$original_encoding}|$original_provider|$original_datcollate|$original_datctype|$original_datlocale",
"check that locales in new cluster match original cluster");
# Second dump from the upgraded instance.
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1122843715..3b327d159a 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -926,7 +926,7 @@ listAllDbs(const char *pattern, bool verbose)
gettext_noop("Encoding"));
if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
- " CASE d.datlocprovider WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
+ " CASE d.datlocprovider WHEN 'b' THEN 'builtin' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
gettext_noop("Locale Provider"));
else
appendPQExpBuffer(&buf,
@@ -4966,7 +4966,7 @@ listCollations(const char *pattern, bool verbose, bool showSystem)
if (pset.sversion >= 100000)
appendPQExpBuffer(&buf,
- " CASE c.collprovider WHEN 'd' THEN 'default' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
+ " CASE c.collprovider WHEN 'd' THEN 'default' WHEN 'b' THEN 'builtin' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
gettext_noop("Provider"));
else
appendPQExpBuffer(&buf,
diff --git a/src/bin/scripts/createdb.c b/src/bin/scripts/createdb.c
index 9ca86a3e53..78d729106a 100644
--- a/src/bin/scripts/createdb.c
+++ b/src/bin/scripts/createdb.c
@@ -40,8 +40,9 @@ main(int argc, char *argv[])
{"locale", required_argument, NULL, 'l'},
{"maintenance-db", required_argument, NULL, 3},
{"locale-provider", required_argument, NULL, 4},
- {"icu-locale", required_argument, NULL, 5},
- {"icu-rules", required_argument, NULL, 6},
+ {"builtin-locale", required_argument, NULL, 5},
+ {"icu-locale", required_argument, NULL, 6},
+ {"icu-rules", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
@@ -67,6 +68,7 @@ main(int argc, char *argv[])
char *lc_ctype = NULL;
char *locale = NULL;
char *locale_provider = NULL;
+ char *builtin_locale = NULL;
char *icu_locale = NULL;
char *icu_rules = NULL;
@@ -134,9 +136,12 @@ main(int argc, char *argv[])
locale_provider = pg_strdup(optarg);
break;
case 5:
- icu_locale = pg_strdup(optarg);
+ builtin_locale = pg_strdup(optarg);
break;
case 6:
+ icu_locale = pg_strdup(optarg);
+ break;
+ case 7:
icu_rules = pg_strdup(optarg);
break;
default:
@@ -216,6 +221,11 @@ main(int argc, char *argv[])
appendPQExpBufferStr(&sql, " LOCALE ");
appendStringLiteralConn(&sql, locale, conn);
}
+ if (builtin_locale)
+ {
+ appendPQExpBufferStr(&sql, " BUILTIN_LOCALE ");
+ appendStringLiteralConn(&sql, builtin_locale, conn);
+ }
if (lc_collate)
{
appendPQExpBufferStr(&sql, " LC_COLLATE ");
@@ -296,7 +306,7 @@ help(const char *progname)
printf(_(" --lc-ctype=LOCALE LC_CTYPE setting for the database\n"));
printf(_(" --icu-locale=LOCALE ICU locale setting for the database\n"));
printf(_(" --icu-rules=RULES ICU rules setting for the database\n"));
- printf(_(" --locale-provider={libc|icu}\n"
+ printf(_(" --locale-provider={builtin|libc|icu}\n"
" locale provider for the database's default collation\n"));
printf(_(" -O, --owner=OWNER database user to own the new database\n"));
printf(_(" -S, --strategy=STRATEGY database creation strategy wal_log or file_copy\n"));
diff --git a/src/bin/scripts/t/020_createdb.pl b/src/bin/scripts/t/020_createdb.pl
index 40291924e5..feee9cf85d 100644
--- a/src/bin/scripts/t/020_createdb.pl
+++ b/src/bin/scripts/t/020_createdb.pl
@@ -105,6 +105,78 @@ else
'create database with ICU fails since no ICU support');
}
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ 'tbuiltin1'
+ ],
+ 'create database with provider "builtin" fails without --locale'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', 'tbuiltin2'
+ ],
+ 'create database with provider "builtin" and locale "C"'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--lc-collate=C', 'tbuiltin3'
+ ],
+ 'create database with provider "builtin" and LC_COLLATE=C'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--lc-ctype=C', 'tbuiltin4'
+ ],
+ 'create database with provider "builtin" and LC_CTYPE=C'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '-E UTF-8', '--builtin-locale=C.UTF8', 'tbuiltin5'
+ ],
+ 'create database with provider "builtin" and --builtin-locale C.UTF-8'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '-E LATIN1', '--builtin-locale=C.UTF-8', 'tbuiltin6'
+ ],
+ 'create database with provider "builtin" and --builtin-locale C.UTF-8'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--icu-locale=en', 'tbuiltin7'
+ ],
+ 'create database with provider "builtin" and ICU_LOCALE="en"'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--icu-rules=""', 'tbuiltin8'
+ ],
+ 'create database with provider "builtin" and ICU_RULES=""'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template1', '--locale-provider=builtin',
+ '--locale=C', 'tbuiltin9'
+ ],
+ 'create database with provider "builtin" not matching template'
+);
+
$node->command_fails([ 'createdb', 'foobar1' ],
'fails if database already exists');
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 2c112cd6bc..fa071ce68a 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -23,12 +23,15 @@
descr => 'standard POSIX collation',
collname => 'POSIX', collprovider => 'c', collencoding => '-1',
collcollate => 'POSIX', collctype => 'POSIX' },
-{ oid => '962', descr => 'sorts by Unicode code point',
- collname => 'ucs_basic', collprovider => 'c', collencoding => '6',
- collcollate => 'C', collctype => 'C' },
+{ oid => '962', descr => 'sorts by Unicode code point, C character semantics',
+ collname => 'ucs_basic', collprovider => 'b', collencoding => '6',
+ colllocale => 'C' },
{ oid => '963',
descr => 'sorts using the Unicode Collation Algorithm with default settings',
collname => 'unicode', collprovider => 'i', collencoding => '-1',
colllocale => 'und' },
+{ oid => '970', descr => 'sorts by Unicode code point; Unicode & POSIX character semantics',
+ collname => 'c_utf8', collprovider => 'b', collencoding => '6',
+ colllocale => 'C.UTF8' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index d357c89ae6..09fc991038 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -65,6 +65,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_collation_oid_index, 3085, CollationOidIndexId, pg_
#ifdef EXPOSE_TO_CLIENT_CODE
#define COLLPROVIDER_DEFAULT 'd'
+#define COLLPROVIDER_BUILTIN 'b'
#define COLLPROVIDER_ICU 'i'
#define COLLPROVIDER_LIBC 'c'
@@ -73,6 +74,8 @@ collprovider_name(char c)
{
switch (c)
{
+ case COLLPROVIDER_BUILTIN:
+ return "builtin";
case COLLPROVIDER_ICU:
return "icu";
case COLLPROVIDER_LIBC:
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 6447bea8e0..22dc38f42e 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -76,6 +76,11 @@ struct pg_locale_struct
bool deterministic;
union
{
+ struct
+ {
+ const char *locale;
+ bool cclass_posix;
+ } builtin;
locale_t lt;
#ifdef USE_ICU
struct
@@ -112,7 +117,7 @@ extern size_t pg_strxfrm_prefix(char *dest, const char *src, size_t destsize,
pg_locale_t locale);
extern size_t pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
size_t srclen, pg_locale_t locale);
-
+extern const char *builtin_validate_locale(int encoding, const char *loc_str);
extern void icu_validate_locale(const char *loc_str);
extern char *icu_language_tag(const char *loc_str, int elevel);
diff --git a/src/test/icu/t/010_database.pl b/src/test/icu/t/010_database.pl
index 67fc3bbf19..af34cfc1dd 100644
--- a/src/test/icu/t/010_database.pl
+++ b/src/test/icu/t/010_database.pl
@@ -63,14 +63,14 @@ is( $node1->psql(
0,
"C locale works for ICU");
-# Test that LOCALE works for ICU locales if LC_COLLATE and LC_CTYPE
-# are specified
-is( $node1->psql(
- 'postgres',
- q{CREATE DATABASE dbicu2 LOCALE_PROVIDER icu LOCALE '@colStrength=primary'
- LC_COLLATE='C' LC_CTYPE='C' TEMPLATE template0 ENCODING UTF8}
- ),
- 0,
- "LOCALE works for ICU locales if LC_COLLATE and LC_CTYPE are specified");
+my ($ret, $stdout, $stderr) = $node1->psql('postgres',
+ q{CREATE DATABASE dbicu LOCALE_PROVIDER builtin LOCALE 'C' TEMPLATE dbicu}
+);
+isnt($ret, 0,
+ "locale provider must match template: exit code not 0");
+like(
+ $stderr,
+ qr/ERROR: new locale provider \(builtin\) does not match locale provider of the template database \(icu\)/,
+ "locale provider must match template: error message");
done_testing();
diff --git a/src/test/regress/expected/collate.out b/src/test/regress/expected/collate.out
index 0649564485..ece4a8e99d 100644
--- a/src/test/regress/expected/collate.out
+++ b/src/test/regress/expected/collate.out
@@ -650,6 +650,26 @@ EXPLAIN (COSTS OFF)
(3 rows)
-- CREATE/DROP COLLATION
+CREATE COLLATION builtin_c ( PROVIDER = builtin, LOCALE = "C" );
+CREATE COLLATION builtin_posix ( PROVIDER = builtin, LOCALE = "POSIX" );
+SELECT b FROM collate_test1 ORDER BY b COLLATE builtin_c;
+ b
+-----
+ ABD
+ Abc
+ abc
+ bbc
+(4 rows)
+
+CREATE COLLATION builtin2 ( PROVIDER = builtin ); -- fails
+ERROR: parameter "locale" must be specified
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "en_US" ); -- fails
+ERROR: invalid locale name "en_US" for builtin provider
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LC_CTYPE = "C", LC_COLLATE = "C" ); -- fails
+ERROR: parameter "locale" must be specified
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "POSIX", LC_CTYPE = "POSIX" ); -- fails
+ERROR: conflicting or redundant options
+DETAIL: LOCALE cannot be specified together with LC_COLLATE or LC_CTYPE.
CREATE COLLATION mycoll1 FROM "C";
CREATE COLLATION mycoll2 ( LC_COLLATE = "POSIX", LC_CTYPE = "POSIX" );
CREATE COLLATION mycoll3 FROM "default"; -- intentionally unsupported
@@ -754,7 +774,7 @@ DETAIL: FROM cannot be specified together with any other options.
-- must get rid of them.
--
DROP SCHEMA collate_tests CASCADE;
-NOTICE: drop cascades to 19 other objects
+NOTICE: drop cascades to 21 other objects
DETAIL: drop cascades to table collate_test1
drop cascades to table collate_test_like
drop cascades to table collate_test2
@@ -771,6 +791,8 @@ drop cascades to function dup(anyelement)
drop cascades to table collate_test20
drop cascades to table collate_test21
drop cascades to table collate_test22
+drop cascades to collation builtin_c
+drop cascades to collation builtin_posix
drop cascades to collation mycoll2
drop cascades to table collate_test23
drop cascades to view collate_on_int
diff --git a/src/test/regress/expected/collate.utf8.out b/src/test/regress/expected/collate.utf8.out
new file mode 100644
index 0000000000..f4a9d26978
--- /dev/null
+++ b/src/test/regress/expected/collate.utf8.out
@@ -0,0 +1,97 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+SET client_encoding TO UTF8;
+--
+-- Test preinstalled C_UTF8 collation.
+--
+CREATE TABLE builtin_test (
+ t TEXT COLLATE C_UTF8
+);
+INSERT INTO builtin_test VALUES
+ ('abc DEF'),
+ ('ábc DÉF'),
+ ('DŽxxDŽ džxxDž Džxxdž');
+SELECT t, lower(t), initcap(t), upper(t) FROM builtin_test;
+ t | lower | initcap | upper
+----------------+----------------+----------------+----------------
+ abc DEF | abc def | Abc Def | ABC DEF
+ ábc DÉF | ábc déf | Ábc Déf | ÁBC DÉF
+ DŽxxDŽ džxxDž Džxxdž | džxxdž džxxdž džxxdž | Džxxdž Džxxdž Džxxdž | DŽXXDŽ DŽXXDŽ DŽXXDŽ
+(3 rows)
+
+DROP TABLE builtin_test;
+-- character classes
+SELECT 'xyz' ~ '[[:alnum:]]' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xyz' !~ '[[:upper:]]' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '@' !~ '[[:alnum:]]' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '@' ~ '[[:punct:]]' COLLATE C_UTF8; -- symbols are punctuation in posix
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'a8a' ~ '[[:digit:]]' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '൧' !~ '\d' COLLATE C_UTF8; -- only 0-9 considered digits in posix
+ ?column?
+----------
+ t
+(1 row)
+
+-- case mapping
+SELECT 'xYz' ~* 'XyZ' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xAb' ~* '[W-Y]' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xAb' !~* '[c-d]' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'Δ' ~* '[α-λ]' COLLATE C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'δ' ~* '[Γ-Λ]' COLLATE C_UTF8; -- same as above with cases reversed
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/collate.utf8_1.out b/src/test/regress/expected/collate.utf8_1.out
new file mode 100644
index 0000000000..e73fdf50c3
--- /dev/null
+++ b/src/test/regress/expected/collate.utf8_1.out
@@ -0,0 +1,8 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f0987ff537..292bc54932 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -78,9 +78,9 @@ test: brin_bloom brin_multi
# psql depends on create_am
# amutils depends on geometry, create_index_spgist, hash_index, brin
# ----------
-test: create_table_like alter_generic alter_operator misc async dbsize merge misc_functions sysviews tsrf tid tidscan tidrangescan collate.icu.utf8 incremental_sort create_role
+test: create_table_like alter_generic alter_operator misc async dbsize merge misc_functions sysviews tsrf tid tidscan tidrangescan collate.utf8 collate.icu.utf8 incremental_sort create_role
-# collate.*.utf8 tests cannot be run in parallel with each other
+# collate.linux.utf8 and collate.icu.utf8 tests cannot be run in parallel with each other
test: rules psql psql_crosstab amutils stats_ext collate.linux.utf8 collate.windows.win1252
# ----------
diff --git a/src/test/regress/sql/collate.sql b/src/test/regress/sql/collate.sql
index c3d40fc195..01d5c69fe4 100644
--- a/src/test/regress/sql/collate.sql
+++ b/src/test/regress/sql/collate.sql
@@ -244,6 +244,16 @@ EXPLAIN (COSTS OFF)
-- CREATE/DROP COLLATION
+CREATE COLLATION builtin_c ( PROVIDER = builtin, LOCALE = "C" );
+CREATE COLLATION builtin_posix ( PROVIDER = builtin, LOCALE = "POSIX" );
+
+SELECT b FROM collate_test1 ORDER BY b COLLATE builtin_c;
+
+CREATE COLLATION builtin2 ( PROVIDER = builtin ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "en_US" ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LC_CTYPE = "C", LC_COLLATE = "C" ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "POSIX", LC_CTYPE = "POSIX" ); -- fails
+
CREATE COLLATION mycoll1 FROM "C";
CREATE COLLATION mycoll2 ( LC_COLLATE = "POSIX", LC_CTYPE = "POSIX" );
CREATE COLLATION mycoll3 FROM "default"; -- intentionally unsupported
diff --git a/src/test/regress/sql/collate.utf8.sql b/src/test/regress/sql/collate.utf8.sql
new file mode 100644
index 0000000000..0f30c5704d
--- /dev/null
+++ b/src/test/regress/sql/collate.utf8.sql
@@ -0,0 +1,45 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+SET client_encoding TO UTF8;
+
+--
+-- Test preinstalled C_UTF8 collation.
+--
+
+CREATE TABLE builtin_test (
+ t TEXT COLLATE C_UTF8
+);
+INSERT INTO builtin_test VALUES
+ ('abc DEF'),
+ ('ábc DÉF'),
+ ('DŽxxDŽ džxxDž Džxxdž');
+
+SELECT t, lower(t), initcap(t), upper(t) FROM builtin_test;
+
+DROP TABLE builtin_test;
+
+-- character classes
+
+SELECT 'xyz' ~ '[[:alnum:]]' COLLATE C_UTF8;
+SELECT 'xyz' !~ '[[:upper:]]' COLLATE C_UTF8;
+SELECT '@' !~ '[[:alnum:]]' COLLATE C_UTF8;
+SELECT '@' ~ '[[:punct:]]' COLLATE C_UTF8; -- symbols are punctuation in posix
+SELECT 'a8a' ~ '[[:digit:]]' COLLATE C_UTF8;
+SELECT '൧' !~ '\d' COLLATE C_UTF8; -- only 0-9 considered digits in posix
+
+-- case mapping
+
+SELECT 'xYz' ~* 'XyZ' COLLATE C_UTF8;
+SELECT 'xAb' ~* '[W-Y]' COLLATE C_UTF8;
+SELECT 'xAb' !~* '[c-d]' COLLATE C_UTF8;
+SELECT 'Δ' ~* '[α-λ]' COLLATE C_UTF8;
+SELECT 'δ' ~* '[Γ-Λ]' COLLATE C_UTF8; -- same as above with cases reversed
--
2.34.1
view thread (9+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: Built-in CTYPE provider
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox