public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v6 6/7] Row pattern recognition patch (tests).
12+ messages / 4 participants
[nested] [flat]
* [PATCH v6 6/7] Row pattern recognition patch (tests).
@ 2023-09-12 05:22 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 12+ messages in thread
From: Tatsuo Ishii @ 2023-09-12 05:22 UTC (permalink / raw)
---
src/test/regress/expected/rpr.out | 594 +++++++++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/rpr.sql | 292 ++++++++++++++
3 files changed, 887 insertions(+), 1 deletion(-)
create mode 100644 src/test/regress/expected/rpr.out
create mode 100644 src/test/regress/sql/rpr.sql
diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out
new file mode 100644
index 0000000000..63bed05f05
--- /dev/null
+++ b/src/test/regress/expected/rpr.out
@@ -0,0 +1,594 @@
+--
+-- Test for row pattern definition clause
+--
+CREATE TEMP TABLE stock (
+ company TEXT,
+ tdate DATE,
+ price INTEGER
+ );
+INSERT INTO stock VALUES ('company1', '2023-07-01', 100);
+INSERT INTO stock VALUES ('company1', '2023-07-02', 200);
+INSERT INTO stock VALUES ('company1', '2023-07-03', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-04', 140);
+INSERT INTO stock VALUES ('company1', '2023-07-05', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-06', 90);
+INSERT INTO stock VALUES ('company1', '2023-07-07', 110);
+INSERT INTO stock VALUES ('company1', '2023-07-08', 130);
+INSERT INTO stock VALUES ('company1', '2023-07-09', 120);
+INSERT INTO stock VALUES ('company1', '2023-07-10', 130);
+INSERT INTO stock VALUES ('company2', '2023-07-01', 50);
+INSERT INTO stock VALUES ('company2', '2023-07-02', 2000);
+INSERT INTO stock VALUES ('company2', '2023-07-03', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-04', 1400);
+INSERT INTO stock VALUES ('company2', '2023-07-05', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-06', 60);
+INSERT INTO stock VALUES ('company2', '2023-07-07', 1100);
+INSERT INTO stock VALUES ('company2', '2023-07-08', 1300);
+INSERT INTO stock VALUES ('company2', '2023-07-09', 1200);
+INSERT INTO stock VALUES ('company2', '2023-07-10', 1300);
+SELECT * FROM stock;
+ company | tdate | price
+----------+------------+-------
+ company1 | 07-01-2023 | 100
+ company1 | 07-02-2023 | 200
+ company1 | 07-03-2023 | 150
+ company1 | 07-04-2023 | 140
+ company1 | 07-05-2023 | 150
+ company1 | 07-06-2023 | 90
+ company1 | 07-07-2023 | 110
+ company1 | 07-08-2023 | 130
+ company1 | 07-09-2023 | 120
+ company1 | 07-10-2023 | 130
+ company2 | 07-01-2023 | 50
+ company2 | 07-02-2023 | 2000
+ company2 | 07-03-2023 | 1500
+ company2 | 07-04-2023 | 1400
+ company2 | 07-05-2023 | 1500
+ company2 | 07-06-2023 | 60
+ company2 | 07-07-2023 | 1100
+ company2 | 07-08-2023 | 1300
+ company2 | 07-09-2023 | 1200
+ company2 | 07-10-2023 | 1300
+(20 rows)
+
+-- basic test using PREV
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | nth_second
+----------+------------+-------+-------------+------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023
+ company1 | 07-02-2023 | 200 | | |
+ company1 | 07-03-2023 | 150 | | |
+ company1 | 07-04-2023 | 140 | | |
+ company1 | 07-05-2023 | 150 | | |
+ company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023
+ company1 | 07-07-2023 | 110 | | |
+ company1 | 07-08-2023 | 130 | | |
+ company1 | 07-09-2023 | 120 | | |
+ company1 | 07-10-2023 | 130 | | |
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023
+ company2 | 07-02-2023 | 2000 | | |
+ company2 | 07-03-2023 | 1500 | | |
+ company2 | 07-04-2023 | 1400 | | |
+ company2 | 07-05-2023 | 1500 | | |
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023
+ company2 | 07-07-2023 | 1100 | | |
+ company2 | 07-08-2023 | 1300 | | |
+ company2 | 07-09-2023 | 1200 | | |
+ company2 | 07-10-2023 | 1300 | | |
+(20 rows)
+
+-- last_value() should remain consistent
+SELECT company, tdate, price, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | last_value
+----------+------------+-------+------------
+ company1 | 07-01-2023 | 100 | 140
+ company1 | 07-02-2023 | 200 |
+ company1 | 07-03-2023 | 150 |
+ company1 | 07-04-2023 | 140 |
+ company1 | 07-05-2023 | 150 |
+ company1 | 07-06-2023 | 90 | 120
+ company1 | 07-07-2023 | 110 |
+ company1 | 07-08-2023 | 130 |
+ company1 | 07-09-2023 | 120 |
+ company1 | 07-10-2023 | 130 |
+ company2 | 07-01-2023 | 50 | 1400
+ company2 | 07-02-2023 | 2000 |
+ company2 | 07-03-2023 | 1500 |
+ company2 | 07-04-2023 | 1400 |
+ company2 | 07-05-2023 | 1500 |
+ company2 | 07-06-2023 | 60 | 1200
+ company2 | 07-07-2023 | 1100 |
+ company2 | 07-08-2023 | 1300 |
+ company2 | 07-09-2023 | 1200 |
+ company2 | 07-10-2023 | 1300 |
+(20 rows)
+
+-- omit "START" in DEFINE but it is ok because "START AS TRUE" is
+-- implicitly defined. per spec.
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | nth_second
+----------+------------+-------+-------------+------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023
+ company1 | 07-02-2023 | 200 | | |
+ company1 | 07-03-2023 | 150 | | |
+ company1 | 07-04-2023 | 140 | | |
+ company1 | 07-05-2023 | 150 | | |
+ company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023
+ company1 | 07-07-2023 | 110 | | |
+ company1 | 07-08-2023 | 130 | | |
+ company1 | 07-09-2023 | 120 | | |
+ company1 | 07-10-2023 | 130 | | |
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023
+ company2 | 07-02-2023 | 2000 | | |
+ company2 | 07-03-2023 | 1500 | | |
+ company2 | 07-04-2023 | 1400 | | |
+ company2 | 07-05-2023 | 1500 | | |
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023
+ company2 | 07-07-2023 | 1100 | | |
+ company2 | 07-08-2023 | 1300 | | |
+ company2 | 07-09-2023 | 1200 | | |
+ company2 | 07-10-2023 | 1300 | | |
+(20 rows)
+
+-- the first row start with less than or equal to 100
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | 90 | 120
+ company1 | 07-07-2023 | 110 | |
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 1400
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | 60 | 1200
+ company2 | 07-07-2023 | 1100 | |
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- second row raises 120%
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price) * 1.2,
+ DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 140
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | |
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 1400
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | |
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- using NEXT
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 200
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | 140 | 150
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 110 | 130
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 2000
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | 1400 | 1500
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 1100 | 1300
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 200
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | 140 | 150
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 110 | 130
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 2000
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | 1400 | 1500
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 1100 | 1300
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- match everything
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+)
+ DEFINE
+ A AS TRUE
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | 100 | 130
+ company1 | 07-02-2023 | 200 | |
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | |
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | 50 | 1300
+ company2 | 07-02-2023 | 2000 | |
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | |
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | |
+ company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023
+ company1 | 07-03-2023 | 150 | |
+ company1 | 07-04-2023 | 140 | |
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023
+ company1 | 07-08-2023 | 130 | |
+ company1 | 07-09-2023 | 120 | |
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | |
+ company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023
+ company2 | 07-03-2023 | 1500 | |
+ company2 | 07-04-2023 | 1400 | |
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023
+ company2 | 07-08-2023 | 1300 | |
+ company2 | 07-09-2023 | 1200 | |
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+ company | tdate | price | first_value | last_value
+----------+------------+-------+-------------+------------
+ company1 | 07-01-2023 | 100 | |
+ company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023
+ company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023
+ company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023
+ company1 | 07-05-2023 | 150 | |
+ company1 | 07-06-2023 | 90 | |
+ company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023
+ company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023
+ company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023
+ company1 | 07-10-2023 | 130 | |
+ company2 | 07-01-2023 | 50 | |
+ company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023
+ company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023
+ company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023
+ company2 | 07-05-2023 | 1500 | |
+ company2 | 07-06-2023 | 60 | |
+ company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023
+ company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023
+ company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023
+ company2 | 07-10-2023 | 1300 | |
+(20 rows)
+
+--
+-- Aggregates
+--
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP PAST LAST ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | max | min | sum | avg | count
+----------+------------+-------+-------------+------------+------+-----+------+-----------------------+-------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4
+ company1 | 07-02-2023 | 200 | | | | | | |
+ company1 | 07-03-2023 | 150 | | | | | | |
+ company1 | 07-04-2023 | 140 | | | | | | |
+ company1 | 07-05-2023 | 150 | | | | | | |
+ company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4
+ company1 | 07-07-2023 | 110 | | | | | | |
+ company1 | 07-08-2023 | 130 | | | | | | |
+ company1 | 07-09-2023 | 120 | | | | | | |
+ company1 | 07-10-2023 | 130 | | | | | | |
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4
+ company2 | 07-02-2023 | 2000 | | | | | | |
+ company2 | 07-03-2023 | 1500 | | | | | | |
+ company2 | 07-04-2023 | 1400 | | | | | | |
+ company2 | 07-05-2023 | 1500 | | | | | | |
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4
+ company2 | 07-07-2023 | 1100 | | | | | | |
+ company2 | 07-08-2023 | 1300 | | | | | | |
+ company2 | 07-09-2023 | 1200 | | | | | | |
+ company2 | 07-10-2023 | 1300 | | | | | | |
+(20 rows)
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP TO NEXT ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+ company | tdate | price | first_value | last_value | max | min | sum | avg | count
+----------+------------+-------+-------------+------------+------+------+------+-----------------------+-------
+ company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4
+ company1 | 07-02-2023 | 200 | | | | | | | 0
+ company1 | 07-03-2023 | 150 | | | | | | | 0
+ company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3
+ company1 | 07-05-2023 | 150 | | | | | | | 0
+ company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4
+ company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3
+ company1 | 07-08-2023 | 130 | | | | | | | 0
+ company1 | 07-09-2023 | 120 | | | | | | | 0
+ company1 | 07-10-2023 | 130 | | | | | | | 0
+ company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4
+ company2 | 07-02-2023 | 2000 | | | | | | | 0
+ company2 | 07-03-2023 | 1500 | | | | | | | 0
+ company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3
+ company2 | 07-05-2023 | 1500 | | | | | | | 0
+ company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4
+ company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3
+ company2 | 07-08-2023 | 1300 | | | | | | | 0
+ company2 | 07-09-2023 | 1200 | | | | | | | 0
+ company2 | 07-10-2023 | 1300 | | | | | | | 0
+(20 rows)
+
+--
+-- Error cases
+--
+-- row pattern definition variable name must not appear more than once
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ ORDER BY tdate
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price),
+ UP AS price > PREV(price)
+);
+ERROR: syntax error at or near "ORDER"
+LINE 6: ORDER BY tdate
+ ^
+-- pattern variable name must appear in DEFINE
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+ END)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ERROR: syntax error at or near "END"
+LINE 8: PATTERN (START UP+ DOWN+ END)
+ ^
+-- FRAME must start at current row when row patttern recognition is used
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ERROR: FRAME must start at current row when row patttern recognition is used
+-- SEEK is not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ SEEK
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+ERROR: SEEK is not supported
+LINE 8: SEEK
+ ^
+HINT: Use INITIAL.
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4df9d8503b..896531002b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -98,7 +98,7 @@ test: publication subscription
# Another group of parallel tests
# select_views depends on create_view
# ----------
-test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass
+test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass rpr
# ----------
# Another group of parallel tests (JSON related)
diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql
new file mode 100644
index 0000000000..bdefd20647
--- /dev/null
+++ b/src/test/regress/sql/rpr.sql
@@ -0,0 +1,292 @@
+--
+-- Test for row pattern definition clause
+--
+
+CREATE TEMP TABLE stock (
+ company TEXT,
+ tdate DATE,
+ price INTEGER
+ );
+INSERT INTO stock VALUES ('company1', '2023-07-01', 100);
+INSERT INTO stock VALUES ('company1', '2023-07-02', 200);
+INSERT INTO stock VALUES ('company1', '2023-07-03', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-04', 140);
+INSERT INTO stock VALUES ('company1', '2023-07-05', 150);
+INSERT INTO stock VALUES ('company1', '2023-07-06', 90);
+INSERT INTO stock VALUES ('company1', '2023-07-07', 110);
+INSERT INTO stock VALUES ('company1', '2023-07-08', 130);
+INSERT INTO stock VALUES ('company1', '2023-07-09', 120);
+INSERT INTO stock VALUES ('company1', '2023-07-10', 130);
+INSERT INTO stock VALUES ('company2', '2023-07-01', 50);
+INSERT INTO stock VALUES ('company2', '2023-07-02', 2000);
+INSERT INTO stock VALUES ('company2', '2023-07-03', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-04', 1400);
+INSERT INTO stock VALUES ('company2', '2023-07-05', 1500);
+INSERT INTO stock VALUES ('company2', '2023-07-06', 60);
+INSERT INTO stock VALUES ('company2', '2023-07-07', 1100);
+INSERT INTO stock VALUES ('company2', '2023-07-08', 1300);
+INSERT INTO stock VALUES ('company2', '2023-07-09', 1200);
+INSERT INTO stock VALUES ('company2', '2023-07-10', 1300);
+
+SELECT * FROM stock;
+
+-- basic test using PREV
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- last_value() should remain consistent
+SELECT company, tdate, price, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- omit "START" in DEFINE but it is ok because "START AS TRUE" is
+-- implicitly defined. per spec.
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w,
+ nth_value(tdate, 2) OVER w AS nth_second
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- the first row start with less than or equal to 100
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- second row raises 120%
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price) * 1.2,
+ DOWN AS price < PREV(price)
+);
+
+-- using NEXT
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (START UPDOWN)
+ DEFINE
+ START AS TRUE,
+ UPDOWN AS price > PREV(price) AND price > NEXT(price)
+);
+
+-- match everything
+
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+)
+ DEFINE
+ A AS TRUE
+);
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+
+-- backtracking with reclassification of rows
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ INITIAL
+ PATTERN (A+ B+)
+ DEFINE
+ A AS price > 100,
+ B AS price > 100
+);
+
+--
+-- Aggregates
+--
+
+-- using AFTER MATCH SKIP PAST LAST ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP PAST LAST ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+
+-- using AFTER MATCH SKIP TO NEXT ROW
+SELECT company, tdate, price,
+ first_value(price) OVER w,
+ last_value(price) OVER w,
+ max(price) OVER w,
+ min(price) OVER w,
+ sum(price) OVER w,
+ avg(price) OVER w,
+ count(price) OVER w
+FROM stock
+WINDOW w AS (
+PARTITION BY company
+ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+AFTER MATCH SKIP TO NEXT ROW
+INITIAL
+PATTERN (START UP+ DOWN+)
+DEFINE
+START AS TRUE,
+UP AS price > PREV(price),
+DOWN AS price < PREV(price)
+);
+
+--
+-- Error cases
+--
+
+-- row pattern definition variable name must not appear more than once
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ ORDER BY tdate
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price),
+ UP AS price > PREV(price)
+);
+
+-- pattern variable name must appear in DEFINE
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+ END)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- FRAME must start at current row when row patttern recognition is used
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
+ INITIAL
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+
+-- SEEK is not supported
+SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w
+ FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ORDER BY tdate
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP TO NEXT ROW
+ SEEK
+ PATTERN (START UP+ DOWN+)
+ DEFINE
+ START AS TRUE,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
--
2.25.1
----Next_Part(Tue_Sep_12_15_18_43_2023_359)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v6-0007-Allow-to-print-raw-parse-tree.patch"
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-12 02:02 Jeff Davis <[email protected]>
0 siblings, 4 replies; 12+ messages in thread
From: Jeff Davis @ 2024-01-12 02:02 UTC (permalink / raw)
To: Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On Wed, 2024-01-10 at 23:56 +0100, Daniel Verite wrote:
> A related comment is about naming the builtin locale C.UTF-8, the
> same
> name as in libc. On one hand this is semantically sound, but on the
> other hand, it's likely to confuse people. What about using
> completely
> different names, like "pg_unicode" or something else prefixed by
> "pg_"
> both for the locale name and the collation name (currently
> C.UTF-8/c_utf8)?
New version attached. Changes:
* Named collation object PG_C_UTF8, which seems like a good idea to
prevent name conflicts with existing collations. The locale name is
still C.UTF-8, which still makes sense to me because it matches the
behavior of the libc locale of the same name so closely.
* Added missing documentation for initdb --builtin-locale
* Refactored the upper/lower/initcap implementations
* Improved tests for case conversions where the byte length of the
UTF8-encoded string changes (the string length doesn't change because
we don't do full case mapping).
* No longer uses titlecase mappings -- libc doesn't do that, so it was
an unnecessary difference in case mapping behavior.
* Improved test report per Jeremy's suggestion: now it reports the
number of codepoints tested.
Jeremy also raised a problem with old versions of psql connecting to a
new server: the \l and \dO won't work. Not sure exactly what to do
there, but I could work around it by adding a new field rather than
renaming (though that's not ideal).
Regards,
Jeff Davis
Attachments:
[text/x-patch] v16-0002-Add-Unicode-property-tables.patch (91.5K, ../../[email protected]/2-v16-0002-Add-Unicode-property-tables.patch)
download | inline diff:
From 5bfc3047f74651f9b31ad899ff91923eada0ec88 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Sat, 18 Nov 2023 15:34:24 -0800
Subject: [PATCH v16 2/5] Add Unicode property tables.
---
src/common/unicode/Makefile | 6 +-
src/common/unicode/category_test.c | 300 +-
.../generate-unicode_category_table.pl | 203 +-
src/common/unicode/meson.build | 4 +-
src/common/unicode_category.c | 258 +-
src/include/common/unicode_category.h | 25 +-
src/include/common/unicode_category_table.h | 2532 +++++++++++++++++
7 files changed, 3265 insertions(+), 63 deletions(-)
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 04d81dd5cb..27f0408d8b 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -29,13 +29,13 @@ update-unicode: unicode_category_table.h unicode_east_asian_fw_table.h unicode_n
# These files are part of the Unicode Character Database. Download
# them on demand. The dependency on Makefile.global is for
# UNICODE_VERSION.
-CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt: $(top_builddir)/src/Makefile.global
+CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt: $(top_builddir)/src/Makefile.global
$(DOWNLOAD) https://www.unicode.org/Public/$(UNICODE_VERSION)/ucd/$(@F)
unicode_version.h: generate-unicode_version.pl
$(PERL) $< --version $(UNICODE_VERSION)
-unicode_category_table.h: generate-unicode_category_table.pl UnicodeData.txt
+unicode_category_table.h: generate-unicode_category_table.pl DerivedCoreProperties.txt PropList.txt UnicodeData.txt
$(PERL) $<
# Generation of conversion tables used for string normalization with
@@ -82,4 +82,4 @@ clean:
rm -f $(OBJS) category_test category_test.o norm_test norm_test.o
distclean: clean
- rm -f CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
+ rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
diff --git a/src/common/unicode/category_test.c b/src/common/unicode/category_test.c
index f1aaac0f61..1d21a8c1cb 100644
--- a/src/common/unicode/category_test.c
+++ b/src/common/unicode/category_test.c
@@ -1,6 +1,7 @@
/*-------------------------------------------------------------------------
* category_test.c
- * Program to test Unicode general category functions.
+ * Program to test Unicode general category and character class
+ * functions.
*
* Portions Copyright (c) 2017-2024, PostgreSQL Global Development Group
*
@@ -14,17 +15,24 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
-
#ifdef USE_ICU
#include <unicode/uchar.h>
#endif
-#include "common/unicode_category.h"
+#include <wctype.h>
+
#include "common/unicode_version.h"
+#include "common/unicode_category.h"
+
+#define LIBC_LOCALE "C.UTF-8"
+
+static int pg_unicode_version = 0;
+#ifdef USE_ICU
+static int icu_unicode_version = 0;
+#endif
/*
* Parse version into integer for easy comparison.
*/
-#ifdef USE_ICU
static int
parse_unicode_version(const char *version)
{
@@ -39,57 +47,165 @@ parse_unicode_version(const char *version)
return major * 100 + minor;
}
-#endif
+#ifdef USE_ICU
/*
- * Exhaustively test that the Unicode category for each codepoint matches that
- * returned by ICU.
+ * Test Postgres Unicode tables by comparing with ICU. Test the General
+ * Category, as well as the properties Alphabetic, Lowercase, Uppercase,
+ * White_Space, and Hex_Digit.
*/
-int
-main(int argc, char **argv)
+static void
+icu_test()
{
-#ifdef USE_ICU
- int pg_unicode_version = parse_unicode_version(PG_UNICODE_VERSION);
- int icu_unicode_version = parse_unicode_version(U_UNICODE_VERSION);
+ int successful = 0;
int pg_skipped_codepoints = 0;
int icu_skipped_codepoints = 0;
- printf("category_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
- printf("category_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
-
- for (UChar32 code = 0; code <= 0x10ffff; code++)
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
{
uint8_t pg_category = unicode_category(code);
uint8_t icu_category = u_charType(code);
+ /* Property tests */
+ bool prop_alphabetic = pg_u_prop_alphabetic(code);
+ bool prop_lowercase = pg_u_prop_lowercase(code);
+ bool prop_uppercase = pg_u_prop_uppercase(code);
+ bool prop_white_space = pg_u_prop_white_space(code);
+ bool prop_hex_digit = pg_u_prop_hex_digit(code);
+ bool prop_join_control = pg_u_prop_join_control(code);
+
+ bool icu_prop_alphabetic = u_hasBinaryProperty(
+ code, UCHAR_ALPHABETIC);
+ bool icu_prop_lowercase = u_hasBinaryProperty(
+ code, UCHAR_LOWERCASE);
+ bool icu_prop_uppercase = u_hasBinaryProperty(
+ code, UCHAR_UPPERCASE);
+ bool icu_prop_white_space = u_hasBinaryProperty(
+ code, UCHAR_WHITE_SPACE);
+ bool icu_prop_hex_digit = u_hasBinaryProperty(
+ code, UCHAR_HEX_DIGIT);
+ bool icu_prop_join_control = u_hasBinaryProperty(
+ code, UCHAR_JOIN_CONTROL);
+
+ /*
+ * Compare with ICU for TR #18 character classes using:
+ *
+ * https://unicode-org.github.io/icu-docs/apidoc/dev/icu4c/uchar_8h.html#details
+ *
+ * which describes how to use ICU to test for membership in regex
+ * character classes ("Standard", not "POSIX Compatible").
+ *
+ * NB: the document suggests testing for some properties such as
+ * UCHAR_POSIX_ALNUM, but that doesn't mean that we're testing for the
+ * "POSIX Compatible" character classes.
+ */
+ bool isalpha = pg_u_isalpha(code);
+ bool islower = pg_u_islower(code);
+ bool isupper = pg_u_isupper(code);
+ bool ispunct = pg_u_ispunct(code, false);
+ bool isdigit = pg_u_isdigit(code, false);
+ bool isxdigit = pg_u_isxdigit(code, false);
+ bool isalnum = pg_u_isalnum(code, false);
+ bool isspace = pg_u_isspace(code);
+ bool isblank = pg_u_isblank(code);
+ bool iscntrl = pg_u_iscntrl(code);
+ bool isgraph = pg_u_isgraph(code);
+ bool isprint = pg_u_isprint(code);
+
+ bool icu_isalpha = u_isUAlphabetic(code);
+ bool icu_islower = u_isULowercase(code);
+ bool icu_isupper = u_isUUppercase(code);
+ bool icu_ispunct = u_ispunct(code);
+ bool icu_isdigit = u_isdigit(code);
+ bool icu_isxdigit = u_hasBinaryProperty(code,
+ UCHAR_POSIX_XDIGIT);
+ bool icu_isalnum = u_hasBinaryProperty(code,
+ UCHAR_POSIX_ALNUM);
+ bool icu_isspace = u_isUWhiteSpace(code);
+ bool icu_isblank = u_isblank(code);
+ bool icu_iscntrl = icu_category == PG_U_CONTROL;
+ bool icu_isgraph = u_hasBinaryProperty(code,
+ UCHAR_POSIX_GRAPH);
+ bool icu_isprint = u_hasBinaryProperty(code,
+ UCHAR_POSIX_PRINT);
+
+ /*
+ * A version mismatch means that some assigned codepoints in the newer
+ * version may be unassigned in the older version. That's OK, though
+ * the test will not cover those codepoints marked unassigned in the
+ * older version (that is, it will no longer be an exhaustive test).
+ */
+ if (pg_category == PG_U_UNASSIGNED &&
+ icu_category != PG_U_UNASSIGNED &&
+ pg_unicode_version < icu_unicode_version)
+ {
+ pg_skipped_codepoints++;
+ continue;
+ }
+
+ if (icu_category == PG_U_UNASSIGNED &&
+ pg_category != PG_U_UNASSIGNED &&
+ icu_unicode_version < pg_unicode_version)
+ {
+ icu_skipped_codepoints++;
+ continue;
+ }
+
if (pg_category != icu_category)
{
- /*
- * A version mismatch means that some assigned codepoints in the
- * newer version may be unassigned in the older version. That's
- * OK, though the test will not cover those codepoints marked
- * unassigned in the older version (that is, it will no longer be
- * an exhaustive test).
- */
- if (pg_category == PG_U_UNASSIGNED &&
- pg_unicode_version < icu_unicode_version)
- pg_skipped_codepoints++;
- else if (icu_category == PG_U_UNASSIGNED &&
- icu_unicode_version < pg_unicode_version)
- icu_skipped_codepoints++;
- else
- {
- printf("category_test: FAILURE for codepoint 0x%06x\n", code);
- printf("category_test: Postgres category: %02d %s %s\n", pg_category,
- unicode_category_abbrev(pg_category),
- unicode_category_string(pg_category));
- printf("category_test: ICU category: %02d %s %s\n", icu_category,
- unicode_category_abbrev(icu_category),
- unicode_category_string(icu_category));
- printf("\n");
- exit(1);
- }
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres category: %02d %s %s\n", pg_category,
+ unicode_category_abbrev(pg_category),
+ unicode_category_string(pg_category));
+ printf("category_test: ICU category: %02d %s %s\n", icu_category,
+ unicode_category_abbrev(icu_category),
+ unicode_category_string(icu_category));
+ printf("\n");
+ exit(1);
}
+
+ if (prop_alphabetic != icu_prop_alphabetic ||
+ prop_lowercase != icu_prop_lowercase ||
+ prop_uppercase != icu_prop_uppercase ||
+ prop_white_space != icu_prop_white_space ||
+ prop_hex_digit != icu_prop_hex_digit ||
+ prop_join_control != icu_prop_join_control)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres property alphabetic/lowercase/uppercase/white_space/hex_digit/join_control: %d/%d/%d/%d/%d/%d\n",
+ prop_alphabetic, prop_lowercase, prop_uppercase,
+ prop_white_space, prop_hex_digit, prop_join_control);
+ printf("category_test: ICU property alphabetic/lowercase/uppercase/white_space/hex_digit/join_control: %d/%d/%d/%d/%d/%d\n",
+ icu_prop_alphabetic, icu_prop_lowercase, icu_prop_uppercase,
+ icu_prop_white_space, icu_prop_hex_digit, icu_prop_join_control);
+ printf("\n");
+ exit(1);
+ }
+
+ if (isalpha != icu_isalpha ||
+ islower != icu_islower ||
+ isupper != icu_isupper ||
+ ispunct != icu_ispunct ||
+ isdigit != icu_isdigit ||
+ isxdigit != icu_isxdigit ||
+ isalnum != icu_isalnum ||
+ isspace != icu_isspace ||
+ isblank != icu_isblank ||
+ iscntrl != icu_iscntrl ||
+ isgraph != icu_isgraph ||
+ isprint != icu_isprint)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres class alpha/lower/upper/punct/digit/xdigit/alnum/space/blank/cntrl/graph/print: %d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d\n",
+ isalpha, islower, isupper, ispunct, isdigit, isxdigit, isalnum, isspace, isblank, iscntrl, isgraph, isprint);
+ printf("category_test: ICU class alpha/lower/upper/punct/digit/xdigit/alnum/space/blank/cntrl/graph/print: %d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d\n",
+ icu_isalpha, icu_islower, icu_isupper, icu_ispunct, icu_isdigit, icu_isxdigit, icu_isalnum, icu_isspace, icu_isblank, icu_iscntrl, icu_isgraph, icu_isprint);
+ printf("\n");
+ exit(1);
+ }
+
+ if (pg_category != PG_U_UNASSIGNED)
+ successful++;
}
if (pg_skipped_codepoints > 0)
@@ -99,10 +215,104 @@ main(int argc, char **argv)
printf("category_test: skipped %d codepoints unassigned in ICU due to Unicode version mismatch\n",
icu_skipped_codepoints);
- printf("category_test: success\n");
- exit(0);
+ printf("category_test: ICU test: %d codepoints successful\n", successful);
+}
+#endif
+
+/*
+ * For libc, test only some characters for membership in the punctuation
+ * class. We have no guarantee that all characters will obey the same rules as
+ * pg_u_ispunct_posix(), though some coverage is still useful.
+ */
+static const unsigned char test_punct[] = {
+ ',', '$', '"', 0x85, 0x00, 'b', '&', 'Z', ' ', '\t', '\n'
+};
+
+/*
+ * Test what we can for libc, which is limited but still useful to cover the
+ * _posix-variant functions.
+ */
+static void
+libc_test()
+{
+ char * libc_locale = setlocale(LC_CTYPE, LIBC_LOCALE);
+
+ if (!libc_locale)
+ {
+ printf("category_test: libc locale \"%s\" not available; skipping\n", LIBC_LOCALE);
+ return;
+ }
+
+ /* non-exhaustive test of pg_u_ispunct_posix() */
+ for (int i = 0; i < sizeof(test_punct)/sizeof(test_punct[0]); i++)
+ {
+ pg_wchar code = (pg_wchar) test_punct[i];
+ bool ispunct = pg_u_ispunct(code, true);
+ bool libc_ispunct = iswpunct(code);
+
+ if (ispunct != libc_ispunct)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres ispunct_posix: %d\n", ispunct);
+ printf("category_test: libc iswpunct: %d\n", libc_ispunct);
+ printf("\n");
+ exit(1);
+ }
+ }
+
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
+ {
+ uint8_t pg_category = unicode_category(code);
+
+ bool isalpha = pg_u_isalpha(code);
+ bool isdigit = pg_u_isdigit(code, true);
+ bool isxdigit = pg_u_isxdigit(code, true);
+ bool isalnum = pg_u_isalnum(code, true);
+
+ bool libc_isdigit = iswdigit(code);
+ bool libc_isxdigit = iswxdigit(code);
+
+ if (pg_category == PG_U_UNASSIGNED)
+ continue;
+
+ /* check that alnum is the same as isdigit OR isalpha */
+ if (((isdigit || isalpha) && !isalnum) ||
+ (!(isdigit || isalpha) && isalnum))
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: isalnum inconsistent: isalpha/isdigit/isalnum: %d/%d/%d\n",
+ isalpha, isdigit, isalnum);
+ exit(1);
+ }
+
+ if (isdigit != libc_isdigit ||
+ isxdigit != libc_isxdigit)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres class digit/xdigit: %d/%d\n",
+ isdigit, isxdigit);
+ printf("category_test: libc class digit/xdigit: %d/%d\n",
+ libc_isdigit, libc_isxdigit);
+ printf("\n");
+ exit(1);
+ }
+ }
+}
+
+int
+main(int argc, char **argv)
+{
+ pg_unicode_version = parse_unicode_version(PG_UNICODE_VERSION);
+ printf("category_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
+
+ libc_test();
+
+#ifdef USE_ICU
+ icu_unicode_version = parse_unicode_version(U_UNICODE_VERSION);
+ printf("category_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
+
+ icu_test();
#else
- printf("category_test: ICU support required for test; skipping\n");
- exit(0);
+ printf("category_test: ICU not available; skipping\n");
#endif
}
diff --git a/src/common/unicode/generate-unicode_category_table.pl b/src/common/unicode/generate-unicode_category_table.pl
index a50c87b7e9..5a96020bcf 100644
--- a/src/common/unicode/generate-unicode_category_table.pl
+++ b/src/common/unicode/generate-unicode_category_table.pl
@@ -120,8 +120,6 @@ if ($range_category ne $CATEGORY_UNASSIGNED) {
category => $range_category});
}
-my $num_ranges = scalar @category_ranges;
-
# See: https://www.unicode.org/reports/tr44/#General_Category_Values
my $categories = {
Cn => 'PG_U_UNASSIGNED',
@@ -156,11 +154,98 @@ my $categories = {
Pf => 'PG_U_FINAL_PUNCTUATION'
};
-# Start writing out the output files
+# Find White_Space and Hex_Digit characters
+my @white_space = ();
+my @hex_digits = ();
+my @join_control = ();
+open($FH, '<', "$output_path/PropList.txt")
+ or die "Could not open $output_path/PropList.txt: $!.";
+while (my $line = <$FH>)
+{
+ my $pattern = qr/([0-9A-F\.]+)\s*;\s*(\w+)\s*#.*/s;
+ next unless $line =~ $pattern;
+
+ my $code = $line =~ s/$pattern/$1/rg;
+ my $property = $line =~ s/$pattern/$2/rg;
+ my $start;
+ my $end;
+
+ if ($code =~ /\.\./) {
+ # code range
+ my @sp = split /\.\./, $code;
+ $start = hex($sp[0]);
+ $end = hex($sp[1]);
+ } else {
+ # single code point
+ $start = hex($code);
+ $end = hex($code);
+ }
+
+ if ($property eq "White_Space") {
+ push @white_space, {start => $start, end => $end};
+ }
+ elsif ($property eq "Hex_Digit") {
+ push @hex_digits, {start => $start, end => $end};
+ }
+ elsif ($property eq "Join_Control") {
+ push @join_control, {start => $start, end => $end};
+ }
+}
+
+# Find Alphabetic, Lowercase, and Uppercase characters
+my @alphabetic = ();
+my @lowercase = ();
+my @uppercase = ();
+open($FH, '<', "$output_path/DerivedCoreProperties.txt")
+ or die "Could not open $output_path/DerivedCoreProperties.txt: $!.";
+while (my $line = <$FH>)
+{
+ my $pattern = qr/^([0-9A-F\.]+)\s*;\s*(\w+)\s*#.*$/s;
+ next unless $line =~ $pattern;
+
+ my $code = $line =~ s/$pattern/$1/rg;
+ my $property = $line =~ s/$pattern/$2/rg;
+ my $start;
+ my $end;
+
+ if ($code =~ /\.\./) {
+ # code range
+ my @sp = split /\.\./, $code;
+ die "line: {$line} code: {$code} sp[0] {$sp[0]} sp[1] {$sp[1]}"
+ unless $sp[0] =~ /^[0-9A-F]+$/ && $sp[1] =~ /^[0-9A-F]+$/;
+ $start = hex($sp[0]);
+ $end = hex($sp[1]);
+ } else {
+ die "line: {$line} code: {$code}" unless $code =~ /^[0-9A-F]+$/;
+ # single code point
+ $start = hex($code);
+ $end = hex($code);
+ }
+
+ if ($property eq "Alphabetic") {
+ push @alphabetic, {start => $start, end => $end};
+ }
+ elsif ($property eq "Lowercase") {
+ push @lowercase, {start => $start, end => $end};
+ }
+ elsif ($property eq "Uppercase") {
+ push @uppercase, {start => $start, end => $end};
+ }
+}
+
+my $num_category_ranges = scalar @category_ranges;
+my $num_alphabetic_ranges = scalar @alphabetic;
+my $num_lowercase_ranges = scalar @lowercase;
+my $num_uppercase_ranges = scalar @uppercase;
+my $num_white_space_ranges = scalar @white_space;
+my $num_hex_digit_ranges = scalar @hex_digits;
+my $num_join_control_ranges = scalar @join_control;
+
+# Start writing out the output file
open my $OT, '>', $output_table_file
or die "Could not open output file $output_table_file: $!\n";
-print $OT <<HEADER;
+print $OT <<"HEADER";
/*-------------------------------------------------------------------------
*
* unicode_category_table.h
@@ -188,11 +273,20 @@ typedef struct
uint8 category; /* General Category */
} pg_category_range;
-/* table of Unicode codepoint ranges and their categories */
-static const pg_category_range unicode_categories[$num_ranges] =
+typedef struct
{
+ uint32 first; /* Unicode codepoint */
+ uint32 last; /* Unicode codepoint */
+} pg_unicode_range;
+
HEADER
+print $OT <<"CATEGORY_TABLE";
+/* table of Unicode codepoint ranges and their categories */
+static const pg_category_range unicode_categories[$num_category_ranges] =
+{
+CATEGORY_TABLE
+
my $firsttime = 1;
foreach my $range (@category_ranges) {
printf $OT ",\n" unless $firsttime;
@@ -202,4 +296,101 @@ foreach my $range (@category_ranges) {
die "category missing: $range->{category}" unless $category;
printf $OT "\t{0x%06x, 0x%06x, %s}", $range->{start}, $range->{end}, $category;
}
+
+print $OT "\n};\n\n";
+
+print $OT <<"ALPHABETIC_TABLE";
+/* table of Unicode codepoint ranges of Alphabetic characters */
+static const pg_unicode_range unicode_alphabetic[$num_alphabetic_ranges] =
+{
+ALPHABETIC_TABLE
+
+$firsttime = 1;
+foreach my $range (@alphabetic) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"LOWERCASE_TABLE";
+/* table of Unicode codepoint ranges of Lowercase characters */
+static const pg_unicode_range unicode_lowercase[$num_lowercase_ranges] =
+{
+LOWERCASE_TABLE
+
+$firsttime = 1;
+foreach my $range (@lowercase) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"UPPERCASE_TABLE";
+/* table of Unicode codepoint ranges of Uppercase characters */
+static const pg_unicode_range unicode_uppercase[$num_uppercase_ranges] =
+{
+UPPERCASE_TABLE
+
+$firsttime = 1;
+foreach my $range (@uppercase) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"WHITE_SPACE_TABLE";
+/* table of Unicode codepoint ranges of White_Space characters */
+static const pg_unicode_range unicode_white_space[$num_white_space_ranges] =
+{
+WHITE_SPACE_TABLE
+
+$firsttime = 1;
+foreach my $range (@white_space) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"HEX_DIGITS_TABLE";
+/* table of Unicode codepoint ranges of Hex_Digit characters */
+static const pg_unicode_range unicode_hex_digit[$num_hex_digit_ranges] =
+{
+HEX_DIGITS_TABLE
+
+$firsttime = 1;
+foreach my $range (@hex_digits) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
+print $OT "\n};\n\n";
+
+print $OT <<"JOIN_CONTROL_TABLE";
+/* table of Unicode codepoint ranges of Join_Control characters */
+static const pg_unicode_range unicode_join_control[$num_join_control_ranges] =
+{
+JOIN_CONTROL_TABLE
+
+$firsttime = 1;
+foreach my $range (@join_control) {
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ printf $OT "\t{0x%06x, 0x%06x}", $range->{start}, $range->{end};
+}
+
print $OT "\n};\n";
diff --git a/src/common/unicode/meson.build b/src/common/unicode/meson.build
index df4f3a4ed1..d7190bb8ca 100644
--- a/src/common/unicode/meson.build
+++ b/src/common/unicode/meson.build
@@ -11,7 +11,7 @@ endif
# These files are part of the Unicode Character Database. Download them on
# demand.
-foreach f : ['CompositionExclusions.txt', 'DerivedNormalizationProps.txt', 'EastAsianWidth.txt', 'NormalizationTest.txt', 'UnicodeData.txt']
+foreach f : ['CompositionExclusions.txt', 'DerivedCoreProperties.txt', 'DerivedNormalizationProps.txt', 'EastAsianWidth.txt', 'NormalizationTest.txt', 'PropList.txt', 'UnicodeData.txt']
url = unicode_baseurl.format(UNICODE_VERSION, f)
target = custom_target(f,
output: f,
@@ -26,7 +26,7 @@ update_unicode_targets = []
update_unicode_targets += \
custom_target('unicode_category_table.h',
- input: [unicode_data['UnicodeData.txt']],
+ input: [unicode_data['UnicodeData.txt'], unicode_data['DerivedCoreProperties.txt'], unicode_data['PropList.txt']],
output: ['unicode_category_table.h'],
command: [
perl, files('generate-unicode_category_table.pl'),
diff --git a/src/common/unicode_category.c b/src/common/unicode_category.c
index 668051b461..fc17c560b1 100644
--- a/src/common/unicode_category.c
+++ b/src/common/unicode_category.c
@@ -1,6 +1,8 @@
/*-------------------------------------------------------------------------
* unicode_category.c
- * Determine general category of Unicode characters.
+ * Determine general category and character class of Unicode
+ * characters. Encoding must be UTF8, where we assume that the pg_wchar
+ * representation is a code point.
*
* Portions Copyright (c) 2017-2024, PostgreSQL Global Development Group
*
@@ -18,24 +20,78 @@
#include "common/unicode_category.h"
#include "common/unicode_category_table.h"
+/*
+ * We use a mask word for convenience when testing for multiple categories at
+ * once. The number of Unicode General Categories should never grow, so a
+ * 32-bit mask is fine.
+ */
+#define PG_U_CATEGORY_MASK(X) ((uint32)(1 << (X)))
+
+#define PG_U_LU_MASK PG_U_CATEGORY_MASK(PG_U_UPPERCASE_LETTER)
+#define PG_U_LL_MASK PG_U_CATEGORY_MASK(PG_U_LOWERCASE_LETTER)
+#define PG_U_LT_MASK PG_U_CATEGORY_MASK(PG_U_TITLECASE_LETTER)
+#define PG_U_LC_MASK (PG_U_LU_MASK|PG_U_LL_MASK|PG_U_LT_MASK)
+#define PG_U_LM_MASK PG_U_CATEGORY_MASK(PG_U_MODIFIER_LETTER)
+#define PG_U_LO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_LETTER)
+#define PG_U_L_MASK (PG_U_LU_MASK|PG_U_LL_MASK|PG_U_LT_MASK|PG_U_LM_MASK|\
+ PG_U_LO_MASK)
+#define PG_U_MN_MASK PG_U_CATEGORY_MASK(PG_U_NONSPACING_MARK)
+#define PG_U_ME_MASK PG_U_CATEGORY_MASK(PG_U_ENCLOSING_MARK)
+#define PG_U_MC_MASK PG_U_CATEGORY_MASK(PG_U_SPACING_MARK)
+#define PG_U_M_MASK (PG_U_MN_MASK|PG_U_MC_MASK|PG_U_ME_MASK)
+#define PG_U_ND_MASK PG_U_CATEGORY_MASK(PG_U_DECIMAL_NUMBER)
+#define PG_U_NL_MASK PG_U_CATEGORY_MASK(PG_U_LETTER_NUMBER)
+#define PG_U_NO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_NUMBER)
+#define PG_U_N_MASK (PG_U_ND_MASK|PG_U_NL_MASK|PG_U_NO_MASK)
+#define PG_U_PC_MASK PG_U_CATEGORY_MASK(PG_U_CONNECTOR_PUNCTUATION)
+#define PG_U_PD_MASK PG_U_CATEGORY_MASK(PG_U_DASH_PUNCTUATION)
+#define PG_U_PS_MASK PG_U_CATEGORY_MASK(PG_U_OPEN_PUNCTUATION)
+#define PG_U_PE_MASK PG_U_CATEGORY_MASK(PG_U_CLOSE_PUNCTUATION)
+#define PG_U_PI_MASK PG_U_CATEGORY_MASK(PG_U_INITIAL_PUNCTUATION)
+#define PG_U_PF_MASK PG_U_CATEGORY_MASK(PG_U_FINAL_PUNCTUATION)
+#define PG_U_PO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_PUNCTUATION)
+#define PG_U_P_MASK (PG_U_PC_MASK|PG_U_PD_MASK|PG_U_PS_MASK|PG_U_PE_MASK|\
+ PG_U_PI_MASK|PG_U_PF_MASK|PG_U_PO_MASK)
+#define PG_U_SM_MASK PG_U_CATEGORY_MASK(PG_U_MATH_SYMBOL)
+#define PG_U_SC_MASK PG_U_CATEGORY_MASK(PG_U_CURRENCY_SYMBOL)
+#define PG_U_SK_MASK PG_U_CATEGORY_MASK(PG_U_MODIFIER_SYMBOL)
+#define PG_U_SO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_SYMBOL)
+#define PG_U_S_MASK (PG_U_SM_MASK|PG_U_SC_MASK|PG_U_SK_MASK|PG_U_SO_MASK)
+#define PG_U_ZS_MASK PG_U_CATEGORY_MASK(PG_U_SPACE_SEPARATOR)
+#define PG_U_ZL_MASK PG_U_CATEGORY_MASK(PG_U_LINE_SEPARATOR)
+#define PG_U_ZP_MASK PG_U_CATEGORY_MASK(PG_U_PARAGRAPH_SEPARATOR)
+#define PG_U_Z_MASK (PG_U_ZS_MASK|PG_U_ZL_MASK|PG_U_ZP_MASK)
+#define PG_U_CC_MASK PG_U_CATEGORY_MASK(PG_U_CONTROL)
+#define PG_U_CF_MASK PG_U_CATEGORY_MASK(PG_U_FORMAT)
+#define PG_U_CS_MASK PG_U_CATEGORY_MASK(PG_U_SURROGATE)
+#define PG_U_CO_MASK PG_U_CATEGORY_MASK(PG_U_PRIVATE_USE)
+#define PG_U_CN_MASK PG_U_CATEGORY_MASK(PG_U_UNASSIGNED)
+#define PG_U_C_MASK (PG_U_CC_MASK|PG_U_CF_MASK|PG_U_CS_MASK|PG_U_CO_MASK|\
+ PG_U_CN_MASK)
+
+#define PG_U_CHARACTER_TAB 0x09
+
+static bool range_search(const pg_unicode_range * tbl, Size size,
+ pg_wchar code);
+
/*
* Unicode general category for the given codepoint.
*/
pg_unicode_category
-unicode_category(pg_wchar ucs)
+unicode_category(pg_wchar code)
{
int min = 0;
int mid;
int max = lengthof(unicode_categories) - 1;
- Assert(ucs <= 0x10ffff);
+ Assert(code <= 0x10ffff);
while (max >= min)
{
mid = (min + max) / 2;
- if (ucs > unicode_categories[mid].last)
+ if (code > unicode_categories[mid].last)
min = mid + 1;
- else if (ucs < unicode_categories[mid].first)
+ else if (code < unicode_categories[mid].first)
max = mid - 1;
else
return unicode_categories[mid].category;
@@ -44,6 +100,171 @@ unicode_category(pg_wchar ucs)
return PG_U_UNASSIGNED;
}
+bool
+pg_u_prop_alphabetic(pg_wchar code)
+{
+ return range_search(unicode_alphabetic, lengthof(unicode_alphabetic),
+ code);
+}
+
+bool
+pg_u_prop_lowercase(pg_wchar code)
+{
+ return range_search(unicode_lowercase, lengthof(unicode_lowercase), code);
+}
+
+bool
+pg_u_prop_uppercase(pg_wchar code)
+{
+ return range_search(unicode_uppercase, lengthof(unicode_uppercase), code);
+}
+
+bool
+pg_u_prop_white_space(pg_wchar code)
+{
+ return range_search(unicode_white_space, lengthof(unicode_white_space),
+ code);
+}
+
+bool
+pg_u_prop_hex_digit(pg_wchar code)
+{
+ return range_search(unicode_hex_digit, lengthof(unicode_hex_digit), code);
+}
+
+bool
+pg_u_prop_join_control(pg_wchar code)
+{
+ return range_search(unicode_join_control, lengthof(unicode_join_control),
+ code);
+}
+
+/*
+ * The following functions implement the regex character classification as
+ * described at: http://www.unicode.org/reports/tr18/#Compatibility_Properties
+ *
+ * If 'posix' is true, implements the "POSIX Compatible" variant, otherwise
+ * the "Standard" variant.
+ */
+
+bool
+pg_u_isdigit(pg_wchar code, bool posix)
+{
+ if (posix)
+ return ('0' <= code && code <= '9');
+ else
+ return unicode_category(code) == PG_U_DECIMAL_NUMBER;
+}
+
+bool
+pg_u_isalpha(pg_wchar code)
+{
+ return pg_u_prop_alphabetic(code);
+}
+
+bool
+pg_u_isalnum(pg_wchar code, bool posix)
+{
+ return pg_u_isalpha(code) || pg_u_isdigit(code, posix);
+}
+
+bool
+pg_u_isword(pg_wchar code)
+{
+ uint32 category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ return
+ category_mask & (PG_U_M_MASK|PG_U_ND_MASK|PG_U_PC_MASK) ||
+ pg_u_isalpha(code) ||
+ pg_u_prop_join_control(code);
+}
+
+bool
+pg_u_isupper(pg_wchar code)
+{
+ return pg_u_prop_uppercase(code);
+}
+
+bool
+pg_u_islower(pg_wchar code)
+{
+ return pg_u_prop_lowercase(code);
+}
+
+bool
+pg_u_isblank(pg_wchar code)
+{
+ return code == PG_U_CHARACTER_TAB ||
+ unicode_category(code) == PG_U_SPACE_SEPARATOR;
+}
+
+bool
+pg_u_iscntrl(pg_wchar code)
+{
+ return unicode_category(code) == PG_U_CONTROL;
+}
+
+bool
+pg_u_isgraph(pg_wchar code)
+{
+ uint32 category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ if (category_mask & (PG_U_CC_MASK|PG_U_CS_MASK|PG_U_CN_MASK) ||
+ pg_u_isspace(code))
+ return false;
+ return true;
+}
+
+bool
+pg_u_isprint(pg_wchar code)
+{
+ pg_unicode_category category = unicode_category(code);
+
+ if (category == PG_U_CONTROL)
+ return false;
+
+ return pg_u_isgraph(code) || pg_u_isblank(code);
+}
+
+bool
+pg_u_ispunct(pg_wchar code, bool posix)
+{
+ uint32 category_mask;
+
+ if (posix)
+ {
+ if (pg_u_isalpha(code))
+ return false;
+
+ category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+ return category_mask & (PG_U_P_MASK|PG_U_S_MASK);
+ }
+ else
+ {
+ category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ return category_mask & PG_U_P_MASK;
+ }
+}
+
+bool
+pg_u_isspace(pg_wchar code)
+{
+ return pg_u_prop_white_space(code);
+}
+
+bool
+pg_u_isxdigit(pg_wchar code, bool posix)
+{
+ if (posix)
+ return (('0' <= code && code <= '9') ||
+ ('A' <= code && code <= 'F') ||
+ ('a' <= code && code <= 'f'));
+ else
+ return unicode_category(code) == PG_U_DECIMAL_NUMBER ||
+ pg_u_prop_hex_digit(code);
+}
+
/*
* Description of Unicode general category.
*/
@@ -191,3 +412,30 @@ unicode_category_abbrev(pg_unicode_category category)
Assert(false);
return "??"; /* keep compiler quiet */
}
+
+/*
+ * Binary search to test if given codepoint exists in one of the ranges in the
+ * given table.
+ */
+static bool
+range_search(const pg_unicode_range * tbl, Size size, pg_wchar code)
+{
+ int min = 0;
+ int mid;
+ int max = size - 1;
+
+ Assert(code <= 0x10ffff);
+
+ while (max >= min)
+ {
+ mid = (min + max) / 2;
+ if (code > tbl[mid].last)
+ min = mid + 1;
+ else if (code < tbl[mid].first)
+ max = mid - 1;
+ else
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/include/common/unicode_category.h b/src/include/common/unicode_category.h
index 5bad280615..a26ff21329 100644
--- a/src/include/common/unicode_category.h
+++ b/src/include/common/unicode_category.h
@@ -62,7 +62,28 @@ typedef enum pg_unicode_category
} pg_unicode_category;
extern pg_unicode_category unicode_category(pg_wchar ucs);
-const char *unicode_category_string(pg_unicode_category category);
-const char *unicode_category_abbrev(pg_unicode_category category);
+extern const char *unicode_category_string(pg_unicode_category category);
+extern const char *unicode_category_abbrev(pg_unicode_category category);
+
+extern bool pg_u_prop_alphabetic(pg_wchar c);
+extern bool pg_u_prop_lowercase(pg_wchar c);
+extern bool pg_u_prop_uppercase(pg_wchar c);
+extern bool pg_u_prop_white_space(pg_wchar c);
+extern bool pg_u_prop_hex_digit(pg_wchar c);
+extern bool pg_u_prop_join_control(pg_wchar c);
+
+extern bool pg_u_isdigit(pg_wchar c, bool posix);
+extern bool pg_u_isalpha(pg_wchar c);
+extern bool pg_u_isalnum(pg_wchar c, bool posix);
+extern bool pg_u_isword(pg_wchar c);
+extern bool pg_u_isupper(pg_wchar c);
+extern bool pg_u_islower(pg_wchar c);
+extern bool pg_u_isblank(pg_wchar c);
+extern bool pg_u_iscntrl(pg_wchar c);
+extern bool pg_u_isgraph(pg_wchar c);
+extern bool pg_u_isprint(pg_wchar c);
+extern bool pg_u_ispunct(pg_wchar c, bool posix);
+extern bool pg_u_isspace(pg_wchar c);
+extern bool pg_u_isxdigit(pg_wchar c, bool posix);
#endif /* UNICODE_CATEGORY_H */
diff --git a/src/include/common/unicode_category_table.h b/src/include/common/unicode_category_table.h
index d7ef996189..40d0e91d42 100644
--- a/src/include/common/unicode_category_table.h
+++ b/src/include/common/unicode_category_table.h
@@ -25,6 +25,12 @@ typedef struct
uint8 category; /* General Category */
} pg_category_range;
+typedef struct
+{
+ uint32 first; /* Unicode codepoint */
+ uint32 last; /* Unicode codepoint */
+} pg_unicode_range;
+
/* table of Unicode codepoint ranges and their categories */
static const pg_category_range unicode_categories[3302] =
{
@@ -3331,3 +3337,2529 @@ static const pg_category_range unicode_categories[3302] =
{0x0f0000, 0x0ffffd, PG_U_PRIVATE_USE},
{0x100000, 0x10fffd, PG_U_PRIVATE_USE}
};
+
+/* table of Unicode codepoint ranges of Alphabetic characters */
+static const pg_unicode_range unicode_alphabetic[1141] =
+{
+ {0x000041, 0x00005a},
+ {0x000061, 0x00007a},
+ {0x0000aa, 0x0000aa},
+ {0x0000b5, 0x0000b5},
+ {0x0000ba, 0x0000ba},
+ {0x0000c0, 0x0000d6},
+ {0x0000d8, 0x0000f6},
+ {0x0000f8, 0x0001ba},
+ {0x0001bb, 0x0001bb},
+ {0x0001bc, 0x0001bf},
+ {0x0001c0, 0x0001c3},
+ {0x0001c4, 0x000293},
+ {0x000294, 0x000294},
+ {0x000295, 0x0002af},
+ {0x0002b0, 0x0002c1},
+ {0x0002c6, 0x0002d1},
+ {0x0002e0, 0x0002e4},
+ {0x0002ec, 0x0002ec},
+ {0x0002ee, 0x0002ee},
+ {0x000345, 0x000345},
+ {0x000370, 0x000373},
+ {0x000374, 0x000374},
+ {0x000376, 0x000377},
+ {0x00037a, 0x00037a},
+ {0x00037b, 0x00037d},
+ {0x00037f, 0x00037f},
+ {0x000386, 0x000386},
+ {0x000388, 0x00038a},
+ {0x00038c, 0x00038c},
+ {0x00038e, 0x0003a1},
+ {0x0003a3, 0x0003f5},
+ {0x0003f7, 0x000481},
+ {0x00048a, 0x00052f},
+ {0x000531, 0x000556},
+ {0x000559, 0x000559},
+ {0x000560, 0x000588},
+ {0x0005b0, 0x0005bd},
+ {0x0005bf, 0x0005bf},
+ {0x0005c1, 0x0005c2},
+ {0x0005c4, 0x0005c5},
+ {0x0005c7, 0x0005c7},
+ {0x0005d0, 0x0005ea},
+ {0x0005ef, 0x0005f2},
+ {0x000610, 0x00061a},
+ {0x000620, 0x00063f},
+ {0x000640, 0x000640},
+ {0x000641, 0x00064a},
+ {0x00064b, 0x000657},
+ {0x000659, 0x00065f},
+ {0x00066e, 0x00066f},
+ {0x000670, 0x000670},
+ {0x000671, 0x0006d3},
+ {0x0006d5, 0x0006d5},
+ {0x0006d6, 0x0006dc},
+ {0x0006e1, 0x0006e4},
+ {0x0006e5, 0x0006e6},
+ {0x0006e7, 0x0006e8},
+ {0x0006ed, 0x0006ed},
+ {0x0006ee, 0x0006ef},
+ {0x0006fa, 0x0006fc},
+ {0x0006ff, 0x0006ff},
+ {0x000710, 0x000710},
+ {0x000711, 0x000711},
+ {0x000712, 0x00072f},
+ {0x000730, 0x00073f},
+ {0x00074d, 0x0007a5},
+ {0x0007a6, 0x0007b0},
+ {0x0007b1, 0x0007b1},
+ {0x0007ca, 0x0007ea},
+ {0x0007f4, 0x0007f5},
+ {0x0007fa, 0x0007fa},
+ {0x000800, 0x000815},
+ {0x000816, 0x000817},
+ {0x00081a, 0x00081a},
+ {0x00081b, 0x000823},
+ {0x000824, 0x000824},
+ {0x000825, 0x000827},
+ {0x000828, 0x000828},
+ {0x000829, 0x00082c},
+ {0x000840, 0x000858},
+ {0x000860, 0x00086a},
+ {0x000870, 0x000887},
+ {0x000889, 0x00088e},
+ {0x0008a0, 0x0008c8},
+ {0x0008c9, 0x0008c9},
+ {0x0008d4, 0x0008df},
+ {0x0008e3, 0x0008e9},
+ {0x0008f0, 0x000902},
+ {0x000903, 0x000903},
+ {0x000904, 0x000939},
+ {0x00093a, 0x00093a},
+ {0x00093b, 0x00093b},
+ {0x00093d, 0x00093d},
+ {0x00093e, 0x000940},
+ {0x000941, 0x000948},
+ {0x000949, 0x00094c},
+ {0x00094e, 0x00094f},
+ {0x000950, 0x000950},
+ {0x000955, 0x000957},
+ {0x000958, 0x000961},
+ {0x000962, 0x000963},
+ {0x000971, 0x000971},
+ {0x000972, 0x000980},
+ {0x000981, 0x000981},
+ {0x000982, 0x000983},
+ {0x000985, 0x00098c},
+ {0x00098f, 0x000990},
+ {0x000993, 0x0009a8},
+ {0x0009aa, 0x0009b0},
+ {0x0009b2, 0x0009b2},
+ {0x0009b6, 0x0009b9},
+ {0x0009bd, 0x0009bd},
+ {0x0009be, 0x0009c0},
+ {0x0009c1, 0x0009c4},
+ {0x0009c7, 0x0009c8},
+ {0x0009cb, 0x0009cc},
+ {0x0009ce, 0x0009ce},
+ {0x0009d7, 0x0009d7},
+ {0x0009dc, 0x0009dd},
+ {0x0009df, 0x0009e1},
+ {0x0009e2, 0x0009e3},
+ {0x0009f0, 0x0009f1},
+ {0x0009fc, 0x0009fc},
+ {0x000a01, 0x000a02},
+ {0x000a03, 0x000a03},
+ {0x000a05, 0x000a0a},
+ {0x000a0f, 0x000a10},
+ {0x000a13, 0x000a28},
+ {0x000a2a, 0x000a30},
+ {0x000a32, 0x000a33},
+ {0x000a35, 0x000a36},
+ {0x000a38, 0x000a39},
+ {0x000a3e, 0x000a40},
+ {0x000a41, 0x000a42},
+ {0x000a47, 0x000a48},
+ {0x000a4b, 0x000a4c},
+ {0x000a51, 0x000a51},
+ {0x000a59, 0x000a5c},
+ {0x000a5e, 0x000a5e},
+ {0x000a70, 0x000a71},
+ {0x000a72, 0x000a74},
+ {0x000a75, 0x000a75},
+ {0x000a81, 0x000a82},
+ {0x000a83, 0x000a83},
+ {0x000a85, 0x000a8d},
+ {0x000a8f, 0x000a91},
+ {0x000a93, 0x000aa8},
+ {0x000aaa, 0x000ab0},
+ {0x000ab2, 0x000ab3},
+ {0x000ab5, 0x000ab9},
+ {0x000abd, 0x000abd},
+ {0x000abe, 0x000ac0},
+ {0x000ac1, 0x000ac5},
+ {0x000ac7, 0x000ac8},
+ {0x000ac9, 0x000ac9},
+ {0x000acb, 0x000acc},
+ {0x000ad0, 0x000ad0},
+ {0x000ae0, 0x000ae1},
+ {0x000ae2, 0x000ae3},
+ {0x000af9, 0x000af9},
+ {0x000afa, 0x000afc},
+ {0x000b01, 0x000b01},
+ {0x000b02, 0x000b03},
+ {0x000b05, 0x000b0c},
+ {0x000b0f, 0x000b10},
+ {0x000b13, 0x000b28},
+ {0x000b2a, 0x000b30},
+ {0x000b32, 0x000b33},
+ {0x000b35, 0x000b39},
+ {0x000b3d, 0x000b3d},
+ {0x000b3e, 0x000b3e},
+ {0x000b3f, 0x000b3f},
+ {0x000b40, 0x000b40},
+ {0x000b41, 0x000b44},
+ {0x000b47, 0x000b48},
+ {0x000b4b, 0x000b4c},
+ {0x000b56, 0x000b56},
+ {0x000b57, 0x000b57},
+ {0x000b5c, 0x000b5d},
+ {0x000b5f, 0x000b61},
+ {0x000b62, 0x000b63},
+ {0x000b71, 0x000b71},
+ {0x000b82, 0x000b82},
+ {0x000b83, 0x000b83},
+ {0x000b85, 0x000b8a},
+ {0x000b8e, 0x000b90},
+ {0x000b92, 0x000b95},
+ {0x000b99, 0x000b9a},
+ {0x000b9c, 0x000b9c},
+ {0x000b9e, 0x000b9f},
+ {0x000ba3, 0x000ba4},
+ {0x000ba8, 0x000baa},
+ {0x000bae, 0x000bb9},
+ {0x000bbe, 0x000bbf},
+ {0x000bc0, 0x000bc0},
+ {0x000bc1, 0x000bc2},
+ {0x000bc6, 0x000bc8},
+ {0x000bca, 0x000bcc},
+ {0x000bd0, 0x000bd0},
+ {0x000bd7, 0x000bd7},
+ {0x000c00, 0x000c00},
+ {0x000c01, 0x000c03},
+ {0x000c04, 0x000c04},
+ {0x000c05, 0x000c0c},
+ {0x000c0e, 0x000c10},
+ {0x000c12, 0x000c28},
+ {0x000c2a, 0x000c39},
+ {0x000c3d, 0x000c3d},
+ {0x000c3e, 0x000c40},
+ {0x000c41, 0x000c44},
+ {0x000c46, 0x000c48},
+ {0x000c4a, 0x000c4c},
+ {0x000c55, 0x000c56},
+ {0x000c58, 0x000c5a},
+ {0x000c5d, 0x000c5d},
+ {0x000c60, 0x000c61},
+ {0x000c62, 0x000c63},
+ {0x000c80, 0x000c80},
+ {0x000c81, 0x000c81},
+ {0x000c82, 0x000c83},
+ {0x000c85, 0x000c8c},
+ {0x000c8e, 0x000c90},
+ {0x000c92, 0x000ca8},
+ {0x000caa, 0x000cb3},
+ {0x000cb5, 0x000cb9},
+ {0x000cbd, 0x000cbd},
+ {0x000cbe, 0x000cbe},
+ {0x000cbf, 0x000cbf},
+ {0x000cc0, 0x000cc4},
+ {0x000cc6, 0x000cc6},
+ {0x000cc7, 0x000cc8},
+ {0x000cca, 0x000ccb},
+ {0x000ccc, 0x000ccc},
+ {0x000cd5, 0x000cd6},
+ {0x000cdd, 0x000cde},
+ {0x000ce0, 0x000ce1},
+ {0x000ce2, 0x000ce3},
+ {0x000cf1, 0x000cf2},
+ {0x000cf3, 0x000cf3},
+ {0x000d00, 0x000d01},
+ {0x000d02, 0x000d03},
+ {0x000d04, 0x000d0c},
+ {0x000d0e, 0x000d10},
+ {0x000d12, 0x000d3a},
+ {0x000d3d, 0x000d3d},
+ {0x000d3e, 0x000d40},
+ {0x000d41, 0x000d44},
+ {0x000d46, 0x000d48},
+ {0x000d4a, 0x000d4c},
+ {0x000d4e, 0x000d4e},
+ {0x000d54, 0x000d56},
+ {0x000d57, 0x000d57},
+ {0x000d5f, 0x000d61},
+ {0x000d62, 0x000d63},
+ {0x000d7a, 0x000d7f},
+ {0x000d81, 0x000d81},
+ {0x000d82, 0x000d83},
+ {0x000d85, 0x000d96},
+ {0x000d9a, 0x000db1},
+ {0x000db3, 0x000dbb},
+ {0x000dbd, 0x000dbd},
+ {0x000dc0, 0x000dc6},
+ {0x000dcf, 0x000dd1},
+ {0x000dd2, 0x000dd4},
+ {0x000dd6, 0x000dd6},
+ {0x000dd8, 0x000ddf},
+ {0x000df2, 0x000df3},
+ {0x000e01, 0x000e30},
+ {0x000e31, 0x000e31},
+ {0x000e32, 0x000e33},
+ {0x000e34, 0x000e3a},
+ {0x000e40, 0x000e45},
+ {0x000e46, 0x000e46},
+ {0x000e4d, 0x000e4d},
+ {0x000e81, 0x000e82},
+ {0x000e84, 0x000e84},
+ {0x000e86, 0x000e8a},
+ {0x000e8c, 0x000ea3},
+ {0x000ea5, 0x000ea5},
+ {0x000ea7, 0x000eb0},
+ {0x000eb1, 0x000eb1},
+ {0x000eb2, 0x000eb3},
+ {0x000eb4, 0x000eb9},
+ {0x000ebb, 0x000ebc},
+ {0x000ebd, 0x000ebd},
+ {0x000ec0, 0x000ec4},
+ {0x000ec6, 0x000ec6},
+ {0x000ecd, 0x000ecd},
+ {0x000edc, 0x000edf},
+ {0x000f00, 0x000f00},
+ {0x000f40, 0x000f47},
+ {0x000f49, 0x000f6c},
+ {0x000f71, 0x000f7e},
+ {0x000f7f, 0x000f7f},
+ {0x000f80, 0x000f83},
+ {0x000f88, 0x000f8c},
+ {0x000f8d, 0x000f97},
+ {0x000f99, 0x000fbc},
+ {0x001000, 0x00102a},
+ {0x00102b, 0x00102c},
+ {0x00102d, 0x001030},
+ {0x001031, 0x001031},
+ {0x001032, 0x001036},
+ {0x001038, 0x001038},
+ {0x00103b, 0x00103c},
+ {0x00103d, 0x00103e},
+ {0x00103f, 0x00103f},
+ {0x001050, 0x001055},
+ {0x001056, 0x001057},
+ {0x001058, 0x001059},
+ {0x00105a, 0x00105d},
+ {0x00105e, 0x001060},
+ {0x001061, 0x001061},
+ {0x001062, 0x001064},
+ {0x001065, 0x001066},
+ {0x001067, 0x00106d},
+ {0x00106e, 0x001070},
+ {0x001071, 0x001074},
+ {0x001075, 0x001081},
+ {0x001082, 0x001082},
+ {0x001083, 0x001084},
+ {0x001085, 0x001086},
+ {0x001087, 0x00108c},
+ {0x00108d, 0x00108d},
+ {0x00108e, 0x00108e},
+ {0x00108f, 0x00108f},
+ {0x00109a, 0x00109c},
+ {0x00109d, 0x00109d},
+ {0x0010a0, 0x0010c5},
+ {0x0010c7, 0x0010c7},
+ {0x0010cd, 0x0010cd},
+ {0x0010d0, 0x0010fa},
+ {0x0010fc, 0x0010fc},
+ {0x0010fd, 0x0010ff},
+ {0x001100, 0x001248},
+ {0x00124a, 0x00124d},
+ {0x001250, 0x001256},
+ {0x001258, 0x001258},
+ {0x00125a, 0x00125d},
+ {0x001260, 0x001288},
+ {0x00128a, 0x00128d},
+ {0x001290, 0x0012b0},
+ {0x0012b2, 0x0012b5},
+ {0x0012b8, 0x0012be},
+ {0x0012c0, 0x0012c0},
+ {0x0012c2, 0x0012c5},
+ {0x0012c8, 0x0012d6},
+ {0x0012d8, 0x001310},
+ {0x001312, 0x001315},
+ {0x001318, 0x00135a},
+ {0x001380, 0x00138f},
+ {0x0013a0, 0x0013f5},
+ {0x0013f8, 0x0013fd},
+ {0x001401, 0x00166c},
+ {0x00166f, 0x00167f},
+ {0x001681, 0x00169a},
+ {0x0016a0, 0x0016ea},
+ {0x0016ee, 0x0016f0},
+ {0x0016f1, 0x0016f8},
+ {0x001700, 0x001711},
+ {0x001712, 0x001713},
+ {0x00171f, 0x001731},
+ {0x001732, 0x001733},
+ {0x001740, 0x001751},
+ {0x001752, 0x001753},
+ {0x001760, 0x00176c},
+ {0x00176e, 0x001770},
+ {0x001772, 0x001773},
+ {0x001780, 0x0017b3},
+ {0x0017b6, 0x0017b6},
+ {0x0017b7, 0x0017bd},
+ {0x0017be, 0x0017c5},
+ {0x0017c6, 0x0017c6},
+ {0x0017c7, 0x0017c8},
+ {0x0017d7, 0x0017d7},
+ {0x0017dc, 0x0017dc},
+ {0x001820, 0x001842},
+ {0x001843, 0x001843},
+ {0x001844, 0x001878},
+ {0x001880, 0x001884},
+ {0x001885, 0x001886},
+ {0x001887, 0x0018a8},
+ {0x0018a9, 0x0018a9},
+ {0x0018aa, 0x0018aa},
+ {0x0018b0, 0x0018f5},
+ {0x001900, 0x00191e},
+ {0x001920, 0x001922},
+ {0x001923, 0x001926},
+ {0x001927, 0x001928},
+ {0x001929, 0x00192b},
+ {0x001930, 0x001931},
+ {0x001932, 0x001932},
+ {0x001933, 0x001938},
+ {0x001950, 0x00196d},
+ {0x001970, 0x001974},
+ {0x001980, 0x0019ab},
+ {0x0019b0, 0x0019c9},
+ {0x001a00, 0x001a16},
+ {0x001a17, 0x001a18},
+ {0x001a19, 0x001a1a},
+ {0x001a1b, 0x001a1b},
+ {0x001a20, 0x001a54},
+ {0x001a55, 0x001a55},
+ {0x001a56, 0x001a56},
+ {0x001a57, 0x001a57},
+ {0x001a58, 0x001a5e},
+ {0x001a61, 0x001a61},
+ {0x001a62, 0x001a62},
+ {0x001a63, 0x001a64},
+ {0x001a65, 0x001a6c},
+ {0x001a6d, 0x001a72},
+ {0x001a73, 0x001a74},
+ {0x001aa7, 0x001aa7},
+ {0x001abf, 0x001ac0},
+ {0x001acc, 0x001ace},
+ {0x001b00, 0x001b03},
+ {0x001b04, 0x001b04},
+ {0x001b05, 0x001b33},
+ {0x001b35, 0x001b35},
+ {0x001b36, 0x001b3a},
+ {0x001b3b, 0x001b3b},
+ {0x001b3c, 0x001b3c},
+ {0x001b3d, 0x001b41},
+ {0x001b42, 0x001b42},
+ {0x001b43, 0x001b43},
+ {0x001b45, 0x001b4c},
+ {0x001b80, 0x001b81},
+ {0x001b82, 0x001b82},
+ {0x001b83, 0x001ba0},
+ {0x001ba1, 0x001ba1},
+ {0x001ba2, 0x001ba5},
+ {0x001ba6, 0x001ba7},
+ {0x001ba8, 0x001ba9},
+ {0x001bac, 0x001bad},
+ {0x001bae, 0x001baf},
+ {0x001bba, 0x001be5},
+ {0x001be7, 0x001be7},
+ {0x001be8, 0x001be9},
+ {0x001bea, 0x001bec},
+ {0x001bed, 0x001bed},
+ {0x001bee, 0x001bee},
+ {0x001bef, 0x001bf1},
+ {0x001c00, 0x001c23},
+ {0x001c24, 0x001c2b},
+ {0x001c2c, 0x001c33},
+ {0x001c34, 0x001c35},
+ {0x001c36, 0x001c36},
+ {0x001c4d, 0x001c4f},
+ {0x001c5a, 0x001c77},
+ {0x001c78, 0x001c7d},
+ {0x001c80, 0x001c88},
+ {0x001c90, 0x001cba},
+ {0x001cbd, 0x001cbf},
+ {0x001ce9, 0x001cec},
+ {0x001cee, 0x001cf3},
+ {0x001cf5, 0x001cf6},
+ {0x001cfa, 0x001cfa},
+ {0x001d00, 0x001d2b},
+ {0x001d2c, 0x001d6a},
+ {0x001d6b, 0x001d77},
+ {0x001d78, 0x001d78},
+ {0x001d79, 0x001d9a},
+ {0x001d9b, 0x001dbf},
+ {0x001de7, 0x001df4},
+ {0x001e00, 0x001f15},
+ {0x001f18, 0x001f1d},
+ {0x001f20, 0x001f45},
+ {0x001f48, 0x001f4d},
+ {0x001f50, 0x001f57},
+ {0x001f59, 0x001f59},
+ {0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f7d},
+ {0x001f80, 0x001fb4},
+ {0x001fb6, 0x001fbc},
+ {0x001fbe, 0x001fbe},
+ {0x001fc2, 0x001fc4},
+ {0x001fc6, 0x001fcc},
+ {0x001fd0, 0x001fd3},
+ {0x001fd6, 0x001fdb},
+ {0x001fe0, 0x001fec},
+ {0x001ff2, 0x001ff4},
+ {0x001ff6, 0x001ffc},
+ {0x002071, 0x002071},
+ {0x00207f, 0x00207f},
+ {0x002090, 0x00209c},
+ {0x002102, 0x002102},
+ {0x002107, 0x002107},
+ {0x00210a, 0x002113},
+ {0x002115, 0x002115},
+ {0x002119, 0x00211d},
+ {0x002124, 0x002124},
+ {0x002126, 0x002126},
+ {0x002128, 0x002128},
+ {0x00212a, 0x00212d},
+ {0x00212f, 0x002134},
+ {0x002135, 0x002138},
+ {0x002139, 0x002139},
+ {0x00213c, 0x00213f},
+ {0x002145, 0x002149},
+ {0x00214e, 0x00214e},
+ {0x002160, 0x002182},
+ {0x002183, 0x002184},
+ {0x002185, 0x002188},
+ {0x0024b6, 0x0024e9},
+ {0x002c00, 0x002c7b},
+ {0x002c7c, 0x002c7d},
+ {0x002c7e, 0x002ce4},
+ {0x002ceb, 0x002cee},
+ {0x002cf2, 0x002cf3},
+ {0x002d00, 0x002d25},
+ {0x002d27, 0x002d27},
+ {0x002d2d, 0x002d2d},
+ {0x002d30, 0x002d67},
+ {0x002d6f, 0x002d6f},
+ {0x002d80, 0x002d96},
+ {0x002da0, 0x002da6},
+ {0x002da8, 0x002dae},
+ {0x002db0, 0x002db6},
+ {0x002db8, 0x002dbe},
+ {0x002dc0, 0x002dc6},
+ {0x002dc8, 0x002dce},
+ {0x002dd0, 0x002dd6},
+ {0x002dd8, 0x002dde},
+ {0x002de0, 0x002dff},
+ {0x002e2f, 0x002e2f},
+ {0x003005, 0x003005},
+ {0x003006, 0x003006},
+ {0x003007, 0x003007},
+ {0x003021, 0x003029},
+ {0x003031, 0x003035},
+ {0x003038, 0x00303a},
+ {0x00303b, 0x00303b},
+ {0x00303c, 0x00303c},
+ {0x003041, 0x003096},
+ {0x00309d, 0x00309e},
+ {0x00309f, 0x00309f},
+ {0x0030a1, 0x0030fa},
+ {0x0030fc, 0x0030fe},
+ {0x0030ff, 0x0030ff},
+ {0x003105, 0x00312f},
+ {0x003131, 0x00318e},
+ {0x0031a0, 0x0031bf},
+ {0x0031f0, 0x0031ff},
+ {0x003400, 0x004dbf},
+ {0x004e00, 0x00a014},
+ {0x00a015, 0x00a015},
+ {0x00a016, 0x00a48c},
+ {0x00a4d0, 0x00a4f7},
+ {0x00a4f8, 0x00a4fd},
+ {0x00a500, 0x00a60b},
+ {0x00a60c, 0x00a60c},
+ {0x00a610, 0x00a61f},
+ {0x00a62a, 0x00a62b},
+ {0x00a640, 0x00a66d},
+ {0x00a66e, 0x00a66e},
+ {0x00a674, 0x00a67b},
+ {0x00a67f, 0x00a67f},
+ {0x00a680, 0x00a69b},
+ {0x00a69c, 0x00a69d},
+ {0x00a69e, 0x00a69f},
+ {0x00a6a0, 0x00a6e5},
+ {0x00a6e6, 0x00a6ef},
+ {0x00a717, 0x00a71f},
+ {0x00a722, 0x00a76f},
+ {0x00a770, 0x00a770},
+ {0x00a771, 0x00a787},
+ {0x00a788, 0x00a788},
+ {0x00a78b, 0x00a78e},
+ {0x00a78f, 0x00a78f},
+ {0x00a790, 0x00a7ca},
+ {0x00a7d0, 0x00a7d1},
+ {0x00a7d3, 0x00a7d3},
+ {0x00a7d5, 0x00a7d9},
+ {0x00a7f2, 0x00a7f4},
+ {0x00a7f5, 0x00a7f6},
+ {0x00a7f7, 0x00a7f7},
+ {0x00a7f8, 0x00a7f9},
+ {0x00a7fa, 0x00a7fa},
+ {0x00a7fb, 0x00a801},
+ {0x00a802, 0x00a802},
+ {0x00a803, 0x00a805},
+ {0x00a807, 0x00a80a},
+ {0x00a80b, 0x00a80b},
+ {0x00a80c, 0x00a822},
+ {0x00a823, 0x00a824},
+ {0x00a825, 0x00a826},
+ {0x00a827, 0x00a827},
+ {0x00a840, 0x00a873},
+ {0x00a880, 0x00a881},
+ {0x00a882, 0x00a8b3},
+ {0x00a8b4, 0x00a8c3},
+ {0x00a8c5, 0x00a8c5},
+ {0x00a8f2, 0x00a8f7},
+ {0x00a8fb, 0x00a8fb},
+ {0x00a8fd, 0x00a8fe},
+ {0x00a8ff, 0x00a8ff},
+ {0x00a90a, 0x00a925},
+ {0x00a926, 0x00a92a},
+ {0x00a930, 0x00a946},
+ {0x00a947, 0x00a951},
+ {0x00a952, 0x00a952},
+ {0x00a960, 0x00a97c},
+ {0x00a980, 0x00a982},
+ {0x00a983, 0x00a983},
+ {0x00a984, 0x00a9b2},
+ {0x00a9b4, 0x00a9b5},
+ {0x00a9b6, 0x00a9b9},
+ {0x00a9ba, 0x00a9bb},
+ {0x00a9bc, 0x00a9bd},
+ {0x00a9be, 0x00a9bf},
+ {0x00a9cf, 0x00a9cf},
+ {0x00a9e0, 0x00a9e4},
+ {0x00a9e5, 0x00a9e5},
+ {0x00a9e6, 0x00a9e6},
+ {0x00a9e7, 0x00a9ef},
+ {0x00a9fa, 0x00a9fe},
+ {0x00aa00, 0x00aa28},
+ {0x00aa29, 0x00aa2e},
+ {0x00aa2f, 0x00aa30},
+ {0x00aa31, 0x00aa32},
+ {0x00aa33, 0x00aa34},
+ {0x00aa35, 0x00aa36},
+ {0x00aa40, 0x00aa42},
+ {0x00aa43, 0x00aa43},
+ {0x00aa44, 0x00aa4b},
+ {0x00aa4c, 0x00aa4c},
+ {0x00aa4d, 0x00aa4d},
+ {0x00aa60, 0x00aa6f},
+ {0x00aa70, 0x00aa70},
+ {0x00aa71, 0x00aa76},
+ {0x00aa7a, 0x00aa7a},
+ {0x00aa7b, 0x00aa7b},
+ {0x00aa7c, 0x00aa7c},
+ {0x00aa7d, 0x00aa7d},
+ {0x00aa7e, 0x00aaaf},
+ {0x00aab0, 0x00aab0},
+ {0x00aab1, 0x00aab1},
+ {0x00aab2, 0x00aab4},
+ {0x00aab5, 0x00aab6},
+ {0x00aab7, 0x00aab8},
+ {0x00aab9, 0x00aabd},
+ {0x00aabe, 0x00aabe},
+ {0x00aac0, 0x00aac0},
+ {0x00aac2, 0x00aac2},
+ {0x00aadb, 0x00aadc},
+ {0x00aadd, 0x00aadd},
+ {0x00aae0, 0x00aaea},
+ {0x00aaeb, 0x00aaeb},
+ {0x00aaec, 0x00aaed},
+ {0x00aaee, 0x00aaef},
+ {0x00aaf2, 0x00aaf2},
+ {0x00aaf3, 0x00aaf4},
+ {0x00aaf5, 0x00aaf5},
+ {0x00ab01, 0x00ab06},
+ {0x00ab09, 0x00ab0e},
+ {0x00ab11, 0x00ab16},
+ {0x00ab20, 0x00ab26},
+ {0x00ab28, 0x00ab2e},
+ {0x00ab30, 0x00ab5a},
+ {0x00ab5c, 0x00ab5f},
+ {0x00ab60, 0x00ab68},
+ {0x00ab69, 0x00ab69},
+ {0x00ab70, 0x00abbf},
+ {0x00abc0, 0x00abe2},
+ {0x00abe3, 0x00abe4},
+ {0x00abe5, 0x00abe5},
+ {0x00abe6, 0x00abe7},
+ {0x00abe8, 0x00abe8},
+ {0x00abe9, 0x00abea},
+ {0x00ac00, 0x00d7a3},
+ {0x00d7b0, 0x00d7c6},
+ {0x00d7cb, 0x00d7fb},
+ {0x00f900, 0x00fa6d},
+ {0x00fa70, 0x00fad9},
+ {0x00fb00, 0x00fb06},
+ {0x00fb13, 0x00fb17},
+ {0x00fb1d, 0x00fb1d},
+ {0x00fb1e, 0x00fb1e},
+ {0x00fb1f, 0x00fb28},
+ {0x00fb2a, 0x00fb36},
+ {0x00fb38, 0x00fb3c},
+ {0x00fb3e, 0x00fb3e},
+ {0x00fb40, 0x00fb41},
+ {0x00fb43, 0x00fb44},
+ {0x00fb46, 0x00fbb1},
+ {0x00fbd3, 0x00fd3d},
+ {0x00fd50, 0x00fd8f},
+ {0x00fd92, 0x00fdc7},
+ {0x00fdf0, 0x00fdfb},
+ {0x00fe70, 0x00fe74},
+ {0x00fe76, 0x00fefc},
+ {0x00ff21, 0x00ff3a},
+ {0x00ff41, 0x00ff5a},
+ {0x00ff66, 0x00ff6f},
+ {0x00ff70, 0x00ff70},
+ {0x00ff71, 0x00ff9d},
+ {0x00ff9e, 0x00ff9f},
+ {0x00ffa0, 0x00ffbe},
+ {0x00ffc2, 0x00ffc7},
+ {0x00ffca, 0x00ffcf},
+ {0x00ffd2, 0x00ffd7},
+ {0x00ffda, 0x00ffdc},
+ {0x010000, 0x01000b},
+ {0x01000d, 0x010026},
+ {0x010028, 0x01003a},
+ {0x01003c, 0x01003d},
+ {0x01003f, 0x01004d},
+ {0x010050, 0x01005d},
+ {0x010080, 0x0100fa},
+ {0x010140, 0x010174},
+ {0x010280, 0x01029c},
+ {0x0102a0, 0x0102d0},
+ {0x010300, 0x01031f},
+ {0x01032d, 0x010340},
+ {0x010341, 0x010341},
+ {0x010342, 0x010349},
+ {0x01034a, 0x01034a},
+ {0x010350, 0x010375},
+ {0x010376, 0x01037a},
+ {0x010380, 0x01039d},
+ {0x0103a0, 0x0103c3},
+ {0x0103c8, 0x0103cf},
+ {0x0103d1, 0x0103d5},
+ {0x010400, 0x01044f},
+ {0x010450, 0x01049d},
+ {0x0104b0, 0x0104d3},
+ {0x0104d8, 0x0104fb},
+ {0x010500, 0x010527},
+ {0x010530, 0x010563},
+ {0x010570, 0x01057a},
+ {0x01057c, 0x01058a},
+ {0x01058c, 0x010592},
+ {0x010594, 0x010595},
+ {0x010597, 0x0105a1},
+ {0x0105a3, 0x0105b1},
+ {0x0105b3, 0x0105b9},
+ {0x0105bb, 0x0105bc},
+ {0x010600, 0x010736},
+ {0x010740, 0x010755},
+ {0x010760, 0x010767},
+ {0x010780, 0x010785},
+ {0x010787, 0x0107b0},
+ {0x0107b2, 0x0107ba},
+ {0x010800, 0x010805},
+ {0x010808, 0x010808},
+ {0x01080a, 0x010835},
+ {0x010837, 0x010838},
+ {0x01083c, 0x01083c},
+ {0x01083f, 0x010855},
+ {0x010860, 0x010876},
+ {0x010880, 0x01089e},
+ {0x0108e0, 0x0108f2},
+ {0x0108f4, 0x0108f5},
+ {0x010900, 0x010915},
+ {0x010920, 0x010939},
+ {0x010980, 0x0109b7},
+ {0x0109be, 0x0109bf},
+ {0x010a00, 0x010a00},
+ {0x010a01, 0x010a03},
+ {0x010a05, 0x010a06},
+ {0x010a0c, 0x010a0f},
+ {0x010a10, 0x010a13},
+ {0x010a15, 0x010a17},
+ {0x010a19, 0x010a35},
+ {0x010a60, 0x010a7c},
+ {0x010a80, 0x010a9c},
+ {0x010ac0, 0x010ac7},
+ {0x010ac9, 0x010ae4},
+ {0x010b00, 0x010b35},
+ {0x010b40, 0x010b55},
+ {0x010b60, 0x010b72},
+ {0x010b80, 0x010b91},
+ {0x010c00, 0x010c48},
+ {0x010c80, 0x010cb2},
+ {0x010cc0, 0x010cf2},
+ {0x010d00, 0x010d23},
+ {0x010d24, 0x010d27},
+ {0x010e80, 0x010ea9},
+ {0x010eab, 0x010eac},
+ {0x010eb0, 0x010eb1},
+ {0x010f00, 0x010f1c},
+ {0x010f27, 0x010f27},
+ {0x010f30, 0x010f45},
+ {0x010f70, 0x010f81},
+ {0x010fb0, 0x010fc4},
+ {0x010fe0, 0x010ff6},
+ {0x011000, 0x011000},
+ {0x011001, 0x011001},
+ {0x011002, 0x011002},
+ {0x011003, 0x011037},
+ {0x011038, 0x011045},
+ {0x011071, 0x011072},
+ {0x011073, 0x011074},
+ {0x011075, 0x011075},
+ {0x011080, 0x011081},
+ {0x011082, 0x011082},
+ {0x011083, 0x0110af},
+ {0x0110b0, 0x0110b2},
+ {0x0110b3, 0x0110b6},
+ {0x0110b7, 0x0110b8},
+ {0x0110c2, 0x0110c2},
+ {0x0110d0, 0x0110e8},
+ {0x011100, 0x011102},
+ {0x011103, 0x011126},
+ {0x011127, 0x01112b},
+ {0x01112c, 0x01112c},
+ {0x01112d, 0x011132},
+ {0x011144, 0x011144},
+ {0x011145, 0x011146},
+ {0x011147, 0x011147},
+ {0x011150, 0x011172},
+ {0x011176, 0x011176},
+ {0x011180, 0x011181},
+ {0x011182, 0x011182},
+ {0x011183, 0x0111b2},
+ {0x0111b3, 0x0111b5},
+ {0x0111b6, 0x0111be},
+ {0x0111bf, 0x0111bf},
+ {0x0111c1, 0x0111c4},
+ {0x0111ce, 0x0111ce},
+ {0x0111cf, 0x0111cf},
+ {0x0111da, 0x0111da},
+ {0x0111dc, 0x0111dc},
+ {0x011200, 0x011211},
+ {0x011213, 0x01122b},
+ {0x01122c, 0x01122e},
+ {0x01122f, 0x011231},
+ {0x011232, 0x011233},
+ {0x011234, 0x011234},
+ {0x011237, 0x011237},
+ {0x01123e, 0x01123e},
+ {0x01123f, 0x011240},
+ {0x011241, 0x011241},
+ {0x011280, 0x011286},
+ {0x011288, 0x011288},
+ {0x01128a, 0x01128d},
+ {0x01128f, 0x01129d},
+ {0x01129f, 0x0112a8},
+ {0x0112b0, 0x0112de},
+ {0x0112df, 0x0112df},
+ {0x0112e0, 0x0112e2},
+ {0x0112e3, 0x0112e8},
+ {0x011300, 0x011301},
+ {0x011302, 0x011303},
+ {0x011305, 0x01130c},
+ {0x01130f, 0x011310},
+ {0x011313, 0x011328},
+ {0x01132a, 0x011330},
+ {0x011332, 0x011333},
+ {0x011335, 0x011339},
+ {0x01133d, 0x01133d},
+ {0x01133e, 0x01133f},
+ {0x011340, 0x011340},
+ {0x011341, 0x011344},
+ {0x011347, 0x011348},
+ {0x01134b, 0x01134c},
+ {0x011350, 0x011350},
+ {0x011357, 0x011357},
+ {0x01135d, 0x011361},
+ {0x011362, 0x011363},
+ {0x011400, 0x011434},
+ {0x011435, 0x011437},
+ {0x011438, 0x01143f},
+ {0x011440, 0x011441},
+ {0x011443, 0x011444},
+ {0x011445, 0x011445},
+ {0x011447, 0x01144a},
+ {0x01145f, 0x011461},
+ {0x011480, 0x0114af},
+ {0x0114b0, 0x0114b2},
+ {0x0114b3, 0x0114b8},
+ {0x0114b9, 0x0114b9},
+ {0x0114ba, 0x0114ba},
+ {0x0114bb, 0x0114be},
+ {0x0114bf, 0x0114c0},
+ {0x0114c1, 0x0114c1},
+ {0x0114c4, 0x0114c5},
+ {0x0114c7, 0x0114c7},
+ {0x011580, 0x0115ae},
+ {0x0115af, 0x0115b1},
+ {0x0115b2, 0x0115b5},
+ {0x0115b8, 0x0115bb},
+ {0x0115bc, 0x0115bd},
+ {0x0115be, 0x0115be},
+ {0x0115d8, 0x0115db},
+ {0x0115dc, 0x0115dd},
+ {0x011600, 0x01162f},
+ {0x011630, 0x011632},
+ {0x011633, 0x01163a},
+ {0x01163b, 0x01163c},
+ {0x01163d, 0x01163d},
+ {0x01163e, 0x01163e},
+ {0x011640, 0x011640},
+ {0x011644, 0x011644},
+ {0x011680, 0x0116aa},
+ {0x0116ab, 0x0116ab},
+ {0x0116ac, 0x0116ac},
+ {0x0116ad, 0x0116ad},
+ {0x0116ae, 0x0116af},
+ {0x0116b0, 0x0116b5},
+ {0x0116b8, 0x0116b8},
+ {0x011700, 0x01171a},
+ {0x01171d, 0x01171f},
+ {0x011720, 0x011721},
+ {0x011722, 0x011725},
+ {0x011726, 0x011726},
+ {0x011727, 0x01172a},
+ {0x011740, 0x011746},
+ {0x011800, 0x01182b},
+ {0x01182c, 0x01182e},
+ {0x01182f, 0x011837},
+ {0x011838, 0x011838},
+ {0x0118a0, 0x0118df},
+ {0x0118ff, 0x011906},
+ {0x011909, 0x011909},
+ {0x01190c, 0x011913},
+ {0x011915, 0x011916},
+ {0x011918, 0x01192f},
+ {0x011930, 0x011935},
+ {0x011937, 0x011938},
+ {0x01193b, 0x01193c},
+ {0x01193f, 0x01193f},
+ {0x011940, 0x011940},
+ {0x011941, 0x011941},
+ {0x011942, 0x011942},
+ {0x0119a0, 0x0119a7},
+ {0x0119aa, 0x0119d0},
+ {0x0119d1, 0x0119d3},
+ {0x0119d4, 0x0119d7},
+ {0x0119da, 0x0119db},
+ {0x0119dc, 0x0119df},
+ {0x0119e1, 0x0119e1},
+ {0x0119e3, 0x0119e3},
+ {0x0119e4, 0x0119e4},
+ {0x011a00, 0x011a00},
+ {0x011a01, 0x011a0a},
+ {0x011a0b, 0x011a32},
+ {0x011a35, 0x011a38},
+ {0x011a39, 0x011a39},
+ {0x011a3a, 0x011a3a},
+ {0x011a3b, 0x011a3e},
+ {0x011a50, 0x011a50},
+ {0x011a51, 0x011a56},
+ {0x011a57, 0x011a58},
+ {0x011a59, 0x011a5b},
+ {0x011a5c, 0x011a89},
+ {0x011a8a, 0x011a96},
+ {0x011a97, 0x011a97},
+ {0x011a9d, 0x011a9d},
+ {0x011ab0, 0x011af8},
+ {0x011c00, 0x011c08},
+ {0x011c0a, 0x011c2e},
+ {0x011c2f, 0x011c2f},
+ {0x011c30, 0x011c36},
+ {0x011c38, 0x011c3d},
+ {0x011c3e, 0x011c3e},
+ {0x011c40, 0x011c40},
+ {0x011c72, 0x011c8f},
+ {0x011c92, 0x011ca7},
+ {0x011ca9, 0x011ca9},
+ {0x011caa, 0x011cb0},
+ {0x011cb1, 0x011cb1},
+ {0x011cb2, 0x011cb3},
+ {0x011cb4, 0x011cb4},
+ {0x011cb5, 0x011cb6},
+ {0x011d00, 0x011d06},
+ {0x011d08, 0x011d09},
+ {0x011d0b, 0x011d30},
+ {0x011d31, 0x011d36},
+ {0x011d3a, 0x011d3a},
+ {0x011d3c, 0x011d3d},
+ {0x011d3f, 0x011d41},
+ {0x011d43, 0x011d43},
+ {0x011d46, 0x011d46},
+ {0x011d47, 0x011d47},
+ {0x011d60, 0x011d65},
+ {0x011d67, 0x011d68},
+ {0x011d6a, 0x011d89},
+ {0x011d8a, 0x011d8e},
+ {0x011d90, 0x011d91},
+ {0x011d93, 0x011d94},
+ {0x011d95, 0x011d95},
+ {0x011d96, 0x011d96},
+ {0x011d98, 0x011d98},
+ {0x011ee0, 0x011ef2},
+ {0x011ef3, 0x011ef4},
+ {0x011ef5, 0x011ef6},
+ {0x011f00, 0x011f01},
+ {0x011f02, 0x011f02},
+ {0x011f03, 0x011f03},
+ {0x011f04, 0x011f10},
+ {0x011f12, 0x011f33},
+ {0x011f34, 0x011f35},
+ {0x011f36, 0x011f3a},
+ {0x011f3e, 0x011f3f},
+ {0x011f40, 0x011f40},
+ {0x011fb0, 0x011fb0},
+ {0x012000, 0x012399},
+ {0x012400, 0x01246e},
+ {0x012480, 0x012543},
+ {0x012f90, 0x012ff0},
+ {0x013000, 0x01342f},
+ {0x013441, 0x013446},
+ {0x014400, 0x014646},
+ {0x016800, 0x016a38},
+ {0x016a40, 0x016a5e},
+ {0x016a70, 0x016abe},
+ {0x016ad0, 0x016aed},
+ {0x016b00, 0x016b2f},
+ {0x016b40, 0x016b43},
+ {0x016b63, 0x016b77},
+ {0x016b7d, 0x016b8f},
+ {0x016e40, 0x016e7f},
+ {0x016f00, 0x016f4a},
+ {0x016f4f, 0x016f4f},
+ {0x016f50, 0x016f50},
+ {0x016f51, 0x016f87},
+ {0x016f8f, 0x016f92},
+ {0x016f93, 0x016f9f},
+ {0x016fe0, 0x016fe1},
+ {0x016fe3, 0x016fe3},
+ {0x016ff0, 0x016ff1},
+ {0x017000, 0x0187f7},
+ {0x018800, 0x018cd5},
+ {0x018d00, 0x018d08},
+ {0x01aff0, 0x01aff3},
+ {0x01aff5, 0x01affb},
+ {0x01affd, 0x01affe},
+ {0x01b000, 0x01b122},
+ {0x01b132, 0x01b132},
+ {0x01b150, 0x01b152},
+ {0x01b155, 0x01b155},
+ {0x01b164, 0x01b167},
+ {0x01b170, 0x01b2fb},
+ {0x01bc00, 0x01bc6a},
+ {0x01bc70, 0x01bc7c},
+ {0x01bc80, 0x01bc88},
+ {0x01bc90, 0x01bc99},
+ {0x01bc9e, 0x01bc9e},
+ {0x01d400, 0x01d454},
+ {0x01d456, 0x01d49c},
+ {0x01d49e, 0x01d49f},
+ {0x01d4a2, 0x01d4a2},
+ {0x01d4a5, 0x01d4a6},
+ {0x01d4a9, 0x01d4ac},
+ {0x01d4ae, 0x01d4b9},
+ {0x01d4bb, 0x01d4bb},
+ {0x01d4bd, 0x01d4c3},
+ {0x01d4c5, 0x01d505},
+ {0x01d507, 0x01d50a},
+ {0x01d50d, 0x01d514},
+ {0x01d516, 0x01d51c},
+ {0x01d51e, 0x01d539},
+ {0x01d53b, 0x01d53e},
+ {0x01d540, 0x01d544},
+ {0x01d546, 0x01d546},
+ {0x01d54a, 0x01d550},
+ {0x01d552, 0x01d6a5},
+ {0x01d6a8, 0x01d6c0},
+ {0x01d6c2, 0x01d6da},
+ {0x01d6dc, 0x01d6fa},
+ {0x01d6fc, 0x01d714},
+ {0x01d716, 0x01d734},
+ {0x01d736, 0x01d74e},
+ {0x01d750, 0x01d76e},
+ {0x01d770, 0x01d788},
+ {0x01d78a, 0x01d7a8},
+ {0x01d7aa, 0x01d7c2},
+ {0x01d7c4, 0x01d7cb},
+ {0x01df00, 0x01df09},
+ {0x01df0a, 0x01df0a},
+ {0x01df0b, 0x01df1e},
+ {0x01df25, 0x01df2a},
+ {0x01e000, 0x01e006},
+ {0x01e008, 0x01e018},
+ {0x01e01b, 0x01e021},
+ {0x01e023, 0x01e024},
+ {0x01e026, 0x01e02a},
+ {0x01e030, 0x01e06d},
+ {0x01e08f, 0x01e08f},
+ {0x01e100, 0x01e12c},
+ {0x01e137, 0x01e13d},
+ {0x01e14e, 0x01e14e},
+ {0x01e290, 0x01e2ad},
+ {0x01e2c0, 0x01e2eb},
+ {0x01e4d0, 0x01e4ea},
+ {0x01e4eb, 0x01e4eb},
+ {0x01e7e0, 0x01e7e6},
+ {0x01e7e8, 0x01e7eb},
+ {0x01e7ed, 0x01e7ee},
+ {0x01e7f0, 0x01e7fe},
+ {0x01e800, 0x01e8c4},
+ {0x01e900, 0x01e943},
+ {0x01e947, 0x01e947},
+ {0x01e94b, 0x01e94b},
+ {0x01ee00, 0x01ee03},
+ {0x01ee05, 0x01ee1f},
+ {0x01ee21, 0x01ee22},
+ {0x01ee24, 0x01ee24},
+ {0x01ee27, 0x01ee27},
+ {0x01ee29, 0x01ee32},
+ {0x01ee34, 0x01ee37},
+ {0x01ee39, 0x01ee39},
+ {0x01ee3b, 0x01ee3b},
+ {0x01ee42, 0x01ee42},
+ {0x01ee47, 0x01ee47},
+ {0x01ee49, 0x01ee49},
+ {0x01ee4b, 0x01ee4b},
+ {0x01ee4d, 0x01ee4f},
+ {0x01ee51, 0x01ee52},
+ {0x01ee54, 0x01ee54},
+ {0x01ee57, 0x01ee57},
+ {0x01ee59, 0x01ee59},
+ {0x01ee5b, 0x01ee5b},
+ {0x01ee5d, 0x01ee5d},
+ {0x01ee5f, 0x01ee5f},
+ {0x01ee61, 0x01ee62},
+ {0x01ee64, 0x01ee64},
+ {0x01ee67, 0x01ee6a},
+ {0x01ee6c, 0x01ee72},
+ {0x01ee74, 0x01ee77},
+ {0x01ee79, 0x01ee7c},
+ {0x01ee7e, 0x01ee7e},
+ {0x01ee80, 0x01ee89},
+ {0x01ee8b, 0x01ee9b},
+ {0x01eea1, 0x01eea3},
+ {0x01eea5, 0x01eea9},
+ {0x01eeab, 0x01eebb},
+ {0x01f130, 0x01f149},
+ {0x01f150, 0x01f169},
+ {0x01f170, 0x01f189},
+ {0x020000, 0x02a6df},
+ {0x02a700, 0x02b739},
+ {0x02b740, 0x02b81d},
+ {0x02b820, 0x02cea1},
+ {0x02ceb0, 0x02ebe0},
+ {0x02ebf0, 0x02ee5d},
+ {0x02f800, 0x02fa1d},
+ {0x030000, 0x03134a},
+ {0x031350, 0x0323af}
+};
+
+/* table of Unicode codepoint ranges of Lowercase characters */
+static const pg_unicode_range unicode_lowercase[686] =
+{
+ {0x000061, 0x00007a},
+ {0x0000aa, 0x0000aa},
+ {0x0000b5, 0x0000b5},
+ {0x0000ba, 0x0000ba},
+ {0x0000df, 0x0000f6},
+ {0x0000f8, 0x0000ff},
+ {0x000101, 0x000101},
+ {0x000103, 0x000103},
+ {0x000105, 0x000105},
+ {0x000107, 0x000107},
+ {0x000109, 0x000109},
+ {0x00010b, 0x00010b},
+ {0x00010d, 0x00010d},
+ {0x00010f, 0x00010f},
+ {0x000111, 0x000111},
+ {0x000113, 0x000113},
+ {0x000115, 0x000115},
+ {0x000117, 0x000117},
+ {0x000119, 0x000119},
+ {0x00011b, 0x00011b},
+ {0x00011d, 0x00011d},
+ {0x00011f, 0x00011f},
+ {0x000121, 0x000121},
+ {0x000123, 0x000123},
+ {0x000125, 0x000125},
+ {0x000127, 0x000127},
+ {0x000129, 0x000129},
+ {0x00012b, 0x00012b},
+ {0x00012d, 0x00012d},
+ {0x00012f, 0x00012f},
+ {0x000131, 0x000131},
+ {0x000133, 0x000133},
+ {0x000135, 0x000135},
+ {0x000137, 0x000138},
+ {0x00013a, 0x00013a},
+ {0x00013c, 0x00013c},
+ {0x00013e, 0x00013e},
+ {0x000140, 0x000140},
+ {0x000142, 0x000142},
+ {0x000144, 0x000144},
+ {0x000146, 0x000146},
+ {0x000148, 0x000149},
+ {0x00014b, 0x00014b},
+ {0x00014d, 0x00014d},
+ {0x00014f, 0x00014f},
+ {0x000151, 0x000151},
+ {0x000153, 0x000153},
+ {0x000155, 0x000155},
+ {0x000157, 0x000157},
+ {0x000159, 0x000159},
+ {0x00015b, 0x00015b},
+ {0x00015d, 0x00015d},
+ {0x00015f, 0x00015f},
+ {0x000161, 0x000161},
+ {0x000163, 0x000163},
+ {0x000165, 0x000165},
+ {0x000167, 0x000167},
+ {0x000169, 0x000169},
+ {0x00016b, 0x00016b},
+ {0x00016d, 0x00016d},
+ {0x00016f, 0x00016f},
+ {0x000171, 0x000171},
+ {0x000173, 0x000173},
+ {0x000175, 0x000175},
+ {0x000177, 0x000177},
+ {0x00017a, 0x00017a},
+ {0x00017c, 0x00017c},
+ {0x00017e, 0x000180},
+ {0x000183, 0x000183},
+ {0x000185, 0x000185},
+ {0x000188, 0x000188},
+ {0x00018c, 0x00018d},
+ {0x000192, 0x000192},
+ {0x000195, 0x000195},
+ {0x000199, 0x00019b},
+ {0x00019e, 0x00019e},
+ {0x0001a1, 0x0001a1},
+ {0x0001a3, 0x0001a3},
+ {0x0001a5, 0x0001a5},
+ {0x0001a8, 0x0001a8},
+ {0x0001aa, 0x0001ab},
+ {0x0001ad, 0x0001ad},
+ {0x0001b0, 0x0001b0},
+ {0x0001b4, 0x0001b4},
+ {0x0001b6, 0x0001b6},
+ {0x0001b9, 0x0001ba},
+ {0x0001bd, 0x0001bf},
+ {0x0001c6, 0x0001c6},
+ {0x0001c9, 0x0001c9},
+ {0x0001cc, 0x0001cc},
+ {0x0001ce, 0x0001ce},
+ {0x0001d0, 0x0001d0},
+ {0x0001d2, 0x0001d2},
+ {0x0001d4, 0x0001d4},
+ {0x0001d6, 0x0001d6},
+ {0x0001d8, 0x0001d8},
+ {0x0001da, 0x0001da},
+ {0x0001dc, 0x0001dd},
+ {0x0001df, 0x0001df},
+ {0x0001e1, 0x0001e1},
+ {0x0001e3, 0x0001e3},
+ {0x0001e5, 0x0001e5},
+ {0x0001e7, 0x0001e7},
+ {0x0001e9, 0x0001e9},
+ {0x0001eb, 0x0001eb},
+ {0x0001ed, 0x0001ed},
+ {0x0001ef, 0x0001f0},
+ {0x0001f3, 0x0001f3},
+ {0x0001f5, 0x0001f5},
+ {0x0001f9, 0x0001f9},
+ {0x0001fb, 0x0001fb},
+ {0x0001fd, 0x0001fd},
+ {0x0001ff, 0x0001ff},
+ {0x000201, 0x000201},
+ {0x000203, 0x000203},
+ {0x000205, 0x000205},
+ {0x000207, 0x000207},
+ {0x000209, 0x000209},
+ {0x00020b, 0x00020b},
+ {0x00020d, 0x00020d},
+ {0x00020f, 0x00020f},
+ {0x000211, 0x000211},
+ {0x000213, 0x000213},
+ {0x000215, 0x000215},
+ {0x000217, 0x000217},
+ {0x000219, 0x000219},
+ {0x00021b, 0x00021b},
+ {0x00021d, 0x00021d},
+ {0x00021f, 0x00021f},
+ {0x000221, 0x000221},
+ {0x000223, 0x000223},
+ {0x000225, 0x000225},
+ {0x000227, 0x000227},
+ {0x000229, 0x000229},
+ {0x00022b, 0x00022b},
+ {0x00022d, 0x00022d},
+ {0x00022f, 0x00022f},
+ {0x000231, 0x000231},
+ {0x000233, 0x000239},
+ {0x00023c, 0x00023c},
+ {0x00023f, 0x000240},
+ {0x000242, 0x000242},
+ {0x000247, 0x000247},
+ {0x000249, 0x000249},
+ {0x00024b, 0x00024b},
+ {0x00024d, 0x00024d},
+ {0x00024f, 0x000293},
+ {0x000295, 0x0002af},
+ {0x0002b0, 0x0002b8},
+ {0x0002c0, 0x0002c1},
+ {0x0002e0, 0x0002e4},
+ {0x000345, 0x000345},
+ {0x000371, 0x000371},
+ {0x000373, 0x000373},
+ {0x000377, 0x000377},
+ {0x00037a, 0x00037a},
+ {0x00037b, 0x00037d},
+ {0x000390, 0x000390},
+ {0x0003ac, 0x0003ce},
+ {0x0003d0, 0x0003d1},
+ {0x0003d5, 0x0003d7},
+ {0x0003d9, 0x0003d9},
+ {0x0003db, 0x0003db},
+ {0x0003dd, 0x0003dd},
+ {0x0003df, 0x0003df},
+ {0x0003e1, 0x0003e1},
+ {0x0003e3, 0x0003e3},
+ {0x0003e5, 0x0003e5},
+ {0x0003e7, 0x0003e7},
+ {0x0003e9, 0x0003e9},
+ {0x0003eb, 0x0003eb},
+ {0x0003ed, 0x0003ed},
+ {0x0003ef, 0x0003f3},
+ {0x0003f5, 0x0003f5},
+ {0x0003f8, 0x0003f8},
+ {0x0003fb, 0x0003fc},
+ {0x000430, 0x00045f},
+ {0x000461, 0x000461},
+ {0x000463, 0x000463},
+ {0x000465, 0x000465},
+ {0x000467, 0x000467},
+ {0x000469, 0x000469},
+ {0x00046b, 0x00046b},
+ {0x00046d, 0x00046d},
+ {0x00046f, 0x00046f},
+ {0x000471, 0x000471},
+ {0x000473, 0x000473},
+ {0x000475, 0x000475},
+ {0x000477, 0x000477},
+ {0x000479, 0x000479},
+ {0x00047b, 0x00047b},
+ {0x00047d, 0x00047d},
+ {0x00047f, 0x00047f},
+ {0x000481, 0x000481},
+ {0x00048b, 0x00048b},
+ {0x00048d, 0x00048d},
+ {0x00048f, 0x00048f},
+ {0x000491, 0x000491},
+ {0x000493, 0x000493},
+ {0x000495, 0x000495},
+ {0x000497, 0x000497},
+ {0x000499, 0x000499},
+ {0x00049b, 0x00049b},
+ {0x00049d, 0x00049d},
+ {0x00049f, 0x00049f},
+ {0x0004a1, 0x0004a1},
+ {0x0004a3, 0x0004a3},
+ {0x0004a5, 0x0004a5},
+ {0x0004a7, 0x0004a7},
+ {0x0004a9, 0x0004a9},
+ {0x0004ab, 0x0004ab},
+ {0x0004ad, 0x0004ad},
+ {0x0004af, 0x0004af},
+ {0x0004b1, 0x0004b1},
+ {0x0004b3, 0x0004b3},
+ {0x0004b5, 0x0004b5},
+ {0x0004b7, 0x0004b7},
+ {0x0004b9, 0x0004b9},
+ {0x0004bb, 0x0004bb},
+ {0x0004bd, 0x0004bd},
+ {0x0004bf, 0x0004bf},
+ {0x0004c2, 0x0004c2},
+ {0x0004c4, 0x0004c4},
+ {0x0004c6, 0x0004c6},
+ {0x0004c8, 0x0004c8},
+ {0x0004ca, 0x0004ca},
+ {0x0004cc, 0x0004cc},
+ {0x0004ce, 0x0004cf},
+ {0x0004d1, 0x0004d1},
+ {0x0004d3, 0x0004d3},
+ {0x0004d5, 0x0004d5},
+ {0x0004d7, 0x0004d7},
+ {0x0004d9, 0x0004d9},
+ {0x0004db, 0x0004db},
+ {0x0004dd, 0x0004dd},
+ {0x0004df, 0x0004df},
+ {0x0004e1, 0x0004e1},
+ {0x0004e3, 0x0004e3},
+ {0x0004e5, 0x0004e5},
+ {0x0004e7, 0x0004e7},
+ {0x0004e9, 0x0004e9},
+ {0x0004eb, 0x0004eb},
+ {0x0004ed, 0x0004ed},
+ {0x0004ef, 0x0004ef},
+ {0x0004f1, 0x0004f1},
+ {0x0004f3, 0x0004f3},
+ {0x0004f5, 0x0004f5},
+ {0x0004f7, 0x0004f7},
+ {0x0004f9, 0x0004f9},
+ {0x0004fb, 0x0004fb},
+ {0x0004fd, 0x0004fd},
+ {0x0004ff, 0x0004ff},
+ {0x000501, 0x000501},
+ {0x000503, 0x000503},
+ {0x000505, 0x000505},
+ {0x000507, 0x000507},
+ {0x000509, 0x000509},
+ {0x00050b, 0x00050b},
+ {0x00050d, 0x00050d},
+ {0x00050f, 0x00050f},
+ {0x000511, 0x000511},
+ {0x000513, 0x000513},
+ {0x000515, 0x000515},
+ {0x000517, 0x000517},
+ {0x000519, 0x000519},
+ {0x00051b, 0x00051b},
+ {0x00051d, 0x00051d},
+ {0x00051f, 0x00051f},
+ {0x000521, 0x000521},
+ {0x000523, 0x000523},
+ {0x000525, 0x000525},
+ {0x000527, 0x000527},
+ {0x000529, 0x000529},
+ {0x00052b, 0x00052b},
+ {0x00052d, 0x00052d},
+ {0x00052f, 0x00052f},
+ {0x000560, 0x000588},
+ {0x0010d0, 0x0010fa},
+ {0x0010fc, 0x0010fc},
+ {0x0010fd, 0x0010ff},
+ {0x0013f8, 0x0013fd},
+ {0x001c80, 0x001c88},
+ {0x001d00, 0x001d2b},
+ {0x001d2c, 0x001d6a},
+ {0x001d6b, 0x001d77},
+ {0x001d78, 0x001d78},
+ {0x001d79, 0x001d9a},
+ {0x001d9b, 0x001dbf},
+ {0x001e01, 0x001e01},
+ {0x001e03, 0x001e03},
+ {0x001e05, 0x001e05},
+ {0x001e07, 0x001e07},
+ {0x001e09, 0x001e09},
+ {0x001e0b, 0x001e0b},
+ {0x001e0d, 0x001e0d},
+ {0x001e0f, 0x001e0f},
+ {0x001e11, 0x001e11},
+ {0x001e13, 0x001e13},
+ {0x001e15, 0x001e15},
+ {0x001e17, 0x001e17},
+ {0x001e19, 0x001e19},
+ {0x001e1b, 0x001e1b},
+ {0x001e1d, 0x001e1d},
+ {0x001e1f, 0x001e1f},
+ {0x001e21, 0x001e21},
+ {0x001e23, 0x001e23},
+ {0x001e25, 0x001e25},
+ {0x001e27, 0x001e27},
+ {0x001e29, 0x001e29},
+ {0x001e2b, 0x001e2b},
+ {0x001e2d, 0x001e2d},
+ {0x001e2f, 0x001e2f},
+ {0x001e31, 0x001e31},
+ {0x001e33, 0x001e33},
+ {0x001e35, 0x001e35},
+ {0x001e37, 0x001e37},
+ {0x001e39, 0x001e39},
+ {0x001e3b, 0x001e3b},
+ {0x001e3d, 0x001e3d},
+ {0x001e3f, 0x001e3f},
+ {0x001e41, 0x001e41},
+ {0x001e43, 0x001e43},
+ {0x001e45, 0x001e45},
+ {0x001e47, 0x001e47},
+ {0x001e49, 0x001e49},
+ {0x001e4b, 0x001e4b},
+ {0x001e4d, 0x001e4d},
+ {0x001e4f, 0x001e4f},
+ {0x001e51, 0x001e51},
+ {0x001e53, 0x001e53},
+ {0x001e55, 0x001e55},
+ {0x001e57, 0x001e57},
+ {0x001e59, 0x001e59},
+ {0x001e5b, 0x001e5b},
+ {0x001e5d, 0x001e5d},
+ {0x001e5f, 0x001e5f},
+ {0x001e61, 0x001e61},
+ {0x001e63, 0x001e63},
+ {0x001e65, 0x001e65},
+ {0x001e67, 0x001e67},
+ {0x001e69, 0x001e69},
+ {0x001e6b, 0x001e6b},
+ {0x001e6d, 0x001e6d},
+ {0x001e6f, 0x001e6f},
+ {0x001e71, 0x001e71},
+ {0x001e73, 0x001e73},
+ {0x001e75, 0x001e75},
+ {0x001e77, 0x001e77},
+ {0x001e79, 0x001e79},
+ {0x001e7b, 0x001e7b},
+ {0x001e7d, 0x001e7d},
+ {0x001e7f, 0x001e7f},
+ {0x001e81, 0x001e81},
+ {0x001e83, 0x001e83},
+ {0x001e85, 0x001e85},
+ {0x001e87, 0x001e87},
+ {0x001e89, 0x001e89},
+ {0x001e8b, 0x001e8b},
+ {0x001e8d, 0x001e8d},
+ {0x001e8f, 0x001e8f},
+ {0x001e91, 0x001e91},
+ {0x001e93, 0x001e93},
+ {0x001e95, 0x001e9d},
+ {0x001e9f, 0x001e9f},
+ {0x001ea1, 0x001ea1},
+ {0x001ea3, 0x001ea3},
+ {0x001ea5, 0x001ea5},
+ {0x001ea7, 0x001ea7},
+ {0x001ea9, 0x001ea9},
+ {0x001eab, 0x001eab},
+ {0x001ead, 0x001ead},
+ {0x001eaf, 0x001eaf},
+ {0x001eb1, 0x001eb1},
+ {0x001eb3, 0x001eb3},
+ {0x001eb5, 0x001eb5},
+ {0x001eb7, 0x001eb7},
+ {0x001eb9, 0x001eb9},
+ {0x001ebb, 0x001ebb},
+ {0x001ebd, 0x001ebd},
+ {0x001ebf, 0x001ebf},
+ {0x001ec1, 0x001ec1},
+ {0x001ec3, 0x001ec3},
+ {0x001ec5, 0x001ec5},
+ {0x001ec7, 0x001ec7},
+ {0x001ec9, 0x001ec9},
+ {0x001ecb, 0x001ecb},
+ {0x001ecd, 0x001ecd},
+ {0x001ecf, 0x001ecf},
+ {0x001ed1, 0x001ed1},
+ {0x001ed3, 0x001ed3},
+ {0x001ed5, 0x001ed5},
+ {0x001ed7, 0x001ed7},
+ {0x001ed9, 0x001ed9},
+ {0x001edb, 0x001edb},
+ {0x001edd, 0x001edd},
+ {0x001edf, 0x001edf},
+ {0x001ee1, 0x001ee1},
+ {0x001ee3, 0x001ee3},
+ {0x001ee5, 0x001ee5},
+ {0x001ee7, 0x001ee7},
+ {0x001ee9, 0x001ee9},
+ {0x001eeb, 0x001eeb},
+ {0x001eed, 0x001eed},
+ {0x001eef, 0x001eef},
+ {0x001ef1, 0x001ef1},
+ {0x001ef3, 0x001ef3},
+ {0x001ef5, 0x001ef5},
+ {0x001ef7, 0x001ef7},
+ {0x001ef9, 0x001ef9},
+ {0x001efb, 0x001efb},
+ {0x001efd, 0x001efd},
+ {0x001eff, 0x001f07},
+ {0x001f10, 0x001f15},
+ {0x001f20, 0x001f27},
+ {0x001f30, 0x001f37},
+ {0x001f40, 0x001f45},
+ {0x001f50, 0x001f57},
+ {0x001f60, 0x001f67},
+ {0x001f70, 0x001f7d},
+ {0x001f80, 0x001f87},
+ {0x001f90, 0x001f97},
+ {0x001fa0, 0x001fa7},
+ {0x001fb0, 0x001fb4},
+ {0x001fb6, 0x001fb7},
+ {0x001fbe, 0x001fbe},
+ {0x001fc2, 0x001fc4},
+ {0x001fc6, 0x001fc7},
+ {0x001fd0, 0x001fd3},
+ {0x001fd6, 0x001fd7},
+ {0x001fe0, 0x001fe7},
+ {0x001ff2, 0x001ff4},
+ {0x001ff6, 0x001ff7},
+ {0x002071, 0x002071},
+ {0x00207f, 0x00207f},
+ {0x002090, 0x00209c},
+ {0x00210a, 0x00210a},
+ {0x00210e, 0x00210f},
+ {0x002113, 0x002113},
+ {0x00212f, 0x00212f},
+ {0x002134, 0x002134},
+ {0x002139, 0x002139},
+ {0x00213c, 0x00213d},
+ {0x002146, 0x002149},
+ {0x00214e, 0x00214e},
+ {0x002170, 0x00217f},
+ {0x002184, 0x002184},
+ {0x0024d0, 0x0024e9},
+ {0x002c30, 0x002c5f},
+ {0x002c61, 0x002c61},
+ {0x002c65, 0x002c66},
+ {0x002c68, 0x002c68},
+ {0x002c6a, 0x002c6a},
+ {0x002c6c, 0x002c6c},
+ {0x002c71, 0x002c71},
+ {0x002c73, 0x002c74},
+ {0x002c76, 0x002c7b},
+ {0x002c7c, 0x002c7d},
+ {0x002c81, 0x002c81},
+ {0x002c83, 0x002c83},
+ {0x002c85, 0x002c85},
+ {0x002c87, 0x002c87},
+ {0x002c89, 0x002c89},
+ {0x002c8b, 0x002c8b},
+ {0x002c8d, 0x002c8d},
+ {0x002c8f, 0x002c8f},
+ {0x002c91, 0x002c91},
+ {0x002c93, 0x002c93},
+ {0x002c95, 0x002c95},
+ {0x002c97, 0x002c97},
+ {0x002c99, 0x002c99},
+ {0x002c9b, 0x002c9b},
+ {0x002c9d, 0x002c9d},
+ {0x002c9f, 0x002c9f},
+ {0x002ca1, 0x002ca1},
+ {0x002ca3, 0x002ca3},
+ {0x002ca5, 0x002ca5},
+ {0x002ca7, 0x002ca7},
+ {0x002ca9, 0x002ca9},
+ {0x002cab, 0x002cab},
+ {0x002cad, 0x002cad},
+ {0x002caf, 0x002caf},
+ {0x002cb1, 0x002cb1},
+ {0x002cb3, 0x002cb3},
+ {0x002cb5, 0x002cb5},
+ {0x002cb7, 0x002cb7},
+ {0x002cb9, 0x002cb9},
+ {0x002cbb, 0x002cbb},
+ {0x002cbd, 0x002cbd},
+ {0x002cbf, 0x002cbf},
+ {0x002cc1, 0x002cc1},
+ {0x002cc3, 0x002cc3},
+ {0x002cc5, 0x002cc5},
+ {0x002cc7, 0x002cc7},
+ {0x002cc9, 0x002cc9},
+ {0x002ccb, 0x002ccb},
+ {0x002ccd, 0x002ccd},
+ {0x002ccf, 0x002ccf},
+ {0x002cd1, 0x002cd1},
+ {0x002cd3, 0x002cd3},
+ {0x002cd5, 0x002cd5},
+ {0x002cd7, 0x002cd7},
+ {0x002cd9, 0x002cd9},
+ {0x002cdb, 0x002cdb},
+ {0x002cdd, 0x002cdd},
+ {0x002cdf, 0x002cdf},
+ {0x002ce1, 0x002ce1},
+ {0x002ce3, 0x002ce4},
+ {0x002cec, 0x002cec},
+ {0x002cee, 0x002cee},
+ {0x002cf3, 0x002cf3},
+ {0x002d00, 0x002d25},
+ {0x002d27, 0x002d27},
+ {0x002d2d, 0x002d2d},
+ {0x00a641, 0x00a641},
+ {0x00a643, 0x00a643},
+ {0x00a645, 0x00a645},
+ {0x00a647, 0x00a647},
+ {0x00a649, 0x00a649},
+ {0x00a64b, 0x00a64b},
+ {0x00a64d, 0x00a64d},
+ {0x00a64f, 0x00a64f},
+ {0x00a651, 0x00a651},
+ {0x00a653, 0x00a653},
+ {0x00a655, 0x00a655},
+ {0x00a657, 0x00a657},
+ {0x00a659, 0x00a659},
+ {0x00a65b, 0x00a65b},
+ {0x00a65d, 0x00a65d},
+ {0x00a65f, 0x00a65f},
+ {0x00a661, 0x00a661},
+ {0x00a663, 0x00a663},
+ {0x00a665, 0x00a665},
+ {0x00a667, 0x00a667},
+ {0x00a669, 0x00a669},
+ {0x00a66b, 0x00a66b},
+ {0x00a66d, 0x00a66d},
+ {0x00a681, 0x00a681},
+ {0x00a683, 0x00a683},
+ {0x00a685, 0x00a685},
+ {0x00a687, 0x00a687},
+ {0x00a689, 0x00a689},
+ {0x00a68b, 0x00a68b},
+ {0x00a68d, 0x00a68d},
+ {0x00a68f, 0x00a68f},
+ {0x00a691, 0x00a691},
+ {0x00a693, 0x00a693},
+ {0x00a695, 0x00a695},
+ {0x00a697, 0x00a697},
+ {0x00a699, 0x00a699},
+ {0x00a69b, 0x00a69b},
+ {0x00a69c, 0x00a69d},
+ {0x00a723, 0x00a723},
+ {0x00a725, 0x00a725},
+ {0x00a727, 0x00a727},
+ {0x00a729, 0x00a729},
+ {0x00a72b, 0x00a72b},
+ {0x00a72d, 0x00a72d},
+ {0x00a72f, 0x00a731},
+ {0x00a733, 0x00a733},
+ {0x00a735, 0x00a735},
+ {0x00a737, 0x00a737},
+ {0x00a739, 0x00a739},
+ {0x00a73b, 0x00a73b},
+ {0x00a73d, 0x00a73d},
+ {0x00a73f, 0x00a73f},
+ {0x00a741, 0x00a741},
+ {0x00a743, 0x00a743},
+ {0x00a745, 0x00a745},
+ {0x00a747, 0x00a747},
+ {0x00a749, 0x00a749},
+ {0x00a74b, 0x00a74b},
+ {0x00a74d, 0x00a74d},
+ {0x00a74f, 0x00a74f},
+ {0x00a751, 0x00a751},
+ {0x00a753, 0x00a753},
+ {0x00a755, 0x00a755},
+ {0x00a757, 0x00a757},
+ {0x00a759, 0x00a759},
+ {0x00a75b, 0x00a75b},
+ {0x00a75d, 0x00a75d},
+ {0x00a75f, 0x00a75f},
+ {0x00a761, 0x00a761},
+ {0x00a763, 0x00a763},
+ {0x00a765, 0x00a765},
+ {0x00a767, 0x00a767},
+ {0x00a769, 0x00a769},
+ {0x00a76b, 0x00a76b},
+ {0x00a76d, 0x00a76d},
+ {0x00a76f, 0x00a76f},
+ {0x00a770, 0x00a770},
+ {0x00a771, 0x00a778},
+ {0x00a77a, 0x00a77a},
+ {0x00a77c, 0x00a77c},
+ {0x00a77f, 0x00a77f},
+ {0x00a781, 0x00a781},
+ {0x00a783, 0x00a783},
+ {0x00a785, 0x00a785},
+ {0x00a787, 0x00a787},
+ {0x00a78c, 0x00a78c},
+ {0x00a78e, 0x00a78e},
+ {0x00a791, 0x00a791},
+ {0x00a793, 0x00a795},
+ {0x00a797, 0x00a797},
+ {0x00a799, 0x00a799},
+ {0x00a79b, 0x00a79b},
+ {0x00a79d, 0x00a79d},
+ {0x00a79f, 0x00a79f},
+ {0x00a7a1, 0x00a7a1},
+ {0x00a7a3, 0x00a7a3},
+ {0x00a7a5, 0x00a7a5},
+ {0x00a7a7, 0x00a7a7},
+ {0x00a7a9, 0x00a7a9},
+ {0x00a7af, 0x00a7af},
+ {0x00a7b5, 0x00a7b5},
+ {0x00a7b7, 0x00a7b7},
+ {0x00a7b9, 0x00a7b9},
+ {0x00a7bb, 0x00a7bb},
+ {0x00a7bd, 0x00a7bd},
+ {0x00a7bf, 0x00a7bf},
+ {0x00a7c1, 0x00a7c1},
+ {0x00a7c3, 0x00a7c3},
+ {0x00a7c8, 0x00a7c8},
+ {0x00a7ca, 0x00a7ca},
+ {0x00a7d1, 0x00a7d1},
+ {0x00a7d3, 0x00a7d3},
+ {0x00a7d5, 0x00a7d5},
+ {0x00a7d7, 0x00a7d7},
+ {0x00a7d9, 0x00a7d9},
+ {0x00a7f2, 0x00a7f4},
+ {0x00a7f6, 0x00a7f6},
+ {0x00a7f8, 0x00a7f9},
+ {0x00a7fa, 0x00a7fa},
+ {0x00ab30, 0x00ab5a},
+ {0x00ab5c, 0x00ab5f},
+ {0x00ab60, 0x00ab68},
+ {0x00ab69, 0x00ab69},
+ {0x00ab70, 0x00abbf},
+ {0x00fb00, 0x00fb06},
+ {0x00fb13, 0x00fb17},
+ {0x00ff41, 0x00ff5a},
+ {0x010428, 0x01044f},
+ {0x0104d8, 0x0104fb},
+ {0x010597, 0x0105a1},
+ {0x0105a3, 0x0105b1},
+ {0x0105b3, 0x0105b9},
+ {0x0105bb, 0x0105bc},
+ {0x010780, 0x010780},
+ {0x010783, 0x010785},
+ {0x010787, 0x0107b0},
+ {0x0107b2, 0x0107ba},
+ {0x010cc0, 0x010cf2},
+ {0x0118c0, 0x0118df},
+ {0x016e60, 0x016e7f},
+ {0x01d41a, 0x01d433},
+ {0x01d44e, 0x01d454},
+ {0x01d456, 0x01d467},
+ {0x01d482, 0x01d49b},
+ {0x01d4b6, 0x01d4b9},
+ {0x01d4bb, 0x01d4bb},
+ {0x01d4bd, 0x01d4c3},
+ {0x01d4c5, 0x01d4cf},
+ {0x01d4ea, 0x01d503},
+ {0x01d51e, 0x01d537},
+ {0x01d552, 0x01d56b},
+ {0x01d586, 0x01d59f},
+ {0x01d5ba, 0x01d5d3},
+ {0x01d5ee, 0x01d607},
+ {0x01d622, 0x01d63b},
+ {0x01d656, 0x01d66f},
+ {0x01d68a, 0x01d6a5},
+ {0x01d6c2, 0x01d6da},
+ {0x01d6dc, 0x01d6e1},
+ {0x01d6fc, 0x01d714},
+ {0x01d716, 0x01d71b},
+ {0x01d736, 0x01d74e},
+ {0x01d750, 0x01d755},
+ {0x01d770, 0x01d788},
+ {0x01d78a, 0x01d78f},
+ {0x01d7aa, 0x01d7c2},
+ {0x01d7c4, 0x01d7c9},
+ {0x01d7cb, 0x01d7cb},
+ {0x01df00, 0x01df09},
+ {0x01df0b, 0x01df1e},
+ {0x01df25, 0x01df2a},
+ {0x01e030, 0x01e06d},
+ {0x01e922, 0x01e943}
+};
+
+/* table of Unicode codepoint ranges of Uppercase characters */
+static const pg_unicode_range unicode_uppercase[651] =
+{
+ {0x000041, 0x00005a},
+ {0x0000c0, 0x0000d6},
+ {0x0000d8, 0x0000de},
+ {0x000100, 0x000100},
+ {0x000102, 0x000102},
+ {0x000104, 0x000104},
+ {0x000106, 0x000106},
+ {0x000108, 0x000108},
+ {0x00010a, 0x00010a},
+ {0x00010c, 0x00010c},
+ {0x00010e, 0x00010e},
+ {0x000110, 0x000110},
+ {0x000112, 0x000112},
+ {0x000114, 0x000114},
+ {0x000116, 0x000116},
+ {0x000118, 0x000118},
+ {0x00011a, 0x00011a},
+ {0x00011c, 0x00011c},
+ {0x00011e, 0x00011e},
+ {0x000120, 0x000120},
+ {0x000122, 0x000122},
+ {0x000124, 0x000124},
+ {0x000126, 0x000126},
+ {0x000128, 0x000128},
+ {0x00012a, 0x00012a},
+ {0x00012c, 0x00012c},
+ {0x00012e, 0x00012e},
+ {0x000130, 0x000130},
+ {0x000132, 0x000132},
+ {0x000134, 0x000134},
+ {0x000136, 0x000136},
+ {0x000139, 0x000139},
+ {0x00013b, 0x00013b},
+ {0x00013d, 0x00013d},
+ {0x00013f, 0x00013f},
+ {0x000141, 0x000141},
+ {0x000143, 0x000143},
+ {0x000145, 0x000145},
+ {0x000147, 0x000147},
+ {0x00014a, 0x00014a},
+ {0x00014c, 0x00014c},
+ {0x00014e, 0x00014e},
+ {0x000150, 0x000150},
+ {0x000152, 0x000152},
+ {0x000154, 0x000154},
+ {0x000156, 0x000156},
+ {0x000158, 0x000158},
+ {0x00015a, 0x00015a},
+ {0x00015c, 0x00015c},
+ {0x00015e, 0x00015e},
+ {0x000160, 0x000160},
+ {0x000162, 0x000162},
+ {0x000164, 0x000164},
+ {0x000166, 0x000166},
+ {0x000168, 0x000168},
+ {0x00016a, 0x00016a},
+ {0x00016c, 0x00016c},
+ {0x00016e, 0x00016e},
+ {0x000170, 0x000170},
+ {0x000172, 0x000172},
+ {0x000174, 0x000174},
+ {0x000176, 0x000176},
+ {0x000178, 0x000179},
+ {0x00017b, 0x00017b},
+ {0x00017d, 0x00017d},
+ {0x000181, 0x000182},
+ {0x000184, 0x000184},
+ {0x000186, 0x000187},
+ {0x000189, 0x00018b},
+ {0x00018e, 0x000191},
+ {0x000193, 0x000194},
+ {0x000196, 0x000198},
+ {0x00019c, 0x00019d},
+ {0x00019f, 0x0001a0},
+ {0x0001a2, 0x0001a2},
+ {0x0001a4, 0x0001a4},
+ {0x0001a6, 0x0001a7},
+ {0x0001a9, 0x0001a9},
+ {0x0001ac, 0x0001ac},
+ {0x0001ae, 0x0001af},
+ {0x0001b1, 0x0001b3},
+ {0x0001b5, 0x0001b5},
+ {0x0001b7, 0x0001b8},
+ {0x0001bc, 0x0001bc},
+ {0x0001c4, 0x0001c4},
+ {0x0001c7, 0x0001c7},
+ {0x0001ca, 0x0001ca},
+ {0x0001cd, 0x0001cd},
+ {0x0001cf, 0x0001cf},
+ {0x0001d1, 0x0001d1},
+ {0x0001d3, 0x0001d3},
+ {0x0001d5, 0x0001d5},
+ {0x0001d7, 0x0001d7},
+ {0x0001d9, 0x0001d9},
+ {0x0001db, 0x0001db},
+ {0x0001de, 0x0001de},
+ {0x0001e0, 0x0001e0},
+ {0x0001e2, 0x0001e2},
+ {0x0001e4, 0x0001e4},
+ {0x0001e6, 0x0001e6},
+ {0x0001e8, 0x0001e8},
+ {0x0001ea, 0x0001ea},
+ {0x0001ec, 0x0001ec},
+ {0x0001ee, 0x0001ee},
+ {0x0001f1, 0x0001f1},
+ {0x0001f4, 0x0001f4},
+ {0x0001f6, 0x0001f8},
+ {0x0001fa, 0x0001fa},
+ {0x0001fc, 0x0001fc},
+ {0x0001fe, 0x0001fe},
+ {0x000200, 0x000200},
+ {0x000202, 0x000202},
+ {0x000204, 0x000204},
+ {0x000206, 0x000206},
+ {0x000208, 0x000208},
+ {0x00020a, 0x00020a},
+ {0x00020c, 0x00020c},
+ {0x00020e, 0x00020e},
+ {0x000210, 0x000210},
+ {0x000212, 0x000212},
+ {0x000214, 0x000214},
+ {0x000216, 0x000216},
+ {0x000218, 0x000218},
+ {0x00021a, 0x00021a},
+ {0x00021c, 0x00021c},
+ {0x00021e, 0x00021e},
+ {0x000220, 0x000220},
+ {0x000222, 0x000222},
+ {0x000224, 0x000224},
+ {0x000226, 0x000226},
+ {0x000228, 0x000228},
+ {0x00022a, 0x00022a},
+ {0x00022c, 0x00022c},
+ {0x00022e, 0x00022e},
+ {0x000230, 0x000230},
+ {0x000232, 0x000232},
+ {0x00023a, 0x00023b},
+ {0x00023d, 0x00023e},
+ {0x000241, 0x000241},
+ {0x000243, 0x000246},
+ {0x000248, 0x000248},
+ {0x00024a, 0x00024a},
+ {0x00024c, 0x00024c},
+ {0x00024e, 0x00024e},
+ {0x000370, 0x000370},
+ {0x000372, 0x000372},
+ {0x000376, 0x000376},
+ {0x00037f, 0x00037f},
+ {0x000386, 0x000386},
+ {0x000388, 0x00038a},
+ {0x00038c, 0x00038c},
+ {0x00038e, 0x00038f},
+ {0x000391, 0x0003a1},
+ {0x0003a3, 0x0003ab},
+ {0x0003cf, 0x0003cf},
+ {0x0003d2, 0x0003d4},
+ {0x0003d8, 0x0003d8},
+ {0x0003da, 0x0003da},
+ {0x0003dc, 0x0003dc},
+ {0x0003de, 0x0003de},
+ {0x0003e0, 0x0003e0},
+ {0x0003e2, 0x0003e2},
+ {0x0003e4, 0x0003e4},
+ {0x0003e6, 0x0003e6},
+ {0x0003e8, 0x0003e8},
+ {0x0003ea, 0x0003ea},
+ {0x0003ec, 0x0003ec},
+ {0x0003ee, 0x0003ee},
+ {0x0003f4, 0x0003f4},
+ {0x0003f7, 0x0003f7},
+ {0x0003f9, 0x0003fa},
+ {0x0003fd, 0x00042f},
+ {0x000460, 0x000460},
+ {0x000462, 0x000462},
+ {0x000464, 0x000464},
+ {0x000466, 0x000466},
+ {0x000468, 0x000468},
+ {0x00046a, 0x00046a},
+ {0x00046c, 0x00046c},
+ {0x00046e, 0x00046e},
+ {0x000470, 0x000470},
+ {0x000472, 0x000472},
+ {0x000474, 0x000474},
+ {0x000476, 0x000476},
+ {0x000478, 0x000478},
+ {0x00047a, 0x00047a},
+ {0x00047c, 0x00047c},
+ {0x00047e, 0x00047e},
+ {0x000480, 0x000480},
+ {0x00048a, 0x00048a},
+ {0x00048c, 0x00048c},
+ {0x00048e, 0x00048e},
+ {0x000490, 0x000490},
+ {0x000492, 0x000492},
+ {0x000494, 0x000494},
+ {0x000496, 0x000496},
+ {0x000498, 0x000498},
+ {0x00049a, 0x00049a},
+ {0x00049c, 0x00049c},
+ {0x00049e, 0x00049e},
+ {0x0004a0, 0x0004a0},
+ {0x0004a2, 0x0004a2},
+ {0x0004a4, 0x0004a4},
+ {0x0004a6, 0x0004a6},
+ {0x0004a8, 0x0004a8},
+ {0x0004aa, 0x0004aa},
+ {0x0004ac, 0x0004ac},
+ {0x0004ae, 0x0004ae},
+ {0x0004b0, 0x0004b0},
+ {0x0004b2, 0x0004b2},
+ {0x0004b4, 0x0004b4},
+ {0x0004b6, 0x0004b6},
+ {0x0004b8, 0x0004b8},
+ {0x0004ba, 0x0004ba},
+ {0x0004bc, 0x0004bc},
+ {0x0004be, 0x0004be},
+ {0x0004c0, 0x0004c1},
+ {0x0004c3, 0x0004c3},
+ {0x0004c5, 0x0004c5},
+ {0x0004c7, 0x0004c7},
+ {0x0004c9, 0x0004c9},
+ {0x0004cb, 0x0004cb},
+ {0x0004cd, 0x0004cd},
+ {0x0004d0, 0x0004d0},
+ {0x0004d2, 0x0004d2},
+ {0x0004d4, 0x0004d4},
+ {0x0004d6, 0x0004d6},
+ {0x0004d8, 0x0004d8},
+ {0x0004da, 0x0004da},
+ {0x0004dc, 0x0004dc},
+ {0x0004de, 0x0004de},
+ {0x0004e0, 0x0004e0},
+ {0x0004e2, 0x0004e2},
+ {0x0004e4, 0x0004e4},
+ {0x0004e6, 0x0004e6},
+ {0x0004e8, 0x0004e8},
+ {0x0004ea, 0x0004ea},
+ {0x0004ec, 0x0004ec},
+ {0x0004ee, 0x0004ee},
+ {0x0004f0, 0x0004f0},
+ {0x0004f2, 0x0004f2},
+ {0x0004f4, 0x0004f4},
+ {0x0004f6, 0x0004f6},
+ {0x0004f8, 0x0004f8},
+ {0x0004fa, 0x0004fa},
+ {0x0004fc, 0x0004fc},
+ {0x0004fe, 0x0004fe},
+ {0x000500, 0x000500},
+ {0x000502, 0x000502},
+ {0x000504, 0x000504},
+ {0x000506, 0x000506},
+ {0x000508, 0x000508},
+ {0x00050a, 0x00050a},
+ {0x00050c, 0x00050c},
+ {0x00050e, 0x00050e},
+ {0x000510, 0x000510},
+ {0x000512, 0x000512},
+ {0x000514, 0x000514},
+ {0x000516, 0x000516},
+ {0x000518, 0x000518},
+ {0x00051a, 0x00051a},
+ {0x00051c, 0x00051c},
+ {0x00051e, 0x00051e},
+ {0x000520, 0x000520},
+ {0x000522, 0x000522},
+ {0x000524, 0x000524},
+ {0x000526, 0x000526},
+ {0x000528, 0x000528},
+ {0x00052a, 0x00052a},
+ {0x00052c, 0x00052c},
+ {0x00052e, 0x00052e},
+ {0x000531, 0x000556},
+ {0x0010a0, 0x0010c5},
+ {0x0010c7, 0x0010c7},
+ {0x0010cd, 0x0010cd},
+ {0x0013a0, 0x0013f5},
+ {0x001c90, 0x001cba},
+ {0x001cbd, 0x001cbf},
+ {0x001e00, 0x001e00},
+ {0x001e02, 0x001e02},
+ {0x001e04, 0x001e04},
+ {0x001e06, 0x001e06},
+ {0x001e08, 0x001e08},
+ {0x001e0a, 0x001e0a},
+ {0x001e0c, 0x001e0c},
+ {0x001e0e, 0x001e0e},
+ {0x001e10, 0x001e10},
+ {0x001e12, 0x001e12},
+ {0x001e14, 0x001e14},
+ {0x001e16, 0x001e16},
+ {0x001e18, 0x001e18},
+ {0x001e1a, 0x001e1a},
+ {0x001e1c, 0x001e1c},
+ {0x001e1e, 0x001e1e},
+ {0x001e20, 0x001e20},
+ {0x001e22, 0x001e22},
+ {0x001e24, 0x001e24},
+ {0x001e26, 0x001e26},
+ {0x001e28, 0x001e28},
+ {0x001e2a, 0x001e2a},
+ {0x001e2c, 0x001e2c},
+ {0x001e2e, 0x001e2e},
+ {0x001e30, 0x001e30},
+ {0x001e32, 0x001e32},
+ {0x001e34, 0x001e34},
+ {0x001e36, 0x001e36},
+ {0x001e38, 0x001e38},
+ {0x001e3a, 0x001e3a},
+ {0x001e3c, 0x001e3c},
+ {0x001e3e, 0x001e3e},
+ {0x001e40, 0x001e40},
+ {0x001e42, 0x001e42},
+ {0x001e44, 0x001e44},
+ {0x001e46, 0x001e46},
+ {0x001e48, 0x001e48},
+ {0x001e4a, 0x001e4a},
+ {0x001e4c, 0x001e4c},
+ {0x001e4e, 0x001e4e},
+ {0x001e50, 0x001e50},
+ {0x001e52, 0x001e52},
+ {0x001e54, 0x001e54},
+ {0x001e56, 0x001e56},
+ {0x001e58, 0x001e58},
+ {0x001e5a, 0x001e5a},
+ {0x001e5c, 0x001e5c},
+ {0x001e5e, 0x001e5e},
+ {0x001e60, 0x001e60},
+ {0x001e62, 0x001e62},
+ {0x001e64, 0x001e64},
+ {0x001e66, 0x001e66},
+ {0x001e68, 0x001e68},
+ {0x001e6a, 0x001e6a},
+ {0x001e6c, 0x001e6c},
+ {0x001e6e, 0x001e6e},
+ {0x001e70, 0x001e70},
+ {0x001e72, 0x001e72},
+ {0x001e74, 0x001e74},
+ {0x001e76, 0x001e76},
+ {0x001e78, 0x001e78},
+ {0x001e7a, 0x001e7a},
+ {0x001e7c, 0x001e7c},
+ {0x001e7e, 0x001e7e},
+ {0x001e80, 0x001e80},
+ {0x001e82, 0x001e82},
+ {0x001e84, 0x001e84},
+ {0x001e86, 0x001e86},
+ {0x001e88, 0x001e88},
+ {0x001e8a, 0x001e8a},
+ {0x001e8c, 0x001e8c},
+ {0x001e8e, 0x001e8e},
+ {0x001e90, 0x001e90},
+ {0x001e92, 0x001e92},
+ {0x001e94, 0x001e94},
+ {0x001e9e, 0x001e9e},
+ {0x001ea0, 0x001ea0},
+ {0x001ea2, 0x001ea2},
+ {0x001ea4, 0x001ea4},
+ {0x001ea6, 0x001ea6},
+ {0x001ea8, 0x001ea8},
+ {0x001eaa, 0x001eaa},
+ {0x001eac, 0x001eac},
+ {0x001eae, 0x001eae},
+ {0x001eb0, 0x001eb0},
+ {0x001eb2, 0x001eb2},
+ {0x001eb4, 0x001eb4},
+ {0x001eb6, 0x001eb6},
+ {0x001eb8, 0x001eb8},
+ {0x001eba, 0x001eba},
+ {0x001ebc, 0x001ebc},
+ {0x001ebe, 0x001ebe},
+ {0x001ec0, 0x001ec0},
+ {0x001ec2, 0x001ec2},
+ {0x001ec4, 0x001ec4},
+ {0x001ec6, 0x001ec6},
+ {0x001ec8, 0x001ec8},
+ {0x001eca, 0x001eca},
+ {0x001ecc, 0x001ecc},
+ {0x001ece, 0x001ece},
+ {0x001ed0, 0x001ed0},
+ {0x001ed2, 0x001ed2},
+ {0x001ed4, 0x001ed4},
+ {0x001ed6, 0x001ed6},
+ {0x001ed8, 0x001ed8},
+ {0x001eda, 0x001eda},
+ {0x001edc, 0x001edc},
+ {0x001ede, 0x001ede},
+ {0x001ee0, 0x001ee0},
+ {0x001ee2, 0x001ee2},
+ {0x001ee4, 0x001ee4},
+ {0x001ee6, 0x001ee6},
+ {0x001ee8, 0x001ee8},
+ {0x001eea, 0x001eea},
+ {0x001eec, 0x001eec},
+ {0x001eee, 0x001eee},
+ {0x001ef0, 0x001ef0},
+ {0x001ef2, 0x001ef2},
+ {0x001ef4, 0x001ef4},
+ {0x001ef6, 0x001ef6},
+ {0x001ef8, 0x001ef8},
+ {0x001efa, 0x001efa},
+ {0x001efc, 0x001efc},
+ {0x001efe, 0x001efe},
+ {0x001f08, 0x001f0f},
+ {0x001f18, 0x001f1d},
+ {0x001f28, 0x001f2f},
+ {0x001f38, 0x001f3f},
+ {0x001f48, 0x001f4d},
+ {0x001f59, 0x001f59},
+ {0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f5f},
+ {0x001f68, 0x001f6f},
+ {0x001fb8, 0x001fbb},
+ {0x001fc8, 0x001fcb},
+ {0x001fd8, 0x001fdb},
+ {0x001fe8, 0x001fec},
+ {0x001ff8, 0x001ffb},
+ {0x002102, 0x002102},
+ {0x002107, 0x002107},
+ {0x00210b, 0x00210d},
+ {0x002110, 0x002112},
+ {0x002115, 0x002115},
+ {0x002119, 0x00211d},
+ {0x002124, 0x002124},
+ {0x002126, 0x002126},
+ {0x002128, 0x002128},
+ {0x00212a, 0x00212d},
+ {0x002130, 0x002133},
+ {0x00213e, 0x00213f},
+ {0x002145, 0x002145},
+ {0x002160, 0x00216f},
+ {0x002183, 0x002183},
+ {0x0024b6, 0x0024cf},
+ {0x002c00, 0x002c2f},
+ {0x002c60, 0x002c60},
+ {0x002c62, 0x002c64},
+ {0x002c67, 0x002c67},
+ {0x002c69, 0x002c69},
+ {0x002c6b, 0x002c6b},
+ {0x002c6d, 0x002c70},
+ {0x002c72, 0x002c72},
+ {0x002c75, 0x002c75},
+ {0x002c7e, 0x002c80},
+ {0x002c82, 0x002c82},
+ {0x002c84, 0x002c84},
+ {0x002c86, 0x002c86},
+ {0x002c88, 0x002c88},
+ {0x002c8a, 0x002c8a},
+ {0x002c8c, 0x002c8c},
+ {0x002c8e, 0x002c8e},
+ {0x002c90, 0x002c90},
+ {0x002c92, 0x002c92},
+ {0x002c94, 0x002c94},
+ {0x002c96, 0x002c96},
+ {0x002c98, 0x002c98},
+ {0x002c9a, 0x002c9a},
+ {0x002c9c, 0x002c9c},
+ {0x002c9e, 0x002c9e},
+ {0x002ca0, 0x002ca0},
+ {0x002ca2, 0x002ca2},
+ {0x002ca4, 0x002ca4},
+ {0x002ca6, 0x002ca6},
+ {0x002ca8, 0x002ca8},
+ {0x002caa, 0x002caa},
+ {0x002cac, 0x002cac},
+ {0x002cae, 0x002cae},
+ {0x002cb0, 0x002cb0},
+ {0x002cb2, 0x002cb2},
+ {0x002cb4, 0x002cb4},
+ {0x002cb6, 0x002cb6},
+ {0x002cb8, 0x002cb8},
+ {0x002cba, 0x002cba},
+ {0x002cbc, 0x002cbc},
+ {0x002cbe, 0x002cbe},
+ {0x002cc0, 0x002cc0},
+ {0x002cc2, 0x002cc2},
+ {0x002cc4, 0x002cc4},
+ {0x002cc6, 0x002cc6},
+ {0x002cc8, 0x002cc8},
+ {0x002cca, 0x002cca},
+ {0x002ccc, 0x002ccc},
+ {0x002cce, 0x002cce},
+ {0x002cd0, 0x002cd0},
+ {0x002cd2, 0x002cd2},
+ {0x002cd4, 0x002cd4},
+ {0x002cd6, 0x002cd6},
+ {0x002cd8, 0x002cd8},
+ {0x002cda, 0x002cda},
+ {0x002cdc, 0x002cdc},
+ {0x002cde, 0x002cde},
+ {0x002ce0, 0x002ce0},
+ {0x002ce2, 0x002ce2},
+ {0x002ceb, 0x002ceb},
+ {0x002ced, 0x002ced},
+ {0x002cf2, 0x002cf2},
+ {0x00a640, 0x00a640},
+ {0x00a642, 0x00a642},
+ {0x00a644, 0x00a644},
+ {0x00a646, 0x00a646},
+ {0x00a648, 0x00a648},
+ {0x00a64a, 0x00a64a},
+ {0x00a64c, 0x00a64c},
+ {0x00a64e, 0x00a64e},
+ {0x00a650, 0x00a650},
+ {0x00a652, 0x00a652},
+ {0x00a654, 0x00a654},
+ {0x00a656, 0x00a656},
+ {0x00a658, 0x00a658},
+ {0x00a65a, 0x00a65a},
+ {0x00a65c, 0x00a65c},
+ {0x00a65e, 0x00a65e},
+ {0x00a660, 0x00a660},
+ {0x00a662, 0x00a662},
+ {0x00a664, 0x00a664},
+ {0x00a666, 0x00a666},
+ {0x00a668, 0x00a668},
+ {0x00a66a, 0x00a66a},
+ {0x00a66c, 0x00a66c},
+ {0x00a680, 0x00a680},
+ {0x00a682, 0x00a682},
+ {0x00a684, 0x00a684},
+ {0x00a686, 0x00a686},
+ {0x00a688, 0x00a688},
+ {0x00a68a, 0x00a68a},
+ {0x00a68c, 0x00a68c},
+ {0x00a68e, 0x00a68e},
+ {0x00a690, 0x00a690},
+ {0x00a692, 0x00a692},
+ {0x00a694, 0x00a694},
+ {0x00a696, 0x00a696},
+ {0x00a698, 0x00a698},
+ {0x00a69a, 0x00a69a},
+ {0x00a722, 0x00a722},
+ {0x00a724, 0x00a724},
+ {0x00a726, 0x00a726},
+ {0x00a728, 0x00a728},
+ {0x00a72a, 0x00a72a},
+ {0x00a72c, 0x00a72c},
+ {0x00a72e, 0x00a72e},
+ {0x00a732, 0x00a732},
+ {0x00a734, 0x00a734},
+ {0x00a736, 0x00a736},
+ {0x00a738, 0x00a738},
+ {0x00a73a, 0x00a73a},
+ {0x00a73c, 0x00a73c},
+ {0x00a73e, 0x00a73e},
+ {0x00a740, 0x00a740},
+ {0x00a742, 0x00a742},
+ {0x00a744, 0x00a744},
+ {0x00a746, 0x00a746},
+ {0x00a748, 0x00a748},
+ {0x00a74a, 0x00a74a},
+ {0x00a74c, 0x00a74c},
+ {0x00a74e, 0x00a74e},
+ {0x00a750, 0x00a750},
+ {0x00a752, 0x00a752},
+ {0x00a754, 0x00a754},
+ {0x00a756, 0x00a756},
+ {0x00a758, 0x00a758},
+ {0x00a75a, 0x00a75a},
+ {0x00a75c, 0x00a75c},
+ {0x00a75e, 0x00a75e},
+ {0x00a760, 0x00a760},
+ {0x00a762, 0x00a762},
+ {0x00a764, 0x00a764},
+ {0x00a766, 0x00a766},
+ {0x00a768, 0x00a768},
+ {0x00a76a, 0x00a76a},
+ {0x00a76c, 0x00a76c},
+ {0x00a76e, 0x00a76e},
+ {0x00a779, 0x00a779},
+ {0x00a77b, 0x00a77b},
+ {0x00a77d, 0x00a77e},
+ {0x00a780, 0x00a780},
+ {0x00a782, 0x00a782},
+ {0x00a784, 0x00a784},
+ {0x00a786, 0x00a786},
+ {0x00a78b, 0x00a78b},
+ {0x00a78d, 0x00a78d},
+ {0x00a790, 0x00a790},
+ {0x00a792, 0x00a792},
+ {0x00a796, 0x00a796},
+ {0x00a798, 0x00a798},
+ {0x00a79a, 0x00a79a},
+ {0x00a79c, 0x00a79c},
+ {0x00a79e, 0x00a79e},
+ {0x00a7a0, 0x00a7a0},
+ {0x00a7a2, 0x00a7a2},
+ {0x00a7a4, 0x00a7a4},
+ {0x00a7a6, 0x00a7a6},
+ {0x00a7a8, 0x00a7a8},
+ {0x00a7aa, 0x00a7ae},
+ {0x00a7b0, 0x00a7b4},
+ {0x00a7b6, 0x00a7b6},
+ {0x00a7b8, 0x00a7b8},
+ {0x00a7ba, 0x00a7ba},
+ {0x00a7bc, 0x00a7bc},
+ {0x00a7be, 0x00a7be},
+ {0x00a7c0, 0x00a7c0},
+ {0x00a7c2, 0x00a7c2},
+ {0x00a7c4, 0x00a7c7},
+ {0x00a7c9, 0x00a7c9},
+ {0x00a7d0, 0x00a7d0},
+ {0x00a7d6, 0x00a7d6},
+ {0x00a7d8, 0x00a7d8},
+ {0x00a7f5, 0x00a7f5},
+ {0x00ff21, 0x00ff3a},
+ {0x010400, 0x010427},
+ {0x0104b0, 0x0104d3},
+ {0x010570, 0x01057a},
+ {0x01057c, 0x01058a},
+ {0x01058c, 0x010592},
+ {0x010594, 0x010595},
+ {0x010c80, 0x010cb2},
+ {0x0118a0, 0x0118bf},
+ {0x016e40, 0x016e5f},
+ {0x01d400, 0x01d419},
+ {0x01d434, 0x01d44d},
+ {0x01d468, 0x01d481},
+ {0x01d49c, 0x01d49c},
+ {0x01d49e, 0x01d49f},
+ {0x01d4a2, 0x01d4a2},
+ {0x01d4a5, 0x01d4a6},
+ {0x01d4a9, 0x01d4ac},
+ {0x01d4ae, 0x01d4b5},
+ {0x01d4d0, 0x01d4e9},
+ {0x01d504, 0x01d505},
+ {0x01d507, 0x01d50a},
+ {0x01d50d, 0x01d514},
+ {0x01d516, 0x01d51c},
+ {0x01d538, 0x01d539},
+ {0x01d53b, 0x01d53e},
+ {0x01d540, 0x01d544},
+ {0x01d546, 0x01d546},
+ {0x01d54a, 0x01d550},
+ {0x01d56c, 0x01d585},
+ {0x01d5a0, 0x01d5b9},
+ {0x01d5d4, 0x01d5ed},
+ {0x01d608, 0x01d621},
+ {0x01d63c, 0x01d655},
+ {0x01d670, 0x01d689},
+ {0x01d6a8, 0x01d6c0},
+ {0x01d6e2, 0x01d6fa},
+ {0x01d71c, 0x01d734},
+ {0x01d756, 0x01d76e},
+ {0x01d790, 0x01d7a8},
+ {0x01d7ca, 0x01d7ca},
+ {0x01e900, 0x01e921},
+ {0x01f130, 0x01f149},
+ {0x01f150, 0x01f169},
+ {0x01f170, 0x01f189}
+};
+
+/* table of Unicode codepoint ranges of White_Space characters */
+static const pg_unicode_range unicode_white_space[11] =
+{
+ {0x000009, 0x00000d},
+ {0x000020, 0x000020},
+ {0x000085, 0x000085},
+ {0x0000a0, 0x0000a0},
+ {0x001680, 0x001680},
+ {0x002000, 0x00200a},
+ {0x002028, 0x002028},
+ {0x002029, 0x002029},
+ {0x00202f, 0x00202f},
+ {0x00205f, 0x00205f},
+ {0x003000, 0x003000}
+};
+
+/* table of Unicode codepoint ranges of Hex_Digit characters */
+static const pg_unicode_range unicode_hex_digit[6] =
+{
+ {0x000030, 0x000039},
+ {0x000041, 0x000046},
+ {0x000061, 0x000066},
+ {0x00ff10, 0x00ff19},
+ {0x00ff21, 0x00ff26},
+ {0x00ff41, 0x00ff46}
+};
+
+/* table of Unicode codepoint ranges of Join_Control characters */
+static const pg_unicode_range unicode_join_control[1] =
+{
+ {0x00200c, 0x00200d}
+};
--
2.34.1
[text/x-patch] v16-0003-Add-unicode-case-mapping-tables-and-functions.patch (145.0K, ../../[email protected]/3-v16-0003-Add-unicode-case-mapping-tables-and-functions.patch)
download | inline diff:
From 67741db46c5601dfdc273e2a0a9db22833fa91f8 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Mon, 30 Oct 2023 17:38:54 -0700
Subject: [PATCH v16 3/5] Add unicode case mapping tables and functions.
---
src/common/Makefile | 1 +
src/common/meson.build | 1 +
src/common/unicode/Makefile | 13 +-
src/common/unicode/case_test.c | 140 +
.../unicode/generate-unicode_case_table.pl | 134 +
src/common/unicode/meson.build | 31 +
src/common/unicode_case.c | 87 +
src/include/common/unicode_case.h | 23 +
src/include/common/unicode_case_table.h | 2994 +++++++++++++++++
9 files changed, 3422 insertions(+), 2 deletions(-)
create mode 100644 src/common/unicode/case_test.c
create mode 100644 src/common/unicode/generate-unicode_case_table.pl
create mode 100644 src/common/unicode_case.c
create mode 100644 src/include/common/unicode_case.h
create mode 100644 src/include/common/unicode_case_table.h
diff --git a/src/common/Makefile b/src/common/Makefile
index 2ba5069dca..3d83299432 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -78,6 +78,7 @@ OBJS_COMMON = \
scram-common.o \
string.o \
stringinfo.o \
+ unicode_case.o \
unicode_category.o \
unicode_norm.o \
username.o \
diff --git a/src/common/meson.build b/src/common/meson.build
index 4eb16024cb..de68e408fa 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -32,6 +32,7 @@ common_sources = files(
'scram-common.c',
'string.c',
'stringinfo.c',
+ 'unicode_case.c',
'unicode_category.c',
'unicode_norm.c',
'username.c',
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 27f0408d8b..8223d02375 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -21,8 +21,9 @@ CPPFLAGS += $(ICU_CFLAGS)
# By default, do nothing.
all:
-update-unicode: unicode_category_table.h unicode_east_asian_fw_table.h unicode_nonspacing_table.h unicode_norm_hashfunc.h unicode_norm_table.h unicode_normprops_table.h unicode_version.h
+update-unicode: unicode_case_table.h unicode_category_table.h unicode_east_asian_fw_table.h unicode_nonspacing_table.h unicode_norm_hashfunc.h unicode_norm_table.h unicode_normprops_table.h unicode_version.h
mv $^ $(top_srcdir)/src/include/common/
+ $(MAKE) case-check
$(MAKE) category-check
$(MAKE) normalization-check
@@ -35,6 +36,9 @@ CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.tx
unicode_version.h: generate-unicode_version.pl
$(PERL) $< --version $(UNICODE_VERSION)
+unicode_case_table.h: generate-unicode_case_table.pl UnicodeData.txt
+ $(PERL) $<
+
unicode_category_table.h: generate-unicode_category_table.pl DerivedCoreProperties.txt PropList.txt UnicodeData.txt
$(PERL) $<
@@ -55,12 +59,17 @@ unicode_normprops_table.h: generate-unicode_normprops_table.pl DerivedNormalizat
$(PERL) $^ >$@
# Test suite
+case-check: case_test
+ ./case_test
+
category-check: category_test
./category_test
normalization-check: norm_test
./norm_test
+case_test: case_test.o ../unicode_case.o | submake-common
+
category_test: category_test.o ../unicode_category.o | submake-common
norm_test: norm_test.o ../unicode_norm.o | submake-common
@@ -82,4 +91,4 @@ clean:
rm -f $(OBJS) category_test category_test.o norm_test norm_test.o
distclean: clean
- rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
+ rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_case_table.h unicode_category_table.h unicode_norm_table.h
diff --git a/src/common/unicode/case_test.c b/src/common/unicode/case_test.c
new file mode 100644
index 0000000000..4758a7f017
--- /dev/null
+++ b/src/common/unicode/case_test.c
@@ -0,0 +1,140 @@
+/*-------------------------------------------------------------------------
+ * case_test.c
+ * Program to test Unicode case mapping functions.
+ *
+ * Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/unicode/case_test.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include <locale.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <wctype.h>
+
+#ifdef USE_ICU
+#include <unicode/uchar.h>
+#endif
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
+#include "common/unicode_version.h"
+
+/*
+ * We expect that C.UTF-8 has the same CTYPE behavior as the simple unicode
+ * mappings, but that's not guaranteed. If there are failures in the libc
+ * test, that's useful information, but does not necessarily indicate a
+ * problem.
+ */
+#define LIBC_LOCALE "C.UTF-8"
+
+#ifdef USE_ICU
+
+/* use root locale for test */
+#define ICU_LOCALE "und"
+
+static void
+icu_test_simple(pg_wchar code)
+{
+ pg_wchar lower = unicode_lowercase_simple(code);
+ pg_wchar title = unicode_titlecase_simple(code);
+ pg_wchar upper = unicode_uppercase_simple(code);
+ pg_wchar iculower = u_tolower(code);
+ pg_wchar icutitle = u_totitle(code);
+ pg_wchar icuupper = u_toupper(code);
+
+ if (lower != iculower || title != icutitle || upper != icuupper)
+ {
+ printf("case_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("case_test: Postgres lower/title/upper: 0x%06x/0x%06x/0x%06x\n",
+ lower, title, upper);
+ printf("case_test: ICU lower/title/upper: 0x%06x/0x%06x/0x%06x\n",
+ iculower, icutitle, icuupper);
+ printf("\n");
+ exit(1);
+ }
+}
+#endif
+
+static void
+libc_test_simple(pg_wchar code)
+{
+ pg_wchar lower = unicode_lowercase_simple(code);
+ pg_wchar upper = unicode_uppercase_simple(code);
+ wchar_t libclower = towlower(code);
+ wchar_t libcupper = towupper(code);
+
+ if (lower != libclower || upper != libcupper)
+ {
+ printf("case_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("case_test: Postgres lower/upper: 0x%06x/0x%06x\n",
+ lower, upper);
+ printf("case_test: libc lower/upper: 0x%06x/0x%06x\n",
+ libclower, libcupper);
+ printf("\n");
+ exit(1);
+ }
+}
+
+/*
+ * Exhaustively compare case mappings with the results from libc and ICU.
+ */
+int
+main(int argc, char **argv)
+{
+ int libc_successful = 0;
+#ifdef USE_ICU
+ int icu_successful = 0;
+#endif
+ char *libc_locale;
+
+ libc_locale = setlocale(LC_CTYPE, LIBC_LOCALE);
+
+ printf("case_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
+#ifdef USE_ICU
+ printf("case_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
+#else
+ printf("case_test: ICU not available; skipping\n");
+#endif
+
+ if (libc_locale)
+ {
+ printf("case_test: comparing with libc locale \"%s\"\n", libc_locale);
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
+ {
+ pg_unicode_category category = unicode_category(code);
+
+ if (category != PG_U_UNASSIGNED && category != PG_U_SURROGATE)
+ {
+ libc_test_simple(code);
+ libc_successful++;
+ }
+ }
+ printf("case_test: libc simple mapping test: %d codepoints successful\n",
+ libc_successful);
+ }
+ else
+ printf("case_test: libc locale \"%s\" not available; skipping\n",
+ LIBC_LOCALE);
+
+#ifdef USE_ICU
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
+ {
+ pg_unicode_category category = unicode_category(code);
+
+ if (category != PG_U_UNASSIGNED && category != PG_U_SURROGATE)
+ {
+ icu_test_simple(code);
+ icu_successful++;
+ }
+ }
+ printf("case_test: ICU simple mapping test: %d codepoints successful\n",
+ icu_successful);
+#endif
+
+ exit(0);
+}
diff --git a/src/common/unicode/generate-unicode_case_table.pl b/src/common/unicode/generate-unicode_case_table.pl
new file mode 100644
index 0000000000..9bf1deb297
--- /dev/null
+++ b/src/common/unicode/generate-unicode_case_table.pl
@@ -0,0 +1,134 @@
+#!/usr/bin/perl
+#
+# Generate Unicode character case mappings. Does not include tailoring
+# or locale-specific mappings.
+#
+# Input: UnicodeData.txt
+# Output: unicode_case_table.h
+#
+# Copyright (c) 2000-2023, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+use Getopt::Long;
+
+use FindBin;
+use lib "$FindBin::RealBin/../../tools/";
+
+my $output_path = '.';
+
+GetOptions('outdir:s' => \$output_path);
+
+my $output_table_file = "$output_path/unicode_case_table.h";
+
+# The maximum number of codepoints that can result from case mapping
+# of a single character. See Unicode section 5.18 "Case Mappings".
+my $MAX_CASE_EXPANSION = 3;
+
+my $FH;
+
+my %simple = ();
+
+open($FH, '<', "$output_path/UnicodeData.txt")
+ or die "Could not open $output_path/UnicodeData.txt: $!.";
+while (my $line = <$FH>)
+{
+ my @elts = split(';', $line);
+ my $code = hex($elts[0]);
+ my $simple_uppercase = hex($elts[12] =~ s/^\s+|\s+$//rg);
+ my $simple_lowercase = hex($elts[13] =~ s/^\s+|\s+$//rg);
+ my $simple_titlecase = hex($elts[14] =~ s/^\s+|\s+$//rg);
+
+ die "codepoint $code out of range" if $code > 0x10FFFF;
+ die "Simple_Lowercase $code out of range" if $simple_lowercase > 0x10FFFF;
+ die "Simple_Titlecase $code out of range" if $simple_titlecase > 0x10FFFF;
+ die "Simple_Uppercase $code out of range" if $simple_uppercase > 0x10FFFF;
+
+ if ($simple_lowercase || $simple_titlecase || $simple_uppercase)
+ {
+ $simple{$code} = {Simple_Lowercase => ($simple_lowercase || $code),
+ Simple_Titlecase => ($simple_titlecase || $code),
+ Simple_Uppercase => ($simple_uppercase || $code)};
+ }
+}
+close $FH;
+
+# Start writing out the output files
+open my $OT, '>', $output_table_file
+ or die "Could not open output file $output_table_file: $!\n";
+
+# determine size of array given that codepoints <= 0x80 are dense and
+# the rest of the entries are sparse
+my $num_simple = 0x80;
+foreach my $code (sort { $a <=> $b } (keys %simple))
+{
+ $num_simple++ unless $code < 0x80;
+}
+
+print $OT <<"HEADER";
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case_table.h
+ * Case mapping and information table.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/unicode_case_table.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+/*
+ * File auto-generated by src/common/unicode/generate-unicode_case_table.pl,
+ * do not edit. There is deliberately not an #ifndef PG_UNICODE_CASE_TABLE_H
+ * here.
+ */
+
+#include "mb/pg_wchar.h"
+
+typedef struct
+{
+ pg_wchar codepoint; /* Unicode codepoint */
+ pg_wchar simple_lowercase;
+ pg_wchar simple_titlecase;
+ pg_wchar simple_uppercase;
+} pg_simple_case_map;
+
+/*
+ * Case mapping table. Dense for codepoints < 0x80 (enabling fast lookup),
+ * sparse for higher codepoints (requiring scan or binary search).
+ */
+static const pg_simple_case_map simple_case_map[$num_simple] =
+{
+HEADER
+
+my $firsttime = 1;
+
+printf $OT "\t/* begin dense entries for codepoints < 0x80 */\n";
+for (my $code = 0; $code < 0x80; $code++)
+{
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ my $lc = ($simple{$code}{Simple_Lowercase} || $code);
+ my $tc = ($simple{$code}{Simple_Titlecase} || $code);
+ my $uc = ($simple{$code}{Simple_Uppercase} || $code);
+ printf $OT "\t{0x%06x, 0x%06x, 0x%06x, 0x%06x}", $code, $lc, $tc, $uc;
+}
+printf $OT ",\n\n";
+
+$firsttime = 1;
+printf $OT "\t/* begin sparse entries for codepoints >= 0x80 */\n";
+foreach my $code (sort { $a <=> $b } (keys %simple))
+{
+ next unless $code >= 0x80; # already output above
+
+ printf $OT ",\n" unless $firsttime;
+ $firsttime = 0;
+
+ my $map = $simple{$code};
+ printf $OT "\t{0x%06x, 0x%06x, 0x%06x, 0x%06x}",
+ $code, $map->{Simple_Lowercase}, $map->{Simple_Titlecase}, $map->{Simple_Uppercase};
+}
+print $OT "\n};\n";
diff --git a/src/common/unicode/meson.build b/src/common/unicode/meson.build
index d7190bb8ca..b9a4181c32 100644
--- a/src/common/unicode/meson.build
+++ b/src/common/unicode/meson.build
@@ -24,6 +24,16 @@ endforeach
update_unicode_targets = []
+update_unicode_targets += \
+ custom_target('unicode_case_table.h',
+ input: [unicode_data['UnicodeData.txt']],
+ output: ['unicode_case_table.h'],
+ command: [
+ perl, files('generate-unicode_case_table.pl'),
+ '--outdir', '@OUTDIR@', '@INPUT@'],
+ build_by_default: false,
+ )
+
update_unicode_targets += \
custom_target('unicode_category_table.h',
input: [unicode_data['UnicodeData.txt'], unicode_data['DerivedCoreProperties.txt'], unicode_data['PropList.txt']],
@@ -92,6 +102,17 @@ norm_test_table = custom_target('norm_test_table.h',
inc = include_directories('.')
+case_test = executable('case_test',
+ ['case_test.c'],
+ dependencies: [frontend_port_code, icu],
+ include_directories: inc,
+ link_with: [common_static, pgport_static],
+ build_by_default: false,
+ kwargs: default_bin_args + {
+ 'install': false,
+ }
+)
+
category_test = executable('category_test',
['category_test.c'],
dependencies: [frontend_port_code, icu],
@@ -116,6 +137,16 @@ norm_test = executable('norm_test',
update_unicode_dep = []
+if not meson.is_cross_build()
+ update_unicode_dep += custom_target('case_test.run',
+ output: 'case_test.run',
+ input: update_unicode_targets,
+ command: [case_test, UNICODE_VERSION],
+ build_by_default: false,
+ build_always_stale: true,
+ )
+endif
+
if not meson.is_cross_build()
update_unicode_dep += custom_target('category_test.run',
output: 'category_test.run',
diff --git a/src/common/unicode_case.c b/src/common/unicode_case.c
new file mode 100644
index 0000000000..714e0453e1
--- /dev/null
+++ b/src/common/unicode_case.c
@@ -0,0 +1,87 @@
+/*-------------------------------------------------------------------------
+ * unicode_case.c
+ * Conversion to upper or lower case.
+ *
+ * Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/unicode_case.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/unicode_case.h"
+#include "common/unicode_case_table.h"
+
+/* find entry in simple case map, if any */
+static inline const pg_simple_case_map *
+find_case_map(pg_wchar ucs)
+{
+ int min = 0;
+ int mid;
+ int max = lengthof(simple_case_map) - 1;
+
+ /* all chars <= 0x80 are stored in array for fast lookup */
+ Assert(max >= 0x7f);
+ if (ucs < 0x80)
+ {
+ const pg_simple_case_map *map = &simple_case_map[ucs];
+ Assert(map->codepoint == ucs);
+ return map;
+ }
+
+ /* otherwise, binary search */
+ while (max >= min)
+ {
+ mid = (min + max) / 2;
+ if (ucs > simple_case_map[mid].codepoint)
+ min = mid + 1;
+ else if (ucs < simple_case_map[mid].codepoint)
+ max = mid - 1;
+ else
+ return &simple_case_map[mid];
+ }
+
+ return NULL;
+}
+
+/*
+ * Returns simple lowercase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_lowercase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_lowercase : ucs;
+}
+
+/*
+ * Returns simple titlecase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_titlecase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_titlecase : ucs;
+}
+
+/*
+ * Returns simple uppercase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_uppercase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_uppercase : ucs;
+}
diff --git a/src/include/common/unicode_case.h b/src/include/common/unicode_case.h
new file mode 100644
index 0000000000..fe5fe6af63
--- /dev/null
+++ b/src/include/common/unicode_case.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case.h
+ * Routines for converting character case.
+ *
+ * These definitions can be used by both frontend and backend code.
+ *
+ * Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * src/include/common/unicode_case.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef UNICODE_CASE_H
+#define UNICODE_CASE_H
+
+#include "mb/pg_wchar.h"
+
+pg_wchar unicode_lowercase_simple(pg_wchar ucs);
+pg_wchar unicode_titlecase_simple(pg_wchar ucs);
+pg_wchar unicode_uppercase_simple(pg_wchar ucs);
+
+#endif /* UNICODE_CASE_H */
diff --git a/src/include/common/unicode_case_table.h b/src/include/common/unicode_case_table.h
new file mode 100644
index 0000000000..7733457bec
--- /dev/null
+++ b/src/include/common/unicode_case_table.h
@@ -0,0 +1,2994 @@
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case_table.h
+ * Case mapping and information table.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/unicode_case_table.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+/*
+ * File auto-generated by src/common/unicode/generate-unicode_case_table.pl,
+ * do not edit. There is deliberately not an #ifndef PG_UNICODE_CASE_TABLE_H
+ * here.
+ */
+
+#include "mb/pg_wchar.h"
+
+typedef struct
+{
+ pg_wchar codepoint; /* Unicode codepoint */
+ pg_wchar simple_lowercase;
+ pg_wchar simple_titlecase;
+ pg_wchar simple_uppercase;
+} pg_simple_case_map;
+
+/*
+ * Case mapping table. Dense for codepoints < 0x80 (enabling fast lookup),
+ * sparse for higher codepoints (requiring scan or binary search).
+ */
+static const pg_simple_case_map simple_case_map[2955] =
+{
+ /* begin dense entries for codepoints < 0x80 */
+ {0x000000, 0x000000, 0x000000, 0x000000},
+ {0x000001, 0x000001, 0x000001, 0x000001},
+ {0x000002, 0x000002, 0x000002, 0x000002},
+ {0x000003, 0x000003, 0x000003, 0x000003},
+ {0x000004, 0x000004, 0x000004, 0x000004},
+ {0x000005, 0x000005, 0x000005, 0x000005},
+ {0x000006, 0x000006, 0x000006, 0x000006},
+ {0x000007, 0x000007, 0x000007, 0x000007},
+ {0x000008, 0x000008, 0x000008, 0x000008},
+ {0x000009, 0x000009, 0x000009, 0x000009},
+ {0x00000a, 0x00000a, 0x00000a, 0x00000a},
+ {0x00000b, 0x00000b, 0x00000b, 0x00000b},
+ {0x00000c, 0x00000c, 0x00000c, 0x00000c},
+ {0x00000d, 0x00000d, 0x00000d, 0x00000d},
+ {0x00000e, 0x00000e, 0x00000e, 0x00000e},
+ {0x00000f, 0x00000f, 0x00000f, 0x00000f},
+ {0x000010, 0x000010, 0x000010, 0x000010},
+ {0x000011, 0x000011, 0x000011, 0x000011},
+ {0x000012, 0x000012, 0x000012, 0x000012},
+ {0x000013, 0x000013, 0x000013, 0x000013},
+ {0x000014, 0x000014, 0x000014, 0x000014},
+ {0x000015, 0x000015, 0x000015, 0x000015},
+ {0x000016, 0x000016, 0x000016, 0x000016},
+ {0x000017, 0x000017, 0x000017, 0x000017},
+ {0x000018, 0x000018, 0x000018, 0x000018},
+ {0x000019, 0x000019, 0x000019, 0x000019},
+ {0x00001a, 0x00001a, 0x00001a, 0x00001a},
+ {0x00001b, 0x00001b, 0x00001b, 0x00001b},
+ {0x00001c, 0x00001c, 0x00001c, 0x00001c},
+ {0x00001d, 0x00001d, 0x00001d, 0x00001d},
+ {0x00001e, 0x00001e, 0x00001e, 0x00001e},
+ {0x00001f, 0x00001f, 0x00001f, 0x00001f},
+ {0x000020, 0x000020, 0x000020, 0x000020},
+ {0x000021, 0x000021, 0x000021, 0x000021},
+ {0x000022, 0x000022, 0x000022, 0x000022},
+ {0x000023, 0x000023, 0x000023, 0x000023},
+ {0x000024, 0x000024, 0x000024, 0x000024},
+ {0x000025, 0x000025, 0x000025, 0x000025},
+ {0x000026, 0x000026, 0x000026, 0x000026},
+ {0x000027, 0x000027, 0x000027, 0x000027},
+ {0x000028, 0x000028, 0x000028, 0x000028},
+ {0x000029, 0x000029, 0x000029, 0x000029},
+ {0x00002a, 0x00002a, 0x00002a, 0x00002a},
+ {0x00002b, 0x00002b, 0x00002b, 0x00002b},
+ {0x00002c, 0x00002c, 0x00002c, 0x00002c},
+ {0x00002d, 0x00002d, 0x00002d, 0x00002d},
+ {0x00002e, 0x00002e, 0x00002e, 0x00002e},
+ {0x00002f, 0x00002f, 0x00002f, 0x00002f},
+ {0x000030, 0x000030, 0x000030, 0x000030},
+ {0x000031, 0x000031, 0x000031, 0x000031},
+ {0x000032, 0x000032, 0x000032, 0x000032},
+ {0x000033, 0x000033, 0x000033, 0x000033},
+ {0x000034, 0x000034, 0x000034, 0x000034},
+ {0x000035, 0x000035, 0x000035, 0x000035},
+ {0x000036, 0x000036, 0x000036, 0x000036},
+ {0x000037, 0x000037, 0x000037, 0x000037},
+ {0x000038, 0x000038, 0x000038, 0x000038},
+ {0x000039, 0x000039, 0x000039, 0x000039},
+ {0x00003a, 0x00003a, 0x00003a, 0x00003a},
+ {0x00003b, 0x00003b, 0x00003b, 0x00003b},
+ {0x00003c, 0x00003c, 0x00003c, 0x00003c},
+ {0x00003d, 0x00003d, 0x00003d, 0x00003d},
+ {0x00003e, 0x00003e, 0x00003e, 0x00003e},
+ {0x00003f, 0x00003f, 0x00003f, 0x00003f},
+ {0x000040, 0x000040, 0x000040, 0x000040},
+ {0x000041, 0x000061, 0x000041, 0x000041},
+ {0x000042, 0x000062, 0x000042, 0x000042},
+ {0x000043, 0x000063, 0x000043, 0x000043},
+ {0x000044, 0x000064, 0x000044, 0x000044},
+ {0x000045, 0x000065, 0x000045, 0x000045},
+ {0x000046, 0x000066, 0x000046, 0x000046},
+ {0x000047, 0x000067, 0x000047, 0x000047},
+ {0x000048, 0x000068, 0x000048, 0x000048},
+ {0x000049, 0x000069, 0x000049, 0x000049},
+ {0x00004a, 0x00006a, 0x00004a, 0x00004a},
+ {0x00004b, 0x00006b, 0x00004b, 0x00004b},
+ {0x00004c, 0x00006c, 0x00004c, 0x00004c},
+ {0x00004d, 0x00006d, 0x00004d, 0x00004d},
+ {0x00004e, 0x00006e, 0x00004e, 0x00004e},
+ {0x00004f, 0x00006f, 0x00004f, 0x00004f},
+ {0x000050, 0x000070, 0x000050, 0x000050},
+ {0x000051, 0x000071, 0x000051, 0x000051},
+ {0x000052, 0x000072, 0x000052, 0x000052},
+ {0x000053, 0x000073, 0x000053, 0x000053},
+ {0x000054, 0x000074, 0x000054, 0x000054},
+ {0x000055, 0x000075, 0x000055, 0x000055},
+ {0x000056, 0x000076, 0x000056, 0x000056},
+ {0x000057, 0x000077, 0x000057, 0x000057},
+ {0x000058, 0x000078, 0x000058, 0x000058},
+ {0x000059, 0x000079, 0x000059, 0x000059},
+ {0x00005a, 0x00007a, 0x00005a, 0x00005a},
+ {0x00005b, 0x00005b, 0x00005b, 0x00005b},
+ {0x00005c, 0x00005c, 0x00005c, 0x00005c},
+ {0x00005d, 0x00005d, 0x00005d, 0x00005d},
+ {0x00005e, 0x00005e, 0x00005e, 0x00005e},
+ {0x00005f, 0x00005f, 0x00005f, 0x00005f},
+ {0x000060, 0x000060, 0x000060, 0x000060},
+ {0x000061, 0x000061, 0x000041, 0x000041},
+ {0x000062, 0x000062, 0x000042, 0x000042},
+ {0x000063, 0x000063, 0x000043, 0x000043},
+ {0x000064, 0x000064, 0x000044, 0x000044},
+ {0x000065, 0x000065, 0x000045, 0x000045},
+ {0x000066, 0x000066, 0x000046, 0x000046},
+ {0x000067, 0x000067, 0x000047, 0x000047},
+ {0x000068, 0x000068, 0x000048, 0x000048},
+ {0x000069, 0x000069, 0x000049, 0x000049},
+ {0x00006a, 0x00006a, 0x00004a, 0x00004a},
+ {0x00006b, 0x00006b, 0x00004b, 0x00004b},
+ {0x00006c, 0x00006c, 0x00004c, 0x00004c},
+ {0x00006d, 0x00006d, 0x00004d, 0x00004d},
+ {0x00006e, 0x00006e, 0x00004e, 0x00004e},
+ {0x00006f, 0x00006f, 0x00004f, 0x00004f},
+ {0x000070, 0x000070, 0x000050, 0x000050},
+ {0x000071, 0x000071, 0x000051, 0x000051},
+ {0x000072, 0x000072, 0x000052, 0x000052},
+ {0x000073, 0x000073, 0x000053, 0x000053},
+ {0x000074, 0x000074, 0x000054, 0x000054},
+ {0x000075, 0x000075, 0x000055, 0x000055},
+ {0x000076, 0x000076, 0x000056, 0x000056},
+ {0x000077, 0x000077, 0x000057, 0x000057},
+ {0x000078, 0x000078, 0x000058, 0x000058},
+ {0x000079, 0x000079, 0x000059, 0x000059},
+ {0x00007a, 0x00007a, 0x00005a, 0x00005a},
+ {0x00007b, 0x00007b, 0x00007b, 0x00007b},
+ {0x00007c, 0x00007c, 0x00007c, 0x00007c},
+ {0x00007d, 0x00007d, 0x00007d, 0x00007d},
+ {0x00007e, 0x00007e, 0x00007e, 0x00007e},
+ {0x00007f, 0x00007f, 0x00007f, 0x00007f},
+
+ /* begin sparse entries for codepoints >= 0x80 */
+ {0x0000b5, 0x0000b5, 0x00039c, 0x00039c},
+ {0x0000c0, 0x0000e0, 0x0000c0, 0x0000c0},
+ {0x0000c1, 0x0000e1, 0x0000c1, 0x0000c1},
+ {0x0000c2, 0x0000e2, 0x0000c2, 0x0000c2},
+ {0x0000c3, 0x0000e3, 0x0000c3, 0x0000c3},
+ {0x0000c4, 0x0000e4, 0x0000c4, 0x0000c4},
+ {0x0000c5, 0x0000e5, 0x0000c5, 0x0000c5},
+ {0x0000c6, 0x0000e6, 0x0000c6, 0x0000c6},
+ {0x0000c7, 0x0000e7, 0x0000c7, 0x0000c7},
+ {0x0000c8, 0x0000e8, 0x0000c8, 0x0000c8},
+ {0x0000c9, 0x0000e9, 0x0000c9, 0x0000c9},
+ {0x0000ca, 0x0000ea, 0x0000ca, 0x0000ca},
+ {0x0000cb, 0x0000eb, 0x0000cb, 0x0000cb},
+ {0x0000cc, 0x0000ec, 0x0000cc, 0x0000cc},
+ {0x0000cd, 0x0000ed, 0x0000cd, 0x0000cd},
+ {0x0000ce, 0x0000ee, 0x0000ce, 0x0000ce},
+ {0x0000cf, 0x0000ef, 0x0000cf, 0x0000cf},
+ {0x0000d0, 0x0000f0, 0x0000d0, 0x0000d0},
+ {0x0000d1, 0x0000f1, 0x0000d1, 0x0000d1},
+ {0x0000d2, 0x0000f2, 0x0000d2, 0x0000d2},
+ {0x0000d3, 0x0000f3, 0x0000d3, 0x0000d3},
+ {0x0000d4, 0x0000f4, 0x0000d4, 0x0000d4},
+ {0x0000d5, 0x0000f5, 0x0000d5, 0x0000d5},
+ {0x0000d6, 0x0000f6, 0x0000d6, 0x0000d6},
+ {0x0000d8, 0x0000f8, 0x0000d8, 0x0000d8},
+ {0x0000d9, 0x0000f9, 0x0000d9, 0x0000d9},
+ {0x0000da, 0x0000fa, 0x0000da, 0x0000da},
+ {0x0000db, 0x0000fb, 0x0000db, 0x0000db},
+ {0x0000dc, 0x0000fc, 0x0000dc, 0x0000dc},
+ {0x0000dd, 0x0000fd, 0x0000dd, 0x0000dd},
+ {0x0000de, 0x0000fe, 0x0000de, 0x0000de},
+ {0x0000e0, 0x0000e0, 0x0000c0, 0x0000c0},
+ {0x0000e1, 0x0000e1, 0x0000c1, 0x0000c1},
+ {0x0000e2, 0x0000e2, 0x0000c2, 0x0000c2},
+ {0x0000e3, 0x0000e3, 0x0000c3, 0x0000c3},
+ {0x0000e4, 0x0000e4, 0x0000c4, 0x0000c4},
+ {0x0000e5, 0x0000e5, 0x0000c5, 0x0000c5},
+ {0x0000e6, 0x0000e6, 0x0000c6, 0x0000c6},
+ {0x0000e7, 0x0000e7, 0x0000c7, 0x0000c7},
+ {0x0000e8, 0x0000e8, 0x0000c8, 0x0000c8},
+ {0x0000e9, 0x0000e9, 0x0000c9, 0x0000c9},
+ {0x0000ea, 0x0000ea, 0x0000ca, 0x0000ca},
+ {0x0000eb, 0x0000eb, 0x0000cb, 0x0000cb},
+ {0x0000ec, 0x0000ec, 0x0000cc, 0x0000cc},
+ {0x0000ed, 0x0000ed, 0x0000cd, 0x0000cd},
+ {0x0000ee, 0x0000ee, 0x0000ce, 0x0000ce},
+ {0x0000ef, 0x0000ef, 0x0000cf, 0x0000cf},
+ {0x0000f0, 0x0000f0, 0x0000d0, 0x0000d0},
+ {0x0000f1, 0x0000f1, 0x0000d1, 0x0000d1},
+ {0x0000f2, 0x0000f2, 0x0000d2, 0x0000d2},
+ {0x0000f3, 0x0000f3, 0x0000d3, 0x0000d3},
+ {0x0000f4, 0x0000f4, 0x0000d4, 0x0000d4},
+ {0x0000f5, 0x0000f5, 0x0000d5, 0x0000d5},
+ {0x0000f6, 0x0000f6, 0x0000d6, 0x0000d6},
+ {0x0000f8, 0x0000f8, 0x0000d8, 0x0000d8},
+ {0x0000f9, 0x0000f9, 0x0000d9, 0x0000d9},
+ {0x0000fa, 0x0000fa, 0x0000da, 0x0000da},
+ {0x0000fb, 0x0000fb, 0x0000db, 0x0000db},
+ {0x0000fc, 0x0000fc, 0x0000dc, 0x0000dc},
+ {0x0000fd, 0x0000fd, 0x0000dd, 0x0000dd},
+ {0x0000fe, 0x0000fe, 0x0000de, 0x0000de},
+ {0x0000ff, 0x0000ff, 0x000178, 0x000178},
+ {0x000100, 0x000101, 0x000100, 0x000100},
+ {0x000101, 0x000101, 0x000100, 0x000100},
+ {0x000102, 0x000103, 0x000102, 0x000102},
+ {0x000103, 0x000103, 0x000102, 0x000102},
+ {0x000104, 0x000105, 0x000104, 0x000104},
+ {0x000105, 0x000105, 0x000104, 0x000104},
+ {0x000106, 0x000107, 0x000106, 0x000106},
+ {0x000107, 0x000107, 0x000106, 0x000106},
+ {0x000108, 0x000109, 0x000108, 0x000108},
+ {0x000109, 0x000109, 0x000108, 0x000108},
+ {0x00010a, 0x00010b, 0x00010a, 0x00010a},
+ {0x00010b, 0x00010b, 0x00010a, 0x00010a},
+ {0x00010c, 0x00010d, 0x00010c, 0x00010c},
+ {0x00010d, 0x00010d, 0x00010c, 0x00010c},
+ {0x00010e, 0x00010f, 0x00010e, 0x00010e},
+ {0x00010f, 0x00010f, 0x00010e, 0x00010e},
+ {0x000110, 0x000111, 0x000110, 0x000110},
+ {0x000111, 0x000111, 0x000110, 0x000110},
+ {0x000112, 0x000113, 0x000112, 0x000112},
+ {0x000113, 0x000113, 0x000112, 0x000112},
+ {0x000114, 0x000115, 0x000114, 0x000114},
+ {0x000115, 0x000115, 0x000114, 0x000114},
+ {0x000116, 0x000117, 0x000116, 0x000116},
+ {0x000117, 0x000117, 0x000116, 0x000116},
+ {0x000118, 0x000119, 0x000118, 0x000118},
+ {0x000119, 0x000119, 0x000118, 0x000118},
+ {0x00011a, 0x00011b, 0x00011a, 0x00011a},
+ {0x00011b, 0x00011b, 0x00011a, 0x00011a},
+ {0x00011c, 0x00011d, 0x00011c, 0x00011c},
+ {0x00011d, 0x00011d, 0x00011c, 0x00011c},
+ {0x00011e, 0x00011f, 0x00011e, 0x00011e},
+ {0x00011f, 0x00011f, 0x00011e, 0x00011e},
+ {0x000120, 0x000121, 0x000120, 0x000120},
+ {0x000121, 0x000121, 0x000120, 0x000120},
+ {0x000122, 0x000123, 0x000122, 0x000122},
+ {0x000123, 0x000123, 0x000122, 0x000122},
+ {0x000124, 0x000125, 0x000124, 0x000124},
+ {0x000125, 0x000125, 0x000124, 0x000124},
+ {0x000126, 0x000127, 0x000126, 0x000126},
+ {0x000127, 0x000127, 0x000126, 0x000126},
+ {0x000128, 0x000129, 0x000128, 0x000128},
+ {0x000129, 0x000129, 0x000128, 0x000128},
+ {0x00012a, 0x00012b, 0x00012a, 0x00012a},
+ {0x00012b, 0x00012b, 0x00012a, 0x00012a},
+ {0x00012c, 0x00012d, 0x00012c, 0x00012c},
+ {0x00012d, 0x00012d, 0x00012c, 0x00012c},
+ {0x00012e, 0x00012f, 0x00012e, 0x00012e},
+ {0x00012f, 0x00012f, 0x00012e, 0x00012e},
+ {0x000130, 0x000069, 0x000130, 0x000130},
+ {0x000131, 0x000131, 0x000049, 0x000049},
+ {0x000132, 0x000133, 0x000132, 0x000132},
+ {0x000133, 0x000133, 0x000132, 0x000132},
+ {0x000134, 0x000135, 0x000134, 0x000134},
+ {0x000135, 0x000135, 0x000134, 0x000134},
+ {0x000136, 0x000137, 0x000136, 0x000136},
+ {0x000137, 0x000137, 0x000136, 0x000136},
+ {0x000139, 0x00013a, 0x000139, 0x000139},
+ {0x00013a, 0x00013a, 0x000139, 0x000139},
+ {0x00013b, 0x00013c, 0x00013b, 0x00013b},
+ {0x00013c, 0x00013c, 0x00013b, 0x00013b},
+ {0x00013d, 0x00013e, 0x00013d, 0x00013d},
+ {0x00013e, 0x00013e, 0x00013d, 0x00013d},
+ {0x00013f, 0x000140, 0x00013f, 0x00013f},
+ {0x000140, 0x000140, 0x00013f, 0x00013f},
+ {0x000141, 0x000142, 0x000141, 0x000141},
+ {0x000142, 0x000142, 0x000141, 0x000141},
+ {0x000143, 0x000144, 0x000143, 0x000143},
+ {0x000144, 0x000144, 0x000143, 0x000143},
+ {0x000145, 0x000146, 0x000145, 0x000145},
+ {0x000146, 0x000146, 0x000145, 0x000145},
+ {0x000147, 0x000148, 0x000147, 0x000147},
+ {0x000148, 0x000148, 0x000147, 0x000147},
+ {0x00014a, 0x00014b, 0x00014a, 0x00014a},
+ {0x00014b, 0x00014b, 0x00014a, 0x00014a},
+ {0x00014c, 0x00014d, 0x00014c, 0x00014c},
+ {0x00014d, 0x00014d, 0x00014c, 0x00014c},
+ {0x00014e, 0x00014f, 0x00014e, 0x00014e},
+ {0x00014f, 0x00014f, 0x00014e, 0x00014e},
+ {0x000150, 0x000151, 0x000150, 0x000150},
+ {0x000151, 0x000151, 0x000150, 0x000150},
+ {0x000152, 0x000153, 0x000152, 0x000152},
+ {0x000153, 0x000153, 0x000152, 0x000152},
+ {0x000154, 0x000155, 0x000154, 0x000154},
+ {0x000155, 0x000155, 0x000154, 0x000154},
+ {0x000156, 0x000157, 0x000156, 0x000156},
+ {0x000157, 0x000157, 0x000156, 0x000156},
+ {0x000158, 0x000159, 0x000158, 0x000158},
+ {0x000159, 0x000159, 0x000158, 0x000158},
+ {0x00015a, 0x00015b, 0x00015a, 0x00015a},
+ {0x00015b, 0x00015b, 0x00015a, 0x00015a},
+ {0x00015c, 0x00015d, 0x00015c, 0x00015c},
+ {0x00015d, 0x00015d, 0x00015c, 0x00015c},
+ {0x00015e, 0x00015f, 0x00015e, 0x00015e},
+ {0x00015f, 0x00015f, 0x00015e, 0x00015e},
+ {0x000160, 0x000161, 0x000160, 0x000160},
+ {0x000161, 0x000161, 0x000160, 0x000160},
+ {0x000162, 0x000163, 0x000162, 0x000162},
+ {0x000163, 0x000163, 0x000162, 0x000162},
+ {0x000164, 0x000165, 0x000164, 0x000164},
+ {0x000165, 0x000165, 0x000164, 0x000164},
+ {0x000166, 0x000167, 0x000166, 0x000166},
+ {0x000167, 0x000167, 0x000166, 0x000166},
+ {0x000168, 0x000169, 0x000168, 0x000168},
+ {0x000169, 0x000169, 0x000168, 0x000168},
+ {0x00016a, 0x00016b, 0x00016a, 0x00016a},
+ {0x00016b, 0x00016b, 0x00016a, 0x00016a},
+ {0x00016c, 0x00016d, 0x00016c, 0x00016c},
+ {0x00016d, 0x00016d, 0x00016c, 0x00016c},
+ {0x00016e, 0x00016f, 0x00016e, 0x00016e},
+ {0x00016f, 0x00016f, 0x00016e, 0x00016e},
+ {0x000170, 0x000171, 0x000170, 0x000170},
+ {0x000171, 0x000171, 0x000170, 0x000170},
+ {0x000172, 0x000173, 0x000172, 0x000172},
+ {0x000173, 0x000173, 0x000172, 0x000172},
+ {0x000174, 0x000175, 0x000174, 0x000174},
+ {0x000175, 0x000175, 0x000174, 0x000174},
+ {0x000176, 0x000177, 0x000176, 0x000176},
+ {0x000177, 0x000177, 0x000176, 0x000176},
+ {0x000178, 0x0000ff, 0x000178, 0x000178},
+ {0x000179, 0x00017a, 0x000179, 0x000179},
+ {0x00017a, 0x00017a, 0x000179, 0x000179},
+ {0x00017b, 0x00017c, 0x00017b, 0x00017b},
+ {0x00017c, 0x00017c, 0x00017b, 0x00017b},
+ {0x00017d, 0x00017e, 0x00017d, 0x00017d},
+ {0x00017e, 0x00017e, 0x00017d, 0x00017d},
+ {0x00017f, 0x00017f, 0x000053, 0x000053},
+ {0x000180, 0x000180, 0x000243, 0x000243},
+ {0x000181, 0x000253, 0x000181, 0x000181},
+ {0x000182, 0x000183, 0x000182, 0x000182},
+ {0x000183, 0x000183, 0x000182, 0x000182},
+ {0x000184, 0x000185, 0x000184, 0x000184},
+ {0x000185, 0x000185, 0x000184, 0x000184},
+ {0x000186, 0x000254, 0x000186, 0x000186},
+ {0x000187, 0x000188, 0x000187, 0x000187},
+ {0x000188, 0x000188, 0x000187, 0x000187},
+ {0x000189, 0x000256, 0x000189, 0x000189},
+ {0x00018a, 0x000257, 0x00018a, 0x00018a},
+ {0x00018b, 0x00018c, 0x00018b, 0x00018b},
+ {0x00018c, 0x00018c, 0x00018b, 0x00018b},
+ {0x00018e, 0x0001dd, 0x00018e, 0x00018e},
+ {0x00018f, 0x000259, 0x00018f, 0x00018f},
+ {0x000190, 0x00025b, 0x000190, 0x000190},
+ {0x000191, 0x000192, 0x000191, 0x000191},
+ {0x000192, 0x000192, 0x000191, 0x000191},
+ {0x000193, 0x000260, 0x000193, 0x000193},
+ {0x000194, 0x000263, 0x000194, 0x000194},
+ {0x000195, 0x000195, 0x0001f6, 0x0001f6},
+ {0x000196, 0x000269, 0x000196, 0x000196},
+ {0x000197, 0x000268, 0x000197, 0x000197},
+ {0x000198, 0x000199, 0x000198, 0x000198},
+ {0x000199, 0x000199, 0x000198, 0x000198},
+ {0x00019a, 0x00019a, 0x00023d, 0x00023d},
+ {0x00019c, 0x00026f, 0x00019c, 0x00019c},
+ {0x00019d, 0x000272, 0x00019d, 0x00019d},
+ {0x00019e, 0x00019e, 0x000220, 0x000220},
+ {0x00019f, 0x000275, 0x00019f, 0x00019f},
+ {0x0001a0, 0x0001a1, 0x0001a0, 0x0001a0},
+ {0x0001a1, 0x0001a1, 0x0001a0, 0x0001a0},
+ {0x0001a2, 0x0001a3, 0x0001a2, 0x0001a2},
+ {0x0001a3, 0x0001a3, 0x0001a2, 0x0001a2},
+ {0x0001a4, 0x0001a5, 0x0001a4, 0x0001a4},
+ {0x0001a5, 0x0001a5, 0x0001a4, 0x0001a4},
+ {0x0001a6, 0x000280, 0x0001a6, 0x0001a6},
+ {0x0001a7, 0x0001a8, 0x0001a7, 0x0001a7},
+ {0x0001a8, 0x0001a8, 0x0001a7, 0x0001a7},
+ {0x0001a9, 0x000283, 0x0001a9, 0x0001a9},
+ {0x0001ac, 0x0001ad, 0x0001ac, 0x0001ac},
+ {0x0001ad, 0x0001ad, 0x0001ac, 0x0001ac},
+ {0x0001ae, 0x000288, 0x0001ae, 0x0001ae},
+ {0x0001af, 0x0001b0, 0x0001af, 0x0001af},
+ {0x0001b0, 0x0001b0, 0x0001af, 0x0001af},
+ {0x0001b1, 0x00028a, 0x0001b1, 0x0001b1},
+ {0x0001b2, 0x00028b, 0x0001b2, 0x0001b2},
+ {0x0001b3, 0x0001b4, 0x0001b3, 0x0001b3},
+ {0x0001b4, 0x0001b4, 0x0001b3, 0x0001b3},
+ {0x0001b5, 0x0001b6, 0x0001b5, 0x0001b5},
+ {0x0001b6, 0x0001b6, 0x0001b5, 0x0001b5},
+ {0x0001b7, 0x000292, 0x0001b7, 0x0001b7},
+ {0x0001b8, 0x0001b9, 0x0001b8, 0x0001b8},
+ {0x0001b9, 0x0001b9, 0x0001b8, 0x0001b8},
+ {0x0001bc, 0x0001bd, 0x0001bc, 0x0001bc},
+ {0x0001bd, 0x0001bd, 0x0001bc, 0x0001bc},
+ {0x0001bf, 0x0001bf, 0x0001f7, 0x0001f7},
+ {0x0001c4, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c5, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c6, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c7, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001c8, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001c9, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001ca, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cb, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cc, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cd, 0x0001ce, 0x0001cd, 0x0001cd},
+ {0x0001ce, 0x0001ce, 0x0001cd, 0x0001cd},
+ {0x0001cf, 0x0001d0, 0x0001cf, 0x0001cf},
+ {0x0001d0, 0x0001d0, 0x0001cf, 0x0001cf},
+ {0x0001d1, 0x0001d2, 0x0001d1, 0x0001d1},
+ {0x0001d2, 0x0001d2, 0x0001d1, 0x0001d1},
+ {0x0001d3, 0x0001d4, 0x0001d3, 0x0001d3},
+ {0x0001d4, 0x0001d4, 0x0001d3, 0x0001d3},
+ {0x0001d5, 0x0001d6, 0x0001d5, 0x0001d5},
+ {0x0001d6, 0x0001d6, 0x0001d5, 0x0001d5},
+ {0x0001d7, 0x0001d8, 0x0001d7, 0x0001d7},
+ {0x0001d8, 0x0001d8, 0x0001d7, 0x0001d7},
+ {0x0001d9, 0x0001da, 0x0001d9, 0x0001d9},
+ {0x0001da, 0x0001da, 0x0001d9, 0x0001d9},
+ {0x0001db, 0x0001dc, 0x0001db, 0x0001db},
+ {0x0001dc, 0x0001dc, 0x0001db, 0x0001db},
+ {0x0001dd, 0x0001dd, 0x00018e, 0x00018e},
+ {0x0001de, 0x0001df, 0x0001de, 0x0001de},
+ {0x0001df, 0x0001df, 0x0001de, 0x0001de},
+ {0x0001e0, 0x0001e1, 0x0001e0, 0x0001e0},
+ {0x0001e1, 0x0001e1, 0x0001e0, 0x0001e0},
+ {0x0001e2, 0x0001e3, 0x0001e2, 0x0001e2},
+ {0x0001e3, 0x0001e3, 0x0001e2, 0x0001e2},
+ {0x0001e4, 0x0001e5, 0x0001e4, 0x0001e4},
+ {0x0001e5, 0x0001e5, 0x0001e4, 0x0001e4},
+ {0x0001e6, 0x0001e7, 0x0001e6, 0x0001e6},
+ {0x0001e7, 0x0001e7, 0x0001e6, 0x0001e6},
+ {0x0001e8, 0x0001e9, 0x0001e8, 0x0001e8},
+ {0x0001e9, 0x0001e9, 0x0001e8, 0x0001e8},
+ {0x0001ea, 0x0001eb, 0x0001ea, 0x0001ea},
+ {0x0001eb, 0x0001eb, 0x0001ea, 0x0001ea},
+ {0x0001ec, 0x0001ed, 0x0001ec, 0x0001ec},
+ {0x0001ed, 0x0001ed, 0x0001ec, 0x0001ec},
+ {0x0001ee, 0x0001ef, 0x0001ee, 0x0001ee},
+ {0x0001ef, 0x0001ef, 0x0001ee, 0x0001ee},
+ {0x0001f1, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f2, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f3, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f4, 0x0001f5, 0x0001f4, 0x0001f4},
+ {0x0001f5, 0x0001f5, 0x0001f4, 0x0001f4},
+ {0x0001f6, 0x000195, 0x0001f6, 0x0001f6},
+ {0x0001f7, 0x0001bf, 0x0001f7, 0x0001f7},
+ {0x0001f8, 0x0001f9, 0x0001f8, 0x0001f8},
+ {0x0001f9, 0x0001f9, 0x0001f8, 0x0001f8},
+ {0x0001fa, 0x0001fb, 0x0001fa, 0x0001fa},
+ {0x0001fb, 0x0001fb, 0x0001fa, 0x0001fa},
+ {0x0001fc, 0x0001fd, 0x0001fc, 0x0001fc},
+ {0x0001fd, 0x0001fd, 0x0001fc, 0x0001fc},
+ {0x0001fe, 0x0001ff, 0x0001fe, 0x0001fe},
+ {0x0001ff, 0x0001ff, 0x0001fe, 0x0001fe},
+ {0x000200, 0x000201, 0x000200, 0x000200},
+ {0x000201, 0x000201, 0x000200, 0x000200},
+ {0x000202, 0x000203, 0x000202, 0x000202},
+ {0x000203, 0x000203, 0x000202, 0x000202},
+ {0x000204, 0x000205, 0x000204, 0x000204},
+ {0x000205, 0x000205, 0x000204, 0x000204},
+ {0x000206, 0x000207, 0x000206, 0x000206},
+ {0x000207, 0x000207, 0x000206, 0x000206},
+ {0x000208, 0x000209, 0x000208, 0x000208},
+ {0x000209, 0x000209, 0x000208, 0x000208},
+ {0x00020a, 0x00020b, 0x00020a, 0x00020a},
+ {0x00020b, 0x00020b, 0x00020a, 0x00020a},
+ {0x00020c, 0x00020d, 0x00020c, 0x00020c},
+ {0x00020d, 0x00020d, 0x00020c, 0x00020c},
+ {0x00020e, 0x00020f, 0x00020e, 0x00020e},
+ {0x00020f, 0x00020f, 0x00020e, 0x00020e},
+ {0x000210, 0x000211, 0x000210, 0x000210},
+ {0x000211, 0x000211, 0x000210, 0x000210},
+ {0x000212, 0x000213, 0x000212, 0x000212},
+ {0x000213, 0x000213, 0x000212, 0x000212},
+ {0x000214, 0x000215, 0x000214, 0x000214},
+ {0x000215, 0x000215, 0x000214, 0x000214},
+ {0x000216, 0x000217, 0x000216, 0x000216},
+ {0x000217, 0x000217, 0x000216, 0x000216},
+ {0x000218, 0x000219, 0x000218, 0x000218},
+ {0x000219, 0x000219, 0x000218, 0x000218},
+ {0x00021a, 0x00021b, 0x00021a, 0x00021a},
+ {0x00021b, 0x00021b, 0x00021a, 0x00021a},
+ {0x00021c, 0x00021d, 0x00021c, 0x00021c},
+ {0x00021d, 0x00021d, 0x00021c, 0x00021c},
+ {0x00021e, 0x00021f, 0x00021e, 0x00021e},
+ {0x00021f, 0x00021f, 0x00021e, 0x00021e},
+ {0x000220, 0x00019e, 0x000220, 0x000220},
+ {0x000222, 0x000223, 0x000222, 0x000222},
+ {0x000223, 0x000223, 0x000222, 0x000222},
+ {0x000224, 0x000225, 0x000224, 0x000224},
+ {0x000225, 0x000225, 0x000224, 0x000224},
+ {0x000226, 0x000227, 0x000226, 0x000226},
+ {0x000227, 0x000227, 0x000226, 0x000226},
+ {0x000228, 0x000229, 0x000228, 0x000228},
+ {0x000229, 0x000229, 0x000228, 0x000228},
+ {0x00022a, 0x00022b, 0x00022a, 0x00022a},
+ {0x00022b, 0x00022b, 0x00022a, 0x00022a},
+ {0x00022c, 0x00022d, 0x00022c, 0x00022c},
+ {0x00022d, 0x00022d, 0x00022c, 0x00022c},
+ {0x00022e, 0x00022f, 0x00022e, 0x00022e},
+ {0x00022f, 0x00022f, 0x00022e, 0x00022e},
+ {0x000230, 0x000231, 0x000230, 0x000230},
+ {0x000231, 0x000231, 0x000230, 0x000230},
+ {0x000232, 0x000233, 0x000232, 0x000232},
+ {0x000233, 0x000233, 0x000232, 0x000232},
+ {0x00023a, 0x002c65, 0x00023a, 0x00023a},
+ {0x00023b, 0x00023c, 0x00023b, 0x00023b},
+ {0x00023c, 0x00023c, 0x00023b, 0x00023b},
+ {0x00023d, 0x00019a, 0x00023d, 0x00023d},
+ {0x00023e, 0x002c66, 0x00023e, 0x00023e},
+ {0x00023f, 0x00023f, 0x002c7e, 0x002c7e},
+ {0x000240, 0x000240, 0x002c7f, 0x002c7f},
+ {0x000241, 0x000242, 0x000241, 0x000241},
+ {0x000242, 0x000242, 0x000241, 0x000241},
+ {0x000243, 0x000180, 0x000243, 0x000243},
+ {0x000244, 0x000289, 0x000244, 0x000244},
+ {0x000245, 0x00028c, 0x000245, 0x000245},
+ {0x000246, 0x000247, 0x000246, 0x000246},
+ {0x000247, 0x000247, 0x000246, 0x000246},
+ {0x000248, 0x000249, 0x000248, 0x000248},
+ {0x000249, 0x000249, 0x000248, 0x000248},
+ {0x00024a, 0x00024b, 0x00024a, 0x00024a},
+ {0x00024b, 0x00024b, 0x00024a, 0x00024a},
+ {0x00024c, 0x00024d, 0x00024c, 0x00024c},
+ {0x00024d, 0x00024d, 0x00024c, 0x00024c},
+ {0x00024e, 0x00024f, 0x00024e, 0x00024e},
+ {0x00024f, 0x00024f, 0x00024e, 0x00024e},
+ {0x000250, 0x000250, 0x002c6f, 0x002c6f},
+ {0x000251, 0x000251, 0x002c6d, 0x002c6d},
+ {0x000252, 0x000252, 0x002c70, 0x002c70},
+ {0x000253, 0x000253, 0x000181, 0x000181},
+ {0x000254, 0x000254, 0x000186, 0x000186},
+ {0x000256, 0x000256, 0x000189, 0x000189},
+ {0x000257, 0x000257, 0x00018a, 0x00018a},
+ {0x000259, 0x000259, 0x00018f, 0x00018f},
+ {0x00025b, 0x00025b, 0x000190, 0x000190},
+ {0x00025c, 0x00025c, 0x00a7ab, 0x00a7ab},
+ {0x000260, 0x000260, 0x000193, 0x000193},
+ {0x000261, 0x000261, 0x00a7ac, 0x00a7ac},
+ {0x000263, 0x000263, 0x000194, 0x000194},
+ {0x000265, 0x000265, 0x00a78d, 0x00a78d},
+ {0x000266, 0x000266, 0x00a7aa, 0x00a7aa},
+ {0x000268, 0x000268, 0x000197, 0x000197},
+ {0x000269, 0x000269, 0x000196, 0x000196},
+ {0x00026a, 0x00026a, 0x00a7ae, 0x00a7ae},
+ {0x00026b, 0x00026b, 0x002c62, 0x002c62},
+ {0x00026c, 0x00026c, 0x00a7ad, 0x00a7ad},
+ {0x00026f, 0x00026f, 0x00019c, 0x00019c},
+ {0x000271, 0x000271, 0x002c6e, 0x002c6e},
+ {0x000272, 0x000272, 0x00019d, 0x00019d},
+ {0x000275, 0x000275, 0x00019f, 0x00019f},
+ {0x00027d, 0x00027d, 0x002c64, 0x002c64},
+ {0x000280, 0x000280, 0x0001a6, 0x0001a6},
+ {0x000282, 0x000282, 0x00a7c5, 0x00a7c5},
+ {0x000283, 0x000283, 0x0001a9, 0x0001a9},
+ {0x000287, 0x000287, 0x00a7b1, 0x00a7b1},
+ {0x000288, 0x000288, 0x0001ae, 0x0001ae},
+ {0x000289, 0x000289, 0x000244, 0x000244},
+ {0x00028a, 0x00028a, 0x0001b1, 0x0001b1},
+ {0x00028b, 0x00028b, 0x0001b2, 0x0001b2},
+ {0x00028c, 0x00028c, 0x000245, 0x000245},
+ {0x000292, 0x000292, 0x0001b7, 0x0001b7},
+ {0x00029d, 0x00029d, 0x00a7b2, 0x00a7b2},
+ {0x00029e, 0x00029e, 0x00a7b0, 0x00a7b0},
+ {0x000345, 0x000345, 0x000399, 0x000399},
+ {0x000370, 0x000371, 0x000370, 0x000370},
+ {0x000371, 0x000371, 0x000370, 0x000370},
+ {0x000372, 0x000373, 0x000372, 0x000372},
+ {0x000373, 0x000373, 0x000372, 0x000372},
+ {0x000376, 0x000377, 0x000376, 0x000376},
+ {0x000377, 0x000377, 0x000376, 0x000376},
+ {0x00037b, 0x00037b, 0x0003fd, 0x0003fd},
+ {0x00037c, 0x00037c, 0x0003fe, 0x0003fe},
+ {0x00037d, 0x00037d, 0x0003ff, 0x0003ff},
+ {0x00037f, 0x0003f3, 0x00037f, 0x00037f},
+ {0x000386, 0x0003ac, 0x000386, 0x000386},
+ {0x000388, 0x0003ad, 0x000388, 0x000388},
+ {0x000389, 0x0003ae, 0x000389, 0x000389},
+ {0x00038a, 0x0003af, 0x00038a, 0x00038a},
+ {0x00038c, 0x0003cc, 0x00038c, 0x00038c},
+ {0x00038e, 0x0003cd, 0x00038e, 0x00038e},
+ {0x00038f, 0x0003ce, 0x00038f, 0x00038f},
+ {0x000391, 0x0003b1, 0x000391, 0x000391},
+ {0x000392, 0x0003b2, 0x000392, 0x000392},
+ {0x000393, 0x0003b3, 0x000393, 0x000393},
+ {0x000394, 0x0003b4, 0x000394, 0x000394},
+ {0x000395, 0x0003b5, 0x000395, 0x000395},
+ {0x000396, 0x0003b6, 0x000396, 0x000396},
+ {0x000397, 0x0003b7, 0x000397, 0x000397},
+ {0x000398, 0x0003b8, 0x000398, 0x000398},
+ {0x000399, 0x0003b9, 0x000399, 0x000399},
+ {0x00039a, 0x0003ba, 0x00039a, 0x00039a},
+ {0x00039b, 0x0003bb, 0x00039b, 0x00039b},
+ {0x00039c, 0x0003bc, 0x00039c, 0x00039c},
+ {0x00039d, 0x0003bd, 0x00039d, 0x00039d},
+ {0x00039e, 0x0003be, 0x00039e, 0x00039e},
+ {0x00039f, 0x0003bf, 0x00039f, 0x00039f},
+ {0x0003a0, 0x0003c0, 0x0003a0, 0x0003a0},
+ {0x0003a1, 0x0003c1, 0x0003a1, 0x0003a1},
+ {0x0003a3, 0x0003c3, 0x0003a3, 0x0003a3},
+ {0x0003a4, 0x0003c4, 0x0003a4, 0x0003a4},
+ {0x0003a5, 0x0003c5, 0x0003a5, 0x0003a5},
+ {0x0003a6, 0x0003c6, 0x0003a6, 0x0003a6},
+ {0x0003a7, 0x0003c7, 0x0003a7, 0x0003a7},
+ {0x0003a8, 0x0003c8, 0x0003a8, 0x0003a8},
+ {0x0003a9, 0x0003c9, 0x0003a9, 0x0003a9},
+ {0x0003aa, 0x0003ca, 0x0003aa, 0x0003aa},
+ {0x0003ab, 0x0003cb, 0x0003ab, 0x0003ab},
+ {0x0003ac, 0x0003ac, 0x000386, 0x000386},
+ {0x0003ad, 0x0003ad, 0x000388, 0x000388},
+ {0x0003ae, 0x0003ae, 0x000389, 0x000389},
+ {0x0003af, 0x0003af, 0x00038a, 0x00038a},
+ {0x0003b1, 0x0003b1, 0x000391, 0x000391},
+ {0x0003b2, 0x0003b2, 0x000392, 0x000392},
+ {0x0003b3, 0x0003b3, 0x000393, 0x000393},
+ {0x0003b4, 0x0003b4, 0x000394, 0x000394},
+ {0x0003b5, 0x0003b5, 0x000395, 0x000395},
+ {0x0003b6, 0x0003b6, 0x000396, 0x000396},
+ {0x0003b7, 0x0003b7, 0x000397, 0x000397},
+ {0x0003b8, 0x0003b8, 0x000398, 0x000398},
+ {0x0003b9, 0x0003b9, 0x000399, 0x000399},
+ {0x0003ba, 0x0003ba, 0x00039a, 0x00039a},
+ {0x0003bb, 0x0003bb, 0x00039b, 0x00039b},
+ {0x0003bc, 0x0003bc, 0x00039c, 0x00039c},
+ {0x0003bd, 0x0003bd, 0x00039d, 0x00039d},
+ {0x0003be, 0x0003be, 0x00039e, 0x00039e},
+ {0x0003bf, 0x0003bf, 0x00039f, 0x00039f},
+ {0x0003c0, 0x0003c0, 0x0003a0, 0x0003a0},
+ {0x0003c1, 0x0003c1, 0x0003a1, 0x0003a1},
+ {0x0003c2, 0x0003c2, 0x0003a3, 0x0003a3},
+ {0x0003c3, 0x0003c3, 0x0003a3, 0x0003a3},
+ {0x0003c4, 0x0003c4, 0x0003a4, 0x0003a4},
+ {0x0003c5, 0x0003c5, 0x0003a5, 0x0003a5},
+ {0x0003c6, 0x0003c6, 0x0003a6, 0x0003a6},
+ {0x0003c7, 0x0003c7, 0x0003a7, 0x0003a7},
+ {0x0003c8, 0x0003c8, 0x0003a8, 0x0003a8},
+ {0x0003c9, 0x0003c9, 0x0003a9, 0x0003a9},
+ {0x0003ca, 0x0003ca, 0x0003aa, 0x0003aa},
+ {0x0003cb, 0x0003cb, 0x0003ab, 0x0003ab},
+ {0x0003cc, 0x0003cc, 0x00038c, 0x00038c},
+ {0x0003cd, 0x0003cd, 0x00038e, 0x00038e},
+ {0x0003ce, 0x0003ce, 0x00038f, 0x00038f},
+ {0x0003cf, 0x0003d7, 0x0003cf, 0x0003cf},
+ {0x0003d0, 0x0003d0, 0x000392, 0x000392},
+ {0x0003d1, 0x0003d1, 0x000398, 0x000398},
+ {0x0003d5, 0x0003d5, 0x0003a6, 0x0003a6},
+ {0x0003d6, 0x0003d6, 0x0003a0, 0x0003a0},
+ {0x0003d7, 0x0003d7, 0x0003cf, 0x0003cf},
+ {0x0003d8, 0x0003d9, 0x0003d8, 0x0003d8},
+ {0x0003d9, 0x0003d9, 0x0003d8, 0x0003d8},
+ {0x0003da, 0x0003db, 0x0003da, 0x0003da},
+ {0x0003db, 0x0003db, 0x0003da, 0x0003da},
+ {0x0003dc, 0x0003dd, 0x0003dc, 0x0003dc},
+ {0x0003dd, 0x0003dd, 0x0003dc, 0x0003dc},
+ {0x0003de, 0x0003df, 0x0003de, 0x0003de},
+ {0x0003df, 0x0003df, 0x0003de, 0x0003de},
+ {0x0003e0, 0x0003e1, 0x0003e0, 0x0003e0},
+ {0x0003e1, 0x0003e1, 0x0003e0, 0x0003e0},
+ {0x0003e2, 0x0003e3, 0x0003e2, 0x0003e2},
+ {0x0003e3, 0x0003e3, 0x0003e2, 0x0003e2},
+ {0x0003e4, 0x0003e5, 0x0003e4, 0x0003e4},
+ {0x0003e5, 0x0003e5, 0x0003e4, 0x0003e4},
+ {0x0003e6, 0x0003e7, 0x0003e6, 0x0003e6},
+ {0x0003e7, 0x0003e7, 0x0003e6, 0x0003e6},
+ {0x0003e8, 0x0003e9, 0x0003e8, 0x0003e8},
+ {0x0003e9, 0x0003e9, 0x0003e8, 0x0003e8},
+ {0x0003ea, 0x0003eb, 0x0003ea, 0x0003ea},
+ {0x0003eb, 0x0003eb, 0x0003ea, 0x0003ea},
+ {0x0003ec, 0x0003ed, 0x0003ec, 0x0003ec},
+ {0x0003ed, 0x0003ed, 0x0003ec, 0x0003ec},
+ {0x0003ee, 0x0003ef, 0x0003ee, 0x0003ee},
+ {0x0003ef, 0x0003ef, 0x0003ee, 0x0003ee},
+ {0x0003f0, 0x0003f0, 0x00039a, 0x00039a},
+ {0x0003f1, 0x0003f1, 0x0003a1, 0x0003a1},
+ {0x0003f2, 0x0003f2, 0x0003f9, 0x0003f9},
+ {0x0003f3, 0x0003f3, 0x00037f, 0x00037f},
+ {0x0003f4, 0x0003b8, 0x0003f4, 0x0003f4},
+ {0x0003f5, 0x0003f5, 0x000395, 0x000395},
+ {0x0003f7, 0x0003f8, 0x0003f7, 0x0003f7},
+ {0x0003f8, 0x0003f8, 0x0003f7, 0x0003f7},
+ {0x0003f9, 0x0003f2, 0x0003f9, 0x0003f9},
+ {0x0003fa, 0x0003fb, 0x0003fa, 0x0003fa},
+ {0x0003fb, 0x0003fb, 0x0003fa, 0x0003fa},
+ {0x0003fd, 0x00037b, 0x0003fd, 0x0003fd},
+ {0x0003fe, 0x00037c, 0x0003fe, 0x0003fe},
+ {0x0003ff, 0x00037d, 0x0003ff, 0x0003ff},
+ {0x000400, 0x000450, 0x000400, 0x000400},
+ {0x000401, 0x000451, 0x000401, 0x000401},
+ {0x000402, 0x000452, 0x000402, 0x000402},
+ {0x000403, 0x000453, 0x000403, 0x000403},
+ {0x000404, 0x000454, 0x000404, 0x000404},
+ {0x000405, 0x000455, 0x000405, 0x000405},
+ {0x000406, 0x000456, 0x000406, 0x000406},
+ {0x000407, 0x000457, 0x000407, 0x000407},
+ {0x000408, 0x000458, 0x000408, 0x000408},
+ {0x000409, 0x000459, 0x000409, 0x000409},
+ {0x00040a, 0x00045a, 0x00040a, 0x00040a},
+ {0x00040b, 0x00045b, 0x00040b, 0x00040b},
+ {0x00040c, 0x00045c, 0x00040c, 0x00040c},
+ {0x00040d, 0x00045d, 0x00040d, 0x00040d},
+ {0x00040e, 0x00045e, 0x00040e, 0x00040e},
+ {0x00040f, 0x00045f, 0x00040f, 0x00040f},
+ {0x000410, 0x000430, 0x000410, 0x000410},
+ {0x000411, 0x000431, 0x000411, 0x000411},
+ {0x000412, 0x000432, 0x000412, 0x000412},
+ {0x000413, 0x000433, 0x000413, 0x000413},
+ {0x000414, 0x000434, 0x000414, 0x000414},
+ {0x000415, 0x000435, 0x000415, 0x000415},
+ {0x000416, 0x000436, 0x000416, 0x000416},
+ {0x000417, 0x000437, 0x000417, 0x000417},
+ {0x000418, 0x000438, 0x000418, 0x000418},
+ {0x000419, 0x000439, 0x000419, 0x000419},
+ {0x00041a, 0x00043a, 0x00041a, 0x00041a},
+ {0x00041b, 0x00043b, 0x00041b, 0x00041b},
+ {0x00041c, 0x00043c, 0x00041c, 0x00041c},
+ {0x00041d, 0x00043d, 0x00041d, 0x00041d},
+ {0x00041e, 0x00043e, 0x00041e, 0x00041e},
+ {0x00041f, 0x00043f, 0x00041f, 0x00041f},
+ {0x000420, 0x000440, 0x000420, 0x000420},
+ {0x000421, 0x000441, 0x000421, 0x000421},
+ {0x000422, 0x000442, 0x000422, 0x000422},
+ {0x000423, 0x000443, 0x000423, 0x000423},
+ {0x000424, 0x000444, 0x000424, 0x000424},
+ {0x000425, 0x000445, 0x000425, 0x000425},
+ {0x000426, 0x000446, 0x000426, 0x000426},
+ {0x000427, 0x000447, 0x000427, 0x000427},
+ {0x000428, 0x000448, 0x000428, 0x000428},
+ {0x000429, 0x000449, 0x000429, 0x000429},
+ {0x00042a, 0x00044a, 0x00042a, 0x00042a},
+ {0x00042b, 0x00044b, 0x00042b, 0x00042b},
+ {0x00042c, 0x00044c, 0x00042c, 0x00042c},
+ {0x00042d, 0x00044d, 0x00042d, 0x00042d},
+ {0x00042e, 0x00044e, 0x00042e, 0x00042e},
+ {0x00042f, 0x00044f, 0x00042f, 0x00042f},
+ {0x000430, 0x000430, 0x000410, 0x000410},
+ {0x000431, 0x000431, 0x000411, 0x000411},
+ {0x000432, 0x000432, 0x000412, 0x000412},
+ {0x000433, 0x000433, 0x000413, 0x000413},
+ {0x000434, 0x000434, 0x000414, 0x000414},
+ {0x000435, 0x000435, 0x000415, 0x000415},
+ {0x000436, 0x000436, 0x000416, 0x000416},
+ {0x000437, 0x000437, 0x000417, 0x000417},
+ {0x000438, 0x000438, 0x000418, 0x000418},
+ {0x000439, 0x000439, 0x000419, 0x000419},
+ {0x00043a, 0x00043a, 0x00041a, 0x00041a},
+ {0x00043b, 0x00043b, 0x00041b, 0x00041b},
+ {0x00043c, 0x00043c, 0x00041c, 0x00041c},
+ {0x00043d, 0x00043d, 0x00041d, 0x00041d},
+ {0x00043e, 0x00043e, 0x00041e, 0x00041e},
+ {0x00043f, 0x00043f, 0x00041f, 0x00041f},
+ {0x000440, 0x000440, 0x000420, 0x000420},
+ {0x000441, 0x000441, 0x000421, 0x000421},
+ {0x000442, 0x000442, 0x000422, 0x000422},
+ {0x000443, 0x000443, 0x000423, 0x000423},
+ {0x000444, 0x000444, 0x000424, 0x000424},
+ {0x000445, 0x000445, 0x000425, 0x000425},
+ {0x000446, 0x000446, 0x000426, 0x000426},
+ {0x000447, 0x000447, 0x000427, 0x000427},
+ {0x000448, 0x000448, 0x000428, 0x000428},
+ {0x000449, 0x000449, 0x000429, 0x000429},
+ {0x00044a, 0x00044a, 0x00042a, 0x00042a},
+ {0x00044b, 0x00044b, 0x00042b, 0x00042b},
+ {0x00044c, 0x00044c, 0x00042c, 0x00042c},
+ {0x00044d, 0x00044d, 0x00042d, 0x00042d},
+ {0x00044e, 0x00044e, 0x00042e, 0x00042e},
+ {0x00044f, 0x00044f, 0x00042f, 0x00042f},
+ {0x000450, 0x000450, 0x000400, 0x000400},
+ {0x000451, 0x000451, 0x000401, 0x000401},
+ {0x000452, 0x000452, 0x000402, 0x000402},
+ {0x000453, 0x000453, 0x000403, 0x000403},
+ {0x000454, 0x000454, 0x000404, 0x000404},
+ {0x000455, 0x000455, 0x000405, 0x000405},
+ {0x000456, 0x000456, 0x000406, 0x000406},
+ {0x000457, 0x000457, 0x000407, 0x000407},
+ {0x000458, 0x000458, 0x000408, 0x000408},
+ {0x000459, 0x000459, 0x000409, 0x000409},
+ {0x00045a, 0x00045a, 0x00040a, 0x00040a},
+ {0x00045b, 0x00045b, 0x00040b, 0x00040b},
+ {0x00045c, 0x00045c, 0x00040c, 0x00040c},
+ {0x00045d, 0x00045d, 0x00040d, 0x00040d},
+ {0x00045e, 0x00045e, 0x00040e, 0x00040e},
+ {0x00045f, 0x00045f, 0x00040f, 0x00040f},
+ {0x000460, 0x000461, 0x000460, 0x000460},
+ {0x000461, 0x000461, 0x000460, 0x000460},
+ {0x000462, 0x000463, 0x000462, 0x000462},
+ {0x000463, 0x000463, 0x000462, 0x000462},
+ {0x000464, 0x000465, 0x000464, 0x000464},
+ {0x000465, 0x000465, 0x000464, 0x000464},
+ {0x000466, 0x000467, 0x000466, 0x000466},
+ {0x000467, 0x000467, 0x000466, 0x000466},
+ {0x000468, 0x000469, 0x000468, 0x000468},
+ {0x000469, 0x000469, 0x000468, 0x000468},
+ {0x00046a, 0x00046b, 0x00046a, 0x00046a},
+ {0x00046b, 0x00046b, 0x00046a, 0x00046a},
+ {0x00046c, 0x00046d, 0x00046c, 0x00046c},
+ {0x00046d, 0x00046d, 0x00046c, 0x00046c},
+ {0x00046e, 0x00046f, 0x00046e, 0x00046e},
+ {0x00046f, 0x00046f, 0x00046e, 0x00046e},
+ {0x000470, 0x000471, 0x000470, 0x000470},
+ {0x000471, 0x000471, 0x000470, 0x000470},
+ {0x000472, 0x000473, 0x000472, 0x000472},
+ {0x000473, 0x000473, 0x000472, 0x000472},
+ {0x000474, 0x000475, 0x000474, 0x000474},
+ {0x000475, 0x000475, 0x000474, 0x000474},
+ {0x000476, 0x000477, 0x000476, 0x000476},
+ {0x000477, 0x000477, 0x000476, 0x000476},
+ {0x000478, 0x000479, 0x000478, 0x000478},
+ {0x000479, 0x000479, 0x000478, 0x000478},
+ {0x00047a, 0x00047b, 0x00047a, 0x00047a},
+ {0x00047b, 0x00047b, 0x00047a, 0x00047a},
+ {0x00047c, 0x00047d, 0x00047c, 0x00047c},
+ {0x00047d, 0x00047d, 0x00047c, 0x00047c},
+ {0x00047e, 0x00047f, 0x00047e, 0x00047e},
+ {0x00047f, 0x00047f, 0x00047e, 0x00047e},
+ {0x000480, 0x000481, 0x000480, 0x000480},
+ {0x000481, 0x000481, 0x000480, 0x000480},
+ {0x00048a, 0x00048b, 0x00048a, 0x00048a},
+ {0x00048b, 0x00048b, 0x00048a, 0x00048a},
+ {0x00048c, 0x00048d, 0x00048c, 0x00048c},
+ {0x00048d, 0x00048d, 0x00048c, 0x00048c},
+ {0x00048e, 0x00048f, 0x00048e, 0x00048e},
+ {0x00048f, 0x00048f, 0x00048e, 0x00048e},
+ {0x000490, 0x000491, 0x000490, 0x000490},
+ {0x000491, 0x000491, 0x000490, 0x000490},
+ {0x000492, 0x000493, 0x000492, 0x000492},
+ {0x000493, 0x000493, 0x000492, 0x000492},
+ {0x000494, 0x000495, 0x000494, 0x000494},
+ {0x000495, 0x000495, 0x000494, 0x000494},
+ {0x000496, 0x000497, 0x000496, 0x000496},
+ {0x000497, 0x000497, 0x000496, 0x000496},
+ {0x000498, 0x000499, 0x000498, 0x000498},
+ {0x000499, 0x000499, 0x000498, 0x000498},
+ {0x00049a, 0x00049b, 0x00049a, 0x00049a},
+ {0x00049b, 0x00049b, 0x00049a, 0x00049a},
+ {0x00049c, 0x00049d, 0x00049c, 0x00049c},
+ {0x00049d, 0x00049d, 0x00049c, 0x00049c},
+ {0x00049e, 0x00049f, 0x00049e, 0x00049e},
+ {0x00049f, 0x00049f, 0x00049e, 0x00049e},
+ {0x0004a0, 0x0004a1, 0x0004a0, 0x0004a0},
+ {0x0004a1, 0x0004a1, 0x0004a0, 0x0004a0},
+ {0x0004a2, 0x0004a3, 0x0004a2, 0x0004a2},
+ {0x0004a3, 0x0004a3, 0x0004a2, 0x0004a2},
+ {0x0004a4, 0x0004a5, 0x0004a4, 0x0004a4},
+ {0x0004a5, 0x0004a5, 0x0004a4, 0x0004a4},
+ {0x0004a6, 0x0004a7, 0x0004a6, 0x0004a6},
+ {0x0004a7, 0x0004a7, 0x0004a6, 0x0004a6},
+ {0x0004a8, 0x0004a9, 0x0004a8, 0x0004a8},
+ {0x0004a9, 0x0004a9, 0x0004a8, 0x0004a8},
+ {0x0004aa, 0x0004ab, 0x0004aa, 0x0004aa},
+ {0x0004ab, 0x0004ab, 0x0004aa, 0x0004aa},
+ {0x0004ac, 0x0004ad, 0x0004ac, 0x0004ac},
+ {0x0004ad, 0x0004ad, 0x0004ac, 0x0004ac},
+ {0x0004ae, 0x0004af, 0x0004ae, 0x0004ae},
+ {0x0004af, 0x0004af, 0x0004ae, 0x0004ae},
+ {0x0004b0, 0x0004b1, 0x0004b0, 0x0004b0},
+ {0x0004b1, 0x0004b1, 0x0004b0, 0x0004b0},
+ {0x0004b2, 0x0004b3, 0x0004b2, 0x0004b2},
+ {0x0004b3, 0x0004b3, 0x0004b2, 0x0004b2},
+ {0x0004b4, 0x0004b5, 0x0004b4, 0x0004b4},
+ {0x0004b5, 0x0004b5, 0x0004b4, 0x0004b4},
+ {0x0004b6, 0x0004b7, 0x0004b6, 0x0004b6},
+ {0x0004b7, 0x0004b7, 0x0004b6, 0x0004b6},
+ {0x0004b8, 0x0004b9, 0x0004b8, 0x0004b8},
+ {0x0004b9, 0x0004b9, 0x0004b8, 0x0004b8},
+ {0x0004ba, 0x0004bb, 0x0004ba, 0x0004ba},
+ {0x0004bb, 0x0004bb, 0x0004ba, 0x0004ba},
+ {0x0004bc, 0x0004bd, 0x0004bc, 0x0004bc},
+ {0x0004bd, 0x0004bd, 0x0004bc, 0x0004bc},
+ {0x0004be, 0x0004bf, 0x0004be, 0x0004be},
+ {0x0004bf, 0x0004bf, 0x0004be, 0x0004be},
+ {0x0004c0, 0x0004cf, 0x0004c0, 0x0004c0},
+ {0x0004c1, 0x0004c2, 0x0004c1, 0x0004c1},
+ {0x0004c2, 0x0004c2, 0x0004c1, 0x0004c1},
+ {0x0004c3, 0x0004c4, 0x0004c3, 0x0004c3},
+ {0x0004c4, 0x0004c4, 0x0004c3, 0x0004c3},
+ {0x0004c5, 0x0004c6, 0x0004c5, 0x0004c5},
+ {0x0004c6, 0x0004c6, 0x0004c5, 0x0004c5},
+ {0x0004c7, 0x0004c8, 0x0004c7, 0x0004c7},
+ {0x0004c8, 0x0004c8, 0x0004c7, 0x0004c7},
+ {0x0004c9, 0x0004ca, 0x0004c9, 0x0004c9},
+ {0x0004ca, 0x0004ca, 0x0004c9, 0x0004c9},
+ {0x0004cb, 0x0004cc, 0x0004cb, 0x0004cb},
+ {0x0004cc, 0x0004cc, 0x0004cb, 0x0004cb},
+ {0x0004cd, 0x0004ce, 0x0004cd, 0x0004cd},
+ {0x0004ce, 0x0004ce, 0x0004cd, 0x0004cd},
+ {0x0004cf, 0x0004cf, 0x0004c0, 0x0004c0},
+ {0x0004d0, 0x0004d1, 0x0004d0, 0x0004d0},
+ {0x0004d1, 0x0004d1, 0x0004d0, 0x0004d0},
+ {0x0004d2, 0x0004d3, 0x0004d2, 0x0004d2},
+ {0x0004d3, 0x0004d3, 0x0004d2, 0x0004d2},
+ {0x0004d4, 0x0004d5, 0x0004d4, 0x0004d4},
+ {0x0004d5, 0x0004d5, 0x0004d4, 0x0004d4},
+ {0x0004d6, 0x0004d7, 0x0004d6, 0x0004d6},
+ {0x0004d7, 0x0004d7, 0x0004d6, 0x0004d6},
+ {0x0004d8, 0x0004d9, 0x0004d8, 0x0004d8},
+ {0x0004d9, 0x0004d9, 0x0004d8, 0x0004d8},
+ {0x0004da, 0x0004db, 0x0004da, 0x0004da},
+ {0x0004db, 0x0004db, 0x0004da, 0x0004da},
+ {0x0004dc, 0x0004dd, 0x0004dc, 0x0004dc},
+ {0x0004dd, 0x0004dd, 0x0004dc, 0x0004dc},
+ {0x0004de, 0x0004df, 0x0004de, 0x0004de},
+ {0x0004df, 0x0004df, 0x0004de, 0x0004de},
+ {0x0004e0, 0x0004e1, 0x0004e0, 0x0004e0},
+ {0x0004e1, 0x0004e1, 0x0004e0, 0x0004e0},
+ {0x0004e2, 0x0004e3, 0x0004e2, 0x0004e2},
+ {0x0004e3, 0x0004e3, 0x0004e2, 0x0004e2},
+ {0x0004e4, 0x0004e5, 0x0004e4, 0x0004e4},
+ {0x0004e5, 0x0004e5, 0x0004e4, 0x0004e4},
+ {0x0004e6, 0x0004e7, 0x0004e6, 0x0004e6},
+ {0x0004e7, 0x0004e7, 0x0004e6, 0x0004e6},
+ {0x0004e8, 0x0004e9, 0x0004e8, 0x0004e8},
+ {0x0004e9, 0x0004e9, 0x0004e8, 0x0004e8},
+ {0x0004ea, 0x0004eb, 0x0004ea, 0x0004ea},
+ {0x0004eb, 0x0004eb, 0x0004ea, 0x0004ea},
+ {0x0004ec, 0x0004ed, 0x0004ec, 0x0004ec},
+ {0x0004ed, 0x0004ed, 0x0004ec, 0x0004ec},
+ {0x0004ee, 0x0004ef, 0x0004ee, 0x0004ee},
+ {0x0004ef, 0x0004ef, 0x0004ee, 0x0004ee},
+ {0x0004f0, 0x0004f1, 0x0004f0, 0x0004f0},
+ {0x0004f1, 0x0004f1, 0x0004f0, 0x0004f0},
+ {0x0004f2, 0x0004f3, 0x0004f2, 0x0004f2},
+ {0x0004f3, 0x0004f3, 0x0004f2, 0x0004f2},
+ {0x0004f4, 0x0004f5, 0x0004f4, 0x0004f4},
+ {0x0004f5, 0x0004f5, 0x0004f4, 0x0004f4},
+ {0x0004f6, 0x0004f7, 0x0004f6, 0x0004f6},
+ {0x0004f7, 0x0004f7, 0x0004f6, 0x0004f6},
+ {0x0004f8, 0x0004f9, 0x0004f8, 0x0004f8},
+ {0x0004f9, 0x0004f9, 0x0004f8, 0x0004f8},
+ {0x0004fa, 0x0004fb, 0x0004fa, 0x0004fa},
+ {0x0004fb, 0x0004fb, 0x0004fa, 0x0004fa},
+ {0x0004fc, 0x0004fd, 0x0004fc, 0x0004fc},
+ {0x0004fd, 0x0004fd, 0x0004fc, 0x0004fc},
+ {0x0004fe, 0x0004ff, 0x0004fe, 0x0004fe},
+ {0x0004ff, 0x0004ff, 0x0004fe, 0x0004fe},
+ {0x000500, 0x000501, 0x000500, 0x000500},
+ {0x000501, 0x000501, 0x000500, 0x000500},
+ {0x000502, 0x000503, 0x000502, 0x000502},
+ {0x000503, 0x000503, 0x000502, 0x000502},
+ {0x000504, 0x000505, 0x000504, 0x000504},
+ {0x000505, 0x000505, 0x000504, 0x000504},
+ {0x000506, 0x000507, 0x000506, 0x000506},
+ {0x000507, 0x000507, 0x000506, 0x000506},
+ {0x000508, 0x000509, 0x000508, 0x000508},
+ {0x000509, 0x000509, 0x000508, 0x000508},
+ {0x00050a, 0x00050b, 0x00050a, 0x00050a},
+ {0x00050b, 0x00050b, 0x00050a, 0x00050a},
+ {0x00050c, 0x00050d, 0x00050c, 0x00050c},
+ {0x00050d, 0x00050d, 0x00050c, 0x00050c},
+ {0x00050e, 0x00050f, 0x00050e, 0x00050e},
+ {0x00050f, 0x00050f, 0x00050e, 0x00050e},
+ {0x000510, 0x000511, 0x000510, 0x000510},
+ {0x000511, 0x000511, 0x000510, 0x000510},
+ {0x000512, 0x000513, 0x000512, 0x000512},
+ {0x000513, 0x000513, 0x000512, 0x000512},
+ {0x000514, 0x000515, 0x000514, 0x000514},
+ {0x000515, 0x000515, 0x000514, 0x000514},
+ {0x000516, 0x000517, 0x000516, 0x000516},
+ {0x000517, 0x000517, 0x000516, 0x000516},
+ {0x000518, 0x000519, 0x000518, 0x000518},
+ {0x000519, 0x000519, 0x000518, 0x000518},
+ {0x00051a, 0x00051b, 0x00051a, 0x00051a},
+ {0x00051b, 0x00051b, 0x00051a, 0x00051a},
+ {0x00051c, 0x00051d, 0x00051c, 0x00051c},
+ {0x00051d, 0x00051d, 0x00051c, 0x00051c},
+ {0x00051e, 0x00051f, 0x00051e, 0x00051e},
+ {0x00051f, 0x00051f, 0x00051e, 0x00051e},
+ {0x000520, 0x000521, 0x000520, 0x000520},
+ {0x000521, 0x000521, 0x000520, 0x000520},
+ {0x000522, 0x000523, 0x000522, 0x000522},
+ {0x000523, 0x000523, 0x000522, 0x000522},
+ {0x000524, 0x000525, 0x000524, 0x000524},
+ {0x000525, 0x000525, 0x000524, 0x000524},
+ {0x000526, 0x000527, 0x000526, 0x000526},
+ {0x000527, 0x000527, 0x000526, 0x000526},
+ {0x000528, 0x000529, 0x000528, 0x000528},
+ {0x000529, 0x000529, 0x000528, 0x000528},
+ {0x00052a, 0x00052b, 0x00052a, 0x00052a},
+ {0x00052b, 0x00052b, 0x00052a, 0x00052a},
+ {0x00052c, 0x00052d, 0x00052c, 0x00052c},
+ {0x00052d, 0x00052d, 0x00052c, 0x00052c},
+ {0x00052e, 0x00052f, 0x00052e, 0x00052e},
+ {0x00052f, 0x00052f, 0x00052e, 0x00052e},
+ {0x000531, 0x000561, 0x000531, 0x000531},
+ {0x000532, 0x000562, 0x000532, 0x000532},
+ {0x000533, 0x000563, 0x000533, 0x000533},
+ {0x000534, 0x000564, 0x000534, 0x000534},
+ {0x000535, 0x000565, 0x000535, 0x000535},
+ {0x000536, 0x000566, 0x000536, 0x000536},
+ {0x000537, 0x000567, 0x000537, 0x000537},
+ {0x000538, 0x000568, 0x000538, 0x000538},
+ {0x000539, 0x000569, 0x000539, 0x000539},
+ {0x00053a, 0x00056a, 0x00053a, 0x00053a},
+ {0x00053b, 0x00056b, 0x00053b, 0x00053b},
+ {0x00053c, 0x00056c, 0x00053c, 0x00053c},
+ {0x00053d, 0x00056d, 0x00053d, 0x00053d},
+ {0x00053e, 0x00056e, 0x00053e, 0x00053e},
+ {0x00053f, 0x00056f, 0x00053f, 0x00053f},
+ {0x000540, 0x000570, 0x000540, 0x000540},
+ {0x000541, 0x000571, 0x000541, 0x000541},
+ {0x000542, 0x000572, 0x000542, 0x000542},
+ {0x000543, 0x000573, 0x000543, 0x000543},
+ {0x000544, 0x000574, 0x000544, 0x000544},
+ {0x000545, 0x000575, 0x000545, 0x000545},
+ {0x000546, 0x000576, 0x000546, 0x000546},
+ {0x000547, 0x000577, 0x000547, 0x000547},
+ {0x000548, 0x000578, 0x000548, 0x000548},
+ {0x000549, 0x000579, 0x000549, 0x000549},
+ {0x00054a, 0x00057a, 0x00054a, 0x00054a},
+ {0x00054b, 0x00057b, 0x00054b, 0x00054b},
+ {0x00054c, 0x00057c, 0x00054c, 0x00054c},
+ {0x00054d, 0x00057d, 0x00054d, 0x00054d},
+ {0x00054e, 0x00057e, 0x00054e, 0x00054e},
+ {0x00054f, 0x00057f, 0x00054f, 0x00054f},
+ {0x000550, 0x000580, 0x000550, 0x000550},
+ {0x000551, 0x000581, 0x000551, 0x000551},
+ {0x000552, 0x000582, 0x000552, 0x000552},
+ {0x000553, 0x000583, 0x000553, 0x000553},
+ {0x000554, 0x000584, 0x000554, 0x000554},
+ {0x000555, 0x000585, 0x000555, 0x000555},
+ {0x000556, 0x000586, 0x000556, 0x000556},
+ {0x000561, 0x000561, 0x000531, 0x000531},
+ {0x000562, 0x000562, 0x000532, 0x000532},
+ {0x000563, 0x000563, 0x000533, 0x000533},
+ {0x000564, 0x000564, 0x000534, 0x000534},
+ {0x000565, 0x000565, 0x000535, 0x000535},
+ {0x000566, 0x000566, 0x000536, 0x000536},
+ {0x000567, 0x000567, 0x000537, 0x000537},
+ {0x000568, 0x000568, 0x000538, 0x000538},
+ {0x000569, 0x000569, 0x000539, 0x000539},
+ {0x00056a, 0x00056a, 0x00053a, 0x00053a},
+ {0x00056b, 0x00056b, 0x00053b, 0x00053b},
+ {0x00056c, 0x00056c, 0x00053c, 0x00053c},
+ {0x00056d, 0x00056d, 0x00053d, 0x00053d},
+ {0x00056e, 0x00056e, 0x00053e, 0x00053e},
+ {0x00056f, 0x00056f, 0x00053f, 0x00053f},
+ {0x000570, 0x000570, 0x000540, 0x000540},
+ {0x000571, 0x000571, 0x000541, 0x000541},
+ {0x000572, 0x000572, 0x000542, 0x000542},
+ {0x000573, 0x000573, 0x000543, 0x000543},
+ {0x000574, 0x000574, 0x000544, 0x000544},
+ {0x000575, 0x000575, 0x000545, 0x000545},
+ {0x000576, 0x000576, 0x000546, 0x000546},
+ {0x000577, 0x000577, 0x000547, 0x000547},
+ {0x000578, 0x000578, 0x000548, 0x000548},
+ {0x000579, 0x000579, 0x000549, 0x000549},
+ {0x00057a, 0x00057a, 0x00054a, 0x00054a},
+ {0x00057b, 0x00057b, 0x00054b, 0x00054b},
+ {0x00057c, 0x00057c, 0x00054c, 0x00054c},
+ {0x00057d, 0x00057d, 0x00054d, 0x00054d},
+ {0x00057e, 0x00057e, 0x00054e, 0x00054e},
+ {0x00057f, 0x00057f, 0x00054f, 0x00054f},
+ {0x000580, 0x000580, 0x000550, 0x000550},
+ {0x000581, 0x000581, 0x000551, 0x000551},
+ {0x000582, 0x000582, 0x000552, 0x000552},
+ {0x000583, 0x000583, 0x000553, 0x000553},
+ {0x000584, 0x000584, 0x000554, 0x000554},
+ {0x000585, 0x000585, 0x000555, 0x000555},
+ {0x000586, 0x000586, 0x000556, 0x000556},
+ {0x0010a0, 0x002d00, 0x0010a0, 0x0010a0},
+ {0x0010a1, 0x002d01, 0x0010a1, 0x0010a1},
+ {0x0010a2, 0x002d02, 0x0010a2, 0x0010a2},
+ {0x0010a3, 0x002d03, 0x0010a3, 0x0010a3},
+ {0x0010a4, 0x002d04, 0x0010a4, 0x0010a4},
+ {0x0010a5, 0x002d05, 0x0010a5, 0x0010a5},
+ {0x0010a6, 0x002d06, 0x0010a6, 0x0010a6},
+ {0x0010a7, 0x002d07, 0x0010a7, 0x0010a7},
+ {0x0010a8, 0x002d08, 0x0010a8, 0x0010a8},
+ {0x0010a9, 0x002d09, 0x0010a9, 0x0010a9},
+ {0x0010aa, 0x002d0a, 0x0010aa, 0x0010aa},
+ {0x0010ab, 0x002d0b, 0x0010ab, 0x0010ab},
+ {0x0010ac, 0x002d0c, 0x0010ac, 0x0010ac},
+ {0x0010ad, 0x002d0d, 0x0010ad, 0x0010ad},
+ {0x0010ae, 0x002d0e, 0x0010ae, 0x0010ae},
+ {0x0010af, 0x002d0f, 0x0010af, 0x0010af},
+ {0x0010b0, 0x002d10, 0x0010b0, 0x0010b0},
+ {0x0010b1, 0x002d11, 0x0010b1, 0x0010b1},
+ {0x0010b2, 0x002d12, 0x0010b2, 0x0010b2},
+ {0x0010b3, 0x002d13, 0x0010b3, 0x0010b3},
+ {0x0010b4, 0x002d14, 0x0010b4, 0x0010b4},
+ {0x0010b5, 0x002d15, 0x0010b5, 0x0010b5},
+ {0x0010b6, 0x002d16, 0x0010b6, 0x0010b6},
+ {0x0010b7, 0x002d17, 0x0010b7, 0x0010b7},
+ {0x0010b8, 0x002d18, 0x0010b8, 0x0010b8},
+ {0x0010b9, 0x002d19, 0x0010b9, 0x0010b9},
+ {0x0010ba, 0x002d1a, 0x0010ba, 0x0010ba},
+ {0x0010bb, 0x002d1b, 0x0010bb, 0x0010bb},
+ {0x0010bc, 0x002d1c, 0x0010bc, 0x0010bc},
+ {0x0010bd, 0x002d1d, 0x0010bd, 0x0010bd},
+ {0x0010be, 0x002d1e, 0x0010be, 0x0010be},
+ {0x0010bf, 0x002d1f, 0x0010bf, 0x0010bf},
+ {0x0010c0, 0x002d20, 0x0010c0, 0x0010c0},
+ {0x0010c1, 0x002d21, 0x0010c1, 0x0010c1},
+ {0x0010c2, 0x002d22, 0x0010c2, 0x0010c2},
+ {0x0010c3, 0x002d23, 0x0010c3, 0x0010c3},
+ {0x0010c4, 0x002d24, 0x0010c4, 0x0010c4},
+ {0x0010c5, 0x002d25, 0x0010c5, 0x0010c5},
+ {0x0010c7, 0x002d27, 0x0010c7, 0x0010c7},
+ {0x0010cd, 0x002d2d, 0x0010cd, 0x0010cd},
+ {0x0010d0, 0x0010d0, 0x0010d0, 0x001c90},
+ {0x0010d1, 0x0010d1, 0x0010d1, 0x001c91},
+ {0x0010d2, 0x0010d2, 0x0010d2, 0x001c92},
+ {0x0010d3, 0x0010d3, 0x0010d3, 0x001c93},
+ {0x0010d4, 0x0010d4, 0x0010d4, 0x001c94},
+ {0x0010d5, 0x0010d5, 0x0010d5, 0x001c95},
+ {0x0010d6, 0x0010d6, 0x0010d6, 0x001c96},
+ {0x0010d7, 0x0010d7, 0x0010d7, 0x001c97},
+ {0x0010d8, 0x0010d8, 0x0010d8, 0x001c98},
+ {0x0010d9, 0x0010d9, 0x0010d9, 0x001c99},
+ {0x0010da, 0x0010da, 0x0010da, 0x001c9a},
+ {0x0010db, 0x0010db, 0x0010db, 0x001c9b},
+ {0x0010dc, 0x0010dc, 0x0010dc, 0x001c9c},
+ {0x0010dd, 0x0010dd, 0x0010dd, 0x001c9d},
+ {0x0010de, 0x0010de, 0x0010de, 0x001c9e},
+ {0x0010df, 0x0010df, 0x0010df, 0x001c9f},
+ {0x0010e0, 0x0010e0, 0x0010e0, 0x001ca0},
+ {0x0010e1, 0x0010e1, 0x0010e1, 0x001ca1},
+ {0x0010e2, 0x0010e2, 0x0010e2, 0x001ca2},
+ {0x0010e3, 0x0010e3, 0x0010e3, 0x001ca3},
+ {0x0010e4, 0x0010e4, 0x0010e4, 0x001ca4},
+ {0x0010e5, 0x0010e5, 0x0010e5, 0x001ca5},
+ {0x0010e6, 0x0010e6, 0x0010e6, 0x001ca6},
+ {0x0010e7, 0x0010e7, 0x0010e7, 0x001ca7},
+ {0x0010e8, 0x0010e8, 0x0010e8, 0x001ca8},
+ {0x0010e9, 0x0010e9, 0x0010e9, 0x001ca9},
+ {0x0010ea, 0x0010ea, 0x0010ea, 0x001caa},
+ {0x0010eb, 0x0010eb, 0x0010eb, 0x001cab},
+ {0x0010ec, 0x0010ec, 0x0010ec, 0x001cac},
+ {0x0010ed, 0x0010ed, 0x0010ed, 0x001cad},
+ {0x0010ee, 0x0010ee, 0x0010ee, 0x001cae},
+ {0x0010ef, 0x0010ef, 0x0010ef, 0x001caf},
+ {0x0010f0, 0x0010f0, 0x0010f0, 0x001cb0},
+ {0x0010f1, 0x0010f1, 0x0010f1, 0x001cb1},
+ {0x0010f2, 0x0010f2, 0x0010f2, 0x001cb2},
+ {0x0010f3, 0x0010f3, 0x0010f3, 0x001cb3},
+ {0x0010f4, 0x0010f4, 0x0010f4, 0x001cb4},
+ {0x0010f5, 0x0010f5, 0x0010f5, 0x001cb5},
+ {0x0010f6, 0x0010f6, 0x0010f6, 0x001cb6},
+ {0x0010f7, 0x0010f7, 0x0010f7, 0x001cb7},
+ {0x0010f8, 0x0010f8, 0x0010f8, 0x001cb8},
+ {0x0010f9, 0x0010f9, 0x0010f9, 0x001cb9},
+ {0x0010fa, 0x0010fa, 0x0010fa, 0x001cba},
+ {0x0010fd, 0x0010fd, 0x0010fd, 0x001cbd},
+ {0x0010fe, 0x0010fe, 0x0010fe, 0x001cbe},
+ {0x0010ff, 0x0010ff, 0x0010ff, 0x001cbf},
+ {0x0013a0, 0x00ab70, 0x0013a0, 0x0013a0},
+ {0x0013a1, 0x00ab71, 0x0013a1, 0x0013a1},
+ {0x0013a2, 0x00ab72, 0x0013a2, 0x0013a2},
+ {0x0013a3, 0x00ab73, 0x0013a3, 0x0013a3},
+ {0x0013a4, 0x00ab74, 0x0013a4, 0x0013a4},
+ {0x0013a5, 0x00ab75, 0x0013a5, 0x0013a5},
+ {0x0013a6, 0x00ab76, 0x0013a6, 0x0013a6},
+ {0x0013a7, 0x00ab77, 0x0013a7, 0x0013a7},
+ {0x0013a8, 0x00ab78, 0x0013a8, 0x0013a8},
+ {0x0013a9, 0x00ab79, 0x0013a9, 0x0013a9},
+ {0x0013aa, 0x00ab7a, 0x0013aa, 0x0013aa},
+ {0x0013ab, 0x00ab7b, 0x0013ab, 0x0013ab},
+ {0x0013ac, 0x00ab7c, 0x0013ac, 0x0013ac},
+ {0x0013ad, 0x00ab7d, 0x0013ad, 0x0013ad},
+ {0x0013ae, 0x00ab7e, 0x0013ae, 0x0013ae},
+ {0x0013af, 0x00ab7f, 0x0013af, 0x0013af},
+ {0x0013b0, 0x00ab80, 0x0013b0, 0x0013b0},
+ {0x0013b1, 0x00ab81, 0x0013b1, 0x0013b1},
+ {0x0013b2, 0x00ab82, 0x0013b2, 0x0013b2},
+ {0x0013b3, 0x00ab83, 0x0013b3, 0x0013b3},
+ {0x0013b4, 0x00ab84, 0x0013b4, 0x0013b4},
+ {0x0013b5, 0x00ab85, 0x0013b5, 0x0013b5},
+ {0x0013b6, 0x00ab86, 0x0013b6, 0x0013b6},
+ {0x0013b7, 0x00ab87, 0x0013b7, 0x0013b7},
+ {0x0013b8, 0x00ab88, 0x0013b8, 0x0013b8},
+ {0x0013b9, 0x00ab89, 0x0013b9, 0x0013b9},
+ {0x0013ba, 0x00ab8a, 0x0013ba, 0x0013ba},
+ {0x0013bb, 0x00ab8b, 0x0013bb, 0x0013bb},
+ {0x0013bc, 0x00ab8c, 0x0013bc, 0x0013bc},
+ {0x0013bd, 0x00ab8d, 0x0013bd, 0x0013bd},
+ {0x0013be, 0x00ab8e, 0x0013be, 0x0013be},
+ {0x0013bf, 0x00ab8f, 0x0013bf, 0x0013bf},
+ {0x0013c0, 0x00ab90, 0x0013c0, 0x0013c0},
+ {0x0013c1, 0x00ab91, 0x0013c1, 0x0013c1},
+ {0x0013c2, 0x00ab92, 0x0013c2, 0x0013c2},
+ {0x0013c3, 0x00ab93, 0x0013c3, 0x0013c3},
+ {0x0013c4, 0x00ab94, 0x0013c4, 0x0013c4},
+ {0x0013c5, 0x00ab95, 0x0013c5, 0x0013c5},
+ {0x0013c6, 0x00ab96, 0x0013c6, 0x0013c6},
+ {0x0013c7, 0x00ab97, 0x0013c7, 0x0013c7},
+ {0x0013c8, 0x00ab98, 0x0013c8, 0x0013c8},
+ {0x0013c9, 0x00ab99, 0x0013c9, 0x0013c9},
+ {0x0013ca, 0x00ab9a, 0x0013ca, 0x0013ca},
+ {0x0013cb, 0x00ab9b, 0x0013cb, 0x0013cb},
+ {0x0013cc, 0x00ab9c, 0x0013cc, 0x0013cc},
+ {0x0013cd, 0x00ab9d, 0x0013cd, 0x0013cd},
+ {0x0013ce, 0x00ab9e, 0x0013ce, 0x0013ce},
+ {0x0013cf, 0x00ab9f, 0x0013cf, 0x0013cf},
+ {0x0013d0, 0x00aba0, 0x0013d0, 0x0013d0},
+ {0x0013d1, 0x00aba1, 0x0013d1, 0x0013d1},
+ {0x0013d2, 0x00aba2, 0x0013d2, 0x0013d2},
+ {0x0013d3, 0x00aba3, 0x0013d3, 0x0013d3},
+ {0x0013d4, 0x00aba4, 0x0013d4, 0x0013d4},
+ {0x0013d5, 0x00aba5, 0x0013d5, 0x0013d5},
+ {0x0013d6, 0x00aba6, 0x0013d6, 0x0013d6},
+ {0x0013d7, 0x00aba7, 0x0013d7, 0x0013d7},
+ {0x0013d8, 0x00aba8, 0x0013d8, 0x0013d8},
+ {0x0013d9, 0x00aba9, 0x0013d9, 0x0013d9},
+ {0x0013da, 0x00abaa, 0x0013da, 0x0013da},
+ {0x0013db, 0x00abab, 0x0013db, 0x0013db},
+ {0x0013dc, 0x00abac, 0x0013dc, 0x0013dc},
+ {0x0013dd, 0x00abad, 0x0013dd, 0x0013dd},
+ {0x0013de, 0x00abae, 0x0013de, 0x0013de},
+ {0x0013df, 0x00abaf, 0x0013df, 0x0013df},
+ {0x0013e0, 0x00abb0, 0x0013e0, 0x0013e0},
+ {0x0013e1, 0x00abb1, 0x0013e1, 0x0013e1},
+ {0x0013e2, 0x00abb2, 0x0013e2, 0x0013e2},
+ {0x0013e3, 0x00abb3, 0x0013e3, 0x0013e3},
+ {0x0013e4, 0x00abb4, 0x0013e4, 0x0013e4},
+ {0x0013e5, 0x00abb5, 0x0013e5, 0x0013e5},
+ {0x0013e6, 0x00abb6, 0x0013e6, 0x0013e6},
+ {0x0013e7, 0x00abb7, 0x0013e7, 0x0013e7},
+ {0x0013e8, 0x00abb8, 0x0013e8, 0x0013e8},
+ {0x0013e9, 0x00abb9, 0x0013e9, 0x0013e9},
+ {0x0013ea, 0x00abba, 0x0013ea, 0x0013ea},
+ {0x0013eb, 0x00abbb, 0x0013eb, 0x0013eb},
+ {0x0013ec, 0x00abbc, 0x0013ec, 0x0013ec},
+ {0x0013ed, 0x00abbd, 0x0013ed, 0x0013ed},
+ {0x0013ee, 0x00abbe, 0x0013ee, 0x0013ee},
+ {0x0013ef, 0x00abbf, 0x0013ef, 0x0013ef},
+ {0x0013f0, 0x0013f8, 0x0013f0, 0x0013f0},
+ {0x0013f1, 0x0013f9, 0x0013f1, 0x0013f1},
+ {0x0013f2, 0x0013fa, 0x0013f2, 0x0013f2},
+ {0x0013f3, 0x0013fb, 0x0013f3, 0x0013f3},
+ {0x0013f4, 0x0013fc, 0x0013f4, 0x0013f4},
+ {0x0013f5, 0x0013fd, 0x0013f5, 0x0013f5},
+ {0x0013f8, 0x0013f8, 0x0013f0, 0x0013f0},
+ {0x0013f9, 0x0013f9, 0x0013f1, 0x0013f1},
+ {0x0013fa, 0x0013fa, 0x0013f2, 0x0013f2},
+ {0x0013fb, 0x0013fb, 0x0013f3, 0x0013f3},
+ {0x0013fc, 0x0013fc, 0x0013f4, 0x0013f4},
+ {0x0013fd, 0x0013fd, 0x0013f5, 0x0013f5},
+ {0x001c80, 0x001c80, 0x000412, 0x000412},
+ {0x001c81, 0x001c81, 0x000414, 0x000414},
+ {0x001c82, 0x001c82, 0x00041e, 0x00041e},
+ {0x001c83, 0x001c83, 0x000421, 0x000421},
+ {0x001c84, 0x001c84, 0x000422, 0x000422},
+ {0x001c85, 0x001c85, 0x000422, 0x000422},
+ {0x001c86, 0x001c86, 0x00042a, 0x00042a},
+ {0x001c87, 0x001c87, 0x000462, 0x000462},
+ {0x001c88, 0x001c88, 0x00a64a, 0x00a64a},
+ {0x001c90, 0x0010d0, 0x001c90, 0x001c90},
+ {0x001c91, 0x0010d1, 0x001c91, 0x001c91},
+ {0x001c92, 0x0010d2, 0x001c92, 0x001c92},
+ {0x001c93, 0x0010d3, 0x001c93, 0x001c93},
+ {0x001c94, 0x0010d4, 0x001c94, 0x001c94},
+ {0x001c95, 0x0010d5, 0x001c95, 0x001c95},
+ {0x001c96, 0x0010d6, 0x001c96, 0x001c96},
+ {0x001c97, 0x0010d7, 0x001c97, 0x001c97},
+ {0x001c98, 0x0010d8, 0x001c98, 0x001c98},
+ {0x001c99, 0x0010d9, 0x001c99, 0x001c99},
+ {0x001c9a, 0x0010da, 0x001c9a, 0x001c9a},
+ {0x001c9b, 0x0010db, 0x001c9b, 0x001c9b},
+ {0x001c9c, 0x0010dc, 0x001c9c, 0x001c9c},
+ {0x001c9d, 0x0010dd, 0x001c9d, 0x001c9d},
+ {0x001c9e, 0x0010de, 0x001c9e, 0x001c9e},
+ {0x001c9f, 0x0010df, 0x001c9f, 0x001c9f},
+ {0x001ca0, 0x0010e0, 0x001ca0, 0x001ca0},
+ {0x001ca1, 0x0010e1, 0x001ca1, 0x001ca1},
+ {0x001ca2, 0x0010e2, 0x001ca2, 0x001ca2},
+ {0x001ca3, 0x0010e3, 0x001ca3, 0x001ca3},
+ {0x001ca4, 0x0010e4, 0x001ca4, 0x001ca4},
+ {0x001ca5, 0x0010e5, 0x001ca5, 0x001ca5},
+ {0x001ca6, 0x0010e6, 0x001ca6, 0x001ca6},
+ {0x001ca7, 0x0010e7, 0x001ca7, 0x001ca7},
+ {0x001ca8, 0x0010e8, 0x001ca8, 0x001ca8},
+ {0x001ca9, 0x0010e9, 0x001ca9, 0x001ca9},
+ {0x001caa, 0x0010ea, 0x001caa, 0x001caa},
+ {0x001cab, 0x0010eb, 0x001cab, 0x001cab},
+ {0x001cac, 0x0010ec, 0x001cac, 0x001cac},
+ {0x001cad, 0x0010ed, 0x001cad, 0x001cad},
+ {0x001cae, 0x0010ee, 0x001cae, 0x001cae},
+ {0x001caf, 0x0010ef, 0x001caf, 0x001caf},
+ {0x001cb0, 0x0010f0, 0x001cb0, 0x001cb0},
+ {0x001cb1, 0x0010f1, 0x001cb1, 0x001cb1},
+ {0x001cb2, 0x0010f2, 0x001cb2, 0x001cb2},
+ {0x001cb3, 0x0010f3, 0x001cb3, 0x001cb3},
+ {0x001cb4, 0x0010f4, 0x001cb4, 0x001cb4},
+ {0x001cb5, 0x0010f5, 0x001cb5, 0x001cb5},
+ {0x001cb6, 0x0010f6, 0x001cb6, 0x001cb6},
+ {0x001cb7, 0x0010f7, 0x001cb7, 0x001cb7},
+ {0x001cb8, 0x0010f8, 0x001cb8, 0x001cb8},
+ {0x001cb9, 0x0010f9, 0x001cb9, 0x001cb9},
+ {0x001cba, 0x0010fa, 0x001cba, 0x001cba},
+ {0x001cbd, 0x0010fd, 0x001cbd, 0x001cbd},
+ {0x001cbe, 0x0010fe, 0x001cbe, 0x001cbe},
+ {0x001cbf, 0x0010ff, 0x001cbf, 0x001cbf},
+ {0x001d79, 0x001d79, 0x00a77d, 0x00a77d},
+ {0x001d7d, 0x001d7d, 0x002c63, 0x002c63},
+ {0x001d8e, 0x001d8e, 0x00a7c6, 0x00a7c6},
+ {0x001e00, 0x001e01, 0x001e00, 0x001e00},
+ {0x001e01, 0x001e01, 0x001e00, 0x001e00},
+ {0x001e02, 0x001e03, 0x001e02, 0x001e02},
+ {0x001e03, 0x001e03, 0x001e02, 0x001e02},
+ {0x001e04, 0x001e05, 0x001e04, 0x001e04},
+ {0x001e05, 0x001e05, 0x001e04, 0x001e04},
+ {0x001e06, 0x001e07, 0x001e06, 0x001e06},
+ {0x001e07, 0x001e07, 0x001e06, 0x001e06},
+ {0x001e08, 0x001e09, 0x001e08, 0x001e08},
+ {0x001e09, 0x001e09, 0x001e08, 0x001e08},
+ {0x001e0a, 0x001e0b, 0x001e0a, 0x001e0a},
+ {0x001e0b, 0x001e0b, 0x001e0a, 0x001e0a},
+ {0x001e0c, 0x001e0d, 0x001e0c, 0x001e0c},
+ {0x001e0d, 0x001e0d, 0x001e0c, 0x001e0c},
+ {0x001e0e, 0x001e0f, 0x001e0e, 0x001e0e},
+ {0x001e0f, 0x001e0f, 0x001e0e, 0x001e0e},
+ {0x001e10, 0x001e11, 0x001e10, 0x001e10},
+ {0x001e11, 0x001e11, 0x001e10, 0x001e10},
+ {0x001e12, 0x001e13, 0x001e12, 0x001e12},
+ {0x001e13, 0x001e13, 0x001e12, 0x001e12},
+ {0x001e14, 0x001e15, 0x001e14, 0x001e14},
+ {0x001e15, 0x001e15, 0x001e14, 0x001e14},
+ {0x001e16, 0x001e17, 0x001e16, 0x001e16},
+ {0x001e17, 0x001e17, 0x001e16, 0x001e16},
+ {0x001e18, 0x001e19, 0x001e18, 0x001e18},
+ {0x001e19, 0x001e19, 0x001e18, 0x001e18},
+ {0x001e1a, 0x001e1b, 0x001e1a, 0x001e1a},
+ {0x001e1b, 0x001e1b, 0x001e1a, 0x001e1a},
+ {0x001e1c, 0x001e1d, 0x001e1c, 0x001e1c},
+ {0x001e1d, 0x001e1d, 0x001e1c, 0x001e1c},
+ {0x001e1e, 0x001e1f, 0x001e1e, 0x001e1e},
+ {0x001e1f, 0x001e1f, 0x001e1e, 0x001e1e},
+ {0x001e20, 0x001e21, 0x001e20, 0x001e20},
+ {0x001e21, 0x001e21, 0x001e20, 0x001e20},
+ {0x001e22, 0x001e23, 0x001e22, 0x001e22},
+ {0x001e23, 0x001e23, 0x001e22, 0x001e22},
+ {0x001e24, 0x001e25, 0x001e24, 0x001e24},
+ {0x001e25, 0x001e25, 0x001e24, 0x001e24},
+ {0x001e26, 0x001e27, 0x001e26, 0x001e26},
+ {0x001e27, 0x001e27, 0x001e26, 0x001e26},
+ {0x001e28, 0x001e29, 0x001e28, 0x001e28},
+ {0x001e29, 0x001e29, 0x001e28, 0x001e28},
+ {0x001e2a, 0x001e2b, 0x001e2a, 0x001e2a},
+ {0x001e2b, 0x001e2b, 0x001e2a, 0x001e2a},
+ {0x001e2c, 0x001e2d, 0x001e2c, 0x001e2c},
+ {0x001e2d, 0x001e2d, 0x001e2c, 0x001e2c},
+ {0x001e2e, 0x001e2f, 0x001e2e, 0x001e2e},
+ {0x001e2f, 0x001e2f, 0x001e2e, 0x001e2e},
+ {0x001e30, 0x001e31, 0x001e30, 0x001e30},
+ {0x001e31, 0x001e31, 0x001e30, 0x001e30},
+ {0x001e32, 0x001e33, 0x001e32, 0x001e32},
+ {0x001e33, 0x001e33, 0x001e32, 0x001e32},
+ {0x001e34, 0x001e35, 0x001e34, 0x001e34},
+ {0x001e35, 0x001e35, 0x001e34, 0x001e34},
+ {0x001e36, 0x001e37, 0x001e36, 0x001e36},
+ {0x001e37, 0x001e37, 0x001e36, 0x001e36},
+ {0x001e38, 0x001e39, 0x001e38, 0x001e38},
+ {0x001e39, 0x001e39, 0x001e38, 0x001e38},
+ {0x001e3a, 0x001e3b, 0x001e3a, 0x001e3a},
+ {0x001e3b, 0x001e3b, 0x001e3a, 0x001e3a},
+ {0x001e3c, 0x001e3d, 0x001e3c, 0x001e3c},
+ {0x001e3d, 0x001e3d, 0x001e3c, 0x001e3c},
+ {0x001e3e, 0x001e3f, 0x001e3e, 0x001e3e},
+ {0x001e3f, 0x001e3f, 0x001e3e, 0x001e3e},
+ {0x001e40, 0x001e41, 0x001e40, 0x001e40},
+ {0x001e41, 0x001e41, 0x001e40, 0x001e40},
+ {0x001e42, 0x001e43, 0x001e42, 0x001e42},
+ {0x001e43, 0x001e43, 0x001e42, 0x001e42},
+ {0x001e44, 0x001e45, 0x001e44, 0x001e44},
+ {0x001e45, 0x001e45, 0x001e44, 0x001e44},
+ {0x001e46, 0x001e47, 0x001e46, 0x001e46},
+ {0x001e47, 0x001e47, 0x001e46, 0x001e46},
+ {0x001e48, 0x001e49, 0x001e48, 0x001e48},
+ {0x001e49, 0x001e49, 0x001e48, 0x001e48},
+ {0x001e4a, 0x001e4b, 0x001e4a, 0x001e4a},
+ {0x001e4b, 0x001e4b, 0x001e4a, 0x001e4a},
+ {0x001e4c, 0x001e4d, 0x001e4c, 0x001e4c},
+ {0x001e4d, 0x001e4d, 0x001e4c, 0x001e4c},
+ {0x001e4e, 0x001e4f, 0x001e4e, 0x001e4e},
+ {0x001e4f, 0x001e4f, 0x001e4e, 0x001e4e},
+ {0x001e50, 0x001e51, 0x001e50, 0x001e50},
+ {0x001e51, 0x001e51, 0x001e50, 0x001e50},
+ {0x001e52, 0x001e53, 0x001e52, 0x001e52},
+ {0x001e53, 0x001e53, 0x001e52, 0x001e52},
+ {0x001e54, 0x001e55, 0x001e54, 0x001e54},
+ {0x001e55, 0x001e55, 0x001e54, 0x001e54},
+ {0x001e56, 0x001e57, 0x001e56, 0x001e56},
+ {0x001e57, 0x001e57, 0x001e56, 0x001e56},
+ {0x001e58, 0x001e59, 0x001e58, 0x001e58},
+ {0x001e59, 0x001e59, 0x001e58, 0x001e58},
+ {0x001e5a, 0x001e5b, 0x001e5a, 0x001e5a},
+ {0x001e5b, 0x001e5b, 0x001e5a, 0x001e5a},
+ {0x001e5c, 0x001e5d, 0x001e5c, 0x001e5c},
+ {0x001e5d, 0x001e5d, 0x001e5c, 0x001e5c},
+ {0x001e5e, 0x001e5f, 0x001e5e, 0x001e5e},
+ {0x001e5f, 0x001e5f, 0x001e5e, 0x001e5e},
+ {0x001e60, 0x001e61, 0x001e60, 0x001e60},
+ {0x001e61, 0x001e61, 0x001e60, 0x001e60},
+ {0x001e62, 0x001e63, 0x001e62, 0x001e62},
+ {0x001e63, 0x001e63, 0x001e62, 0x001e62},
+ {0x001e64, 0x001e65, 0x001e64, 0x001e64},
+ {0x001e65, 0x001e65, 0x001e64, 0x001e64},
+ {0x001e66, 0x001e67, 0x001e66, 0x001e66},
+ {0x001e67, 0x001e67, 0x001e66, 0x001e66},
+ {0x001e68, 0x001e69, 0x001e68, 0x001e68},
+ {0x001e69, 0x001e69, 0x001e68, 0x001e68},
+ {0x001e6a, 0x001e6b, 0x001e6a, 0x001e6a},
+ {0x001e6b, 0x001e6b, 0x001e6a, 0x001e6a},
+ {0x001e6c, 0x001e6d, 0x001e6c, 0x001e6c},
+ {0x001e6d, 0x001e6d, 0x001e6c, 0x001e6c},
+ {0x001e6e, 0x001e6f, 0x001e6e, 0x001e6e},
+ {0x001e6f, 0x001e6f, 0x001e6e, 0x001e6e},
+ {0x001e70, 0x001e71, 0x001e70, 0x001e70},
+ {0x001e71, 0x001e71, 0x001e70, 0x001e70},
+ {0x001e72, 0x001e73, 0x001e72, 0x001e72},
+ {0x001e73, 0x001e73, 0x001e72, 0x001e72},
+ {0x001e74, 0x001e75, 0x001e74, 0x001e74},
+ {0x001e75, 0x001e75, 0x001e74, 0x001e74},
+ {0x001e76, 0x001e77, 0x001e76, 0x001e76},
+ {0x001e77, 0x001e77, 0x001e76, 0x001e76},
+ {0x001e78, 0x001e79, 0x001e78, 0x001e78},
+ {0x001e79, 0x001e79, 0x001e78, 0x001e78},
+ {0x001e7a, 0x001e7b, 0x001e7a, 0x001e7a},
+ {0x001e7b, 0x001e7b, 0x001e7a, 0x001e7a},
+ {0x001e7c, 0x001e7d, 0x001e7c, 0x001e7c},
+ {0x001e7d, 0x001e7d, 0x001e7c, 0x001e7c},
+ {0x001e7e, 0x001e7f, 0x001e7e, 0x001e7e},
+ {0x001e7f, 0x001e7f, 0x001e7e, 0x001e7e},
+ {0x001e80, 0x001e81, 0x001e80, 0x001e80},
+ {0x001e81, 0x001e81, 0x001e80, 0x001e80},
+ {0x001e82, 0x001e83, 0x001e82, 0x001e82},
+ {0x001e83, 0x001e83, 0x001e82, 0x001e82},
+ {0x001e84, 0x001e85, 0x001e84, 0x001e84},
+ {0x001e85, 0x001e85, 0x001e84, 0x001e84},
+ {0x001e86, 0x001e87, 0x001e86, 0x001e86},
+ {0x001e87, 0x001e87, 0x001e86, 0x001e86},
+ {0x001e88, 0x001e89, 0x001e88, 0x001e88},
+ {0x001e89, 0x001e89, 0x001e88, 0x001e88},
+ {0x001e8a, 0x001e8b, 0x001e8a, 0x001e8a},
+ {0x001e8b, 0x001e8b, 0x001e8a, 0x001e8a},
+ {0x001e8c, 0x001e8d, 0x001e8c, 0x001e8c},
+ {0x001e8d, 0x001e8d, 0x001e8c, 0x001e8c},
+ {0x001e8e, 0x001e8f, 0x001e8e, 0x001e8e},
+ {0x001e8f, 0x001e8f, 0x001e8e, 0x001e8e},
+ {0x001e90, 0x001e91, 0x001e90, 0x001e90},
+ {0x001e91, 0x001e91, 0x001e90, 0x001e90},
+ {0x001e92, 0x001e93, 0x001e92, 0x001e92},
+ {0x001e93, 0x001e93, 0x001e92, 0x001e92},
+ {0x001e94, 0x001e95, 0x001e94, 0x001e94},
+ {0x001e95, 0x001e95, 0x001e94, 0x001e94},
+ {0x001e9b, 0x001e9b, 0x001e60, 0x001e60},
+ {0x001e9e, 0x0000df, 0x001e9e, 0x001e9e},
+ {0x001ea0, 0x001ea1, 0x001ea0, 0x001ea0},
+ {0x001ea1, 0x001ea1, 0x001ea0, 0x001ea0},
+ {0x001ea2, 0x001ea3, 0x001ea2, 0x001ea2},
+ {0x001ea3, 0x001ea3, 0x001ea2, 0x001ea2},
+ {0x001ea4, 0x001ea5, 0x001ea4, 0x001ea4},
+ {0x001ea5, 0x001ea5, 0x001ea4, 0x001ea4},
+ {0x001ea6, 0x001ea7, 0x001ea6, 0x001ea6},
+ {0x001ea7, 0x001ea7, 0x001ea6, 0x001ea6},
+ {0x001ea8, 0x001ea9, 0x001ea8, 0x001ea8},
+ {0x001ea9, 0x001ea9, 0x001ea8, 0x001ea8},
+ {0x001eaa, 0x001eab, 0x001eaa, 0x001eaa},
+ {0x001eab, 0x001eab, 0x001eaa, 0x001eaa},
+ {0x001eac, 0x001ead, 0x001eac, 0x001eac},
+ {0x001ead, 0x001ead, 0x001eac, 0x001eac},
+ {0x001eae, 0x001eaf, 0x001eae, 0x001eae},
+ {0x001eaf, 0x001eaf, 0x001eae, 0x001eae},
+ {0x001eb0, 0x001eb1, 0x001eb0, 0x001eb0},
+ {0x001eb1, 0x001eb1, 0x001eb0, 0x001eb0},
+ {0x001eb2, 0x001eb3, 0x001eb2, 0x001eb2},
+ {0x001eb3, 0x001eb3, 0x001eb2, 0x001eb2},
+ {0x001eb4, 0x001eb5, 0x001eb4, 0x001eb4},
+ {0x001eb5, 0x001eb5, 0x001eb4, 0x001eb4},
+ {0x001eb6, 0x001eb7, 0x001eb6, 0x001eb6},
+ {0x001eb7, 0x001eb7, 0x001eb6, 0x001eb6},
+ {0x001eb8, 0x001eb9, 0x001eb8, 0x001eb8},
+ {0x001eb9, 0x001eb9, 0x001eb8, 0x001eb8},
+ {0x001eba, 0x001ebb, 0x001eba, 0x001eba},
+ {0x001ebb, 0x001ebb, 0x001eba, 0x001eba},
+ {0x001ebc, 0x001ebd, 0x001ebc, 0x001ebc},
+ {0x001ebd, 0x001ebd, 0x001ebc, 0x001ebc},
+ {0x001ebe, 0x001ebf, 0x001ebe, 0x001ebe},
+ {0x001ebf, 0x001ebf, 0x001ebe, 0x001ebe},
+ {0x001ec0, 0x001ec1, 0x001ec0, 0x001ec0},
+ {0x001ec1, 0x001ec1, 0x001ec0, 0x001ec0},
+ {0x001ec2, 0x001ec3, 0x001ec2, 0x001ec2},
+ {0x001ec3, 0x001ec3, 0x001ec2, 0x001ec2},
+ {0x001ec4, 0x001ec5, 0x001ec4, 0x001ec4},
+ {0x001ec5, 0x001ec5, 0x001ec4, 0x001ec4},
+ {0x001ec6, 0x001ec7, 0x001ec6, 0x001ec6},
+ {0x001ec7, 0x001ec7, 0x001ec6, 0x001ec6},
+ {0x001ec8, 0x001ec9, 0x001ec8, 0x001ec8},
+ {0x001ec9, 0x001ec9, 0x001ec8, 0x001ec8},
+ {0x001eca, 0x001ecb, 0x001eca, 0x001eca},
+ {0x001ecb, 0x001ecb, 0x001eca, 0x001eca},
+ {0x001ecc, 0x001ecd, 0x001ecc, 0x001ecc},
+ {0x001ecd, 0x001ecd, 0x001ecc, 0x001ecc},
+ {0x001ece, 0x001ecf, 0x001ece, 0x001ece},
+ {0x001ecf, 0x001ecf, 0x001ece, 0x001ece},
+ {0x001ed0, 0x001ed1, 0x001ed0, 0x001ed0},
+ {0x001ed1, 0x001ed1, 0x001ed0, 0x001ed0},
+ {0x001ed2, 0x001ed3, 0x001ed2, 0x001ed2},
+ {0x001ed3, 0x001ed3, 0x001ed2, 0x001ed2},
+ {0x001ed4, 0x001ed5, 0x001ed4, 0x001ed4},
+ {0x001ed5, 0x001ed5, 0x001ed4, 0x001ed4},
+ {0x001ed6, 0x001ed7, 0x001ed6, 0x001ed6},
+ {0x001ed7, 0x001ed7, 0x001ed6, 0x001ed6},
+ {0x001ed8, 0x001ed9, 0x001ed8, 0x001ed8},
+ {0x001ed9, 0x001ed9, 0x001ed8, 0x001ed8},
+ {0x001eda, 0x001edb, 0x001eda, 0x001eda},
+ {0x001edb, 0x001edb, 0x001eda, 0x001eda},
+ {0x001edc, 0x001edd, 0x001edc, 0x001edc},
+ {0x001edd, 0x001edd, 0x001edc, 0x001edc},
+ {0x001ede, 0x001edf, 0x001ede, 0x001ede},
+ {0x001edf, 0x001edf, 0x001ede, 0x001ede},
+ {0x001ee0, 0x001ee1, 0x001ee0, 0x001ee0},
+ {0x001ee1, 0x001ee1, 0x001ee0, 0x001ee0},
+ {0x001ee2, 0x001ee3, 0x001ee2, 0x001ee2},
+ {0x001ee3, 0x001ee3, 0x001ee2, 0x001ee2},
+ {0x001ee4, 0x001ee5, 0x001ee4, 0x001ee4},
+ {0x001ee5, 0x001ee5, 0x001ee4, 0x001ee4},
+ {0x001ee6, 0x001ee7, 0x001ee6, 0x001ee6},
+ {0x001ee7, 0x001ee7, 0x001ee6, 0x001ee6},
+ {0x001ee8, 0x001ee9, 0x001ee8, 0x001ee8},
+ {0x001ee9, 0x001ee9, 0x001ee8, 0x001ee8},
+ {0x001eea, 0x001eeb, 0x001eea, 0x001eea},
+ {0x001eeb, 0x001eeb, 0x001eea, 0x001eea},
+ {0x001eec, 0x001eed, 0x001eec, 0x001eec},
+ {0x001eed, 0x001eed, 0x001eec, 0x001eec},
+ {0x001eee, 0x001eef, 0x001eee, 0x001eee},
+ {0x001eef, 0x001eef, 0x001eee, 0x001eee},
+ {0x001ef0, 0x001ef1, 0x001ef0, 0x001ef0},
+ {0x001ef1, 0x001ef1, 0x001ef0, 0x001ef0},
+ {0x001ef2, 0x001ef3, 0x001ef2, 0x001ef2},
+ {0x001ef3, 0x001ef3, 0x001ef2, 0x001ef2},
+ {0x001ef4, 0x001ef5, 0x001ef4, 0x001ef4},
+ {0x001ef5, 0x001ef5, 0x001ef4, 0x001ef4},
+ {0x001ef6, 0x001ef7, 0x001ef6, 0x001ef6},
+ {0x001ef7, 0x001ef7, 0x001ef6, 0x001ef6},
+ {0x001ef8, 0x001ef9, 0x001ef8, 0x001ef8},
+ {0x001ef9, 0x001ef9, 0x001ef8, 0x001ef8},
+ {0x001efa, 0x001efb, 0x001efa, 0x001efa},
+ {0x001efb, 0x001efb, 0x001efa, 0x001efa},
+ {0x001efc, 0x001efd, 0x001efc, 0x001efc},
+ {0x001efd, 0x001efd, 0x001efc, 0x001efc},
+ {0x001efe, 0x001eff, 0x001efe, 0x001efe},
+ {0x001eff, 0x001eff, 0x001efe, 0x001efe},
+ {0x001f00, 0x001f00, 0x001f08, 0x001f08},
+ {0x001f01, 0x001f01, 0x001f09, 0x001f09},
+ {0x001f02, 0x001f02, 0x001f0a, 0x001f0a},
+ {0x001f03, 0x001f03, 0x001f0b, 0x001f0b},
+ {0x001f04, 0x001f04, 0x001f0c, 0x001f0c},
+ {0x001f05, 0x001f05, 0x001f0d, 0x001f0d},
+ {0x001f06, 0x001f06, 0x001f0e, 0x001f0e},
+ {0x001f07, 0x001f07, 0x001f0f, 0x001f0f},
+ {0x001f08, 0x001f00, 0x001f08, 0x001f08},
+ {0x001f09, 0x001f01, 0x001f09, 0x001f09},
+ {0x001f0a, 0x001f02, 0x001f0a, 0x001f0a},
+ {0x001f0b, 0x001f03, 0x001f0b, 0x001f0b},
+ {0x001f0c, 0x001f04, 0x001f0c, 0x001f0c},
+ {0x001f0d, 0x001f05, 0x001f0d, 0x001f0d},
+ {0x001f0e, 0x001f06, 0x001f0e, 0x001f0e},
+ {0x001f0f, 0x001f07, 0x001f0f, 0x001f0f},
+ {0x001f10, 0x001f10, 0x001f18, 0x001f18},
+ {0x001f11, 0x001f11, 0x001f19, 0x001f19},
+ {0x001f12, 0x001f12, 0x001f1a, 0x001f1a},
+ {0x001f13, 0x001f13, 0x001f1b, 0x001f1b},
+ {0x001f14, 0x001f14, 0x001f1c, 0x001f1c},
+ {0x001f15, 0x001f15, 0x001f1d, 0x001f1d},
+ {0x001f18, 0x001f10, 0x001f18, 0x001f18},
+ {0x001f19, 0x001f11, 0x001f19, 0x001f19},
+ {0x001f1a, 0x001f12, 0x001f1a, 0x001f1a},
+ {0x001f1b, 0x001f13, 0x001f1b, 0x001f1b},
+ {0x001f1c, 0x001f14, 0x001f1c, 0x001f1c},
+ {0x001f1d, 0x001f15, 0x001f1d, 0x001f1d},
+ {0x001f20, 0x001f20, 0x001f28, 0x001f28},
+ {0x001f21, 0x001f21, 0x001f29, 0x001f29},
+ {0x001f22, 0x001f22, 0x001f2a, 0x001f2a},
+ {0x001f23, 0x001f23, 0x001f2b, 0x001f2b},
+ {0x001f24, 0x001f24, 0x001f2c, 0x001f2c},
+ {0x001f25, 0x001f25, 0x001f2d, 0x001f2d},
+ {0x001f26, 0x001f26, 0x001f2e, 0x001f2e},
+ {0x001f27, 0x001f27, 0x001f2f, 0x001f2f},
+ {0x001f28, 0x001f20, 0x001f28, 0x001f28},
+ {0x001f29, 0x001f21, 0x001f29, 0x001f29},
+ {0x001f2a, 0x001f22, 0x001f2a, 0x001f2a},
+ {0x001f2b, 0x001f23, 0x001f2b, 0x001f2b},
+ {0x001f2c, 0x001f24, 0x001f2c, 0x001f2c},
+ {0x001f2d, 0x001f25, 0x001f2d, 0x001f2d},
+ {0x001f2e, 0x001f26, 0x001f2e, 0x001f2e},
+ {0x001f2f, 0x001f27, 0x001f2f, 0x001f2f},
+ {0x001f30, 0x001f30, 0x001f38, 0x001f38},
+ {0x001f31, 0x001f31, 0x001f39, 0x001f39},
+ {0x001f32, 0x001f32, 0x001f3a, 0x001f3a},
+ {0x001f33, 0x001f33, 0x001f3b, 0x001f3b},
+ {0x001f34, 0x001f34, 0x001f3c, 0x001f3c},
+ {0x001f35, 0x001f35, 0x001f3d, 0x001f3d},
+ {0x001f36, 0x001f36, 0x001f3e, 0x001f3e},
+ {0x001f37, 0x001f37, 0x001f3f, 0x001f3f},
+ {0x001f38, 0x001f30, 0x001f38, 0x001f38},
+ {0x001f39, 0x001f31, 0x001f39, 0x001f39},
+ {0x001f3a, 0x001f32, 0x001f3a, 0x001f3a},
+ {0x001f3b, 0x001f33, 0x001f3b, 0x001f3b},
+ {0x001f3c, 0x001f34, 0x001f3c, 0x001f3c},
+ {0x001f3d, 0x001f35, 0x001f3d, 0x001f3d},
+ {0x001f3e, 0x001f36, 0x001f3e, 0x001f3e},
+ {0x001f3f, 0x001f37, 0x001f3f, 0x001f3f},
+ {0x001f40, 0x001f40, 0x001f48, 0x001f48},
+ {0x001f41, 0x001f41, 0x001f49, 0x001f49},
+ {0x001f42, 0x001f42, 0x001f4a, 0x001f4a},
+ {0x001f43, 0x001f43, 0x001f4b, 0x001f4b},
+ {0x001f44, 0x001f44, 0x001f4c, 0x001f4c},
+ {0x001f45, 0x001f45, 0x001f4d, 0x001f4d},
+ {0x001f48, 0x001f40, 0x001f48, 0x001f48},
+ {0x001f49, 0x001f41, 0x001f49, 0x001f49},
+ {0x001f4a, 0x001f42, 0x001f4a, 0x001f4a},
+ {0x001f4b, 0x001f43, 0x001f4b, 0x001f4b},
+ {0x001f4c, 0x001f44, 0x001f4c, 0x001f4c},
+ {0x001f4d, 0x001f45, 0x001f4d, 0x001f4d},
+ {0x001f51, 0x001f51, 0x001f59, 0x001f59},
+ {0x001f53, 0x001f53, 0x001f5b, 0x001f5b},
+ {0x001f55, 0x001f55, 0x001f5d, 0x001f5d},
+ {0x001f57, 0x001f57, 0x001f5f, 0x001f5f},
+ {0x001f59, 0x001f51, 0x001f59, 0x001f59},
+ {0x001f5b, 0x001f53, 0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f55, 0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f57, 0x001f5f, 0x001f5f},
+ {0x001f60, 0x001f60, 0x001f68, 0x001f68},
+ {0x001f61, 0x001f61, 0x001f69, 0x001f69},
+ {0x001f62, 0x001f62, 0x001f6a, 0x001f6a},
+ {0x001f63, 0x001f63, 0x001f6b, 0x001f6b},
+ {0x001f64, 0x001f64, 0x001f6c, 0x001f6c},
+ {0x001f65, 0x001f65, 0x001f6d, 0x001f6d},
+ {0x001f66, 0x001f66, 0x001f6e, 0x001f6e},
+ {0x001f67, 0x001f67, 0x001f6f, 0x001f6f},
+ {0x001f68, 0x001f60, 0x001f68, 0x001f68},
+ {0x001f69, 0x001f61, 0x001f69, 0x001f69},
+ {0x001f6a, 0x001f62, 0x001f6a, 0x001f6a},
+ {0x001f6b, 0x001f63, 0x001f6b, 0x001f6b},
+ {0x001f6c, 0x001f64, 0x001f6c, 0x001f6c},
+ {0x001f6d, 0x001f65, 0x001f6d, 0x001f6d},
+ {0x001f6e, 0x001f66, 0x001f6e, 0x001f6e},
+ {0x001f6f, 0x001f67, 0x001f6f, 0x001f6f},
+ {0x001f70, 0x001f70, 0x001fba, 0x001fba},
+ {0x001f71, 0x001f71, 0x001fbb, 0x001fbb},
+ {0x001f72, 0x001f72, 0x001fc8, 0x001fc8},
+ {0x001f73, 0x001f73, 0x001fc9, 0x001fc9},
+ {0x001f74, 0x001f74, 0x001fca, 0x001fca},
+ {0x001f75, 0x001f75, 0x001fcb, 0x001fcb},
+ {0x001f76, 0x001f76, 0x001fda, 0x001fda},
+ {0x001f77, 0x001f77, 0x001fdb, 0x001fdb},
+ {0x001f78, 0x001f78, 0x001ff8, 0x001ff8},
+ {0x001f79, 0x001f79, 0x001ff9, 0x001ff9},
+ {0x001f7a, 0x001f7a, 0x001fea, 0x001fea},
+ {0x001f7b, 0x001f7b, 0x001feb, 0x001feb},
+ {0x001f7c, 0x001f7c, 0x001ffa, 0x001ffa},
+ {0x001f7d, 0x001f7d, 0x001ffb, 0x001ffb},
+ {0x001f80, 0x001f80, 0x001f88, 0x001f88},
+ {0x001f81, 0x001f81, 0x001f89, 0x001f89},
+ {0x001f82, 0x001f82, 0x001f8a, 0x001f8a},
+ {0x001f83, 0x001f83, 0x001f8b, 0x001f8b},
+ {0x001f84, 0x001f84, 0x001f8c, 0x001f8c},
+ {0x001f85, 0x001f85, 0x001f8d, 0x001f8d},
+ {0x001f86, 0x001f86, 0x001f8e, 0x001f8e},
+ {0x001f87, 0x001f87, 0x001f8f, 0x001f8f},
+ {0x001f88, 0x001f80, 0x001f88, 0x001f88},
+ {0x001f89, 0x001f81, 0x001f89, 0x001f89},
+ {0x001f8a, 0x001f82, 0x001f8a, 0x001f8a},
+ {0x001f8b, 0x001f83, 0x001f8b, 0x001f8b},
+ {0x001f8c, 0x001f84, 0x001f8c, 0x001f8c},
+ {0x001f8d, 0x001f85, 0x001f8d, 0x001f8d},
+ {0x001f8e, 0x001f86, 0x001f8e, 0x001f8e},
+ {0x001f8f, 0x001f87, 0x001f8f, 0x001f8f},
+ {0x001f90, 0x001f90, 0x001f98, 0x001f98},
+ {0x001f91, 0x001f91, 0x001f99, 0x001f99},
+ {0x001f92, 0x001f92, 0x001f9a, 0x001f9a},
+ {0x001f93, 0x001f93, 0x001f9b, 0x001f9b},
+ {0x001f94, 0x001f94, 0x001f9c, 0x001f9c},
+ {0x001f95, 0x001f95, 0x001f9d, 0x001f9d},
+ {0x001f96, 0x001f96, 0x001f9e, 0x001f9e},
+ {0x001f97, 0x001f97, 0x001f9f, 0x001f9f},
+ {0x001f98, 0x001f90, 0x001f98, 0x001f98},
+ {0x001f99, 0x001f91, 0x001f99, 0x001f99},
+ {0x001f9a, 0x001f92, 0x001f9a, 0x001f9a},
+ {0x001f9b, 0x001f93, 0x001f9b, 0x001f9b},
+ {0x001f9c, 0x001f94, 0x001f9c, 0x001f9c},
+ {0x001f9d, 0x001f95, 0x001f9d, 0x001f9d},
+ {0x001f9e, 0x001f96, 0x001f9e, 0x001f9e},
+ {0x001f9f, 0x001f97, 0x001f9f, 0x001f9f},
+ {0x001fa0, 0x001fa0, 0x001fa8, 0x001fa8},
+ {0x001fa1, 0x001fa1, 0x001fa9, 0x001fa9},
+ {0x001fa2, 0x001fa2, 0x001faa, 0x001faa},
+ {0x001fa3, 0x001fa3, 0x001fab, 0x001fab},
+ {0x001fa4, 0x001fa4, 0x001fac, 0x001fac},
+ {0x001fa5, 0x001fa5, 0x001fad, 0x001fad},
+ {0x001fa6, 0x001fa6, 0x001fae, 0x001fae},
+ {0x001fa7, 0x001fa7, 0x001faf, 0x001faf},
+ {0x001fa8, 0x001fa0, 0x001fa8, 0x001fa8},
+ {0x001fa9, 0x001fa1, 0x001fa9, 0x001fa9},
+ {0x001faa, 0x001fa2, 0x001faa, 0x001faa},
+ {0x001fab, 0x001fa3, 0x001fab, 0x001fab},
+ {0x001fac, 0x001fa4, 0x001fac, 0x001fac},
+ {0x001fad, 0x001fa5, 0x001fad, 0x001fad},
+ {0x001fae, 0x001fa6, 0x001fae, 0x001fae},
+ {0x001faf, 0x001fa7, 0x001faf, 0x001faf},
+ {0x001fb0, 0x001fb0, 0x001fb8, 0x001fb8},
+ {0x001fb1, 0x001fb1, 0x001fb9, 0x001fb9},
+ {0x001fb3, 0x001fb3, 0x001fbc, 0x001fbc},
+ {0x001fb8, 0x001fb0, 0x001fb8, 0x001fb8},
+ {0x001fb9, 0x001fb1, 0x001fb9, 0x001fb9},
+ {0x001fba, 0x001f70, 0x001fba, 0x001fba},
+ {0x001fbb, 0x001f71, 0x001fbb, 0x001fbb},
+ {0x001fbc, 0x001fb3, 0x001fbc, 0x001fbc},
+ {0x001fbe, 0x001fbe, 0x000399, 0x000399},
+ {0x001fc3, 0x001fc3, 0x001fcc, 0x001fcc},
+ {0x001fc8, 0x001f72, 0x001fc8, 0x001fc8},
+ {0x001fc9, 0x001f73, 0x001fc9, 0x001fc9},
+ {0x001fca, 0x001f74, 0x001fca, 0x001fca},
+ {0x001fcb, 0x001f75, 0x001fcb, 0x001fcb},
+ {0x001fcc, 0x001fc3, 0x001fcc, 0x001fcc},
+ {0x001fd0, 0x001fd0, 0x001fd8, 0x001fd8},
+ {0x001fd1, 0x001fd1, 0x001fd9, 0x001fd9},
+ {0x001fd8, 0x001fd0, 0x001fd8, 0x001fd8},
+ {0x001fd9, 0x001fd1, 0x001fd9, 0x001fd9},
+ {0x001fda, 0x001f76, 0x001fda, 0x001fda},
+ {0x001fdb, 0x001f77, 0x001fdb, 0x001fdb},
+ {0x001fe0, 0x001fe0, 0x001fe8, 0x001fe8},
+ {0x001fe1, 0x001fe1, 0x001fe9, 0x001fe9},
+ {0x001fe5, 0x001fe5, 0x001fec, 0x001fec},
+ {0x001fe8, 0x001fe0, 0x001fe8, 0x001fe8},
+ {0x001fe9, 0x001fe1, 0x001fe9, 0x001fe9},
+ {0x001fea, 0x001f7a, 0x001fea, 0x001fea},
+ {0x001feb, 0x001f7b, 0x001feb, 0x001feb},
+ {0x001fec, 0x001fe5, 0x001fec, 0x001fec},
+ {0x001ff3, 0x001ff3, 0x001ffc, 0x001ffc},
+ {0x001ff8, 0x001f78, 0x001ff8, 0x001ff8},
+ {0x001ff9, 0x001f79, 0x001ff9, 0x001ff9},
+ {0x001ffa, 0x001f7c, 0x001ffa, 0x001ffa},
+ {0x001ffb, 0x001f7d, 0x001ffb, 0x001ffb},
+ {0x001ffc, 0x001ff3, 0x001ffc, 0x001ffc},
+ {0x002126, 0x0003c9, 0x002126, 0x002126},
+ {0x00212a, 0x00006b, 0x00212a, 0x00212a},
+ {0x00212b, 0x0000e5, 0x00212b, 0x00212b},
+ {0x002132, 0x00214e, 0x002132, 0x002132},
+ {0x00214e, 0x00214e, 0x002132, 0x002132},
+ {0x002160, 0x002170, 0x002160, 0x002160},
+ {0x002161, 0x002171, 0x002161, 0x002161},
+ {0x002162, 0x002172, 0x002162, 0x002162},
+ {0x002163, 0x002173, 0x002163, 0x002163},
+ {0x002164, 0x002174, 0x002164, 0x002164},
+ {0x002165, 0x002175, 0x002165, 0x002165},
+ {0x002166, 0x002176, 0x002166, 0x002166},
+ {0x002167, 0x002177, 0x002167, 0x002167},
+ {0x002168, 0x002178, 0x002168, 0x002168},
+ {0x002169, 0x002179, 0x002169, 0x002169},
+ {0x00216a, 0x00217a, 0x00216a, 0x00216a},
+ {0x00216b, 0x00217b, 0x00216b, 0x00216b},
+ {0x00216c, 0x00217c, 0x00216c, 0x00216c},
+ {0x00216d, 0x00217d, 0x00216d, 0x00216d},
+ {0x00216e, 0x00217e, 0x00216e, 0x00216e},
+ {0x00216f, 0x00217f, 0x00216f, 0x00216f},
+ {0x002170, 0x002170, 0x002160, 0x002160},
+ {0x002171, 0x002171, 0x002161, 0x002161},
+ {0x002172, 0x002172, 0x002162, 0x002162},
+ {0x002173, 0x002173, 0x002163, 0x002163},
+ {0x002174, 0x002174, 0x002164, 0x002164},
+ {0x002175, 0x002175, 0x002165, 0x002165},
+ {0x002176, 0x002176, 0x002166, 0x002166},
+ {0x002177, 0x002177, 0x002167, 0x002167},
+ {0x002178, 0x002178, 0x002168, 0x002168},
+ {0x002179, 0x002179, 0x002169, 0x002169},
+ {0x00217a, 0x00217a, 0x00216a, 0x00216a},
+ {0x00217b, 0x00217b, 0x00216b, 0x00216b},
+ {0x00217c, 0x00217c, 0x00216c, 0x00216c},
+ {0x00217d, 0x00217d, 0x00216d, 0x00216d},
+ {0x00217e, 0x00217e, 0x00216e, 0x00216e},
+ {0x00217f, 0x00217f, 0x00216f, 0x00216f},
+ {0x002183, 0x002184, 0x002183, 0x002183},
+ {0x002184, 0x002184, 0x002183, 0x002183},
+ {0x0024b6, 0x0024d0, 0x0024b6, 0x0024b6},
+ {0x0024b7, 0x0024d1, 0x0024b7, 0x0024b7},
+ {0x0024b8, 0x0024d2, 0x0024b8, 0x0024b8},
+ {0x0024b9, 0x0024d3, 0x0024b9, 0x0024b9},
+ {0x0024ba, 0x0024d4, 0x0024ba, 0x0024ba},
+ {0x0024bb, 0x0024d5, 0x0024bb, 0x0024bb},
+ {0x0024bc, 0x0024d6, 0x0024bc, 0x0024bc},
+ {0x0024bd, 0x0024d7, 0x0024bd, 0x0024bd},
+ {0x0024be, 0x0024d8, 0x0024be, 0x0024be},
+ {0x0024bf, 0x0024d9, 0x0024bf, 0x0024bf},
+ {0x0024c0, 0x0024da, 0x0024c0, 0x0024c0},
+ {0x0024c1, 0x0024db, 0x0024c1, 0x0024c1},
+ {0x0024c2, 0x0024dc, 0x0024c2, 0x0024c2},
+ {0x0024c3, 0x0024dd, 0x0024c3, 0x0024c3},
+ {0x0024c4, 0x0024de, 0x0024c4, 0x0024c4},
+ {0x0024c5, 0x0024df, 0x0024c5, 0x0024c5},
+ {0x0024c6, 0x0024e0, 0x0024c6, 0x0024c6},
+ {0x0024c7, 0x0024e1, 0x0024c7, 0x0024c7},
+ {0x0024c8, 0x0024e2, 0x0024c8, 0x0024c8},
+ {0x0024c9, 0x0024e3, 0x0024c9, 0x0024c9},
+ {0x0024ca, 0x0024e4, 0x0024ca, 0x0024ca},
+ {0x0024cb, 0x0024e5, 0x0024cb, 0x0024cb},
+ {0x0024cc, 0x0024e6, 0x0024cc, 0x0024cc},
+ {0x0024cd, 0x0024e7, 0x0024cd, 0x0024cd},
+ {0x0024ce, 0x0024e8, 0x0024ce, 0x0024ce},
+ {0x0024cf, 0x0024e9, 0x0024cf, 0x0024cf},
+ {0x0024d0, 0x0024d0, 0x0024b6, 0x0024b6},
+ {0x0024d1, 0x0024d1, 0x0024b7, 0x0024b7},
+ {0x0024d2, 0x0024d2, 0x0024b8, 0x0024b8},
+ {0x0024d3, 0x0024d3, 0x0024b9, 0x0024b9},
+ {0x0024d4, 0x0024d4, 0x0024ba, 0x0024ba},
+ {0x0024d5, 0x0024d5, 0x0024bb, 0x0024bb},
+ {0x0024d6, 0x0024d6, 0x0024bc, 0x0024bc},
+ {0x0024d7, 0x0024d7, 0x0024bd, 0x0024bd},
+ {0x0024d8, 0x0024d8, 0x0024be, 0x0024be},
+ {0x0024d9, 0x0024d9, 0x0024bf, 0x0024bf},
+ {0x0024da, 0x0024da, 0x0024c0, 0x0024c0},
+ {0x0024db, 0x0024db, 0x0024c1, 0x0024c1},
+ {0x0024dc, 0x0024dc, 0x0024c2, 0x0024c2},
+ {0x0024dd, 0x0024dd, 0x0024c3, 0x0024c3},
+ {0x0024de, 0x0024de, 0x0024c4, 0x0024c4},
+ {0x0024df, 0x0024df, 0x0024c5, 0x0024c5},
+ {0x0024e0, 0x0024e0, 0x0024c6, 0x0024c6},
+ {0x0024e1, 0x0024e1, 0x0024c7, 0x0024c7},
+ {0x0024e2, 0x0024e2, 0x0024c8, 0x0024c8},
+ {0x0024e3, 0x0024e3, 0x0024c9, 0x0024c9},
+ {0x0024e4, 0x0024e4, 0x0024ca, 0x0024ca},
+ {0x0024e5, 0x0024e5, 0x0024cb, 0x0024cb},
+ {0x0024e6, 0x0024e6, 0x0024cc, 0x0024cc},
+ {0x0024e7, 0x0024e7, 0x0024cd, 0x0024cd},
+ {0x0024e8, 0x0024e8, 0x0024ce, 0x0024ce},
+ {0x0024e9, 0x0024e9, 0x0024cf, 0x0024cf},
+ {0x002c00, 0x002c30, 0x002c00, 0x002c00},
+ {0x002c01, 0x002c31, 0x002c01, 0x002c01},
+ {0x002c02, 0x002c32, 0x002c02, 0x002c02},
+ {0x002c03, 0x002c33, 0x002c03, 0x002c03},
+ {0x002c04, 0x002c34, 0x002c04, 0x002c04},
+ {0x002c05, 0x002c35, 0x002c05, 0x002c05},
+ {0x002c06, 0x002c36, 0x002c06, 0x002c06},
+ {0x002c07, 0x002c37, 0x002c07, 0x002c07},
+ {0x002c08, 0x002c38, 0x002c08, 0x002c08},
+ {0x002c09, 0x002c39, 0x002c09, 0x002c09},
+ {0x002c0a, 0x002c3a, 0x002c0a, 0x002c0a},
+ {0x002c0b, 0x002c3b, 0x002c0b, 0x002c0b},
+ {0x002c0c, 0x002c3c, 0x002c0c, 0x002c0c},
+ {0x002c0d, 0x002c3d, 0x002c0d, 0x002c0d},
+ {0x002c0e, 0x002c3e, 0x002c0e, 0x002c0e},
+ {0x002c0f, 0x002c3f, 0x002c0f, 0x002c0f},
+ {0x002c10, 0x002c40, 0x002c10, 0x002c10},
+ {0x002c11, 0x002c41, 0x002c11, 0x002c11},
+ {0x002c12, 0x002c42, 0x002c12, 0x002c12},
+ {0x002c13, 0x002c43, 0x002c13, 0x002c13},
+ {0x002c14, 0x002c44, 0x002c14, 0x002c14},
+ {0x002c15, 0x002c45, 0x002c15, 0x002c15},
+ {0x002c16, 0x002c46, 0x002c16, 0x002c16},
+ {0x002c17, 0x002c47, 0x002c17, 0x002c17},
+ {0x002c18, 0x002c48, 0x002c18, 0x002c18},
+ {0x002c19, 0x002c49, 0x002c19, 0x002c19},
+ {0x002c1a, 0x002c4a, 0x002c1a, 0x002c1a},
+ {0x002c1b, 0x002c4b, 0x002c1b, 0x002c1b},
+ {0x002c1c, 0x002c4c, 0x002c1c, 0x002c1c},
+ {0x002c1d, 0x002c4d, 0x002c1d, 0x002c1d},
+ {0x002c1e, 0x002c4e, 0x002c1e, 0x002c1e},
+ {0x002c1f, 0x002c4f, 0x002c1f, 0x002c1f},
+ {0x002c20, 0x002c50, 0x002c20, 0x002c20},
+ {0x002c21, 0x002c51, 0x002c21, 0x002c21},
+ {0x002c22, 0x002c52, 0x002c22, 0x002c22},
+ {0x002c23, 0x002c53, 0x002c23, 0x002c23},
+ {0x002c24, 0x002c54, 0x002c24, 0x002c24},
+ {0x002c25, 0x002c55, 0x002c25, 0x002c25},
+ {0x002c26, 0x002c56, 0x002c26, 0x002c26},
+ {0x002c27, 0x002c57, 0x002c27, 0x002c27},
+ {0x002c28, 0x002c58, 0x002c28, 0x002c28},
+ {0x002c29, 0x002c59, 0x002c29, 0x002c29},
+ {0x002c2a, 0x002c5a, 0x002c2a, 0x002c2a},
+ {0x002c2b, 0x002c5b, 0x002c2b, 0x002c2b},
+ {0x002c2c, 0x002c5c, 0x002c2c, 0x002c2c},
+ {0x002c2d, 0x002c5d, 0x002c2d, 0x002c2d},
+ {0x002c2e, 0x002c5e, 0x002c2e, 0x002c2e},
+ {0x002c2f, 0x002c5f, 0x002c2f, 0x002c2f},
+ {0x002c30, 0x002c30, 0x002c00, 0x002c00},
+ {0x002c31, 0x002c31, 0x002c01, 0x002c01},
+ {0x002c32, 0x002c32, 0x002c02, 0x002c02},
+ {0x002c33, 0x002c33, 0x002c03, 0x002c03},
+ {0x002c34, 0x002c34, 0x002c04, 0x002c04},
+ {0x002c35, 0x002c35, 0x002c05, 0x002c05},
+ {0x002c36, 0x002c36, 0x002c06, 0x002c06},
+ {0x002c37, 0x002c37, 0x002c07, 0x002c07},
+ {0x002c38, 0x002c38, 0x002c08, 0x002c08},
+ {0x002c39, 0x002c39, 0x002c09, 0x002c09},
+ {0x002c3a, 0x002c3a, 0x002c0a, 0x002c0a},
+ {0x002c3b, 0x002c3b, 0x002c0b, 0x002c0b},
+ {0x002c3c, 0x002c3c, 0x002c0c, 0x002c0c},
+ {0x002c3d, 0x002c3d, 0x002c0d, 0x002c0d},
+ {0x002c3e, 0x002c3e, 0x002c0e, 0x002c0e},
+ {0x002c3f, 0x002c3f, 0x002c0f, 0x002c0f},
+ {0x002c40, 0x002c40, 0x002c10, 0x002c10},
+ {0x002c41, 0x002c41, 0x002c11, 0x002c11},
+ {0x002c42, 0x002c42, 0x002c12, 0x002c12},
+ {0x002c43, 0x002c43, 0x002c13, 0x002c13},
+ {0x002c44, 0x002c44, 0x002c14, 0x002c14},
+ {0x002c45, 0x002c45, 0x002c15, 0x002c15},
+ {0x002c46, 0x002c46, 0x002c16, 0x002c16},
+ {0x002c47, 0x002c47, 0x002c17, 0x002c17},
+ {0x002c48, 0x002c48, 0x002c18, 0x002c18},
+ {0x002c49, 0x002c49, 0x002c19, 0x002c19},
+ {0x002c4a, 0x002c4a, 0x002c1a, 0x002c1a},
+ {0x002c4b, 0x002c4b, 0x002c1b, 0x002c1b},
+ {0x002c4c, 0x002c4c, 0x002c1c, 0x002c1c},
+ {0x002c4d, 0x002c4d, 0x002c1d, 0x002c1d},
+ {0x002c4e, 0x002c4e, 0x002c1e, 0x002c1e},
+ {0x002c4f, 0x002c4f, 0x002c1f, 0x002c1f},
+ {0x002c50, 0x002c50, 0x002c20, 0x002c20},
+ {0x002c51, 0x002c51, 0x002c21, 0x002c21},
+ {0x002c52, 0x002c52, 0x002c22, 0x002c22},
+ {0x002c53, 0x002c53, 0x002c23, 0x002c23},
+ {0x002c54, 0x002c54, 0x002c24, 0x002c24},
+ {0x002c55, 0x002c55, 0x002c25, 0x002c25},
+ {0x002c56, 0x002c56, 0x002c26, 0x002c26},
+ {0x002c57, 0x002c57, 0x002c27, 0x002c27},
+ {0x002c58, 0x002c58, 0x002c28, 0x002c28},
+ {0x002c59, 0x002c59, 0x002c29, 0x002c29},
+ {0x002c5a, 0x002c5a, 0x002c2a, 0x002c2a},
+ {0x002c5b, 0x002c5b, 0x002c2b, 0x002c2b},
+ {0x002c5c, 0x002c5c, 0x002c2c, 0x002c2c},
+ {0x002c5d, 0x002c5d, 0x002c2d, 0x002c2d},
+ {0x002c5e, 0x002c5e, 0x002c2e, 0x002c2e},
+ {0x002c5f, 0x002c5f, 0x002c2f, 0x002c2f},
+ {0x002c60, 0x002c61, 0x002c60, 0x002c60},
+ {0x002c61, 0x002c61, 0x002c60, 0x002c60},
+ {0x002c62, 0x00026b, 0x002c62, 0x002c62},
+ {0x002c63, 0x001d7d, 0x002c63, 0x002c63},
+ {0x002c64, 0x00027d, 0x002c64, 0x002c64},
+ {0x002c65, 0x002c65, 0x00023a, 0x00023a},
+ {0x002c66, 0x002c66, 0x00023e, 0x00023e},
+ {0x002c67, 0x002c68, 0x002c67, 0x002c67},
+ {0x002c68, 0x002c68, 0x002c67, 0x002c67},
+ {0x002c69, 0x002c6a, 0x002c69, 0x002c69},
+ {0x002c6a, 0x002c6a, 0x002c69, 0x002c69},
+ {0x002c6b, 0x002c6c, 0x002c6b, 0x002c6b},
+ {0x002c6c, 0x002c6c, 0x002c6b, 0x002c6b},
+ {0x002c6d, 0x000251, 0x002c6d, 0x002c6d},
+ {0x002c6e, 0x000271, 0x002c6e, 0x002c6e},
+ {0x002c6f, 0x000250, 0x002c6f, 0x002c6f},
+ {0x002c70, 0x000252, 0x002c70, 0x002c70},
+ {0x002c72, 0x002c73, 0x002c72, 0x002c72},
+ {0x002c73, 0x002c73, 0x002c72, 0x002c72},
+ {0x002c75, 0x002c76, 0x002c75, 0x002c75},
+ {0x002c76, 0x002c76, 0x002c75, 0x002c75},
+ {0x002c7e, 0x00023f, 0x002c7e, 0x002c7e},
+ {0x002c7f, 0x000240, 0x002c7f, 0x002c7f},
+ {0x002c80, 0x002c81, 0x002c80, 0x002c80},
+ {0x002c81, 0x002c81, 0x002c80, 0x002c80},
+ {0x002c82, 0x002c83, 0x002c82, 0x002c82},
+ {0x002c83, 0x002c83, 0x002c82, 0x002c82},
+ {0x002c84, 0x002c85, 0x002c84, 0x002c84},
+ {0x002c85, 0x002c85, 0x002c84, 0x002c84},
+ {0x002c86, 0x002c87, 0x002c86, 0x002c86},
+ {0x002c87, 0x002c87, 0x002c86, 0x002c86},
+ {0x002c88, 0x002c89, 0x002c88, 0x002c88},
+ {0x002c89, 0x002c89, 0x002c88, 0x002c88},
+ {0x002c8a, 0x002c8b, 0x002c8a, 0x002c8a},
+ {0x002c8b, 0x002c8b, 0x002c8a, 0x002c8a},
+ {0x002c8c, 0x002c8d, 0x002c8c, 0x002c8c},
+ {0x002c8d, 0x002c8d, 0x002c8c, 0x002c8c},
+ {0x002c8e, 0x002c8f, 0x002c8e, 0x002c8e},
+ {0x002c8f, 0x002c8f, 0x002c8e, 0x002c8e},
+ {0x002c90, 0x002c91, 0x002c90, 0x002c90},
+ {0x002c91, 0x002c91, 0x002c90, 0x002c90},
+ {0x002c92, 0x002c93, 0x002c92, 0x002c92},
+ {0x002c93, 0x002c93, 0x002c92, 0x002c92},
+ {0x002c94, 0x002c95, 0x002c94, 0x002c94},
+ {0x002c95, 0x002c95, 0x002c94, 0x002c94},
+ {0x002c96, 0x002c97, 0x002c96, 0x002c96},
+ {0x002c97, 0x002c97, 0x002c96, 0x002c96},
+ {0x002c98, 0x002c99, 0x002c98, 0x002c98},
+ {0x002c99, 0x002c99, 0x002c98, 0x002c98},
+ {0x002c9a, 0x002c9b, 0x002c9a, 0x002c9a},
+ {0x002c9b, 0x002c9b, 0x002c9a, 0x002c9a},
+ {0x002c9c, 0x002c9d, 0x002c9c, 0x002c9c},
+ {0x002c9d, 0x002c9d, 0x002c9c, 0x002c9c},
+ {0x002c9e, 0x002c9f, 0x002c9e, 0x002c9e},
+ {0x002c9f, 0x002c9f, 0x002c9e, 0x002c9e},
+ {0x002ca0, 0x002ca1, 0x002ca0, 0x002ca0},
+ {0x002ca1, 0x002ca1, 0x002ca0, 0x002ca0},
+ {0x002ca2, 0x002ca3, 0x002ca2, 0x002ca2},
+ {0x002ca3, 0x002ca3, 0x002ca2, 0x002ca2},
+ {0x002ca4, 0x002ca5, 0x002ca4, 0x002ca4},
+ {0x002ca5, 0x002ca5, 0x002ca4, 0x002ca4},
+ {0x002ca6, 0x002ca7, 0x002ca6, 0x002ca6},
+ {0x002ca7, 0x002ca7, 0x002ca6, 0x002ca6},
+ {0x002ca8, 0x002ca9, 0x002ca8, 0x002ca8},
+ {0x002ca9, 0x002ca9, 0x002ca8, 0x002ca8},
+ {0x002caa, 0x002cab, 0x002caa, 0x002caa},
+ {0x002cab, 0x002cab, 0x002caa, 0x002caa},
+ {0x002cac, 0x002cad, 0x002cac, 0x002cac},
+ {0x002cad, 0x002cad, 0x002cac, 0x002cac},
+ {0x002cae, 0x002caf, 0x002cae, 0x002cae},
+ {0x002caf, 0x002caf, 0x002cae, 0x002cae},
+ {0x002cb0, 0x002cb1, 0x002cb0, 0x002cb0},
+ {0x002cb1, 0x002cb1, 0x002cb0, 0x002cb0},
+ {0x002cb2, 0x002cb3, 0x002cb2, 0x002cb2},
+ {0x002cb3, 0x002cb3, 0x002cb2, 0x002cb2},
+ {0x002cb4, 0x002cb5, 0x002cb4, 0x002cb4},
+ {0x002cb5, 0x002cb5, 0x002cb4, 0x002cb4},
+ {0x002cb6, 0x002cb7, 0x002cb6, 0x002cb6},
+ {0x002cb7, 0x002cb7, 0x002cb6, 0x002cb6},
+ {0x002cb8, 0x002cb9, 0x002cb8, 0x002cb8},
+ {0x002cb9, 0x002cb9, 0x002cb8, 0x002cb8},
+ {0x002cba, 0x002cbb, 0x002cba, 0x002cba},
+ {0x002cbb, 0x002cbb, 0x002cba, 0x002cba},
+ {0x002cbc, 0x002cbd, 0x002cbc, 0x002cbc},
+ {0x002cbd, 0x002cbd, 0x002cbc, 0x002cbc},
+ {0x002cbe, 0x002cbf, 0x002cbe, 0x002cbe},
+ {0x002cbf, 0x002cbf, 0x002cbe, 0x002cbe},
+ {0x002cc0, 0x002cc1, 0x002cc0, 0x002cc0},
+ {0x002cc1, 0x002cc1, 0x002cc0, 0x002cc0},
+ {0x002cc2, 0x002cc3, 0x002cc2, 0x002cc2},
+ {0x002cc3, 0x002cc3, 0x002cc2, 0x002cc2},
+ {0x002cc4, 0x002cc5, 0x002cc4, 0x002cc4},
+ {0x002cc5, 0x002cc5, 0x002cc4, 0x002cc4},
+ {0x002cc6, 0x002cc7, 0x002cc6, 0x002cc6},
+ {0x002cc7, 0x002cc7, 0x002cc6, 0x002cc6},
+ {0x002cc8, 0x002cc9, 0x002cc8, 0x002cc8},
+ {0x002cc9, 0x002cc9, 0x002cc8, 0x002cc8},
+ {0x002cca, 0x002ccb, 0x002cca, 0x002cca},
+ {0x002ccb, 0x002ccb, 0x002cca, 0x002cca},
+ {0x002ccc, 0x002ccd, 0x002ccc, 0x002ccc},
+ {0x002ccd, 0x002ccd, 0x002ccc, 0x002ccc},
+ {0x002cce, 0x002ccf, 0x002cce, 0x002cce},
+ {0x002ccf, 0x002ccf, 0x002cce, 0x002cce},
+ {0x002cd0, 0x002cd1, 0x002cd0, 0x002cd0},
+ {0x002cd1, 0x002cd1, 0x002cd0, 0x002cd0},
+ {0x002cd2, 0x002cd3, 0x002cd2, 0x002cd2},
+ {0x002cd3, 0x002cd3, 0x002cd2, 0x002cd2},
+ {0x002cd4, 0x002cd5, 0x002cd4, 0x002cd4},
+ {0x002cd5, 0x002cd5, 0x002cd4, 0x002cd4},
+ {0x002cd6, 0x002cd7, 0x002cd6, 0x002cd6},
+ {0x002cd7, 0x002cd7, 0x002cd6, 0x002cd6},
+ {0x002cd8, 0x002cd9, 0x002cd8, 0x002cd8},
+ {0x002cd9, 0x002cd9, 0x002cd8, 0x002cd8},
+ {0x002cda, 0x002cdb, 0x002cda, 0x002cda},
+ {0x002cdb, 0x002cdb, 0x002cda, 0x002cda},
+ {0x002cdc, 0x002cdd, 0x002cdc, 0x002cdc},
+ {0x002cdd, 0x002cdd, 0x002cdc, 0x002cdc},
+ {0x002cde, 0x002cdf, 0x002cde, 0x002cde},
+ {0x002cdf, 0x002cdf, 0x002cde, 0x002cde},
+ {0x002ce0, 0x002ce1, 0x002ce0, 0x002ce0},
+ {0x002ce1, 0x002ce1, 0x002ce0, 0x002ce0},
+ {0x002ce2, 0x002ce3, 0x002ce2, 0x002ce2},
+ {0x002ce3, 0x002ce3, 0x002ce2, 0x002ce2},
+ {0x002ceb, 0x002cec, 0x002ceb, 0x002ceb},
+ {0x002cec, 0x002cec, 0x002ceb, 0x002ceb},
+ {0x002ced, 0x002cee, 0x002ced, 0x002ced},
+ {0x002cee, 0x002cee, 0x002ced, 0x002ced},
+ {0x002cf2, 0x002cf3, 0x002cf2, 0x002cf2},
+ {0x002cf3, 0x002cf3, 0x002cf2, 0x002cf2},
+ {0x002d00, 0x002d00, 0x0010a0, 0x0010a0},
+ {0x002d01, 0x002d01, 0x0010a1, 0x0010a1},
+ {0x002d02, 0x002d02, 0x0010a2, 0x0010a2},
+ {0x002d03, 0x002d03, 0x0010a3, 0x0010a3},
+ {0x002d04, 0x002d04, 0x0010a4, 0x0010a4},
+ {0x002d05, 0x002d05, 0x0010a5, 0x0010a5},
+ {0x002d06, 0x002d06, 0x0010a6, 0x0010a6},
+ {0x002d07, 0x002d07, 0x0010a7, 0x0010a7},
+ {0x002d08, 0x002d08, 0x0010a8, 0x0010a8},
+ {0x002d09, 0x002d09, 0x0010a9, 0x0010a9},
+ {0x002d0a, 0x002d0a, 0x0010aa, 0x0010aa},
+ {0x002d0b, 0x002d0b, 0x0010ab, 0x0010ab},
+ {0x002d0c, 0x002d0c, 0x0010ac, 0x0010ac},
+ {0x002d0d, 0x002d0d, 0x0010ad, 0x0010ad},
+ {0x002d0e, 0x002d0e, 0x0010ae, 0x0010ae},
+ {0x002d0f, 0x002d0f, 0x0010af, 0x0010af},
+ {0x002d10, 0x002d10, 0x0010b0, 0x0010b0},
+ {0x002d11, 0x002d11, 0x0010b1, 0x0010b1},
+ {0x002d12, 0x002d12, 0x0010b2, 0x0010b2},
+ {0x002d13, 0x002d13, 0x0010b3, 0x0010b3},
+ {0x002d14, 0x002d14, 0x0010b4, 0x0010b4},
+ {0x002d15, 0x002d15, 0x0010b5, 0x0010b5},
+ {0x002d16, 0x002d16, 0x0010b6, 0x0010b6},
+ {0x002d17, 0x002d17, 0x0010b7, 0x0010b7},
+ {0x002d18, 0x002d18, 0x0010b8, 0x0010b8},
+ {0x002d19, 0x002d19, 0x0010b9, 0x0010b9},
+ {0x002d1a, 0x002d1a, 0x0010ba, 0x0010ba},
+ {0x002d1b, 0x002d1b, 0x0010bb, 0x0010bb},
+ {0x002d1c, 0x002d1c, 0x0010bc, 0x0010bc},
+ {0x002d1d, 0x002d1d, 0x0010bd, 0x0010bd},
+ {0x002d1e, 0x002d1e, 0x0010be, 0x0010be},
+ {0x002d1f, 0x002d1f, 0x0010bf, 0x0010bf},
+ {0x002d20, 0x002d20, 0x0010c0, 0x0010c0},
+ {0x002d21, 0x002d21, 0x0010c1, 0x0010c1},
+ {0x002d22, 0x002d22, 0x0010c2, 0x0010c2},
+ {0x002d23, 0x002d23, 0x0010c3, 0x0010c3},
+ {0x002d24, 0x002d24, 0x0010c4, 0x0010c4},
+ {0x002d25, 0x002d25, 0x0010c5, 0x0010c5},
+ {0x002d27, 0x002d27, 0x0010c7, 0x0010c7},
+ {0x002d2d, 0x002d2d, 0x0010cd, 0x0010cd},
+ {0x00a640, 0x00a641, 0x00a640, 0x00a640},
+ {0x00a641, 0x00a641, 0x00a640, 0x00a640},
+ {0x00a642, 0x00a643, 0x00a642, 0x00a642},
+ {0x00a643, 0x00a643, 0x00a642, 0x00a642},
+ {0x00a644, 0x00a645, 0x00a644, 0x00a644},
+ {0x00a645, 0x00a645, 0x00a644, 0x00a644},
+ {0x00a646, 0x00a647, 0x00a646, 0x00a646},
+ {0x00a647, 0x00a647, 0x00a646, 0x00a646},
+ {0x00a648, 0x00a649, 0x00a648, 0x00a648},
+ {0x00a649, 0x00a649, 0x00a648, 0x00a648},
+ {0x00a64a, 0x00a64b, 0x00a64a, 0x00a64a},
+ {0x00a64b, 0x00a64b, 0x00a64a, 0x00a64a},
+ {0x00a64c, 0x00a64d, 0x00a64c, 0x00a64c},
+ {0x00a64d, 0x00a64d, 0x00a64c, 0x00a64c},
+ {0x00a64e, 0x00a64f, 0x00a64e, 0x00a64e},
+ {0x00a64f, 0x00a64f, 0x00a64e, 0x00a64e},
+ {0x00a650, 0x00a651, 0x00a650, 0x00a650},
+ {0x00a651, 0x00a651, 0x00a650, 0x00a650},
+ {0x00a652, 0x00a653, 0x00a652, 0x00a652},
+ {0x00a653, 0x00a653, 0x00a652, 0x00a652},
+ {0x00a654, 0x00a655, 0x00a654, 0x00a654},
+ {0x00a655, 0x00a655, 0x00a654, 0x00a654},
+ {0x00a656, 0x00a657, 0x00a656, 0x00a656},
+ {0x00a657, 0x00a657, 0x00a656, 0x00a656},
+ {0x00a658, 0x00a659, 0x00a658, 0x00a658},
+ {0x00a659, 0x00a659, 0x00a658, 0x00a658},
+ {0x00a65a, 0x00a65b, 0x00a65a, 0x00a65a},
+ {0x00a65b, 0x00a65b, 0x00a65a, 0x00a65a},
+ {0x00a65c, 0x00a65d, 0x00a65c, 0x00a65c},
+ {0x00a65d, 0x00a65d, 0x00a65c, 0x00a65c},
+ {0x00a65e, 0x00a65f, 0x00a65e, 0x00a65e},
+ {0x00a65f, 0x00a65f, 0x00a65e, 0x00a65e},
+ {0x00a660, 0x00a661, 0x00a660, 0x00a660},
+ {0x00a661, 0x00a661, 0x00a660, 0x00a660},
+ {0x00a662, 0x00a663, 0x00a662, 0x00a662},
+ {0x00a663, 0x00a663, 0x00a662, 0x00a662},
+ {0x00a664, 0x00a665, 0x00a664, 0x00a664},
+ {0x00a665, 0x00a665, 0x00a664, 0x00a664},
+ {0x00a666, 0x00a667, 0x00a666, 0x00a666},
+ {0x00a667, 0x00a667, 0x00a666, 0x00a666},
+ {0x00a668, 0x00a669, 0x00a668, 0x00a668},
+ {0x00a669, 0x00a669, 0x00a668, 0x00a668},
+ {0x00a66a, 0x00a66b, 0x00a66a, 0x00a66a},
+ {0x00a66b, 0x00a66b, 0x00a66a, 0x00a66a},
+ {0x00a66c, 0x00a66d, 0x00a66c, 0x00a66c},
+ {0x00a66d, 0x00a66d, 0x00a66c, 0x00a66c},
+ {0x00a680, 0x00a681, 0x00a680, 0x00a680},
+ {0x00a681, 0x00a681, 0x00a680, 0x00a680},
+ {0x00a682, 0x00a683, 0x00a682, 0x00a682},
+ {0x00a683, 0x00a683, 0x00a682, 0x00a682},
+ {0x00a684, 0x00a685, 0x00a684, 0x00a684},
+ {0x00a685, 0x00a685, 0x00a684, 0x00a684},
+ {0x00a686, 0x00a687, 0x00a686, 0x00a686},
+ {0x00a687, 0x00a687, 0x00a686, 0x00a686},
+ {0x00a688, 0x00a689, 0x00a688, 0x00a688},
+ {0x00a689, 0x00a689, 0x00a688, 0x00a688},
+ {0x00a68a, 0x00a68b, 0x00a68a, 0x00a68a},
+ {0x00a68b, 0x00a68b, 0x00a68a, 0x00a68a},
+ {0x00a68c, 0x00a68d, 0x00a68c, 0x00a68c},
+ {0x00a68d, 0x00a68d, 0x00a68c, 0x00a68c},
+ {0x00a68e, 0x00a68f, 0x00a68e, 0x00a68e},
+ {0x00a68f, 0x00a68f, 0x00a68e, 0x00a68e},
+ {0x00a690, 0x00a691, 0x00a690, 0x00a690},
+ {0x00a691, 0x00a691, 0x00a690, 0x00a690},
+ {0x00a692, 0x00a693, 0x00a692, 0x00a692},
+ {0x00a693, 0x00a693, 0x00a692, 0x00a692},
+ {0x00a694, 0x00a695, 0x00a694, 0x00a694},
+ {0x00a695, 0x00a695, 0x00a694, 0x00a694},
+ {0x00a696, 0x00a697, 0x00a696, 0x00a696},
+ {0x00a697, 0x00a697, 0x00a696, 0x00a696},
+ {0x00a698, 0x00a699, 0x00a698, 0x00a698},
+ {0x00a699, 0x00a699, 0x00a698, 0x00a698},
+ {0x00a69a, 0x00a69b, 0x00a69a, 0x00a69a},
+ {0x00a69b, 0x00a69b, 0x00a69a, 0x00a69a},
+ {0x00a722, 0x00a723, 0x00a722, 0x00a722},
+ {0x00a723, 0x00a723, 0x00a722, 0x00a722},
+ {0x00a724, 0x00a725, 0x00a724, 0x00a724},
+ {0x00a725, 0x00a725, 0x00a724, 0x00a724},
+ {0x00a726, 0x00a727, 0x00a726, 0x00a726},
+ {0x00a727, 0x00a727, 0x00a726, 0x00a726},
+ {0x00a728, 0x00a729, 0x00a728, 0x00a728},
+ {0x00a729, 0x00a729, 0x00a728, 0x00a728},
+ {0x00a72a, 0x00a72b, 0x00a72a, 0x00a72a},
+ {0x00a72b, 0x00a72b, 0x00a72a, 0x00a72a},
+ {0x00a72c, 0x00a72d, 0x00a72c, 0x00a72c},
+ {0x00a72d, 0x00a72d, 0x00a72c, 0x00a72c},
+ {0x00a72e, 0x00a72f, 0x00a72e, 0x00a72e},
+ {0x00a72f, 0x00a72f, 0x00a72e, 0x00a72e},
+ {0x00a732, 0x00a733, 0x00a732, 0x00a732},
+ {0x00a733, 0x00a733, 0x00a732, 0x00a732},
+ {0x00a734, 0x00a735, 0x00a734, 0x00a734},
+ {0x00a735, 0x00a735, 0x00a734, 0x00a734},
+ {0x00a736, 0x00a737, 0x00a736, 0x00a736},
+ {0x00a737, 0x00a737, 0x00a736, 0x00a736},
+ {0x00a738, 0x00a739, 0x00a738, 0x00a738},
+ {0x00a739, 0x00a739, 0x00a738, 0x00a738},
+ {0x00a73a, 0x00a73b, 0x00a73a, 0x00a73a},
+ {0x00a73b, 0x00a73b, 0x00a73a, 0x00a73a},
+ {0x00a73c, 0x00a73d, 0x00a73c, 0x00a73c},
+ {0x00a73d, 0x00a73d, 0x00a73c, 0x00a73c},
+ {0x00a73e, 0x00a73f, 0x00a73e, 0x00a73e},
+ {0x00a73f, 0x00a73f, 0x00a73e, 0x00a73e},
+ {0x00a740, 0x00a741, 0x00a740, 0x00a740},
+ {0x00a741, 0x00a741, 0x00a740, 0x00a740},
+ {0x00a742, 0x00a743, 0x00a742, 0x00a742},
+ {0x00a743, 0x00a743, 0x00a742, 0x00a742},
+ {0x00a744, 0x00a745, 0x00a744, 0x00a744},
+ {0x00a745, 0x00a745, 0x00a744, 0x00a744},
+ {0x00a746, 0x00a747, 0x00a746, 0x00a746},
+ {0x00a747, 0x00a747, 0x00a746, 0x00a746},
+ {0x00a748, 0x00a749, 0x00a748, 0x00a748},
+ {0x00a749, 0x00a749, 0x00a748, 0x00a748},
+ {0x00a74a, 0x00a74b, 0x00a74a, 0x00a74a},
+ {0x00a74b, 0x00a74b, 0x00a74a, 0x00a74a},
+ {0x00a74c, 0x00a74d, 0x00a74c, 0x00a74c},
+ {0x00a74d, 0x00a74d, 0x00a74c, 0x00a74c},
+ {0x00a74e, 0x00a74f, 0x00a74e, 0x00a74e},
+ {0x00a74f, 0x00a74f, 0x00a74e, 0x00a74e},
+ {0x00a750, 0x00a751, 0x00a750, 0x00a750},
+ {0x00a751, 0x00a751, 0x00a750, 0x00a750},
+ {0x00a752, 0x00a753, 0x00a752, 0x00a752},
+ {0x00a753, 0x00a753, 0x00a752, 0x00a752},
+ {0x00a754, 0x00a755, 0x00a754, 0x00a754},
+ {0x00a755, 0x00a755, 0x00a754, 0x00a754},
+ {0x00a756, 0x00a757, 0x00a756, 0x00a756},
+ {0x00a757, 0x00a757, 0x00a756, 0x00a756},
+ {0x00a758, 0x00a759, 0x00a758, 0x00a758},
+ {0x00a759, 0x00a759, 0x00a758, 0x00a758},
+ {0x00a75a, 0x00a75b, 0x00a75a, 0x00a75a},
+ {0x00a75b, 0x00a75b, 0x00a75a, 0x00a75a},
+ {0x00a75c, 0x00a75d, 0x00a75c, 0x00a75c},
+ {0x00a75d, 0x00a75d, 0x00a75c, 0x00a75c},
+ {0x00a75e, 0x00a75f, 0x00a75e, 0x00a75e},
+ {0x00a75f, 0x00a75f, 0x00a75e, 0x00a75e},
+ {0x00a760, 0x00a761, 0x00a760, 0x00a760},
+ {0x00a761, 0x00a761, 0x00a760, 0x00a760},
+ {0x00a762, 0x00a763, 0x00a762, 0x00a762},
+ {0x00a763, 0x00a763, 0x00a762, 0x00a762},
+ {0x00a764, 0x00a765, 0x00a764, 0x00a764},
+ {0x00a765, 0x00a765, 0x00a764, 0x00a764},
+ {0x00a766, 0x00a767, 0x00a766, 0x00a766},
+ {0x00a767, 0x00a767, 0x00a766, 0x00a766},
+ {0x00a768, 0x00a769, 0x00a768, 0x00a768},
+ {0x00a769, 0x00a769, 0x00a768, 0x00a768},
+ {0x00a76a, 0x00a76b, 0x00a76a, 0x00a76a},
+ {0x00a76b, 0x00a76b, 0x00a76a, 0x00a76a},
+ {0x00a76c, 0x00a76d, 0x00a76c, 0x00a76c},
+ {0x00a76d, 0x00a76d, 0x00a76c, 0x00a76c},
+ {0x00a76e, 0x00a76f, 0x00a76e, 0x00a76e},
+ {0x00a76f, 0x00a76f, 0x00a76e, 0x00a76e},
+ {0x00a779, 0x00a77a, 0x00a779, 0x00a779},
+ {0x00a77a, 0x00a77a, 0x00a779, 0x00a779},
+ {0x00a77b, 0x00a77c, 0x00a77b, 0x00a77b},
+ {0x00a77c, 0x00a77c, 0x00a77b, 0x00a77b},
+ {0x00a77d, 0x001d79, 0x00a77d, 0x00a77d},
+ {0x00a77e, 0x00a77f, 0x00a77e, 0x00a77e},
+ {0x00a77f, 0x00a77f, 0x00a77e, 0x00a77e},
+ {0x00a780, 0x00a781, 0x00a780, 0x00a780},
+ {0x00a781, 0x00a781, 0x00a780, 0x00a780},
+ {0x00a782, 0x00a783, 0x00a782, 0x00a782},
+ {0x00a783, 0x00a783, 0x00a782, 0x00a782},
+ {0x00a784, 0x00a785, 0x00a784, 0x00a784},
+ {0x00a785, 0x00a785, 0x00a784, 0x00a784},
+ {0x00a786, 0x00a787, 0x00a786, 0x00a786},
+ {0x00a787, 0x00a787, 0x00a786, 0x00a786},
+ {0x00a78b, 0x00a78c, 0x00a78b, 0x00a78b},
+ {0x00a78c, 0x00a78c, 0x00a78b, 0x00a78b},
+ {0x00a78d, 0x000265, 0x00a78d, 0x00a78d},
+ {0x00a790, 0x00a791, 0x00a790, 0x00a790},
+ {0x00a791, 0x00a791, 0x00a790, 0x00a790},
+ {0x00a792, 0x00a793, 0x00a792, 0x00a792},
+ {0x00a793, 0x00a793, 0x00a792, 0x00a792},
+ {0x00a794, 0x00a794, 0x00a7c4, 0x00a7c4},
+ {0x00a796, 0x00a797, 0x00a796, 0x00a796},
+ {0x00a797, 0x00a797, 0x00a796, 0x00a796},
+ {0x00a798, 0x00a799, 0x00a798, 0x00a798},
+ {0x00a799, 0x00a799, 0x00a798, 0x00a798},
+ {0x00a79a, 0x00a79b, 0x00a79a, 0x00a79a},
+ {0x00a79b, 0x00a79b, 0x00a79a, 0x00a79a},
+ {0x00a79c, 0x00a79d, 0x00a79c, 0x00a79c},
+ {0x00a79d, 0x00a79d, 0x00a79c, 0x00a79c},
+ {0x00a79e, 0x00a79f, 0x00a79e, 0x00a79e},
+ {0x00a79f, 0x00a79f, 0x00a79e, 0x00a79e},
+ {0x00a7a0, 0x00a7a1, 0x00a7a0, 0x00a7a0},
+ {0x00a7a1, 0x00a7a1, 0x00a7a0, 0x00a7a0},
+ {0x00a7a2, 0x00a7a3, 0x00a7a2, 0x00a7a2},
+ {0x00a7a3, 0x00a7a3, 0x00a7a2, 0x00a7a2},
+ {0x00a7a4, 0x00a7a5, 0x00a7a4, 0x00a7a4},
+ {0x00a7a5, 0x00a7a5, 0x00a7a4, 0x00a7a4},
+ {0x00a7a6, 0x00a7a7, 0x00a7a6, 0x00a7a6},
+ {0x00a7a7, 0x00a7a7, 0x00a7a6, 0x00a7a6},
+ {0x00a7a8, 0x00a7a9, 0x00a7a8, 0x00a7a8},
+ {0x00a7a9, 0x00a7a9, 0x00a7a8, 0x00a7a8},
+ {0x00a7aa, 0x000266, 0x00a7aa, 0x00a7aa},
+ {0x00a7ab, 0x00025c, 0x00a7ab, 0x00a7ab},
+ {0x00a7ac, 0x000261, 0x00a7ac, 0x00a7ac},
+ {0x00a7ad, 0x00026c, 0x00a7ad, 0x00a7ad},
+ {0x00a7ae, 0x00026a, 0x00a7ae, 0x00a7ae},
+ {0x00a7b0, 0x00029e, 0x00a7b0, 0x00a7b0},
+ {0x00a7b1, 0x000287, 0x00a7b1, 0x00a7b1},
+ {0x00a7b2, 0x00029d, 0x00a7b2, 0x00a7b2},
+ {0x00a7b3, 0x00ab53, 0x00a7b3, 0x00a7b3},
+ {0x00a7b4, 0x00a7b5, 0x00a7b4, 0x00a7b4},
+ {0x00a7b5, 0x00a7b5, 0x00a7b4, 0x00a7b4},
+ {0x00a7b6, 0x00a7b7, 0x00a7b6, 0x00a7b6},
+ {0x00a7b7, 0x00a7b7, 0x00a7b6, 0x00a7b6},
+ {0x00a7b8, 0x00a7b9, 0x00a7b8, 0x00a7b8},
+ {0x00a7b9, 0x00a7b9, 0x00a7b8, 0x00a7b8},
+ {0x00a7ba, 0x00a7bb, 0x00a7ba, 0x00a7ba},
+ {0x00a7bb, 0x00a7bb, 0x00a7ba, 0x00a7ba},
+ {0x00a7bc, 0x00a7bd, 0x00a7bc, 0x00a7bc},
+ {0x00a7bd, 0x00a7bd, 0x00a7bc, 0x00a7bc},
+ {0x00a7be, 0x00a7bf, 0x00a7be, 0x00a7be},
+ {0x00a7bf, 0x00a7bf, 0x00a7be, 0x00a7be},
+ {0x00a7c0, 0x00a7c1, 0x00a7c0, 0x00a7c0},
+ {0x00a7c1, 0x00a7c1, 0x00a7c0, 0x00a7c0},
+ {0x00a7c2, 0x00a7c3, 0x00a7c2, 0x00a7c2},
+ {0x00a7c3, 0x00a7c3, 0x00a7c2, 0x00a7c2},
+ {0x00a7c4, 0x00a794, 0x00a7c4, 0x00a7c4},
+ {0x00a7c5, 0x000282, 0x00a7c5, 0x00a7c5},
+ {0x00a7c6, 0x001d8e, 0x00a7c6, 0x00a7c6},
+ {0x00a7c7, 0x00a7c8, 0x00a7c7, 0x00a7c7},
+ {0x00a7c8, 0x00a7c8, 0x00a7c7, 0x00a7c7},
+ {0x00a7c9, 0x00a7ca, 0x00a7c9, 0x00a7c9},
+ {0x00a7ca, 0x00a7ca, 0x00a7c9, 0x00a7c9},
+ {0x00a7d0, 0x00a7d1, 0x00a7d0, 0x00a7d0},
+ {0x00a7d1, 0x00a7d1, 0x00a7d0, 0x00a7d0},
+ {0x00a7d6, 0x00a7d7, 0x00a7d6, 0x00a7d6},
+ {0x00a7d7, 0x00a7d7, 0x00a7d6, 0x00a7d6},
+ {0x00a7d8, 0x00a7d9, 0x00a7d8, 0x00a7d8},
+ {0x00a7d9, 0x00a7d9, 0x00a7d8, 0x00a7d8},
+ {0x00a7f5, 0x00a7f6, 0x00a7f5, 0x00a7f5},
+ {0x00a7f6, 0x00a7f6, 0x00a7f5, 0x00a7f5},
+ {0x00ab53, 0x00ab53, 0x00a7b3, 0x00a7b3},
+ {0x00ab70, 0x00ab70, 0x0013a0, 0x0013a0},
+ {0x00ab71, 0x00ab71, 0x0013a1, 0x0013a1},
+ {0x00ab72, 0x00ab72, 0x0013a2, 0x0013a2},
+ {0x00ab73, 0x00ab73, 0x0013a3, 0x0013a3},
+ {0x00ab74, 0x00ab74, 0x0013a4, 0x0013a4},
+ {0x00ab75, 0x00ab75, 0x0013a5, 0x0013a5},
+ {0x00ab76, 0x00ab76, 0x0013a6, 0x0013a6},
+ {0x00ab77, 0x00ab77, 0x0013a7, 0x0013a7},
+ {0x00ab78, 0x00ab78, 0x0013a8, 0x0013a8},
+ {0x00ab79, 0x00ab79, 0x0013a9, 0x0013a9},
+ {0x00ab7a, 0x00ab7a, 0x0013aa, 0x0013aa},
+ {0x00ab7b, 0x00ab7b, 0x0013ab, 0x0013ab},
+ {0x00ab7c, 0x00ab7c, 0x0013ac, 0x0013ac},
+ {0x00ab7d, 0x00ab7d, 0x0013ad, 0x0013ad},
+ {0x00ab7e, 0x00ab7e, 0x0013ae, 0x0013ae},
+ {0x00ab7f, 0x00ab7f, 0x0013af, 0x0013af},
+ {0x00ab80, 0x00ab80, 0x0013b0, 0x0013b0},
+ {0x00ab81, 0x00ab81, 0x0013b1, 0x0013b1},
+ {0x00ab82, 0x00ab82, 0x0013b2, 0x0013b2},
+ {0x00ab83, 0x00ab83, 0x0013b3, 0x0013b3},
+ {0x00ab84, 0x00ab84, 0x0013b4, 0x0013b4},
+ {0x00ab85, 0x00ab85, 0x0013b5, 0x0013b5},
+ {0x00ab86, 0x00ab86, 0x0013b6, 0x0013b6},
+ {0x00ab87, 0x00ab87, 0x0013b7, 0x0013b7},
+ {0x00ab88, 0x00ab88, 0x0013b8, 0x0013b8},
+ {0x00ab89, 0x00ab89, 0x0013b9, 0x0013b9},
+ {0x00ab8a, 0x00ab8a, 0x0013ba, 0x0013ba},
+ {0x00ab8b, 0x00ab8b, 0x0013bb, 0x0013bb},
+ {0x00ab8c, 0x00ab8c, 0x0013bc, 0x0013bc},
+ {0x00ab8d, 0x00ab8d, 0x0013bd, 0x0013bd},
+ {0x00ab8e, 0x00ab8e, 0x0013be, 0x0013be},
+ {0x00ab8f, 0x00ab8f, 0x0013bf, 0x0013bf},
+ {0x00ab90, 0x00ab90, 0x0013c0, 0x0013c0},
+ {0x00ab91, 0x00ab91, 0x0013c1, 0x0013c1},
+ {0x00ab92, 0x00ab92, 0x0013c2, 0x0013c2},
+ {0x00ab93, 0x00ab93, 0x0013c3, 0x0013c3},
+ {0x00ab94, 0x00ab94, 0x0013c4, 0x0013c4},
+ {0x00ab95, 0x00ab95, 0x0013c5, 0x0013c5},
+ {0x00ab96, 0x00ab96, 0x0013c6, 0x0013c6},
+ {0x00ab97, 0x00ab97, 0x0013c7, 0x0013c7},
+ {0x00ab98, 0x00ab98, 0x0013c8, 0x0013c8},
+ {0x00ab99, 0x00ab99, 0x0013c9, 0x0013c9},
+ {0x00ab9a, 0x00ab9a, 0x0013ca, 0x0013ca},
+ {0x00ab9b, 0x00ab9b, 0x0013cb, 0x0013cb},
+ {0x00ab9c, 0x00ab9c, 0x0013cc, 0x0013cc},
+ {0x00ab9d, 0x00ab9d, 0x0013cd, 0x0013cd},
+ {0x00ab9e, 0x00ab9e, 0x0013ce, 0x0013ce},
+ {0x00ab9f, 0x00ab9f, 0x0013cf, 0x0013cf},
+ {0x00aba0, 0x00aba0, 0x0013d0, 0x0013d0},
+ {0x00aba1, 0x00aba1, 0x0013d1, 0x0013d1},
+ {0x00aba2, 0x00aba2, 0x0013d2, 0x0013d2},
+ {0x00aba3, 0x00aba3, 0x0013d3, 0x0013d3},
+ {0x00aba4, 0x00aba4, 0x0013d4, 0x0013d4},
+ {0x00aba5, 0x00aba5, 0x0013d5, 0x0013d5},
+ {0x00aba6, 0x00aba6, 0x0013d6, 0x0013d6},
+ {0x00aba7, 0x00aba7, 0x0013d7, 0x0013d7},
+ {0x00aba8, 0x00aba8, 0x0013d8, 0x0013d8},
+ {0x00aba9, 0x00aba9, 0x0013d9, 0x0013d9},
+ {0x00abaa, 0x00abaa, 0x0013da, 0x0013da},
+ {0x00abab, 0x00abab, 0x0013db, 0x0013db},
+ {0x00abac, 0x00abac, 0x0013dc, 0x0013dc},
+ {0x00abad, 0x00abad, 0x0013dd, 0x0013dd},
+ {0x00abae, 0x00abae, 0x0013de, 0x0013de},
+ {0x00abaf, 0x00abaf, 0x0013df, 0x0013df},
+ {0x00abb0, 0x00abb0, 0x0013e0, 0x0013e0},
+ {0x00abb1, 0x00abb1, 0x0013e1, 0x0013e1},
+ {0x00abb2, 0x00abb2, 0x0013e2, 0x0013e2},
+ {0x00abb3, 0x00abb3, 0x0013e3, 0x0013e3},
+ {0x00abb4, 0x00abb4, 0x0013e4, 0x0013e4},
+ {0x00abb5, 0x00abb5, 0x0013e5, 0x0013e5},
+ {0x00abb6, 0x00abb6, 0x0013e6, 0x0013e6},
+ {0x00abb7, 0x00abb7, 0x0013e7, 0x0013e7},
+ {0x00abb8, 0x00abb8, 0x0013e8, 0x0013e8},
+ {0x00abb9, 0x00abb9, 0x0013e9, 0x0013e9},
+ {0x00abba, 0x00abba, 0x0013ea, 0x0013ea},
+ {0x00abbb, 0x00abbb, 0x0013eb, 0x0013eb},
+ {0x00abbc, 0x00abbc, 0x0013ec, 0x0013ec},
+ {0x00abbd, 0x00abbd, 0x0013ed, 0x0013ed},
+ {0x00abbe, 0x00abbe, 0x0013ee, 0x0013ee},
+ {0x00abbf, 0x00abbf, 0x0013ef, 0x0013ef},
+ {0x00ff21, 0x00ff41, 0x00ff21, 0x00ff21},
+ {0x00ff22, 0x00ff42, 0x00ff22, 0x00ff22},
+ {0x00ff23, 0x00ff43, 0x00ff23, 0x00ff23},
+ {0x00ff24, 0x00ff44, 0x00ff24, 0x00ff24},
+ {0x00ff25, 0x00ff45, 0x00ff25, 0x00ff25},
+ {0x00ff26, 0x00ff46, 0x00ff26, 0x00ff26},
+ {0x00ff27, 0x00ff47, 0x00ff27, 0x00ff27},
+ {0x00ff28, 0x00ff48, 0x00ff28, 0x00ff28},
+ {0x00ff29, 0x00ff49, 0x00ff29, 0x00ff29},
+ {0x00ff2a, 0x00ff4a, 0x00ff2a, 0x00ff2a},
+ {0x00ff2b, 0x00ff4b, 0x00ff2b, 0x00ff2b},
+ {0x00ff2c, 0x00ff4c, 0x00ff2c, 0x00ff2c},
+ {0x00ff2d, 0x00ff4d, 0x00ff2d, 0x00ff2d},
+ {0x00ff2e, 0x00ff4e, 0x00ff2e, 0x00ff2e},
+ {0x00ff2f, 0x00ff4f, 0x00ff2f, 0x00ff2f},
+ {0x00ff30, 0x00ff50, 0x00ff30, 0x00ff30},
+ {0x00ff31, 0x00ff51, 0x00ff31, 0x00ff31},
+ {0x00ff32, 0x00ff52, 0x00ff32, 0x00ff32},
+ {0x00ff33, 0x00ff53, 0x00ff33, 0x00ff33},
+ {0x00ff34, 0x00ff54, 0x00ff34, 0x00ff34},
+ {0x00ff35, 0x00ff55, 0x00ff35, 0x00ff35},
+ {0x00ff36, 0x00ff56, 0x00ff36, 0x00ff36},
+ {0x00ff37, 0x00ff57, 0x00ff37, 0x00ff37},
+ {0x00ff38, 0x00ff58, 0x00ff38, 0x00ff38},
+ {0x00ff39, 0x00ff59, 0x00ff39, 0x00ff39},
+ {0x00ff3a, 0x00ff5a, 0x00ff3a, 0x00ff3a},
+ {0x00ff41, 0x00ff41, 0x00ff21, 0x00ff21},
+ {0x00ff42, 0x00ff42, 0x00ff22, 0x00ff22},
+ {0x00ff43, 0x00ff43, 0x00ff23, 0x00ff23},
+ {0x00ff44, 0x00ff44, 0x00ff24, 0x00ff24},
+ {0x00ff45, 0x00ff45, 0x00ff25, 0x00ff25},
+ {0x00ff46, 0x00ff46, 0x00ff26, 0x00ff26},
+ {0x00ff47, 0x00ff47, 0x00ff27, 0x00ff27},
+ {0x00ff48, 0x00ff48, 0x00ff28, 0x00ff28},
+ {0x00ff49, 0x00ff49, 0x00ff29, 0x00ff29},
+ {0x00ff4a, 0x00ff4a, 0x00ff2a, 0x00ff2a},
+ {0x00ff4b, 0x00ff4b, 0x00ff2b, 0x00ff2b},
+ {0x00ff4c, 0x00ff4c, 0x00ff2c, 0x00ff2c},
+ {0x00ff4d, 0x00ff4d, 0x00ff2d, 0x00ff2d},
+ {0x00ff4e, 0x00ff4e, 0x00ff2e, 0x00ff2e},
+ {0x00ff4f, 0x00ff4f, 0x00ff2f, 0x00ff2f},
+ {0x00ff50, 0x00ff50, 0x00ff30, 0x00ff30},
+ {0x00ff51, 0x00ff51, 0x00ff31, 0x00ff31},
+ {0x00ff52, 0x00ff52, 0x00ff32, 0x00ff32},
+ {0x00ff53, 0x00ff53, 0x00ff33, 0x00ff33},
+ {0x00ff54, 0x00ff54, 0x00ff34, 0x00ff34},
+ {0x00ff55, 0x00ff55, 0x00ff35, 0x00ff35},
+ {0x00ff56, 0x00ff56, 0x00ff36, 0x00ff36},
+ {0x00ff57, 0x00ff57, 0x00ff37, 0x00ff37},
+ {0x00ff58, 0x00ff58, 0x00ff38, 0x00ff38},
+ {0x00ff59, 0x00ff59, 0x00ff39, 0x00ff39},
+ {0x00ff5a, 0x00ff5a, 0x00ff3a, 0x00ff3a},
+ {0x010400, 0x010428, 0x010400, 0x010400},
+ {0x010401, 0x010429, 0x010401, 0x010401},
+ {0x010402, 0x01042a, 0x010402, 0x010402},
+ {0x010403, 0x01042b, 0x010403, 0x010403},
+ {0x010404, 0x01042c, 0x010404, 0x010404},
+ {0x010405, 0x01042d, 0x010405, 0x010405},
+ {0x010406, 0x01042e, 0x010406, 0x010406},
+ {0x010407, 0x01042f, 0x010407, 0x010407},
+ {0x010408, 0x010430, 0x010408, 0x010408},
+ {0x010409, 0x010431, 0x010409, 0x010409},
+ {0x01040a, 0x010432, 0x01040a, 0x01040a},
+ {0x01040b, 0x010433, 0x01040b, 0x01040b},
+ {0x01040c, 0x010434, 0x01040c, 0x01040c},
+ {0x01040d, 0x010435, 0x01040d, 0x01040d},
+ {0x01040e, 0x010436, 0x01040e, 0x01040e},
+ {0x01040f, 0x010437, 0x01040f, 0x01040f},
+ {0x010410, 0x010438, 0x010410, 0x010410},
+ {0x010411, 0x010439, 0x010411, 0x010411},
+ {0x010412, 0x01043a, 0x010412, 0x010412},
+ {0x010413, 0x01043b, 0x010413, 0x010413},
+ {0x010414, 0x01043c, 0x010414, 0x010414},
+ {0x010415, 0x01043d, 0x010415, 0x010415},
+ {0x010416, 0x01043e, 0x010416, 0x010416},
+ {0x010417, 0x01043f, 0x010417, 0x010417},
+ {0x010418, 0x010440, 0x010418, 0x010418},
+ {0x010419, 0x010441, 0x010419, 0x010419},
+ {0x01041a, 0x010442, 0x01041a, 0x01041a},
+ {0x01041b, 0x010443, 0x01041b, 0x01041b},
+ {0x01041c, 0x010444, 0x01041c, 0x01041c},
+ {0x01041d, 0x010445, 0x01041d, 0x01041d},
+ {0x01041e, 0x010446, 0x01041e, 0x01041e},
+ {0x01041f, 0x010447, 0x01041f, 0x01041f},
+ {0x010420, 0x010448, 0x010420, 0x010420},
+ {0x010421, 0x010449, 0x010421, 0x010421},
+ {0x010422, 0x01044a, 0x010422, 0x010422},
+ {0x010423, 0x01044b, 0x010423, 0x010423},
+ {0x010424, 0x01044c, 0x010424, 0x010424},
+ {0x010425, 0x01044d, 0x010425, 0x010425},
+ {0x010426, 0x01044e, 0x010426, 0x010426},
+ {0x010427, 0x01044f, 0x010427, 0x010427},
+ {0x010428, 0x010428, 0x010400, 0x010400},
+ {0x010429, 0x010429, 0x010401, 0x010401},
+ {0x01042a, 0x01042a, 0x010402, 0x010402},
+ {0x01042b, 0x01042b, 0x010403, 0x010403},
+ {0x01042c, 0x01042c, 0x010404, 0x010404},
+ {0x01042d, 0x01042d, 0x010405, 0x010405},
+ {0x01042e, 0x01042e, 0x010406, 0x010406},
+ {0x01042f, 0x01042f, 0x010407, 0x010407},
+ {0x010430, 0x010430, 0x010408, 0x010408},
+ {0x010431, 0x010431, 0x010409, 0x010409},
+ {0x010432, 0x010432, 0x01040a, 0x01040a},
+ {0x010433, 0x010433, 0x01040b, 0x01040b},
+ {0x010434, 0x010434, 0x01040c, 0x01040c},
+ {0x010435, 0x010435, 0x01040d, 0x01040d},
+ {0x010436, 0x010436, 0x01040e, 0x01040e},
+ {0x010437, 0x010437, 0x01040f, 0x01040f},
+ {0x010438, 0x010438, 0x010410, 0x010410},
+ {0x010439, 0x010439, 0x010411, 0x010411},
+ {0x01043a, 0x01043a, 0x010412, 0x010412},
+ {0x01043b, 0x01043b, 0x010413, 0x010413},
+ {0x01043c, 0x01043c, 0x010414, 0x010414},
+ {0x01043d, 0x01043d, 0x010415, 0x010415},
+ {0x01043e, 0x01043e, 0x010416, 0x010416},
+ {0x01043f, 0x01043f, 0x010417, 0x010417},
+ {0x010440, 0x010440, 0x010418, 0x010418},
+ {0x010441, 0x010441, 0x010419, 0x010419},
+ {0x010442, 0x010442, 0x01041a, 0x01041a},
+ {0x010443, 0x010443, 0x01041b, 0x01041b},
+ {0x010444, 0x010444, 0x01041c, 0x01041c},
+ {0x010445, 0x010445, 0x01041d, 0x01041d},
+ {0x010446, 0x010446, 0x01041e, 0x01041e},
+ {0x010447, 0x010447, 0x01041f, 0x01041f},
+ {0x010448, 0x010448, 0x010420, 0x010420},
+ {0x010449, 0x010449, 0x010421, 0x010421},
+ {0x01044a, 0x01044a, 0x010422, 0x010422},
+ {0x01044b, 0x01044b, 0x010423, 0x010423},
+ {0x01044c, 0x01044c, 0x010424, 0x010424},
+ {0x01044d, 0x01044d, 0x010425, 0x010425},
+ {0x01044e, 0x01044e, 0x010426, 0x010426},
+ {0x01044f, 0x01044f, 0x010427, 0x010427},
+ {0x0104b0, 0x0104d8, 0x0104b0, 0x0104b0},
+ {0x0104b1, 0x0104d9, 0x0104b1, 0x0104b1},
+ {0x0104b2, 0x0104da, 0x0104b2, 0x0104b2},
+ {0x0104b3, 0x0104db, 0x0104b3, 0x0104b3},
+ {0x0104b4, 0x0104dc, 0x0104b4, 0x0104b4},
+ {0x0104b5, 0x0104dd, 0x0104b5, 0x0104b5},
+ {0x0104b6, 0x0104de, 0x0104b6, 0x0104b6},
+ {0x0104b7, 0x0104df, 0x0104b7, 0x0104b7},
+ {0x0104b8, 0x0104e0, 0x0104b8, 0x0104b8},
+ {0x0104b9, 0x0104e1, 0x0104b9, 0x0104b9},
+ {0x0104ba, 0x0104e2, 0x0104ba, 0x0104ba},
+ {0x0104bb, 0x0104e3, 0x0104bb, 0x0104bb},
+ {0x0104bc, 0x0104e4, 0x0104bc, 0x0104bc},
+ {0x0104bd, 0x0104e5, 0x0104bd, 0x0104bd},
+ {0x0104be, 0x0104e6, 0x0104be, 0x0104be},
+ {0x0104bf, 0x0104e7, 0x0104bf, 0x0104bf},
+ {0x0104c0, 0x0104e8, 0x0104c0, 0x0104c0},
+ {0x0104c1, 0x0104e9, 0x0104c1, 0x0104c1},
+ {0x0104c2, 0x0104ea, 0x0104c2, 0x0104c2},
+ {0x0104c3, 0x0104eb, 0x0104c3, 0x0104c3},
+ {0x0104c4, 0x0104ec, 0x0104c4, 0x0104c4},
+ {0x0104c5, 0x0104ed, 0x0104c5, 0x0104c5},
+ {0x0104c6, 0x0104ee, 0x0104c6, 0x0104c6},
+ {0x0104c7, 0x0104ef, 0x0104c7, 0x0104c7},
+ {0x0104c8, 0x0104f0, 0x0104c8, 0x0104c8},
+ {0x0104c9, 0x0104f1, 0x0104c9, 0x0104c9},
+ {0x0104ca, 0x0104f2, 0x0104ca, 0x0104ca},
+ {0x0104cb, 0x0104f3, 0x0104cb, 0x0104cb},
+ {0x0104cc, 0x0104f4, 0x0104cc, 0x0104cc},
+ {0x0104cd, 0x0104f5, 0x0104cd, 0x0104cd},
+ {0x0104ce, 0x0104f6, 0x0104ce, 0x0104ce},
+ {0x0104cf, 0x0104f7, 0x0104cf, 0x0104cf},
+ {0x0104d0, 0x0104f8, 0x0104d0, 0x0104d0},
+ {0x0104d1, 0x0104f9, 0x0104d1, 0x0104d1},
+ {0x0104d2, 0x0104fa, 0x0104d2, 0x0104d2},
+ {0x0104d3, 0x0104fb, 0x0104d3, 0x0104d3},
+ {0x0104d8, 0x0104d8, 0x0104b0, 0x0104b0},
+ {0x0104d9, 0x0104d9, 0x0104b1, 0x0104b1},
+ {0x0104da, 0x0104da, 0x0104b2, 0x0104b2},
+ {0x0104db, 0x0104db, 0x0104b3, 0x0104b3},
+ {0x0104dc, 0x0104dc, 0x0104b4, 0x0104b4},
+ {0x0104dd, 0x0104dd, 0x0104b5, 0x0104b5},
+ {0x0104de, 0x0104de, 0x0104b6, 0x0104b6},
+ {0x0104df, 0x0104df, 0x0104b7, 0x0104b7},
+ {0x0104e0, 0x0104e0, 0x0104b8, 0x0104b8},
+ {0x0104e1, 0x0104e1, 0x0104b9, 0x0104b9},
+ {0x0104e2, 0x0104e2, 0x0104ba, 0x0104ba},
+ {0x0104e3, 0x0104e3, 0x0104bb, 0x0104bb},
+ {0x0104e4, 0x0104e4, 0x0104bc, 0x0104bc},
+ {0x0104e5, 0x0104e5, 0x0104bd, 0x0104bd},
+ {0x0104e6, 0x0104e6, 0x0104be, 0x0104be},
+ {0x0104e7, 0x0104e7, 0x0104bf, 0x0104bf},
+ {0x0104e8, 0x0104e8, 0x0104c0, 0x0104c0},
+ {0x0104e9, 0x0104e9, 0x0104c1, 0x0104c1},
+ {0x0104ea, 0x0104ea, 0x0104c2, 0x0104c2},
+ {0x0104eb, 0x0104eb, 0x0104c3, 0x0104c3},
+ {0x0104ec, 0x0104ec, 0x0104c4, 0x0104c4},
+ {0x0104ed, 0x0104ed, 0x0104c5, 0x0104c5},
+ {0x0104ee, 0x0104ee, 0x0104c6, 0x0104c6},
+ {0x0104ef, 0x0104ef, 0x0104c7, 0x0104c7},
+ {0x0104f0, 0x0104f0, 0x0104c8, 0x0104c8},
+ {0x0104f1, 0x0104f1, 0x0104c9, 0x0104c9},
+ {0x0104f2, 0x0104f2, 0x0104ca, 0x0104ca},
+ {0x0104f3, 0x0104f3, 0x0104cb, 0x0104cb},
+ {0x0104f4, 0x0104f4, 0x0104cc, 0x0104cc},
+ {0x0104f5, 0x0104f5, 0x0104cd, 0x0104cd},
+ {0x0104f6, 0x0104f6, 0x0104ce, 0x0104ce},
+ {0x0104f7, 0x0104f7, 0x0104cf, 0x0104cf},
+ {0x0104f8, 0x0104f8, 0x0104d0, 0x0104d0},
+ {0x0104f9, 0x0104f9, 0x0104d1, 0x0104d1},
+ {0x0104fa, 0x0104fa, 0x0104d2, 0x0104d2},
+ {0x0104fb, 0x0104fb, 0x0104d3, 0x0104d3},
+ {0x010570, 0x010597, 0x010570, 0x010570},
+ {0x010571, 0x010598, 0x010571, 0x010571},
+ {0x010572, 0x010599, 0x010572, 0x010572},
+ {0x010573, 0x01059a, 0x010573, 0x010573},
+ {0x010574, 0x01059b, 0x010574, 0x010574},
+ {0x010575, 0x01059c, 0x010575, 0x010575},
+ {0x010576, 0x01059d, 0x010576, 0x010576},
+ {0x010577, 0x01059e, 0x010577, 0x010577},
+ {0x010578, 0x01059f, 0x010578, 0x010578},
+ {0x010579, 0x0105a0, 0x010579, 0x010579},
+ {0x01057a, 0x0105a1, 0x01057a, 0x01057a},
+ {0x01057c, 0x0105a3, 0x01057c, 0x01057c},
+ {0x01057d, 0x0105a4, 0x01057d, 0x01057d},
+ {0x01057e, 0x0105a5, 0x01057e, 0x01057e},
+ {0x01057f, 0x0105a6, 0x01057f, 0x01057f},
+ {0x010580, 0x0105a7, 0x010580, 0x010580},
+ {0x010581, 0x0105a8, 0x010581, 0x010581},
+ {0x010582, 0x0105a9, 0x010582, 0x010582},
+ {0x010583, 0x0105aa, 0x010583, 0x010583},
+ {0x010584, 0x0105ab, 0x010584, 0x010584},
+ {0x010585, 0x0105ac, 0x010585, 0x010585},
+ {0x010586, 0x0105ad, 0x010586, 0x010586},
+ {0x010587, 0x0105ae, 0x010587, 0x010587},
+ {0x010588, 0x0105af, 0x010588, 0x010588},
+ {0x010589, 0x0105b0, 0x010589, 0x010589},
+ {0x01058a, 0x0105b1, 0x01058a, 0x01058a},
+ {0x01058c, 0x0105b3, 0x01058c, 0x01058c},
+ {0x01058d, 0x0105b4, 0x01058d, 0x01058d},
+ {0x01058e, 0x0105b5, 0x01058e, 0x01058e},
+ {0x01058f, 0x0105b6, 0x01058f, 0x01058f},
+ {0x010590, 0x0105b7, 0x010590, 0x010590},
+ {0x010591, 0x0105b8, 0x010591, 0x010591},
+ {0x010592, 0x0105b9, 0x010592, 0x010592},
+ {0x010594, 0x0105bb, 0x010594, 0x010594},
+ {0x010595, 0x0105bc, 0x010595, 0x010595},
+ {0x010597, 0x010597, 0x010570, 0x010570},
+ {0x010598, 0x010598, 0x010571, 0x010571},
+ {0x010599, 0x010599, 0x010572, 0x010572},
+ {0x01059a, 0x01059a, 0x010573, 0x010573},
+ {0x01059b, 0x01059b, 0x010574, 0x010574},
+ {0x01059c, 0x01059c, 0x010575, 0x010575},
+ {0x01059d, 0x01059d, 0x010576, 0x010576},
+ {0x01059e, 0x01059e, 0x010577, 0x010577},
+ {0x01059f, 0x01059f, 0x010578, 0x010578},
+ {0x0105a0, 0x0105a0, 0x010579, 0x010579},
+ {0x0105a1, 0x0105a1, 0x01057a, 0x01057a},
+ {0x0105a3, 0x0105a3, 0x01057c, 0x01057c},
+ {0x0105a4, 0x0105a4, 0x01057d, 0x01057d},
+ {0x0105a5, 0x0105a5, 0x01057e, 0x01057e},
+ {0x0105a6, 0x0105a6, 0x01057f, 0x01057f},
+ {0x0105a7, 0x0105a7, 0x010580, 0x010580},
+ {0x0105a8, 0x0105a8, 0x010581, 0x010581},
+ {0x0105a9, 0x0105a9, 0x010582, 0x010582},
+ {0x0105aa, 0x0105aa, 0x010583, 0x010583},
+ {0x0105ab, 0x0105ab, 0x010584, 0x010584},
+ {0x0105ac, 0x0105ac, 0x010585, 0x010585},
+ {0x0105ad, 0x0105ad, 0x010586, 0x010586},
+ {0x0105ae, 0x0105ae, 0x010587, 0x010587},
+ {0x0105af, 0x0105af, 0x010588, 0x010588},
+ {0x0105b0, 0x0105b0, 0x010589, 0x010589},
+ {0x0105b1, 0x0105b1, 0x01058a, 0x01058a},
+ {0x0105b3, 0x0105b3, 0x01058c, 0x01058c},
+ {0x0105b4, 0x0105b4, 0x01058d, 0x01058d},
+ {0x0105b5, 0x0105b5, 0x01058e, 0x01058e},
+ {0x0105b6, 0x0105b6, 0x01058f, 0x01058f},
+ {0x0105b7, 0x0105b7, 0x010590, 0x010590},
+ {0x0105b8, 0x0105b8, 0x010591, 0x010591},
+ {0x0105b9, 0x0105b9, 0x010592, 0x010592},
+ {0x0105bb, 0x0105bb, 0x010594, 0x010594},
+ {0x0105bc, 0x0105bc, 0x010595, 0x010595},
+ {0x010c80, 0x010cc0, 0x010c80, 0x010c80},
+ {0x010c81, 0x010cc1, 0x010c81, 0x010c81},
+ {0x010c82, 0x010cc2, 0x010c82, 0x010c82},
+ {0x010c83, 0x010cc3, 0x010c83, 0x010c83},
+ {0x010c84, 0x010cc4, 0x010c84, 0x010c84},
+ {0x010c85, 0x010cc5, 0x010c85, 0x010c85},
+ {0x010c86, 0x010cc6, 0x010c86, 0x010c86},
+ {0x010c87, 0x010cc7, 0x010c87, 0x010c87},
+ {0x010c88, 0x010cc8, 0x010c88, 0x010c88},
+ {0x010c89, 0x010cc9, 0x010c89, 0x010c89},
+ {0x010c8a, 0x010cca, 0x010c8a, 0x010c8a},
+ {0x010c8b, 0x010ccb, 0x010c8b, 0x010c8b},
+ {0x010c8c, 0x010ccc, 0x010c8c, 0x010c8c},
+ {0x010c8d, 0x010ccd, 0x010c8d, 0x010c8d},
+ {0x010c8e, 0x010cce, 0x010c8e, 0x010c8e},
+ {0x010c8f, 0x010ccf, 0x010c8f, 0x010c8f},
+ {0x010c90, 0x010cd0, 0x010c90, 0x010c90},
+ {0x010c91, 0x010cd1, 0x010c91, 0x010c91},
+ {0x010c92, 0x010cd2, 0x010c92, 0x010c92},
+ {0x010c93, 0x010cd3, 0x010c93, 0x010c93},
+ {0x010c94, 0x010cd4, 0x010c94, 0x010c94},
+ {0x010c95, 0x010cd5, 0x010c95, 0x010c95},
+ {0x010c96, 0x010cd6, 0x010c96, 0x010c96},
+ {0x010c97, 0x010cd7, 0x010c97, 0x010c97},
+ {0x010c98, 0x010cd8, 0x010c98, 0x010c98},
+ {0x010c99, 0x010cd9, 0x010c99, 0x010c99},
+ {0x010c9a, 0x010cda, 0x010c9a, 0x010c9a},
+ {0x010c9b, 0x010cdb, 0x010c9b, 0x010c9b},
+ {0x010c9c, 0x010cdc, 0x010c9c, 0x010c9c},
+ {0x010c9d, 0x010cdd, 0x010c9d, 0x010c9d},
+ {0x010c9e, 0x010cde, 0x010c9e, 0x010c9e},
+ {0x010c9f, 0x010cdf, 0x010c9f, 0x010c9f},
+ {0x010ca0, 0x010ce0, 0x010ca0, 0x010ca0},
+ {0x010ca1, 0x010ce1, 0x010ca1, 0x010ca1},
+ {0x010ca2, 0x010ce2, 0x010ca2, 0x010ca2},
+ {0x010ca3, 0x010ce3, 0x010ca3, 0x010ca3},
+ {0x010ca4, 0x010ce4, 0x010ca4, 0x010ca4},
+ {0x010ca5, 0x010ce5, 0x010ca5, 0x010ca5},
+ {0x010ca6, 0x010ce6, 0x010ca6, 0x010ca6},
+ {0x010ca7, 0x010ce7, 0x010ca7, 0x010ca7},
+ {0x010ca8, 0x010ce8, 0x010ca8, 0x010ca8},
+ {0x010ca9, 0x010ce9, 0x010ca9, 0x010ca9},
+ {0x010caa, 0x010cea, 0x010caa, 0x010caa},
+ {0x010cab, 0x010ceb, 0x010cab, 0x010cab},
+ {0x010cac, 0x010cec, 0x010cac, 0x010cac},
+ {0x010cad, 0x010ced, 0x010cad, 0x010cad},
+ {0x010cae, 0x010cee, 0x010cae, 0x010cae},
+ {0x010caf, 0x010cef, 0x010caf, 0x010caf},
+ {0x010cb0, 0x010cf0, 0x010cb0, 0x010cb0},
+ {0x010cb1, 0x010cf1, 0x010cb1, 0x010cb1},
+ {0x010cb2, 0x010cf2, 0x010cb2, 0x010cb2},
+ {0x010cc0, 0x010cc0, 0x010c80, 0x010c80},
+ {0x010cc1, 0x010cc1, 0x010c81, 0x010c81},
+ {0x010cc2, 0x010cc2, 0x010c82, 0x010c82},
+ {0x010cc3, 0x010cc3, 0x010c83, 0x010c83},
+ {0x010cc4, 0x010cc4, 0x010c84, 0x010c84},
+ {0x010cc5, 0x010cc5, 0x010c85, 0x010c85},
+ {0x010cc6, 0x010cc6, 0x010c86, 0x010c86},
+ {0x010cc7, 0x010cc7, 0x010c87, 0x010c87},
+ {0x010cc8, 0x010cc8, 0x010c88, 0x010c88},
+ {0x010cc9, 0x010cc9, 0x010c89, 0x010c89},
+ {0x010cca, 0x010cca, 0x010c8a, 0x010c8a},
+ {0x010ccb, 0x010ccb, 0x010c8b, 0x010c8b},
+ {0x010ccc, 0x010ccc, 0x010c8c, 0x010c8c},
+ {0x010ccd, 0x010ccd, 0x010c8d, 0x010c8d},
+ {0x010cce, 0x010cce, 0x010c8e, 0x010c8e},
+ {0x010ccf, 0x010ccf, 0x010c8f, 0x010c8f},
+ {0x010cd0, 0x010cd0, 0x010c90, 0x010c90},
+ {0x010cd1, 0x010cd1, 0x010c91, 0x010c91},
+ {0x010cd2, 0x010cd2, 0x010c92, 0x010c92},
+ {0x010cd3, 0x010cd3, 0x010c93, 0x010c93},
+ {0x010cd4, 0x010cd4, 0x010c94, 0x010c94},
+ {0x010cd5, 0x010cd5, 0x010c95, 0x010c95},
+ {0x010cd6, 0x010cd6, 0x010c96, 0x010c96},
+ {0x010cd7, 0x010cd7, 0x010c97, 0x010c97},
+ {0x010cd8, 0x010cd8, 0x010c98, 0x010c98},
+ {0x010cd9, 0x010cd9, 0x010c99, 0x010c99},
+ {0x010cda, 0x010cda, 0x010c9a, 0x010c9a},
+ {0x010cdb, 0x010cdb, 0x010c9b, 0x010c9b},
+ {0x010cdc, 0x010cdc, 0x010c9c, 0x010c9c},
+ {0x010cdd, 0x010cdd, 0x010c9d, 0x010c9d},
+ {0x010cde, 0x010cde, 0x010c9e, 0x010c9e},
+ {0x010cdf, 0x010cdf, 0x010c9f, 0x010c9f},
+ {0x010ce0, 0x010ce0, 0x010ca0, 0x010ca0},
+ {0x010ce1, 0x010ce1, 0x010ca1, 0x010ca1},
+ {0x010ce2, 0x010ce2, 0x010ca2, 0x010ca2},
+ {0x010ce3, 0x010ce3, 0x010ca3, 0x010ca3},
+ {0x010ce4, 0x010ce4, 0x010ca4, 0x010ca4},
+ {0x010ce5, 0x010ce5, 0x010ca5, 0x010ca5},
+ {0x010ce6, 0x010ce6, 0x010ca6, 0x010ca6},
+ {0x010ce7, 0x010ce7, 0x010ca7, 0x010ca7},
+ {0x010ce8, 0x010ce8, 0x010ca8, 0x010ca8},
+ {0x010ce9, 0x010ce9, 0x010ca9, 0x010ca9},
+ {0x010cea, 0x010cea, 0x010caa, 0x010caa},
+ {0x010ceb, 0x010ceb, 0x010cab, 0x010cab},
+ {0x010cec, 0x010cec, 0x010cac, 0x010cac},
+ {0x010ced, 0x010ced, 0x010cad, 0x010cad},
+ {0x010cee, 0x010cee, 0x010cae, 0x010cae},
+ {0x010cef, 0x010cef, 0x010caf, 0x010caf},
+ {0x010cf0, 0x010cf0, 0x010cb0, 0x010cb0},
+ {0x010cf1, 0x010cf1, 0x010cb1, 0x010cb1},
+ {0x010cf2, 0x010cf2, 0x010cb2, 0x010cb2},
+ {0x0118a0, 0x0118c0, 0x0118a0, 0x0118a0},
+ {0x0118a1, 0x0118c1, 0x0118a1, 0x0118a1},
+ {0x0118a2, 0x0118c2, 0x0118a2, 0x0118a2},
+ {0x0118a3, 0x0118c3, 0x0118a3, 0x0118a3},
+ {0x0118a4, 0x0118c4, 0x0118a4, 0x0118a4},
+ {0x0118a5, 0x0118c5, 0x0118a5, 0x0118a5},
+ {0x0118a6, 0x0118c6, 0x0118a6, 0x0118a6},
+ {0x0118a7, 0x0118c7, 0x0118a7, 0x0118a7},
+ {0x0118a8, 0x0118c8, 0x0118a8, 0x0118a8},
+ {0x0118a9, 0x0118c9, 0x0118a9, 0x0118a9},
+ {0x0118aa, 0x0118ca, 0x0118aa, 0x0118aa},
+ {0x0118ab, 0x0118cb, 0x0118ab, 0x0118ab},
+ {0x0118ac, 0x0118cc, 0x0118ac, 0x0118ac},
+ {0x0118ad, 0x0118cd, 0x0118ad, 0x0118ad},
+ {0x0118ae, 0x0118ce, 0x0118ae, 0x0118ae},
+ {0x0118af, 0x0118cf, 0x0118af, 0x0118af},
+ {0x0118b0, 0x0118d0, 0x0118b0, 0x0118b0},
+ {0x0118b1, 0x0118d1, 0x0118b1, 0x0118b1},
+ {0x0118b2, 0x0118d2, 0x0118b2, 0x0118b2},
+ {0x0118b3, 0x0118d3, 0x0118b3, 0x0118b3},
+ {0x0118b4, 0x0118d4, 0x0118b4, 0x0118b4},
+ {0x0118b5, 0x0118d5, 0x0118b5, 0x0118b5},
+ {0x0118b6, 0x0118d6, 0x0118b6, 0x0118b6},
+ {0x0118b7, 0x0118d7, 0x0118b7, 0x0118b7},
+ {0x0118b8, 0x0118d8, 0x0118b8, 0x0118b8},
+ {0x0118b9, 0x0118d9, 0x0118b9, 0x0118b9},
+ {0x0118ba, 0x0118da, 0x0118ba, 0x0118ba},
+ {0x0118bb, 0x0118db, 0x0118bb, 0x0118bb},
+ {0x0118bc, 0x0118dc, 0x0118bc, 0x0118bc},
+ {0x0118bd, 0x0118dd, 0x0118bd, 0x0118bd},
+ {0x0118be, 0x0118de, 0x0118be, 0x0118be},
+ {0x0118bf, 0x0118df, 0x0118bf, 0x0118bf},
+ {0x0118c0, 0x0118c0, 0x0118a0, 0x0118a0},
+ {0x0118c1, 0x0118c1, 0x0118a1, 0x0118a1},
+ {0x0118c2, 0x0118c2, 0x0118a2, 0x0118a2},
+ {0x0118c3, 0x0118c3, 0x0118a3, 0x0118a3},
+ {0x0118c4, 0x0118c4, 0x0118a4, 0x0118a4},
+ {0x0118c5, 0x0118c5, 0x0118a5, 0x0118a5},
+ {0x0118c6, 0x0118c6, 0x0118a6, 0x0118a6},
+ {0x0118c7, 0x0118c7, 0x0118a7, 0x0118a7},
+ {0x0118c8, 0x0118c8, 0x0118a8, 0x0118a8},
+ {0x0118c9, 0x0118c9, 0x0118a9, 0x0118a9},
+ {0x0118ca, 0x0118ca, 0x0118aa, 0x0118aa},
+ {0x0118cb, 0x0118cb, 0x0118ab, 0x0118ab},
+ {0x0118cc, 0x0118cc, 0x0118ac, 0x0118ac},
+ {0x0118cd, 0x0118cd, 0x0118ad, 0x0118ad},
+ {0x0118ce, 0x0118ce, 0x0118ae, 0x0118ae},
+ {0x0118cf, 0x0118cf, 0x0118af, 0x0118af},
+ {0x0118d0, 0x0118d0, 0x0118b0, 0x0118b0},
+ {0x0118d1, 0x0118d1, 0x0118b1, 0x0118b1},
+ {0x0118d2, 0x0118d2, 0x0118b2, 0x0118b2},
+ {0x0118d3, 0x0118d3, 0x0118b3, 0x0118b3},
+ {0x0118d4, 0x0118d4, 0x0118b4, 0x0118b4},
+ {0x0118d5, 0x0118d5, 0x0118b5, 0x0118b5},
+ {0x0118d6, 0x0118d6, 0x0118b6, 0x0118b6},
+ {0x0118d7, 0x0118d7, 0x0118b7, 0x0118b7},
+ {0x0118d8, 0x0118d8, 0x0118b8, 0x0118b8},
+ {0x0118d9, 0x0118d9, 0x0118b9, 0x0118b9},
+ {0x0118da, 0x0118da, 0x0118ba, 0x0118ba},
+ {0x0118db, 0x0118db, 0x0118bb, 0x0118bb},
+ {0x0118dc, 0x0118dc, 0x0118bc, 0x0118bc},
+ {0x0118dd, 0x0118dd, 0x0118bd, 0x0118bd},
+ {0x0118de, 0x0118de, 0x0118be, 0x0118be},
+ {0x0118df, 0x0118df, 0x0118bf, 0x0118bf},
+ {0x016e40, 0x016e60, 0x016e40, 0x016e40},
+ {0x016e41, 0x016e61, 0x016e41, 0x016e41},
+ {0x016e42, 0x016e62, 0x016e42, 0x016e42},
+ {0x016e43, 0x016e63, 0x016e43, 0x016e43},
+ {0x016e44, 0x016e64, 0x016e44, 0x016e44},
+ {0x016e45, 0x016e65, 0x016e45, 0x016e45},
+ {0x016e46, 0x016e66, 0x016e46, 0x016e46},
+ {0x016e47, 0x016e67, 0x016e47, 0x016e47},
+ {0x016e48, 0x016e68, 0x016e48, 0x016e48},
+ {0x016e49, 0x016e69, 0x016e49, 0x016e49},
+ {0x016e4a, 0x016e6a, 0x016e4a, 0x016e4a},
+ {0x016e4b, 0x016e6b, 0x016e4b, 0x016e4b},
+ {0x016e4c, 0x016e6c, 0x016e4c, 0x016e4c},
+ {0x016e4d, 0x016e6d, 0x016e4d, 0x016e4d},
+ {0x016e4e, 0x016e6e, 0x016e4e, 0x016e4e},
+ {0x016e4f, 0x016e6f, 0x016e4f, 0x016e4f},
+ {0x016e50, 0x016e70, 0x016e50, 0x016e50},
+ {0x016e51, 0x016e71, 0x016e51, 0x016e51},
+ {0x016e52, 0x016e72, 0x016e52, 0x016e52},
+ {0x016e53, 0x016e73, 0x016e53, 0x016e53},
+ {0x016e54, 0x016e74, 0x016e54, 0x016e54},
+ {0x016e55, 0x016e75, 0x016e55, 0x016e55},
+ {0x016e56, 0x016e76, 0x016e56, 0x016e56},
+ {0x016e57, 0x016e77, 0x016e57, 0x016e57},
+ {0x016e58, 0x016e78, 0x016e58, 0x016e58},
+ {0x016e59, 0x016e79, 0x016e59, 0x016e59},
+ {0x016e5a, 0x016e7a, 0x016e5a, 0x016e5a},
+ {0x016e5b, 0x016e7b, 0x016e5b, 0x016e5b},
+ {0x016e5c, 0x016e7c, 0x016e5c, 0x016e5c},
+ {0x016e5d, 0x016e7d, 0x016e5d, 0x016e5d},
+ {0x016e5e, 0x016e7e, 0x016e5e, 0x016e5e},
+ {0x016e5f, 0x016e7f, 0x016e5f, 0x016e5f},
+ {0x016e60, 0x016e60, 0x016e40, 0x016e40},
+ {0x016e61, 0x016e61, 0x016e41, 0x016e41},
+ {0x016e62, 0x016e62, 0x016e42, 0x016e42},
+ {0x016e63, 0x016e63, 0x016e43, 0x016e43},
+ {0x016e64, 0x016e64, 0x016e44, 0x016e44},
+ {0x016e65, 0x016e65, 0x016e45, 0x016e45},
+ {0x016e66, 0x016e66, 0x016e46, 0x016e46},
+ {0x016e67, 0x016e67, 0x016e47, 0x016e47},
+ {0x016e68, 0x016e68, 0x016e48, 0x016e48},
+ {0x016e69, 0x016e69, 0x016e49, 0x016e49},
+ {0x016e6a, 0x016e6a, 0x016e4a, 0x016e4a},
+ {0x016e6b, 0x016e6b, 0x016e4b, 0x016e4b},
+ {0x016e6c, 0x016e6c, 0x016e4c, 0x016e4c},
+ {0x016e6d, 0x016e6d, 0x016e4d, 0x016e4d},
+ {0x016e6e, 0x016e6e, 0x016e4e, 0x016e4e},
+ {0x016e6f, 0x016e6f, 0x016e4f, 0x016e4f},
+ {0x016e70, 0x016e70, 0x016e50, 0x016e50},
+ {0x016e71, 0x016e71, 0x016e51, 0x016e51},
+ {0x016e72, 0x016e72, 0x016e52, 0x016e52},
+ {0x016e73, 0x016e73, 0x016e53, 0x016e53},
+ {0x016e74, 0x016e74, 0x016e54, 0x016e54},
+ {0x016e75, 0x016e75, 0x016e55, 0x016e55},
+ {0x016e76, 0x016e76, 0x016e56, 0x016e56},
+ {0x016e77, 0x016e77, 0x016e57, 0x016e57},
+ {0x016e78, 0x016e78, 0x016e58, 0x016e58},
+ {0x016e79, 0x016e79, 0x016e59, 0x016e59},
+ {0x016e7a, 0x016e7a, 0x016e5a, 0x016e5a},
+ {0x016e7b, 0x016e7b, 0x016e5b, 0x016e5b},
+ {0x016e7c, 0x016e7c, 0x016e5c, 0x016e5c},
+ {0x016e7d, 0x016e7d, 0x016e5d, 0x016e5d},
+ {0x016e7e, 0x016e7e, 0x016e5e, 0x016e5e},
+ {0x016e7f, 0x016e7f, 0x016e5f, 0x016e5f},
+ {0x01e900, 0x01e922, 0x01e900, 0x01e900},
+ {0x01e901, 0x01e923, 0x01e901, 0x01e901},
+ {0x01e902, 0x01e924, 0x01e902, 0x01e902},
+ {0x01e903, 0x01e925, 0x01e903, 0x01e903},
+ {0x01e904, 0x01e926, 0x01e904, 0x01e904},
+ {0x01e905, 0x01e927, 0x01e905, 0x01e905},
+ {0x01e906, 0x01e928, 0x01e906, 0x01e906},
+ {0x01e907, 0x01e929, 0x01e907, 0x01e907},
+ {0x01e908, 0x01e92a, 0x01e908, 0x01e908},
+ {0x01e909, 0x01e92b, 0x01e909, 0x01e909},
+ {0x01e90a, 0x01e92c, 0x01e90a, 0x01e90a},
+ {0x01e90b, 0x01e92d, 0x01e90b, 0x01e90b},
+ {0x01e90c, 0x01e92e, 0x01e90c, 0x01e90c},
+ {0x01e90d, 0x01e92f, 0x01e90d, 0x01e90d},
+ {0x01e90e, 0x01e930, 0x01e90e, 0x01e90e},
+ {0x01e90f, 0x01e931, 0x01e90f, 0x01e90f},
+ {0x01e910, 0x01e932, 0x01e910, 0x01e910},
+ {0x01e911, 0x01e933, 0x01e911, 0x01e911},
+ {0x01e912, 0x01e934, 0x01e912, 0x01e912},
+ {0x01e913, 0x01e935, 0x01e913, 0x01e913},
+ {0x01e914, 0x01e936, 0x01e914, 0x01e914},
+ {0x01e915, 0x01e937, 0x01e915, 0x01e915},
+ {0x01e916, 0x01e938, 0x01e916, 0x01e916},
+ {0x01e917, 0x01e939, 0x01e917, 0x01e917},
+ {0x01e918, 0x01e93a, 0x01e918, 0x01e918},
+ {0x01e919, 0x01e93b, 0x01e919, 0x01e919},
+ {0x01e91a, 0x01e93c, 0x01e91a, 0x01e91a},
+ {0x01e91b, 0x01e93d, 0x01e91b, 0x01e91b},
+ {0x01e91c, 0x01e93e, 0x01e91c, 0x01e91c},
+ {0x01e91d, 0x01e93f, 0x01e91d, 0x01e91d},
+ {0x01e91e, 0x01e940, 0x01e91e, 0x01e91e},
+ {0x01e91f, 0x01e941, 0x01e91f, 0x01e91f},
+ {0x01e920, 0x01e942, 0x01e920, 0x01e920},
+ {0x01e921, 0x01e943, 0x01e921, 0x01e921},
+ {0x01e922, 0x01e922, 0x01e900, 0x01e900},
+ {0x01e923, 0x01e923, 0x01e901, 0x01e901},
+ {0x01e924, 0x01e924, 0x01e902, 0x01e902},
+ {0x01e925, 0x01e925, 0x01e903, 0x01e903},
+ {0x01e926, 0x01e926, 0x01e904, 0x01e904},
+ {0x01e927, 0x01e927, 0x01e905, 0x01e905},
+ {0x01e928, 0x01e928, 0x01e906, 0x01e906},
+ {0x01e929, 0x01e929, 0x01e907, 0x01e907},
+ {0x01e92a, 0x01e92a, 0x01e908, 0x01e908},
+ {0x01e92b, 0x01e92b, 0x01e909, 0x01e909},
+ {0x01e92c, 0x01e92c, 0x01e90a, 0x01e90a},
+ {0x01e92d, 0x01e92d, 0x01e90b, 0x01e90b},
+ {0x01e92e, 0x01e92e, 0x01e90c, 0x01e90c},
+ {0x01e92f, 0x01e92f, 0x01e90d, 0x01e90d},
+ {0x01e930, 0x01e930, 0x01e90e, 0x01e90e},
+ {0x01e931, 0x01e931, 0x01e90f, 0x01e90f},
+ {0x01e932, 0x01e932, 0x01e910, 0x01e910},
+ {0x01e933, 0x01e933, 0x01e911, 0x01e911},
+ {0x01e934, 0x01e934, 0x01e912, 0x01e912},
+ {0x01e935, 0x01e935, 0x01e913, 0x01e913},
+ {0x01e936, 0x01e936, 0x01e914, 0x01e914},
+ {0x01e937, 0x01e937, 0x01e915, 0x01e915},
+ {0x01e938, 0x01e938, 0x01e916, 0x01e916},
+ {0x01e939, 0x01e939, 0x01e917, 0x01e917},
+ {0x01e93a, 0x01e93a, 0x01e918, 0x01e918},
+ {0x01e93b, 0x01e93b, 0x01e919, 0x01e919},
+ {0x01e93c, 0x01e93c, 0x01e91a, 0x01e91a},
+ {0x01e93d, 0x01e93d, 0x01e91b, 0x01e91b},
+ {0x01e93e, 0x01e93e, 0x01e91c, 0x01e91c},
+ {0x01e93f, 0x01e93f, 0x01e91d, 0x01e91d},
+ {0x01e940, 0x01e940, 0x01e91e, 0x01e91e},
+ {0x01e941, 0x01e941, 0x01e91f, 0x01e91f},
+ {0x01e942, 0x01e942, 0x01e920, 0x01e920},
+ {0x01e943, 0x01e943, 0x01e921, 0x01e921}
+};
--
2.34.1
[text/x-patch] v16-0004-Catalog-changes-preparing-for-builtin-collation-.patch (48.3K, ../../[email protected]/4-v16-0004-Catalog-changes-preparing-for-builtin-collation-.patch)
download | inline diff:
From 3456ed4e8639371a06ab9539d6e1e34762caa297 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Tue, 26 Dec 2023 13:32:48 -0800
Subject: [PATCH v16 4/5] Catalog changes preparing for builtin collation
provider.
daticulocale -> datlocale, colliculocale -> colllocale.
---
doc/src/sgml/bki.sgml | 2 +-
doc/src/sgml/catalogs.sgml | 8 +-
src/backend/catalog/pg_collation.c | 10 +--
src/backend/commands/collationcmds.c | 34 +++----
src/backend/commands/dbcommands.c | 68 +++++++-------
src/backend/utils/adt/pg_locale.c | 4 +-
src/backend/utils/init/postinit.c | 2 +-
src/bin/initdb/initdb.c | 41 ++++-----
src/bin/initdb/t/001_initdb.pl | 4 +-
src/bin/pg_dump/pg_dump.c | 90 +++++++++++--------
src/bin/pg_upgrade/info.c | 31 ++++---
src/bin/pg_upgrade/pg_upgrade.c | 34 ++++---
src/bin/pg_upgrade/pg_upgrade.h | 2 +-
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 15 +++-
src/bin/psql/describe.c | 20 +++--
src/include/catalog/pg_collation.dat | 2 +-
src/include/catalog/pg_collation.h | 4 +-
src/include/catalog/pg_database.dat | 2 +-
src/include/catalog/pg_database.h | 2 +-
.../regress/expected/collate.icu.utf8.out | 4 +-
src/test/regress/expected/psql.out | 18 ++--
src/test/regress/sql/collate.icu.utf8.sql | 4 +-
22 files changed, 227 insertions(+), 174 deletions(-)
diff --git a/doc/src/sgml/bki.sgml b/doc/src/sgml/bki.sgml
index 315ba81951..3cd5bee7ff 100644
--- a/doc/src/sgml/bki.sgml
+++ b/doc/src/sgml/bki.sgml
@@ -186,7 +186,7 @@
datlocprovider => 'LOCALE_PROVIDER', datistemplate => 't',
datallowconn => 't', dathasloginevt => 'f', datconnlimit => '-1', datfrozenxid => '0',
datminmxid => '1', dattablespace => 'pg_default', datcollate => 'LC_COLLATE',
- datctype => 'LC_CTYPE', daticulocale => 'ICU_LOCALE', datacl => '_null_' },
+ datctype => 'LC_CTYPE', datlocale => 'DATLOCALE', datacl => '_null_' },
]
]]></programlisting>
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3ec7391ec5..26d16d25c8 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2422,10 +2422,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>colliculocale</structfield> <type>text</type>
+ <structfield>colllocale</structfield> <type>text</type>
</para>
<para>
- ICU locale ID for this collation object
+ Locale name for builtin or ICU provider
</para></entry>
</row>
@@ -3121,10 +3121,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>daticulocale</structfield> <type>text</type>
+ <structfield>datlocale</structfield> <type>text</type>
</para>
<para>
- ICU locale ID for this database
+ Locale name for builtin or ICU provider
</para></entry>
</row>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5c8ccb8b3b..7bad94f908 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,7 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *colliculocale,
+ const char *colllocale,
const char *collicurules,
const char *collversion,
bool if_not_exists,
@@ -68,7 +68,7 @@ CollationCreate(const char *collname, Oid collnamespace,
Assert(collname);
Assert(collnamespace);
Assert(collowner);
- Assert((collcollate && collctype) || colliculocale);
+ Assert((collcollate && collctype) || colllocale);
/*
* Make sure there is no existing collation of same name & encoding.
@@ -191,10 +191,10 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collctype - 1] = CStringGetTextDatum(collctype);
else
nulls[Anum_pg_collation_collctype - 1] = true;
- if (colliculocale)
- values[Anum_pg_collation_colliculocale - 1] = CStringGetTextDatum(colliculocale);
+ if (colllocale)
+ values[Anum_pg_collation_colllocale - 1] = CStringGetTextDatum(colllocale);
else
- nulls[Anum_pg_collation_colliculocale - 1] = true;
+ nulls[Anum_pg_collation_colllocale - 1] = true;
if (collicurules)
values[Anum_pg_collation_collicurules - 1] = CStringGetTextDatum(collicurules);
else
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 58c059fdb7..27564e569a 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *versionEl = NULL;
char *collcollate;
char *collctype;
- char *colliculocale;
+ char *colllocale;
char *collicurules;
bool collisdeterministic;
int collencoding;
@@ -159,11 +159,11 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
else
collctype = NULL;
- datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_colliculocale, &isnull);
+ datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_colllocale, &isnull);
if (!isnull)
- colliculocale = TextDatumGetCString(datum);
+ colllocale = TextDatumGetCString(datum);
else
- colliculocale = NULL;
+ colllocale = NULL;
/*
* When the ICU locale comes from an existing collation, do not
@@ -196,7 +196,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collcollate = NULL;
collctype = NULL;
- colliculocale = NULL;
+ colllocale = NULL;
collicurules = NULL;
if (providerEl)
@@ -236,7 +236,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collctype = defGetString(localeEl);
}
else
- colliculocale = defGetString(localeEl);
+ colllocale = defGetString(localeEl);
}
if (lccollateEl)
@@ -261,7 +261,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
else if (collprovider == COLLPROVIDER_ICU)
{
- if (!colliculocale)
+ if (!colllocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("parameter \"%s\" must be specified",
@@ -273,20 +273,20 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
*/
if (!IsBinaryUpgrade)
{
- char *langtag = icu_language_tag(colliculocale,
+ char *langtag = icu_language_tag(colllocale,
icu_validation_level);
- if (langtag && strcmp(colliculocale, langtag) != 0)
+ if (langtag && strcmp(colllocale, langtag) != 0)
{
ereport(NOTICE,
(errmsg("using standard form \"%s\" for ICU locale \"%s\"",
- langtag, colliculocale)));
+ langtag, colllocale)));
- colliculocale = langtag;
+ colllocale = langtag;
}
}
- icu_validate_locale(colliculocale);
+ icu_validate_locale(colllocale);
}
/*
@@ -334,7 +334,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
if (!collversion)
- collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colliculocale : collcollate);
+ collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colllocale : collcollate);
newoid = CollationCreate(collName,
collNamespace,
@@ -344,7 +344,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- colliculocale,
+ colllocale,
collicurules,
collversion,
if_not_exists,
@@ -435,7 +435,7 @@ AlterCollation(AlterCollationStmt *stmt)
datum = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion, &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
newversion = get_collation_actual_version(collForm->collprovider, TextDatumGetCString(datum));
/* cannot change from NULL to non-NULL or vice versa */
@@ -502,7 +502,7 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup,
provider == COLLPROVIDER_ICU ?
- Anum_pg_database_daticulocale : Anum_pg_database_datcollate);
+ Anum_pg_database_datlocale : Anum_pg_database_datcollate);
locale = TextDatumGetCString(datum);
@@ -523,7 +523,7 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
Assert(provider != COLLPROVIDER_DEFAULT);
datum = SysCacheGetAttrNotNull(COLLOID, colltp,
provider == COLLPROVIDER_ICU ?
- Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
locale = TextDatumGetCString(datum);
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 0a97a11314..8d327243b3 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -118,7 +118,7 @@ static bool get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, bool *dbHasLoginEvtP,
TransactionId *dbFrozenXidP, MultiXactId *dbMinMultiP,
- Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbIculocale,
+ Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbLocale,
char **dbIcurules,
char *dbLocProvider,
char **dbCollversion);
@@ -675,7 +675,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
int src_encoding = -1;
char *src_collate = NULL;
char *src_ctype = NULL;
- char *src_iculocale = NULL;
+ char *src_locale = NULL;
char *src_icurules = NULL;
char src_locprovider = '\0';
char *src_collversion = NULL;
@@ -713,7 +713,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
const char *dbtemplate = NULL;
char *dbcollate = NULL;
char *dbctype = NULL;
- char *dbiculocale = NULL;
+ char *dblocale = NULL;
char *dbicurules = NULL;
char dblocprovider = '\0';
char *canonname;
@@ -903,7 +903,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
if (dctype && dctype->arg)
dbctype = defGetString(dctype);
if (diculocale && diculocale->arg)
- dbiculocale = defGetString(diculocale);
+ dblocale = defGetString(diculocale);
if (dicurules && dicurules->arg)
dbicurules = defGetString(dicurules);
if (dlocprovider && dlocprovider->arg)
@@ -971,7 +971,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
&src_dboid, &src_owner, &src_encoding,
&src_istemplate, &src_allowconn, &src_hasloginevt,
&src_frozenxid, &src_minmxid, &src_deftablespace,
- &src_collate, &src_ctype, &src_iculocale, &src_icurules, &src_locprovider,
+ &src_collate, &src_ctype, &src_locale, &src_icurules, &src_locprovider,
&src_collversion))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_DATABASE),
@@ -1027,12 +1027,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
dbctype = src_ctype;
if (dblocprovider == '\0')
dblocprovider = src_locprovider;
- if (dbiculocale == NULL && dblocprovider == COLLPROVIDER_ICU)
+ if (dblocale == NULL && dblocprovider == COLLPROVIDER_ICU)
{
if (dlocale && dlocale->arg)
- dbiculocale = defGetString(dlocale);
+ dblocale = defGetString(dlocale);
else
- dbiculocale = src_iculocale;
+ dblocale = src_locale;
}
if (dbicurules == NULL && dblocprovider == COLLPROVIDER_ICU)
dbicurules = src_icurules;
@@ -1071,7 +1071,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* This would happen if template0 uses the libc provider but the new
* database uses icu.
*/
- if (!dbiculocale)
+ if (!dblocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("LOCALE or ICU_LOCALE must be specified")));
@@ -1081,26 +1081,26 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* database, preserve locale string. Otherwise, canonicalize to a
* language tag.
*/
- if (!IsBinaryUpgrade && dbiculocale != src_iculocale)
+ if (!IsBinaryUpgrade && dblocale != src_locale)
{
- char *langtag = icu_language_tag(dbiculocale,
+ char *langtag = icu_language_tag(dblocale,
icu_validation_level);
- if (langtag && strcmp(dbiculocale, langtag) != 0)
+ if (langtag && strcmp(dblocale, langtag) != 0)
{
ereport(NOTICE,
(errmsg("using standard form \"%s\" for ICU locale \"%s\"",
- langtag, dbiculocale)));
+ langtag, dblocale)));
- dbiculocale = langtag;
+ dblocale = langtag;
}
}
- icu_validate_locale(dbiculocale);
+ icu_validate_locale(dblocale);
}
else
{
- if (dbiculocale)
+ if (dblocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU locale cannot be specified unless locale provider is ICU")));
@@ -1157,13 +1157,13 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
char *val1;
char *val2;
- Assert(dbiculocale);
- Assert(src_iculocale);
- if (strcmp(dbiculocale, src_iculocale) != 0)
+ Assert(dblocale);
+ Assert(src_locale);
+ if (strcmp(dblocale, src_locale) != 0)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("new ICU locale (%s) is incompatible with the ICU locale of the template database (%s)",
- dbiculocale, src_iculocale),
+ dblocale, src_locale),
errhint("Use the same ICU locale as in the template database, or use template0 as template.")));
val1 = dbicurules;
@@ -1197,7 +1197,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
char *actual_versionstr;
- actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dbiculocale : dbcollate);
+ actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
if (!actual_versionstr)
ereport(ERROR,
(errmsg("template database \"%s\" has a collation version, but no actual collation version could be determined",
@@ -1225,7 +1225,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* collation version, which is normally only the case for template0.
*/
if (dbcollversion == NULL)
- dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dbiculocale : dbcollate);
+ dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
/* Resolve default tablespace for new database */
if (dtablespacename && dtablespacename->arg)
@@ -1364,8 +1364,8 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* block on the unique index, and fail after we commit).
*/
- Assert((dblocprovider == COLLPROVIDER_ICU && dbiculocale) ||
- (dblocprovider != COLLPROVIDER_ICU && !dbiculocale));
+ Assert((dblocprovider == COLLPROVIDER_ICU && dblocale) ||
+ (dblocprovider != COLLPROVIDER_ICU && !dblocale));
/* Form tuple */
new_record[Anum_pg_database_oid - 1] = ObjectIdGetDatum(dboid);
@@ -1383,10 +1383,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
new_record[Anum_pg_database_dattablespace - 1] = ObjectIdGetDatum(dst_deftablespace);
new_record[Anum_pg_database_datcollate - 1] = CStringGetTextDatum(dbcollate);
new_record[Anum_pg_database_datctype - 1] = CStringGetTextDatum(dbctype);
- if (dbiculocale)
- new_record[Anum_pg_database_daticulocale - 1] = CStringGetTextDatum(dbiculocale);
+ if (dblocale)
+ new_record[Anum_pg_database_datlocale - 1] = CStringGetTextDatum(dblocale);
else
- new_record_nulls[Anum_pg_database_daticulocale - 1] = true;
+ new_record_nulls[Anum_pg_database_datlocale - 1] = true;
if (dbicurules)
new_record[Anum_pg_database_daticurules - 1] = CStringGetTextDatum(dbicurules);
else
@@ -2472,7 +2472,7 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
datum = heap_getattr(tuple, Anum_pg_database_datcollversion, RelationGetDescr(rel), &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_daticulocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
+ datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
if (isnull)
elog(ERROR, "unexpected null in pg_database");
newversion = get_collation_actual_version(datForm->datlocprovider, TextDatumGetCString(datum));
@@ -2670,7 +2670,7 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
datlocprovider = ((Form_pg_database) GETSTRUCT(tp))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_daticulocale : Anum_pg_database_datcollate);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate);
version = get_collation_actual_version(datlocprovider, TextDatumGetCString(datum));
ReleaseSysCache(tp);
@@ -2697,7 +2697,7 @@ get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, bool *dbHasLoginEvtP,
TransactionId *dbFrozenXidP, MultiXactId *dbMinMultiP,
- Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbIculocale,
+ Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbLocale,
char **dbIcurules,
char *dbLocProvider,
char **dbCollversion)
@@ -2808,13 +2808,13 @@ get_db_info(const char *name, LOCKMODE lockmode,
datum = SysCacheGetAttrNotNull(DATABASEOID, tuple, Anum_pg_database_datctype);
*dbCtype = TextDatumGetCString(datum);
}
- if (dbIculocale)
+ if (dbLocale)
{
- datum = SysCacheGetAttr(DATABASEOID, tuple, Anum_pg_database_daticulocale, &isnull);
+ datum = SysCacheGetAttr(DATABASEOID, tuple, Anum_pg_database_datlocale, &isnull);
if (isnull)
- *dbIculocale = NULL;
+ *dbLocale = NULL;
else
- *dbIculocale = TextDatumGetCString(datum);
+ *dbLocale = TextDatumGetCString(datum);
}
if (dbIcurules)
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 79b59b0af7..45fe847320 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1606,7 +1606,7 @@ pg_newlocale_from_collation(Oid collid)
const char *iculocstr;
const char *icurules;
- datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colliculocale);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
iculocstr = TextDatumGetCString(datum);
datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collicurules, &isnull);
@@ -1627,7 +1627,7 @@ pg_newlocale_from_collation(Oid collid)
collversionstr = TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
actual_versionstr = get_collation_actual_version(collform->collprovider,
TextDatumGetCString(datum));
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1ad3367159..6dbee5556a 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -427,7 +427,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
{
char *icurules;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_daticulocale);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
iculocale = TextDatumGetCString(datum);
datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_daticurules, &isnull);
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index ac409b0006..8f4b456995 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -145,7 +145,8 @@ static char *lc_numeric = NULL;
static char *lc_time = NULL;
static char *lc_messages = NULL;
static char locale_provider = COLLPROVIDER_LIBC;
-static char *icu_locale = NULL;
+static char *datlocale = NULL;
+static bool icu_locale_specified = false;
static char *icu_rules = NULL;
static const char *default_text_search_config = NULL;
static char *username = NULL;
@@ -1515,8 +1516,8 @@ bootstrap_template1(void)
bki_lines = replace_token(bki_lines, "LC_CTYPE",
escape_quotes_bki(lc_ctype));
- bki_lines = replace_token(bki_lines, "ICU_LOCALE",
- icu_locale ? escape_quotes_bki(icu_locale) : "_null_");
+ bki_lines = replace_token(bki_lines, "DATLOCALE",
+ datlocale ? escape_quotes_bki(datlocale) : "_null_");
bki_lines = replace_token(bki_lines, "ICU_RULES",
icu_rules ? escape_quotes_bki(icu_rules) : "_null_");
@@ -2363,8 +2364,8 @@ setlocales(void)
lc_monetary = locale;
if (!lc_messages)
lc_messages = locale;
- if (!icu_locale && locale_provider == COLLPROVIDER_ICU)
- icu_locale = locale;
+ if (!datlocale && locale_provider != COLLPROVIDER_LIBC)
+ datlocale = locale;
}
/*
@@ -2390,22 +2391,21 @@ setlocales(void)
lc_messages = canonname;
#endif
+ if (locale_provider != COLLPROVIDER_LIBC && datlocale == NULL)
+ pg_fatal("locale must be specified unless provider is libc");
+
if (locale_provider == COLLPROVIDER_ICU)
{
char *langtag;
- /* acquire default locale from the environment, if not specified */
- if (icu_locale == NULL)
- pg_fatal("ICU locale must be specified");
-
/* canonicalize to a language tag */
- langtag = icu_language_tag(icu_locale);
+ langtag = icu_language_tag(datlocale);
printf(_("Using language tag \"%s\" for ICU locale \"%s\".\n"),
- langtag, icu_locale);
- pg_free(icu_locale);
- icu_locale = langtag;
+ langtag, datlocale);
+ pg_free(datlocale);
+ datlocale = langtag;
- icu_validate_locale(icu_locale);
+ icu_validate_locale(datlocale);
/*
* In supported builds, the ICU locale ID will be opened during
@@ -2599,14 +2599,14 @@ setup_locale_encoding(void)
strcmp(lc_ctype, lc_numeric) == 0 &&
strcmp(lc_ctype, lc_monetary) == 0 &&
strcmp(lc_ctype, lc_messages) == 0 &&
- (!icu_locale || strcmp(lc_ctype, icu_locale) == 0))
+ (!datlocale || strcmp(lc_ctype, datlocale) == 0))
printf(_("The database cluster will be initialized with locale \"%s\".\n"), lc_ctype);
else
{
printf(_("The database cluster will be initialized with this locale configuration:\n"));
- printf(_(" provider: %s\n"), collprovider_name(locale_provider));
- if (icu_locale)
- printf(_(" ICU locale: %s\n"), icu_locale);
+ printf(_(" default collation provider: %s\n"), collprovider_name(locale_provider));
+ if (locale_provider != COLLPROVIDER_LIBC)
+ printf(_(" default collation locale: %s\n"), datlocale);
printf(_(" LC_COLLATE: %s\n"
" LC_CTYPE: %s\n"
" LC_MESSAGES: %s\n"
@@ -3277,7 +3277,8 @@ main(int argc, char *argv[])
pg_fatal("unrecognized locale provider: %s", optarg);
break;
case 16:
- icu_locale = pg_strdup(optarg);
+ datlocale = pg_strdup(optarg);
+ icu_locale_specified = true;
break;
case 17:
icu_rules = pg_strdup(optarg);
@@ -3312,7 +3313,7 @@ main(int argc, char *argv[])
exit(1);
}
- if (icu_locale && locale_provider != COLLPROVIDER_ICU)
+ if (icu_locale_specified && locale_provider != COLLPROVIDER_ICU)
pg_fatal("%s cannot be specified unless locale provider \"%s\" is chosen",
"--icu-locale", "icu");
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index 03376cc0f7..eb17c0e8f9 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -117,7 +117,7 @@ if ($ENV{with_icu} eq 'yes')
{
command_fails_like(
[ 'initdb', '--no-sync', '--locale-provider=icu', "$tempdir/data2" ],
- qr/initdb: error: ICU locale must be specified/,
+ qr/initdb: error: locale must be specified unless provider is libc/,
'locale provider ICU requires --icu-locale');
command_ok(
@@ -138,7 +138,7 @@ if ($ENV{with_icu} eq 'yes')
'--lc-monetary=C', '--lc-time=C',
"$tempdir/data4"
],
- qr/^\s+ICU locale:\s+und\n/ms,
+ qr/^\s+default collation locale:\s+und\n/ms,
'options --locale-provider=icu --locale=und --lc-*=C');
command_fails_like(
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 22d1e6cf92..939ab62ff9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -2982,7 +2982,7 @@ dumpDatabase(Archive *fout)
i_datlocprovider,
i_collate,
i_ctype,
- i_daticulocale,
+ i_datlocale,
i_daticurules,
i_frozenxid,
i_minmxid,
@@ -3001,7 +3001,7 @@ dumpDatabase(Archive *fout)
*datlocprovider,
*collate,
*ctype,
- *iculocale,
+ *locale,
*icurules,
*datistemplate,
*datconnlimit,
@@ -3025,10 +3025,12 @@ dumpDatabase(Archive *fout)
appendPQExpBufferStr(dbQry, "datminmxid, ");
else
appendPQExpBufferStr(dbQry, "0 AS datminmxid, ");
- if (fout->remoteVersion >= 150000)
- appendPQExpBufferStr(dbQry, "datlocprovider, daticulocale, datcollversion, ");
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(dbQry, "datlocprovider, datlocale, datcollversion, ");
+ else if (fout->remoteVersion >= 150000)
+ appendPQExpBufferStr(dbQry, "datlocprovider, daticulocale AS datlocale, datcollversion, ");
else
- appendPQExpBufferStr(dbQry, "'c' AS datlocprovider, NULL AS daticulocale, NULL AS datcollversion, ");
+ appendPQExpBufferStr(dbQry, "'c' AS datlocprovider, NULL AS datlocale, NULL AS datcollversion, ");
if (fout->remoteVersion >= 160000)
appendPQExpBufferStr(dbQry, "daticurules, ");
else
@@ -3049,7 +3051,7 @@ dumpDatabase(Archive *fout)
i_datlocprovider = PQfnumber(res, "datlocprovider");
i_collate = PQfnumber(res, "datcollate");
i_ctype = PQfnumber(res, "datctype");
- i_daticulocale = PQfnumber(res, "daticulocale");
+ i_datlocale = PQfnumber(res, "datlocale");
i_daticurules = PQfnumber(res, "daticurules");
i_frozenxid = PQfnumber(res, "datfrozenxid");
i_minmxid = PQfnumber(res, "datminmxid");
@@ -3068,10 +3070,10 @@ dumpDatabase(Archive *fout)
datlocprovider = PQgetvalue(res, 0, i_datlocprovider);
collate = PQgetvalue(res, 0, i_collate);
ctype = PQgetvalue(res, 0, i_ctype);
- if (!PQgetisnull(res, 0, i_daticulocale))
- iculocale = PQgetvalue(res, 0, i_daticulocale);
+ if (!PQgetisnull(res, 0, i_datlocale))
+ locale = PQgetvalue(res, 0, i_datlocale);
else
- iculocale = NULL;
+ locale = NULL;
if (!PQgetisnull(res, 0, i_daticurules))
icurules = PQgetvalue(res, 0, i_daticurules);
else
@@ -3118,29 +3120,36 @@ dumpDatabase(Archive *fout)
pg_fatal("unrecognized locale provider: %s",
datlocprovider);
- if (strlen(collate) > 0 && strcmp(collate, ctype) == 0)
+ if (!locale && datlocprovider[0] != 'c')
+ pg_log_warning("database '%s' with provider '%s' missing datlocale",
+ datname, datlocprovider);
+
+ if (locale && datlocprovider[0] == 'c')
+ pg_log_warning("database '%s' with provider 'c' has non-NULL locale '%s'",
+ datname, locale);
+
+ /* if collate and ctype are equal, and locale is NULL, use LOCALE */
+ if (!locale && strlen(collate) > 0 && strcmp(collate, ctype) == 0)
+ locale = collate;
+
+ /* output LC_COLLATE and LC_CTYPE if different from LOCALE */
+ if (strlen(collate) > 0 && (!locale || strcmp(collate, locale) != 0))
{
- appendPQExpBufferStr(creaQry, " LOCALE = ");
+ appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
appendStringLiteralAH(creaQry, collate, fout);
}
- else
+ if (strlen(ctype) > 0 && (!locale || strcmp(ctype, locale) != 0))
{
- if (strlen(collate) > 0)
- {
- appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
- appendStringLiteralAH(creaQry, collate, fout);
- }
- if (strlen(ctype) > 0)
- {
- appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
- appendStringLiteralAH(creaQry, ctype, fout);
- }
+ appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
+ appendStringLiteralAH(creaQry, ctype, fout);
}
- if (iculocale)
+
+ if (locale)
{
- appendPQExpBufferStr(creaQry, " ICU_LOCALE = ");
- appendStringLiteralAH(creaQry, iculocale, fout);
+ appendPQExpBufferStr(creaQry, " LOCALE = ");
+ appendStringLiteralAH(creaQry, locale, fout);
}
+
if (icurules)
{
appendPQExpBufferStr(creaQry, " ICU_RULES = ");
@@ -13798,12 +13807,12 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
int i_collisdeterministic;
int i_collcollate;
int i_collctype;
- int i_colliculocale;
+ int i_colllocale;
int i_collicurules;
const char *collprovider;
const char *collcollate;
const char *collctype;
- const char *colliculocale;
+ const char *colllocale;
const char *collicurules;
/* Do nothing in data-only dump */
@@ -13835,12 +13844,15 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
appendPQExpBufferStr(query,
"true AS collisdeterministic, ");
- if (fout->remoteVersion >= 150000)
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(query,
+ "colllocale, ");
+ else if (fout->remoteVersion >= 150000)
appendPQExpBufferStr(query,
- "colliculocale, ");
+ "colliculocale AS colllocale, ");
else
appendPQExpBufferStr(query,
- "NULL AS colliculocale, ");
+ "NULL AS colllocale, ");
if (fout->remoteVersion >= 160000)
appendPQExpBufferStr(query,
@@ -13862,7 +13874,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
i_collisdeterministic = PQfnumber(res, "collisdeterministic");
i_collcollate = PQfnumber(res, "collcollate");
i_collctype = PQfnumber(res, "collctype");
- i_colliculocale = PQfnumber(res, "colliculocale");
+ i_colllocale = PQfnumber(res, "colllocale");
i_collicurules = PQfnumber(res, "collicurules");
collprovider = PQgetvalue(res, 0, i_collprovider);
@@ -13889,10 +13901,10 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
collctype = NULL;
}
- if (!PQgetisnull(res, 0, i_colliculocale))
- colliculocale = PQgetvalue(res, 0, i_colliculocale);
+ if (!PQgetisnull(res, 0, i_colllocale))
+ colllocale = PQgetvalue(res, 0, i_colllocale);
else
- colliculocale = NULL;
+ colllocale = NULL;
if (!PQgetisnull(res, 0, i_collicurules))
collicurules = PQgetvalue(res, 0, i_collicurules);
@@ -13922,7 +13934,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
if (collprovider[0] == 'd')
{
- if (collcollate || collctype || colliculocale || collicurules)
+ if (collcollate || collctype || colllocale || collicurules)
pg_log_warning("invalid collation \"%s\"", qcollname);
/* no locale -- the default collation cannot be reloaded anyway */
@@ -13931,16 +13943,16 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
{
if (fout->remoteVersion >= 150000)
{
- if (collcollate || collctype || !colliculocale)
+ if (collcollate || collctype || !colllocale)
pg_log_warning("invalid collation \"%s\"", qcollname);
appendPQExpBufferStr(q, ", locale = ");
- appendStringLiteralAH(q, colliculocale ? colliculocale : "",
+ appendStringLiteralAH(q, colllocale ? colllocale : "",
fout);
}
else
{
- if (!collcollate || !collctype || colliculocale ||
+ if (!collcollate || !collctype || colllocale ||
strcmp(collcollate, collctype) != 0)
pg_log_warning("invalid collation \"%s\"", qcollname);
@@ -13956,7 +13968,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
}
else if (collprovider[0] == 'c')
{
- if (colliculocale || collicurules || !collcollate || !collctype)
+ if (colllocale || collicurules || !collcollate || !collctype)
pg_log_warning("invalid collation \"%s\"", qcollname);
if (collcollate && collctype && strcmp(collcollate, collctype) == 0)
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 74e02b3f82..0a6e763b71 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -328,18 +328,24 @@ get_template0_info(ClusterInfo *cluster)
int i_datlocprovider;
int i_datcollate;
int i_datctype;
- int i_daticulocale;
+ int i_datlocale;
- if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
+ if (GET_MAJOR_VERSION(cluster->major_version) >= 1700)
dbres = executeQueryOrDie(conn,
"SELECT encoding, datlocprovider, "
- " datcollate, datctype, daticulocale "
+ " datcollate, datctype, datlocale "
+ "FROM pg_catalog.pg_database "
+ "WHERE datname='template0'");
+ else if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
+ dbres = executeQueryOrDie(conn,
+ "SELECT encoding, datlocprovider, "
+ " datcollate, datctype, daticulocale AS datlocale"
"FROM pg_catalog.pg_database "
"WHERE datname='template0'");
else
dbres = executeQueryOrDie(conn,
"SELECT encoding, 'c' AS datlocprovider, "
- " datcollate, datctype, NULL AS daticulocale "
+ " datcollate, datctype, NULL AS datlocale "
"FROM pg_catalog.pg_database "
"WHERE datname='template0'");
@@ -353,16 +359,16 @@ get_template0_info(ClusterInfo *cluster)
i_datlocprovider = PQfnumber(dbres, "datlocprovider");
i_datcollate = PQfnumber(dbres, "datcollate");
i_datctype = PQfnumber(dbres, "datctype");
- i_daticulocale = PQfnumber(dbres, "daticulocale");
+ i_datlocale = PQfnumber(dbres, "datlocale");
locale->db_encoding = atoi(PQgetvalue(dbres, 0, i_datencoding));
locale->db_collprovider = PQgetvalue(dbres, 0, i_datlocprovider)[0];
locale->db_collate = pg_strdup(PQgetvalue(dbres, 0, i_datcollate));
locale->db_ctype = pg_strdup(PQgetvalue(dbres, 0, i_datctype));
- if (PQgetisnull(dbres, 0, i_daticulocale))
- locale->db_iculocale = NULL;
+ if (PQgetisnull(dbres, 0, i_datlocale))
+ locale->db_locale = NULL;
else
- locale->db_iculocale = pg_strdup(PQgetvalue(dbres, 0, i_daticulocale));
+ locale->db_locale = pg_strdup(PQgetvalue(dbres, 0, i_datlocale));
cluster->template0 = locale;
@@ -392,12 +398,15 @@ get_db_infos(ClusterInfo *cluster)
snprintf(query, sizeof(query),
"SELECT d.oid, d.datname, d.encoding, d.datcollate, d.datctype, ");
- if (GET_MAJOR_VERSION(cluster->major_version) < 1500)
+ if (GET_MAJOR_VERSION(cluster->major_version) >= 1700)
+ snprintf(query + strlen(query), sizeof(query) - strlen(query),
+ "datlocprovider, datlocale, ");
+ else if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
snprintf(query + strlen(query), sizeof(query) - strlen(query),
- "'c' AS datlocprovider, NULL AS daticulocale, ");
+ "datlocprovider, daticulocale AS datlocale, ");
else
snprintf(query + strlen(query), sizeof(query) - strlen(query),
- "datlocprovider, daticulocale, ");
+ "'c' AS datlocprovider, NULL AS datlocale, ");
snprintf(query + strlen(query), sizeof(query) - strlen(query),
"pg_catalog.pg_tablespace_location(t.oid) AS spclocation "
"FROM pg_catalog.pg_database d "
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 14a36f0503..fdea0c45a4 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -391,7 +391,7 @@ setup(char *argv0, bool *live_check)
* Copy locale and encoding information into the new cluster's template0.
*
* We need to copy the encoding, datlocprovider, datcollate, datctype, and
- * daticulocale. We don't need datcollversion because that's never set for
+ * datlocale. We don't need datcollversion because that's never set for
* template0.
*/
static void
@@ -400,7 +400,7 @@ set_locale_and_encoding(void)
PGconn *conn_new_template1;
char *datcollate_literal;
char *datctype_literal;
- char *daticulocale_literal = NULL;
+ char *datlocale_literal = NULL;
DbLocaleInfo *locale = old_cluster.template0;
prep_status("Setting locale and encoding for new cluster");
@@ -414,15 +414,29 @@ set_locale_and_encoding(void)
datctype_literal = PQescapeLiteral(conn_new_template1,
locale->db_ctype,
strlen(locale->db_ctype));
- if (locale->db_iculocale)
- daticulocale_literal = PQescapeLiteral(conn_new_template1,
- locale->db_iculocale,
- strlen(locale->db_iculocale));
+ if (locale->db_locale)
+ datlocale_literal = PQescapeLiteral(conn_new_template1,
+ locale->db_locale,
+ strlen(locale->db_locale));
else
- daticulocale_literal = pg_strdup("NULL");
+ datlocale_literal = pg_strdup("NULL");
/* update template0 in new cluster */
- if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1500)
+ if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1700)
+ PQclear(executeQueryOrDie(conn_new_template1,
+ "UPDATE pg_catalog.pg_database "
+ " SET encoding = %d, "
+ " datlocprovider = '%c', "
+ " datcollate = %s, "
+ " datctype = %s, "
+ " datlocale = %s "
+ " WHERE datname = 'template0' ",
+ locale->db_encoding,
+ locale->db_collprovider,
+ datcollate_literal,
+ datctype_literal,
+ datlocale_literal));
+ else if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1500)
PQclear(executeQueryOrDie(conn_new_template1,
"UPDATE pg_catalog.pg_database "
" SET encoding = %d, "
@@ -435,7 +449,7 @@ set_locale_and_encoding(void)
locale->db_collprovider,
datcollate_literal,
datctype_literal,
- daticulocale_literal));
+ datlocale_literal));
else
PQclear(executeQueryOrDie(conn_new_template1,
"UPDATE pg_catalog.pg_database "
@@ -449,7 +463,7 @@ set_locale_and_encoding(void)
PQfreemem(datcollate_literal);
PQfreemem(datctype_literal);
- PQfreemem(daticulocale_literal);
+ PQfreemem(datlocale_literal);
PQfinish(conn_new_template1);
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index a1d08c3dab..bc40b88782 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -206,7 +206,7 @@ typedef struct
char *db_collate;
char *db_ctype;
char db_collprovider;
- char *db_iculocale;
+ char *db_locale;
int db_encoding;
} DbLocaleInfo;
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index d951ed3af0..ec7b244d4f 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -104,6 +104,8 @@ if ($oldnode->pg_version >= 11)
push @custom_opts, '--allow-group-access';
}
+my $oldversion = int($oldnode->pg_version =~ s/([0-9]*).*/$1/rg);
+
# Set up the locale settings for the original cluster, so that we
# can test that pg_upgrade copies the locale settings of template0
# from the old to the new cluster.
@@ -113,11 +115,18 @@ my $original_provider = "c";
my $original_locale = "C";
my $original_iculocale = "";
my $provider_field = "'c' AS datlocprovider";
-my $iculocale_field = "NULL AS daticulocale";
-if ($oldnode->pg_version >= 15 && $ENV{with_icu} eq 'yes')
+my $iculocale_field = "NULL AS datlocale";
+if ($oldversion >= 15 && $ENV{with_icu} eq 'yes')
{
$provider_field = "datlocprovider";
- $iculocale_field = "daticulocale";
+ if ($oldversion >= 17)
+ {
+ $iculocale_field = "datlocale";
+ }
+ else
+ {
+ $iculocale_field = "daticulocale AS datlocale";
+ }
$original_provider = "i";
$original_iculocale = "fr-CA";
}
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 37f9516320..254fce8ff1 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -937,14 +937,18 @@ listAllDbs(const char *pattern, bool verbose)
" d.datctype as \"%s\",\n",
gettext_noop("Collate"),
gettext_noop("Ctype"));
- if (pset.sversion >= 150000)
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ " d.datlocale as \"%s\",\n",
+ gettext_noop("Locale"));
+ else if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
" d.daticulocale as \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
else
appendPQExpBuffer(&buf,
" NULL as \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
" d.daticurules as \"%s\",\n",
@@ -4975,14 +4979,18 @@ listCollations(const char *pattern, bool verbose, bool showSystem)
gettext_noop("Collate"),
gettext_noop("Ctype"));
- if (pset.sversion >= 150000)
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ " c.colllocale AS \"%s\",\n",
+ gettext_noop("Locale"));
+ else if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
" c.colliculocale AS \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
else
appendPQExpBuffer(&buf,
" c.collcollate AS \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 10c363d2ee..7396ff10c4 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -29,6 +29,6 @@
{ oid => '963',
descr => 'sorts using the Unicode Collation Algorithm with default settings',
collname => 'unicode', collprovider => 'i', collencoding => '-1',
- colliculocale => 'und' },
+ colllocale => 'und' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 744e64c777..1ee2d264a1 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -42,7 +42,7 @@ CATALOG(pg_collation,3456,CollationRelationId)
#ifdef CATALOG_VARLEN /* variable-length fields start here */
text collcollate BKI_DEFAULT(_null_); /* LC_COLLATE setting */
text collctype BKI_DEFAULT(_null_); /* LC_CTYPE setting */
- text colliculocale BKI_DEFAULT(_null_); /* ICU locale ID */
+ text colllocale BKI_DEFAULT(_null_); /* locale ID */
text collicurules BKI_DEFAULT(_null_); /* ICU collation rules */
text collversion BKI_DEFAULT(_null_); /* provider-dependent
* version of collation
@@ -91,7 +91,7 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *colliculocale,
+ const char *colllocale,
const char *collicurules,
const char *collversion,
bool if_not_exists,
diff --git a/src/include/catalog/pg_database.dat b/src/include/catalog/pg_database.dat
index 4306e8a3e8..c2ba636f8d 100644
--- a/src/include/catalog/pg_database.dat
+++ b/src/include/catalog/pg_database.dat
@@ -18,7 +18,7 @@
datlocprovider => 'LOCALE_PROVIDER', datistemplate => 't',
datallowconn => 't', dathasloginevt => 'f', datconnlimit => '-1', datfrozenxid => '0',
datminmxid => '1', dattablespace => 'pg_default', datcollate => 'LC_COLLATE',
- datctype => 'LC_CTYPE', daticulocale => 'ICU_LOCALE',
+ datctype => 'LC_CTYPE', datlocale => 'DATLOCALE',
daticurules => 'ICU_RULES', datacl => '_null_' },
]
diff --git a/src/include/catalog/pg_database.h b/src/include/catalog/pg_database.h
index f0ed776b60..65a5635275 100644
--- a/src/include/catalog/pg_database.h
+++ b/src/include/catalog/pg_database.h
@@ -75,7 +75,7 @@ CATALOG(pg_database,1262,DatabaseRelationId) BKI_SHARED_RELATION BKI_ROWTYPE_OID
text datctype BKI_FORCE_NOT_NULL;
/* ICU locale ID */
- text daticulocale;
+ text datlocale;
/* ICU collation rules */
text daticurules;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 7a05c75967..8ca93f4dea 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1024,7 +1024,7 @@ SET icu_validation_level = disabled;
do $$
BEGIN
EXECUTE 'CREATE COLLATION test0 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
CREATE COLLATION test0 FROM "C"; -- fail, duplicate name
@@ -1032,7 +1032,7 @@ ERROR: collation "test0" already exists
do $$
BEGIN
EXECUTE 'CREATE COLLATION test1 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
RESET icu_validation_level;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 4f3fd46420..2235c24729 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6220,9 +6220,9 @@ List of schemas
(0 rows)
\dO "no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp "no.such.access.privilege"
@@ -6409,9 +6409,9 @@ cross-database references are not implemented: "no.such.schema"."no.such.languag
(0 rows)
\dO "no.such.schema"."no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp "no.such.schema"."no.such.access.privilege"
@@ -6552,9 +6552,9 @@ List of text search templates
(0 rows)
\dO regression."no.such.schema"."no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp regression."no.such.schema"."no.such.access.privilege"
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 3db9e25913..03837de846 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -363,14 +363,14 @@ SET icu_validation_level = disabled;
do $$
BEGIN
EXECUTE 'CREATE COLLATION test0 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
CREATE COLLATION test0 FROM "C"; -- fail, duplicate name
do $$
BEGIN
EXECUTE 'CREATE COLLATION test1 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
--
2.34.1
[text/x-patch] v16-0005-Introduce-collation-provider-builtin-for-C-and-C.patch (75.6K, ../../[email protected]/5-v16-0005-Introduce-collation-provider-builtin-for-C-and-C.patch)
download | inline diff:
From 8d35489935406b9e1d4e0b09d6599644cba3ad23 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Mon, 1 May 2023 15:38:29 -0700
Subject: [PATCH v16 5/5] Introduce collation provider "builtin" for "C" and
"C.UTF-8".
The builtin "C" locale is equal (in semantics and implementation) to
the libc "C" locale.
The builtin "C.UTF-8" locale is especially useful. It provides a fast
memcmp-based collation (like "C") that supports abbrevated keys, while
also providing richer ctype semantics (upper/lower and regexes). The
semantics are derived from Unicode by building in lookup tables in the
same way as for text normalization. By using built-in semantics, the
behavior is stabilized within a Postgres major version, and also
matches the behavior of other built-in Unicode functionality, such as
normalization.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
---
doc/src/sgml/charset.sgml | 88 +++++++--
doc/src/sgml/ref/create_collation.sgml | 11 +-
doc/src/sgml/ref/create_database.sgml | 8 +-
doc/src/sgml/ref/createdb.sgml | 2 +-
doc/src/sgml/ref/initdb.sgml | 17 +-
src/backend/catalog/pg_collation.c | 5 +-
src/backend/commands/collationcmds.c | 93 ++++++++--
src/backend/commands/dbcommands.c | 123 +++++++++---
src/backend/regex/regc_pg_locale.c | 41 +++-
src/backend/utils/adt/formatting.c | 185 +++++++++++++++++++
src/backend/utils/adt/pg_locale.c | 130 +++++++++++--
src/backend/utils/init/postinit.c | 27 ++-
src/bin/initdb/initdb.c | 31 +++-
src/bin/initdb/t/001_initdb.pl | 55 ++++++
src/bin/pg_dump/pg_dump.c | 15 +-
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 74 ++++++--
src/bin/psql/describe.c | 4 +-
src/bin/scripts/createdb.c | 18 +-
src/bin/scripts/t/020_createdb.pl | 72 ++++++++
src/common/wchar.c | 4 +-
src/include/catalog/pg_collation.dat | 9 +-
src/include/catalog/pg_collation.h | 3 +
src/include/mb/pg_wchar.h | 16 ++
src/include/utils/pg_locale.h | 7 +-
src/test/icu/t/010_database.pl | 18 +-
src/test/regress/expected/collate.out | 24 ++-
src/test/regress/expected/collate.utf8.out | 109 +++++++++++
src/test/regress/expected/collate.utf8_1.out | 8 +
src/test/regress/parallel_schedule | 4 +-
src/test/regress/sql/collate.sql | 10 +
src/test/regress/sql/collate.utf8.sql | 54 ++++++
31 files changed, 1124 insertions(+), 141 deletions(-)
create mode 100644 src/test/regress/expected/collate.utf8.out
create mode 100644 src/test/regress/expected/collate.utf8_1.out
create mode 100644 src/test/regress/sql/collate.utf8.sql
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 74783d148f..1553deea20 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -342,22 +342,14 @@ initdb --locale=sv_SE
<title>Locale Providers</title>
<para>
- <productname>PostgreSQL</productname> supports multiple <firstterm>locale
- providers</firstterm>. This specifies which library supplies the locale
- data. One standard provider name is <literal>libc</literal>, which uses
- the locales provided by the operating system C library. These are the
- locales used by most tools provided by the operating system. Another
- provider is <literal>icu</literal>, which uses the external
- ICU<indexterm><primary>ICU</primary></indexterm> library. ICU locales can
- only be used if support for ICU was configured when PostgreSQL was built.
+ A locale provider specifies which library defines the locale behavior for
+ collations and character classifications.
</para>
<para>
The commands and tools that select the locale settings, as described
- above, each have an option to select the locale provider. The examples
- shown earlier all use the <literal>libc</literal> provider, which is the
- default. Here is an example to initialize a database cluster using the
- ICU provider:
+ above, each have an option to select the locale provider. Here is an
+ example to initialize a database cluster using the ICU provider:
<programlisting>
initdb --locale-provider=icu --icu-locale=en
</programlisting>
@@ -370,12 +362,74 @@ initdb --locale-provider=icu --icu-locale=en
</para>
<para>
- Which locale provider to use depends on individual requirements. For most
- basic uses, either provider will give adequate results. For the libc
- provider, it depends on what the operating system offers; some operating
- systems are better than others. For advanced uses, ICU offers more locale
- variants and customization options.
+ Regardless of the locale provider, the operating system is still used to
+ provide some locale-aware behavior, such as messages (see <xref
+ linkend="guc-lc-messages"/>).
</para>
+
+ <para>
+ The available locale providers are listed below.
+ </para>
+
+ <sect3 id="locale-provider-builtin">
+ <title>Builtin</title>
+ <para>
+ The <literal>builtin</literal> provider uses built-in operations. Only
+ the <literal>C</literal> and <literal>C.UTF-8</literal> locales are
+ supported for this provider.
+ </para>
+ <para>
+ The <literal>C</literal> locale behavior is identical to the
+ <literal>C</literal> locale in the libc provider. When using this locale,
+ the behavior may depend on the database encoding.
+ </para>
+ <para>
+ The <literal>C.UTF-8</literal> locale is available only for when the
+ database encoding is <literal>UTF-8</literal>, and the behavior is based
+ on Unicode. The collation uses the code point values only. The regular
+ expression character classes are based on the "POSIX Compatible"
+ semantics, and the case mapping is the "simple" variant.
+ </para>
+ </sect3>
+ <sect3 id="locale-provider-icu">
+ <title>ICU</title>
+ <para>
+ The <literal>icu</literal> provider uses the external
+ ICU<indexterm><primary>ICU</primary></indexterm>
+ library. <productname>PostgreSQL</productname> must have been configured
+ with support.
+ </para>
+ <para>
+ ICU provides collation and character classification behavior that is
+ independent of the operating system and database encoding, which is
+ preferable if you expect to transition to other platforms without any
+ change in results. <literal>LC_COLLATE</literal> and
+ <literal>LC_CTYPE</literal> can be set independently of the ICU locale.
+ </para>
+ <note>
+ <para>
+ For the ICU provider, results may depend on the version of the ICU
+ library used, as it is updated to reflect changes in natural language
+ over time.
+ </para>
+ </note>
+ </sect3>
+ <sect3 id="locale-provider-libc">
+ <title>libc</title>
+ <para>
+ The <literal>libc</literal> provider uses the operating system's C
+ library. The collation and character classification behavior is
+ controlled by the settings <literal>LC_COLLATE</literal> and
+ <literal>LC_CTYPE</literal>, so they cannot be set independently.
+ </para>
+ <note>
+ <para>
+ The same locale name may have different behavior on different platforms
+ when using the libc provider.
+ </para>
+ </note>
+ </sect3>
+
</sect2>
<sect2 id="icu-locales">
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 5cf9777764..85f18cbbe5 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -96,6 +96,11 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
<replaceable>locale</replaceable>, you cannot specify either of those
parameters.
</para>
+ <para>
+ If <replaceable>provider</replaceable> is <literal>builtin</literal>,
+ then <replaceable>locale</replaceable> must be specified and set to
+ either <literal>C</literal> or <literal>C.UTF-8</literal>.
+ </para>
</listitem>
</varlistentry>
@@ -129,9 +134,9 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
<listitem>
<para>
Specifies the provider to use for locale services associated with this
- collation. Possible values are
- <literal>icu</literal><indexterm><primary>ICU</primary></indexterm>
- (if the server was built with ICU support) or <literal>libc</literal>.
+ collation. Possible values are <literal>builtin</literal>,
+ <literal>icu</literal><indexterm><primary>ICU</primary></indexterm> (if
+ the server was built with ICU support) or <literal>libc</literal>.
<literal>libc</literal> is the default. See <xref
linkend="locale-providers"/> for details.
</para>
diff --git a/doc/src/sgml/ref/create_database.sgml b/doc/src/sgml/ref/create_database.sgml
index ce7317f81b..6dc3348d1b 100644
--- a/doc/src/sgml/ref/create_database.sgml
+++ b/doc/src/sgml/ref/create_database.sgml
@@ -162,6 +162,12 @@ CREATE DATABASE <replaceable class="parameter">name</replaceable>
linkend="create-database-lc-ctype"/>, or <xref
linkend="create-database-icu-locale"/> individually.
</para>
+ <para>
+ If <xref linkend="create-database-locale-provider"/> is
+ <literal>builtin</literal>, then <replaceable>locale</replaceable>
+ must be specified and set to either <literal>C</literal> or
+ <literal>C.UTF-8</literal>.
+ </para>
<tip>
<para>
The other locale settings <xref linkend="guc-lc-messages"/>, <xref
@@ -243,7 +249,7 @@ CREATE DATABASE <replaceable class="parameter">name</replaceable>
<listitem>
<para>
Specifies the provider to use for the default collation in this
- database. Possible values are
+ database. Possible values are <literal>builtin</literal>,
<literal>icu</literal><indexterm><primary>ICU</primary></indexterm>
(if the server was built with ICU support) or <literal>libc</literal>.
By default, the provider is the same as that of the <xref
diff --git a/doc/src/sgml/ref/createdb.sgml b/doc/src/sgml/ref/createdb.sgml
index e4647d5ce7..d3e815f659 100644
--- a/doc/src/sgml/ref/createdb.sgml
+++ b/doc/src/sgml/ref/createdb.sgml
@@ -171,7 +171,7 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
- <term><option>--locale-provider={<literal>libc</literal>|<literal>icu</literal>}</option></term>
+ <term><option>--locale-provider={<literal>builtin</literal>|<literal>libc</literal>|<literal>icu</literal>}</option></term>
<listitem>
<para>
Specifies the locale provider for the database's default collation.
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
index cd75cae10e..08a1c2538f 100644
--- a/doc/src/sgml/ref/initdb.sgml
+++ b/doc/src/sgml/ref/initdb.sgml
@@ -286,6 +286,11 @@ PostgreSQL documentation
environment that <command>initdb</command> runs in. Locale
support is described in <xref linkend="locale"/>.
</para>
+ <para>
+ If <option>--locale-provider</option> is <literal>builtin</literal>,
+ <option>--locale</option> must be specified and set to
+ <literal>C</literal> or <literal>C.UTF-8</literal>.
+ </para>
</listitem>
</varlistentry>
@@ -314,8 +319,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-builtin-locale">
+ <term><option>--builtin-locale=<replaceable>locale</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the locale name when the builtin provider is used. Locale support
+ is described in <xref linkend="locale"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="app-initdb-option-locale-provider">
- <term><option>--locale-provider={<literal>libc</literal>|<literal>icu</literal>}</option></term>
+ <term><option>--locale-provider={<literal>builtin</literal>|<literal>libc</literal>|<literal>icu</literal>}</option></term>
<listitem>
<para>
This option sets the locale provider for databases created in the new
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 7bad94f908..d9b44c14ed 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -68,7 +68,10 @@ CollationCreate(const char *collname, Oid collnamespace,
Assert(collname);
Assert(collnamespace);
Assert(collowner);
- Assert((collcollate && collctype) || colllocale);
+ Assert((collprovider == COLLPROVIDER_LIBC &&
+ collcollate && collctype && !colllocale) ||
+ (collprovider != COLLPROVIDER_LIBC &&
+ !collcollate && !collctype && colllocale));
/*
* Make sure there is no existing collation of same name & encoding.
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 27564e569a..acb5071ab7 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *versionEl = NULL;
char *collcollate;
char *collctype;
- char *colllocale;
+ const char *colllocale;
char *collicurules;
bool collisdeterministic;
int collencoding;
@@ -215,7 +215,9 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (collproviderstr)
{
- if (pg_strcasecmp(collproviderstr, "icu") == 0)
+ if (pg_strcasecmp(collproviderstr, "builtin") == 0)
+ collprovider = COLLPROVIDER_BUILTIN;
+ else if (pg_strcasecmp(collproviderstr, "icu") == 0)
collprovider = COLLPROVIDER_ICU;
else if (pg_strcasecmp(collproviderstr, "libc") == 0)
collprovider = COLLPROVIDER_LIBC;
@@ -245,7 +247,17 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (lcctypeEl)
collctype = defGetString(lcctypeEl);
- if (collprovider == COLLPROVIDER_LIBC)
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ {
+ if (!colllocale)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("parameter \"locale\" must be specified")));
+
+ colllocale = builtin_validate_locale(GetDatabaseEncoding(),
+ colllocale);
+ }
+ else if (collprovider == COLLPROVIDER_LIBC)
{
if (!collcollate)
ereport(ERROR,
@@ -305,7 +317,17 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU rules cannot be specified unless locale provider is ICU")));
- if (collprovider == COLLPROVIDER_ICU)
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ {
+ /*
+ * Behavior may be different in different encodings, so set
+ * collencoding to the current database encoding. No validation is
+ * required, because the "builtin" provider is compatible with any
+ * encoding.
+ */
+ collencoding = GetDatabaseEncoding();
+ }
+ else if (collprovider == COLLPROVIDER_ICU)
{
#ifdef USE_ICU
/*
@@ -334,7 +356,18 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
if (!collversion)
- collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colllocale : collcollate);
+ {
+ const char *locale;
+
+ if (collprovider == COLLPROVIDER_ICU)
+ locale = colllocale;
+ else if (collprovider == COLLPROVIDER_LIBC)
+ locale = collcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ collversion = get_collation_actual_version(collprovider, locale);
+ }
newoid = CollationCreate(collName,
collNamespace,
@@ -409,6 +442,7 @@ AlterCollation(AlterCollationStmt *stmt)
Form_pg_collation collForm;
Datum datum;
bool isnull;
+ char *locale;
char *oldversion;
char *newversion;
ObjectAddress address;
@@ -435,8 +469,20 @@ AlterCollation(AlterCollationStmt *stmt)
datum = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion, &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
- newversion = get_collation_actual_version(collForm->collprovider, TextDatumGetCString(datum));
+ if (collForm->collprovider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, Anum_pg_collation_colllocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (collForm->collprovider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, Anum_pg_collation_collcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ newversion = get_collation_actual_version(collForm->collprovider, locale);
/* cannot change from NULL to non-NULL or vice versa */
if ((!oldversion && newversion) || (oldversion && !newversion))
@@ -500,11 +546,18 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
provider = ((Form_pg_database) GETSTRUCT(dbtup))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup,
- provider == COLLPROVIDER_ICU ?
- Anum_pg_database_datlocale : Anum_pg_database_datcollate);
-
- locale = TextDatumGetCString(datum);
+ if (provider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup, Anum_pg_database_datlocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (provider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup, Anum_pg_database_datcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
ReleaseSysCache(dbtup);
}
@@ -521,11 +574,19 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
provider = ((Form_pg_collation) GETSTRUCT(colltp))->collprovider;
Assert(provider != COLLPROVIDER_DEFAULT);
- datum = SysCacheGetAttrNotNull(COLLOID, colltp,
- provider == COLLPROVIDER_ICU ?
- Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
- locale = TextDatumGetCString(datum);
+ if (provider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, colltp, Anum_pg_collation_colllocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (provider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, colltp, Anum_pg_collation_collcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
ReleaseSysCache(colltp);
}
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 8d327243b3..0b133627be 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -698,6 +698,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
DefElem *dtemplate = NULL;
DefElem *dencoding = NULL;
DefElem *dlocale = NULL;
+ DefElem *dbuiltinlocale = NULL;
DefElem *dcollate = NULL;
DefElem *dctype = NULL;
DefElem *diculocale = NULL;
@@ -713,7 +714,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
const char *dbtemplate = NULL;
char *dbcollate = NULL;
char *dbctype = NULL;
- char *dblocale = NULL;
+ const char *dblocale = NULL;
char *dbicurules = NULL;
char dblocprovider = '\0';
char *canonname;
@@ -762,6 +763,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
errorConflictingDefElem(defel, pstate);
dlocale = defel;
}
+ else if (strcmp(defel->defname, "builtin_locale") == 0)
+ {
+ if (dbuiltinlocale)
+ errorConflictingDefElem(defel, pstate);
+ dbuiltinlocale = defel;
+ }
else if (strcmp(defel->defname, "lc_collate") == 0)
{
if (dcollate)
@@ -897,7 +904,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
dbcollate = defGetString(dlocale);
dbctype = defGetString(dlocale);
+ dblocale = defGetString(dlocale);
}
+ if (dbuiltinlocale && dbuiltinlocale->arg)
+ dblocale = defGetString(dbuiltinlocale);
if (dcollate && dcollate->arg)
dbcollate = defGetString(dcollate);
if (dctype && dctype->arg)
@@ -910,7 +920,9 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
char *locproviderstr = defGetString(dlocprovider);
- if (pg_strcasecmp(locproviderstr, "icu") == 0)
+ if (pg_strcasecmp(locproviderstr, "builtin") == 0)
+ dblocprovider = COLLPROVIDER_BUILTIN;
+ else if (pg_strcasecmp(locproviderstr, "icu") == 0)
dblocprovider = COLLPROVIDER_ICU;
else if (pg_strcasecmp(locproviderstr, "libc") == 0)
dblocprovider = COLLPROVIDER_LIBC;
@@ -1027,14 +1039,9 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
dbctype = src_ctype;
if (dblocprovider == '\0')
dblocprovider = src_locprovider;
- if (dblocale == NULL && dblocprovider == COLLPROVIDER_ICU)
- {
- if (dlocale && dlocale->arg)
- dblocale = defGetString(dlocale);
- else
- dblocale = src_locale;
- }
- if (dbicurules == NULL && dblocprovider == COLLPROVIDER_ICU)
+ if (dblocale == NULL)
+ dblocale = src_locale;
+ if (dbicurules == NULL)
dbicurules = src_icurules;
/* Some encodings are client only */
@@ -1059,6 +1066,27 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
check_encoding_locale_matches(encoding, dbcollate, dbctype);
+ if (dblocprovider == COLLPROVIDER_BUILTIN)
+ {
+ /*
+ * This would happen if template0 uses the libc provider but the new
+ * database uses builtin.
+ */
+ if (!dblocale)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("LOCALE must be specified for the builtin provider")));
+
+ dblocale = builtin_validate_locale(encoding, dblocale);
+ }
+ else
+ {
+ if (dbuiltinlocale && dbuiltinlocale->arg)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("BUILTIN_LOCALE cannot be specified unless locale provider is builtin")));
+ }
+
if (dblocprovider == COLLPROVIDER_ICU)
{
if (!(is_encoding_supported_by_icu(encoding)))
@@ -1100,7 +1128,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
}
else
{
- if (dblocale)
+ if (diculocale && diculocale->arg)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU locale cannot be specified unless locale provider is ICU")));
@@ -1111,6 +1139,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
errmsg("ICU rules cannot be specified unless locale provider is ICU")));
}
+ /* for libc, locale comes from datcollate and datctype */
+ if (dblocprovider == COLLPROVIDER_LIBC)
+ dblocale = NULL;
+
/*
* Check that the new encoding and locale settings match the source
* database. We insist on this because we simply copy the source data ---
@@ -1195,9 +1227,17 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
*/
if (src_collversion && !dcollversion)
{
- char *actual_versionstr;
+ char *actual_versionstr;
+ const char *locale;
- actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
+ if (dblocprovider == COLLPROVIDER_ICU)
+ locale = dblocale;
+ else if (dblocprovider == COLLPROVIDER_LIBC)
+ locale = dbcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ actual_versionstr = get_collation_actual_version(dblocprovider, locale);
if (!actual_versionstr)
ereport(ERROR,
(errmsg("template database \"%s\" has a collation version, but no actual collation version could be determined",
@@ -1225,7 +1265,18 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* collation version, which is normally only the case for template0.
*/
if (dbcollversion == NULL)
- dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
+ {
+ const char *locale;
+
+ if (dblocprovider == COLLPROVIDER_ICU)
+ locale = dblocale;
+ else if (dblocprovider == COLLPROVIDER_LIBC)
+ locale = dbcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ dbcollversion = get_collation_actual_version(dblocprovider, locale);
+ }
/* Resolve default tablespace for new database */
if (dtablespacename && dtablespacename->arg)
@@ -1364,8 +1415,8 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* block on the unique index, and fail after we commit).
*/
- Assert((dblocprovider == COLLPROVIDER_ICU && dblocale) ||
- (dblocprovider != COLLPROVIDER_ICU && !dblocale));
+ Assert((dblocprovider != COLLPROVIDER_LIBC && dblocale) ||
+ (dblocprovider == COLLPROVIDER_LIBC && !dblocale));
/* Form tuple */
new_record[Anum_pg_database_oid - 1] = ObjectIdGetDatum(dboid);
@@ -2446,6 +2497,7 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
ObjectAddress address;
Datum datum;
bool isnull;
+ char *locale;
char *oldversion;
char *newversion;
@@ -2472,10 +2524,24 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
datum = heap_getattr(tuple, Anum_pg_database_datcollversion, RelationGetDescr(rel), &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
- if (isnull)
- elog(ERROR, "unexpected null in pg_database");
- newversion = get_collation_actual_version(datForm->datlocprovider, TextDatumGetCString(datum));
+ if (datForm->datlocprovider == COLLPROVIDER_ICU)
+ {
+ datum = heap_getattr(tuple, Anum_pg_database_datlocale, RelationGetDescr(rel), &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected null in pg_database");
+ locale = TextDatumGetCString(datum);
+ }
+ else if (datForm->datlocprovider == COLLPROVIDER_LIBC)
+ {
+ datum = heap_getattr(tuple, Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected null in pg_database");
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ newversion = get_collation_actual_version(datForm->datlocprovider, locale);
/* cannot change from NULL to non-NULL or vice versa */
if ((!oldversion && newversion) || (oldversion && !newversion))
@@ -2660,6 +2726,7 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
HeapTuple tp;
char datlocprovider;
Datum datum;
+ char *locale;
char *version;
tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(dbid));
@@ -2670,8 +2737,20 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
datlocprovider = ((Form_pg_database) GETSTRUCT(tp))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate);
- version = get_collation_actual_version(datlocprovider, TextDatumGetCString(datum));
+ if (datlocprovider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, Anum_pg_database_datlocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (datlocprovider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, Anum_pg_database_datcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ version = get_collation_actual_version(datlocprovider, locale);
ReleaseSysCache(tp);
diff --git a/src/backend/regex/regc_pg_locale.c b/src/backend/regex/regc_pg_locale.c
index 6a26388bfa..3cb8678298 100644
--- a/src/backend/regex/regc_pg_locale.c
+++ b/src/backend/regex/regc_pg_locale.c
@@ -16,6 +16,8 @@
*/
#include "catalog/pg_collation.h"
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
#include "utils/pg_locale.h"
/*
@@ -64,6 +66,7 @@
typedef enum
{
PG_REGEX_LOCALE_C, /* C locale (encoding independent) */
+ PG_REGEX_BUILTIN, /* built-in Unicode semantics */
PG_REGEX_LOCALE_WIDE, /* Use <wctype.h> functions */
PG_REGEX_LOCALE_1BYTE, /* Use <ctype.h> functions */
PG_REGEX_LOCALE_WIDE_L, /* Use locale_t <wctype.h> functions */
@@ -75,6 +78,8 @@ static PG_Locale_Strategy pg_regex_strategy;
static pg_locale_t pg_regex_locale;
static Oid pg_regex_collation;
+static bool regex_builtin_cclass_posix = false;
+
/*
* Hard-wired character properties for C locale
*/
@@ -266,7 +271,15 @@ pg_set_regex_collation(Oid collation)
if (GetDatabaseEncoding() == PG_UTF8)
{
if (pg_regex_locale)
- pg_regex_strategy = PG_REGEX_LOCALE_WIDE_L;
+ {
+ if (pg_regex_locale->provider == COLLPROVIDER_BUILTIN)
+ {
+ pg_regex_strategy = PG_REGEX_BUILTIN;
+ regex_builtin_cclass_posix = pg_regex_locale->info.builtin.cclass_posix;
+ }
+ else
+ pg_regex_strategy = PG_REGEX_LOCALE_WIDE_L;
+ }
else
pg_regex_strategy = PG_REGEX_LOCALE_WIDE;
}
@@ -290,6 +303,8 @@ pg_wc_isdigit(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISDIGIT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isdigit(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswdigit((wint_t) c);
@@ -322,6 +337,8 @@ pg_wc_isalpha(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISALPHA));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isalpha(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswalpha((wint_t) c);
@@ -354,6 +371,8 @@ pg_wc_isalnum(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISALNUM));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isalnum(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswalnum((wint_t) c);
@@ -395,6 +414,8 @@ pg_wc_isupper(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISUPPER));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isupper(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswupper((wint_t) c);
@@ -427,6 +448,8 @@ pg_wc_islower(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISLOWER));
+ case PG_REGEX_BUILTIN:
+ return pg_u_islower(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswlower((wint_t) c);
@@ -459,6 +482,8 @@ pg_wc_isgraph(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISGRAPH));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isgraph(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswgraph((wint_t) c);
@@ -491,6 +516,8 @@ pg_wc_isprint(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISPRINT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isprint(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswprint((wint_t) c);
@@ -523,6 +550,8 @@ pg_wc_ispunct(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISPUNCT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_ispunct(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswpunct((wint_t) c);
@@ -555,6 +584,8 @@ pg_wc_isspace(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISSPACE));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isspace(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswspace((wint_t) c);
@@ -588,6 +619,8 @@ pg_wc_toupper(pg_wchar c)
if (c <= (pg_wchar) 127)
return pg_ascii_toupper((unsigned char) c);
return c;
+ case PG_REGEX_BUILTIN:
+ return unicode_uppercase_simple(c);
case PG_REGEX_LOCALE_WIDE:
/* force C behavior for ASCII characters, per comments above */
if (c <= (pg_wchar) 127)
@@ -628,6 +661,8 @@ pg_wc_tolower(pg_wchar c)
if (c <= (pg_wchar) 127)
return pg_ascii_tolower((unsigned char) c);
return c;
+ case PG_REGEX_BUILTIN:
+ return unicode_lowercase_simple(c);
case PG_REGEX_LOCALE_WIDE:
/* force C behavior for ASCII characters, per comments above */
if (c <= (pg_wchar) 127)
@@ -792,6 +827,9 @@ pg_ctype_get_cache(pg_wc_probefunc probefunc, int cclasscode)
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
#endif
break;
+ case PG_REGEX_BUILTIN:
+ max_chr = (pg_wchar) MAX_SIMPLE_CHR;
+ break;
case PG_REGEX_LOCALE_WIDE:
case PG_REGEX_LOCALE_WIDE_L:
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
@@ -809,6 +847,7 @@ pg_ctype_get_cache(pg_wc_probefunc probefunc, int cclasscode)
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
break;
default:
+ Assert(false);
max_chr = 0; /* can't get here, but keep compiler quiet */
break;
}
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index 83e1f1265c..c58055212c 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -77,6 +77,8 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
#include "mb/pg_wchar.h"
#include "nodes/miscnodes.h"
#include "parser/scansup.h"
@@ -1670,6 +1672,64 @@ str_tolower(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *src = (unsigned char *) buff;
+ unsigned char *dst;
+ size_t dstsize = nbytes + 1;
+ int srcoff = 0;
+ int dstoff = 0;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ dst = (unsigned char *)palloc(dstsize);
+
+ while (srcoff < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(src + srcoff);
+ pg_wchar u2 = unicode_lowercase_simple(u1);
+ int u1len = unicode_utf8len(u1);
+ int u2len = unicode_utf8len(u2);
+
+ /*
+ * If we can't fit the necessary bytes and a terminating NUL,
+ * reallocate buffer to the maximum size we might need, and
+ * shrink it later.
+ */
+ if (dstoff + u2len + 1 > dstsize)
+ {
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ dstsize = (nbytes + 1) * sizeof(pg_wchar);
+ dst = repalloc(dst, dstsize);
+ }
+
+ unicode_to_utf8(u2, dst + dstoff);
+ srcoff += u1len;
+ dstoff += u2len;
+ }
+
+ *(dst + dstoff) = '\0';
+ dstoff++;
+
+ if (dstsize == dstoff)
+ {
+ result = (char *) dst;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(dstoff);
+ memcpy(result, dst, dstoff);
+ pfree(dst);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
@@ -1788,6 +1848,64 @@ str_toupper(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *src = (unsigned char *) buff;
+ unsigned char *dst;
+ size_t dstsize = nbytes + 1;
+ int srcoff = 0;
+ int dstoff = 0;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ dst = (unsigned char *)palloc(dstsize);
+
+ while (srcoff < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(src + srcoff);
+ pg_wchar u2 = unicode_uppercase_simple(u1);
+ int u1len = unicode_utf8len(u1);
+ int u2len = unicode_utf8len(u2);
+
+ /*
+ * If we can't fit the necessary bytes and a terminating NUL,
+ * reallocate buffer to the maximum size we might need, and
+ * shrink it later.
+ */
+ if (dstoff + u2len + 1 > dstsize)
+ {
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ dstsize = (nbytes + 1) * sizeof(pg_wchar);
+ dst = repalloc(dst, dstsize);
+ }
+
+ unicode_to_utf8(u2, dst + dstoff);
+ srcoff += u1len;
+ dstoff += u2len;
+ }
+
+ *(dst + dstoff) = '\0';
+ dstoff++;
+
+ if (dstsize == dstoff)
+ {
+ result = (char *) dst;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(dstoff);
+ memcpy(result, dst, dstoff);
+ pfree(dst);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
@@ -1907,6 +2025,73 @@ str_initcap(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *src = (unsigned char *) buff;
+ unsigned char *dst;
+ size_t dstsize = nbytes + 1;
+ int srcoff = 0;
+ int dstoff = 0;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ dst = (unsigned char *)palloc(dstsize);
+
+ while (srcoff < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(src + srcoff);
+ pg_wchar u2;
+ int u1len = unicode_utf8len(u1);
+ int u2len;
+
+ if (wasalnum)
+ u2 = unicode_lowercase_simple(u1);
+ else
+ u2 = unicode_uppercase_simple(u1);
+
+ u2len = unicode_utf8len(u2);
+
+ wasalnum = pg_u_isalnum(u2, mylocale->info.builtin.cclass_posix);
+
+ /*
+ * If we can't fit the necessary bytes and a terminating NUL,
+ * reallocate buffer to the maximum size we might need, and
+ * shrink it later.
+ */
+ if (dstoff + u2len + 1 > dstsize)
+ {
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ dstsize = (nbytes + 1) * sizeof(pg_wchar);
+ dst = repalloc(dst, dstsize);
+ }
+
+ unicode_to_utf8(u2, dst + dstoff);
+ srcoff += u1len;
+ dstoff += u2len;
+ }
+
+ *(dst + dstoff) = '\0';
+ dstoff++;
+
+ if (dstsize == dstoff)
+ {
+ result = (char *) dst;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(dstoff);
+ memcpy(result, dst, dstoff);
+ pfree(dst);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 45fe847320..292f77bc1a 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1269,7 +1269,19 @@ lookup_collation_cache(Oid collation, bool set_flags)
elog(ERROR, "cache lookup failed for collation %u", collation);
collform = (Form_pg_collation) GETSTRUCT(tp);
- if (collform->collprovider == COLLPROVIDER_LIBC)
+ if (collform->collprovider == COLLPROVIDER_BUILTIN)
+ {
+ Datum datum;
+ const char *colllocale;
+
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
+ colllocale = TextDatumGetCString(datum);
+
+ cache_entry->collate_is_c = true;
+ cache_entry->ctype_is_c = ((strcmp(colllocale, "C") == 0) ||
+ (strcmp(colllocale, "POSIX") == 0));
+ }
+ else if (collform->collprovider == COLLPROVIDER_LIBC)
{
Datum datum;
const char *collcollate;
@@ -1320,16 +1332,30 @@ lc_collate_is_c(Oid collation)
if (collation == DEFAULT_COLLATION_OID)
{
static int result = -1;
- char *localeptr;
-
- if (default_locale.provider == COLLPROVIDER_ICU)
- return false;
+ const char *localeptr;
if (result >= 0)
return (bool) result;
- localeptr = setlocale(LC_COLLATE, NULL);
- if (!localeptr)
- elog(ERROR, "invalid LC_COLLATE setting");
+
+ if (default_locale.provider == COLLPROVIDER_BUILTIN)
+ {
+ result = true;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_ICU)
+ {
+ result = false;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_LIBC)
+ {
+ localeptr = setlocale(LC_CTYPE, NULL);
+ if (!localeptr)
+ elog(ERROR, "invalid LC_CTYPE setting");
+ }
+ else
+ elog(ERROR, "unexpected collation provider '%c'",
+ default_locale.provider);
if (strcmp(localeptr, "C") == 0)
result = true;
@@ -1373,16 +1399,29 @@ lc_ctype_is_c(Oid collation)
if (collation == DEFAULT_COLLATION_OID)
{
static int result = -1;
- char *localeptr;
-
- if (default_locale.provider == COLLPROVIDER_ICU)
- return false;
+ const char *localeptr;
if (result >= 0)
return (bool) result;
- localeptr = setlocale(LC_CTYPE, NULL);
- if (!localeptr)
- elog(ERROR, "invalid LC_CTYPE setting");
+
+ if (default_locale.provider == COLLPROVIDER_BUILTIN)
+ {
+ localeptr = default_locale.info.builtin.locale;
+ }
+ else if (default_locale.provider == COLLPROVIDER_ICU)
+ {
+ result = false;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_LIBC)
+ {
+ localeptr = setlocale(LC_CTYPE, NULL);
+ if (!localeptr)
+ elog(ERROR, "invalid LC_CTYPE setting");
+ }
+ else
+ elog(ERROR, "unexpected collation provider '%c'",
+ default_locale.provider);
if (strcmp(localeptr, "C") == 0)
result = true;
@@ -1390,6 +1429,7 @@ lc_ctype_is_c(Oid collation)
result = true;
else
result = false;
+
return (bool) result;
}
@@ -1520,10 +1560,10 @@ pg_newlocale_from_collation(Oid collid)
if (collid == DEFAULT_COLLATION_OID)
{
- if (default_locale.provider == COLLPROVIDER_ICU)
- return &default_locale;
- else
+ if (default_locale.provider == COLLPROVIDER_LIBC)
return (pg_locale_t) 0;
+ else
+ return &default_locale;
}
cache_entry = lookup_collation_cache(collid, false);
@@ -1548,7 +1588,18 @@ pg_newlocale_from_collation(Oid collid)
result.provider = collform->collprovider;
result.deterministic = collform->collisdeterministic;
- if (collform->collprovider == COLLPROVIDER_LIBC)
+ if (collform->collprovider == COLLPROVIDER_BUILTIN)
+ {
+ const char *locstr;
+
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
+ locstr = TextDatumGetCString(datum);
+
+ result.info.builtin.locale = MemoryContextStrdup(TopMemoryContext,
+ locstr);
+ result.info.builtin.cclass_posix = true;
+ }
+ else if (collform->collprovider == COLLPROVIDER_LIBC)
{
const char *collcollate;
const char *collctype pg_attribute_unused();
@@ -1627,6 +1678,7 @@ pg_newlocale_from_collation(Oid collid)
collversionstr = TextDatumGetCString(datum);
+ Assert(collform->collprovider != COLLPROVIDER_BUILTIN);
datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
actual_versionstr = get_collation_actual_version(collform->collprovider,
@@ -1678,6 +1730,14 @@ get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
+ /*
+ * The only two supported locales (C and C.UTF-8) are both based on memcmp
+ * and do not change. (The ctype behavior can change, but the versioning
+ * does not track that.)
+ */
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ return NULL;
+
#ifdef USE_ICU
if (collprovider == COLLPROVIDER_ICU)
{
@@ -2444,6 +2504,38 @@ pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
return result;
}
+const char *
+builtin_validate_locale(int encoding, const char *locale)
+{
+ const char *canonical_name = NULL;
+ int required_encoding = -1;
+
+ if (strcmp(locale, "C") == 0 || strcmp(locale, "POSIX") == 0)
+ {
+ canonical_name = "C";
+ }
+ else if (strcmp(locale, "C.UTF-8") == 0 || strcmp(locale, "C.UTF8") == 0)
+ {
+ required_encoding = PG_UTF8;
+ canonical_name = "C.UTF-8";
+ }
+
+ if (!canonical_name)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("invalid locale name \"%s\" for builtin provider",
+ locale)));
+
+ if (required_encoding >= 0 && encoding != required_encoding)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("encoding \"%s\" does not match locale \"%s\"",
+ pg_encoding_to_char(encoding), locale)));
+
+ return canonical_name;
+}
+
+
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 6dbee5556a..44314b8f00 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -318,7 +318,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
bool isnull;
char *collate;
char *ctype;
- char *iculocale;
+ char *datlocale;
/* Fetch our pg_database row normally, via syscache */
tup = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
@@ -423,12 +423,21 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
strcmp(ctype, "POSIX") == 0)
database_ctype_is_c = true;
- if (dbform->datlocprovider == COLLPROVIDER_ICU)
+ if (dbform->datlocprovider == COLLPROVIDER_BUILTIN)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
+ datlocale = TextDatumGetCString(datum);
+
+ default_locale.info.builtin.locale = MemoryContextStrdup(
+ TopMemoryContext, datlocale);
+ default_locale.info.builtin.cclass_posix = true;
+ }
+ else if (dbform->datlocprovider == COLLPROVIDER_ICU)
{
char *icurules;
datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
- iculocale = TextDatumGetCString(datum);
+ datlocale = TextDatumGetCString(datum);
datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_daticurules, &isnull);
if (!isnull)
@@ -436,10 +445,10 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
else
icurules = NULL;
- make_icu_collator(iculocale, icurules, &default_locale);
+ make_icu_collator(datlocale, icurules, &default_locale);
}
else
- iculocale = NULL;
+ datlocale = NULL;
default_locale.provider = dbform->datlocprovider;
@@ -461,10 +470,16 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
{
char *actual_versionstr;
char *collversionstr;
+ char *locale;
collversionstr = TextDatumGetCString(datum);
- actual_versionstr = get_collation_actual_version(dbform->datlocprovider, dbform->datlocprovider == COLLPROVIDER_ICU ? iculocale : collate);
+ if (dbform->datlocprovider == COLLPROVIDER_LIBC)
+ locale = collate;
+ else
+ locale = datlocale;
+
+ actual_versionstr = get_collation_actual_version(dbform->datlocprovider, locale);
if (!actual_versionstr)
/* should not happen */
elog(WARNING,
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 8f4b456995..daf0ff9653 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -2442,7 +2442,8 @@ usage(const char *progname)
" set default locale in the respective category for\n"
" new databases (default taken from environment)\n"));
printf(_(" --no-locale equivalent to --locale=C\n"));
- printf(_(" --locale-provider={libc|icu}\n"
+ printf(_(" --builtin-locale=LOCALE set builtin locale name for new databases\n"));
+ printf(_(" --locale-provider={builtin|libc|icu}\n"
" set default locale provider for new databases\n"));
printf(_(" --pwfile=FILE read password for the new superuser from file\n"));
printf(_(" -T, --text-search-config=CFG\n"
@@ -2593,7 +2594,15 @@ setup_locale_encoding(void)
{
setlocales();
- if (locale_provider == COLLPROVIDER_LIBC &&
+ if (locale_provider == COLLPROVIDER_BUILTIN &&
+ strcmp(lc_ctype, "C") == 0 &&
+ strcmp(lc_collate, "C") == 0 &&
+ strcmp(lc_time, "C") == 0 &&
+ strcmp(lc_numeric, "C") == 0 &&
+ strcmp(lc_monetary, "C") == 0 &&
+ strcmp(lc_messages, "C") == 0)
+ printf(_("The database cluster will be initialized with no locale.\n"));
+ else if (locale_provider == COLLPROVIDER_LIBC &&
strcmp(lc_ctype, lc_collate) == 0 &&
strcmp(lc_ctype, lc_time) == 0 &&
strcmp(lc_ctype, lc_numeric) == 0 &&
@@ -3099,9 +3108,10 @@ main(int argc, char *argv[])
{"allow-group-access", no_argument, NULL, 'g'},
{"discard-caches", no_argument, NULL, 14},
{"locale-provider", required_argument, NULL, 15},
- {"icu-locale", required_argument, NULL, 16},
- {"icu-rules", required_argument, NULL, 17},
- {"sync-method", required_argument, NULL, 18},
+ {"builtin-locale", required_argument, NULL, 16},
+ {"icu-locale", required_argument, NULL, 17},
+ {"icu-rules", required_argument, NULL, 18},
+ {"sync-method", required_argument, NULL, 19},
{NULL, 0, NULL, 0}
};
@@ -3269,7 +3279,9 @@ main(int argc, char *argv[])
"-c debug_discard_caches=1");
break;
case 15:
- if (strcmp(optarg, "icu") == 0)
+ if (strcmp(optarg, "builtin") == 0)
+ locale_provider = COLLPROVIDER_BUILTIN;
+ else if (strcmp(optarg, "icu") == 0)
locale_provider = COLLPROVIDER_ICU;
else if (strcmp(optarg, "libc") == 0)
locale_provider = COLLPROVIDER_LIBC;
@@ -3278,12 +3290,15 @@ main(int argc, char *argv[])
break;
case 16:
datlocale = pg_strdup(optarg);
- icu_locale_specified = true;
break;
case 17:
- icu_rules = pg_strdup(optarg);
+ datlocale = pg_strdup(optarg);
+ icu_locale_specified = true;
break;
case 18:
+ icu_rules = pg_strdup(optarg);
+ break;
+ case 19:
if (!parse_sync_method(optarg, &sync_method))
exit(1);
break;
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index eb17c0e8f9..540dbac621 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -184,6 +184,61 @@ else
'locale provider ICU fails since no ICU support');
}
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', "$tempdir/data6"
+ ],
+ 'locale provider builtin fails without --locale'
+);
+
+command_ok(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--locale=C',
+ "$tempdir/data7"
+ ],
+ 'locale provider builtin with --locale'
+);
+
+command_ok(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '-E UTF-8',
+ '--builtin-locale=C.UTF-8', "$tempdir/data8"
+ ],
+ 'locale provider builtin with -E UTF-8 --builtin-locale=C.UTF-8'
+);
+
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '-E SQL_ASCII',
+ '--builtin-locale=C.UTF-8', "$tempdir/data9"
+ ],
+ 'locale provider builtin with --builtin-locale=C.UTF-8 fails for SQL_ASCII'
+);
+
+command_ok(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--lc-ctype=C',
+ '--locale=C', "$tempdir/data10"
+ ],
+ 'locale provider builtin with --lc-ctype'
+);
+
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--icu-locale=en',
+ "$tempdir/dataX"
+ ],
+ 'fails for locale provider builtin with ICU locale'
+);
+
+command_fails(
+ [
+ 'initdb', '--no-sync', '--locale-provider=builtin', '--icu-rules=""',
+ "$tempdir/dataX"
+ ],
+ 'fails for locale provider builtin with ICU rules'
+);
+
command_fails(
[ 'initdb', '--no-sync', '--locale-provider=xyz', "$tempdir/dataX" ],
'fails for invalid locale provider');
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 939ab62ff9..a6e3abee81 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3112,7 +3112,9 @@ dumpDatabase(Archive *fout)
}
appendPQExpBufferStr(creaQry, " LOCALE_PROVIDER = ");
- if (datlocprovider[0] == 'c')
+ if (datlocprovider[0] == 'b')
+ appendPQExpBufferStr(creaQry, "builtin");
+ else if (datlocprovider[0] == 'c')
appendPQExpBufferStr(creaQry, "libc");
else if (datlocprovider[0] == 'i')
appendPQExpBufferStr(creaQry, "icu");
@@ -13918,7 +13920,9 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
fmtQualifiedDumpable(collinfo));
appendPQExpBufferStr(q, "provider = ");
- if (collprovider[0] == 'c')
+ if (collprovider[0] == 'b')
+ appendPQExpBufferStr(q, "builtin");
+ else if (collprovider[0] == 'c')
appendPQExpBufferStr(q, "libc");
else if (collprovider[0] == 'i')
appendPQExpBufferStr(q, "icu");
@@ -13939,6 +13943,13 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
/* no locale -- the default collation cannot be reloaded anyway */
}
+ else if (collprovider[0] == 'b')
+ {
+ if (collcollate || collctype || colllocale || collicurules)
+ pg_log_warning("invalid collation \"%s\"", qcollname);
+
+ appendPQExpBufferStr(q, ", locale = 'C'");
+ }
else if (collprovider[0] == 'i')
{
if (fout->remoteVersion >= 150000)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index ec7b244d4f..f4401ba82c 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -110,35 +110,71 @@ my $oldversion = int($oldnode->pg_version =~ s/([0-9]*).*/$1/rg);
# can test that pg_upgrade copies the locale settings of template0
# from the old to the new cluster.
-my $original_encoding = "6"; # UTF-8
-my $original_provider = "c";
-my $original_locale = "C";
-my $original_iculocale = "";
-my $provider_field = "'c' AS datlocprovider";
-my $iculocale_field = "NULL AS datlocale";
-if ($oldversion >= 15 && $ENV{with_icu} eq 'yes')
+my %encoding_number = ('UTF-8' => 6, 'SQL_ASCII' => 0);
+my $provider_field;
+my $datlocale_field;
+my $original_encoding;
+my $original_provider;
+my $original_datcollate = "C";
+my $original_datctype = "C";
+my $original_datlocale;
+
+if ($oldversion >= 15)
{
$provider_field = "datlocprovider";
if ($oldversion >= 17)
{
- $iculocale_field = "datlocale";
+ $datlocale_field = "datlocale";
}
else
{
- $iculocale_field = "daticulocale AS datlocale";
+ $datlocale_field = "daticulocale AS datlocale";
}
+}
+else
+{
+ $provider_field = "'c' AS datlocprovider";
+ $datlocale_field = "NULL AS datlocale";
+}
+
+if ($oldversion >= 17)
+{
+ $original_encoding = "UTF-8";
+ $original_provider = "b";
+ $original_datlocale = "C.UTF-8";
+}
+elsif ($oldversion >= 15 && $ENV{with_icu} eq 'yes')
+{
+ $original_encoding = "UTF-8";
$original_provider = "i";
- $original_iculocale = "fr-CA";
+ $original_datlocale = "fr-CA";
+}
+else
+{
+ my $original_encoding = "SQL_ASCII";
+ my $original_provider = "c";
+ my $original_datlocale = "";
}
my @initdb_params = @custom_opts;
-push @initdb_params, ('--encoding', 'UTF-8');
-push @initdb_params, ('--locale', $original_locale);
-if ($original_provider eq "i")
+push @initdb_params, ('--encoding', $original_encoding);
+push @initdb_params, ('--lc-collate', $original_datcollate);
+push @initdb_params, ('--lc-ctype', $original_datctype);
+
+# add --locale-provider, if supported
+my %provider_name = ('b' => 'builtin', 'i' => 'icu', 'c' => 'libc');
+if ($oldnode->pg_version >= 15)
{
- push @initdb_params, ('--locale-provider', 'icu');
- push @initdb_params, ('--icu-locale', 'fr-CA');
+ push @initdb_params, ('--locale-provider', $provider_name{$original_provider});
+ if ($original_provider eq 'b')
+ {
+ push @initdb_params, ('--builtin-locale', $original_datlocale);
+ }
+ elsif ($original_provider eq 'i')
+ {
+ push @initdb_params, ('--icu-locale', $original_datlocale);
+ }
}
$node_params{extra} = \@initdb_params;
@@ -148,10 +184,10 @@ $oldnode->start;
my $result;
$result = $oldnode->safe_psql(
'postgres',
- "SELECT encoding, $provider_field, datcollate, datctype, $iculocale_field
+ "SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_iculocale",
+ "$encoding_number{$original_encoding}|$original_provider|$original_datcollate|$original_datctype|$original_datlocale",
"check locales in original cluster");
# The default location of the source code is the root of this directory.
@@ -431,10 +467,10 @@ if (-d $log_path)
# Test that upgraded cluster has original locale settings.
$result = $newnode->safe_psql(
'postgres',
- "SELECT encoding, $provider_field, datcollate, datctype, $iculocale_field
+ "SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_iculocale",
+ "$encoding_number{$original_encoding}|$original_provider|$original_datcollate|$original_datctype|$original_datlocale",
"check that locales in new cluster match original cluster");
# Second dump from the upgraded instance.
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 254fce8ff1..42967b344d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -926,7 +926,7 @@ listAllDbs(const char *pattern, bool verbose)
gettext_noop("Encoding"));
if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
- " CASE d.datlocprovider WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
+ " CASE d.datlocprovider WHEN 'b' THEN 'builtin' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
gettext_noop("Locale Provider"));
else
appendPQExpBuffer(&buf,
@@ -4966,7 +4966,7 @@ listCollations(const char *pattern, bool verbose, bool showSystem)
if (pset.sversion >= 100000)
appendPQExpBuffer(&buf,
- " CASE c.collprovider WHEN 'd' THEN 'default' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
+ " CASE c.collprovider WHEN 'd' THEN 'default' WHEN 'b' THEN 'builtin' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
gettext_noop("Provider"));
else
appendPQExpBuffer(&buf,
diff --git a/src/bin/scripts/createdb.c b/src/bin/scripts/createdb.c
index 14970a6a5f..4af4b98181 100644
--- a/src/bin/scripts/createdb.c
+++ b/src/bin/scripts/createdb.c
@@ -40,8 +40,9 @@ main(int argc, char *argv[])
{"locale", required_argument, NULL, 'l'},
{"maintenance-db", required_argument, NULL, 3},
{"locale-provider", required_argument, NULL, 4},
- {"icu-locale", required_argument, NULL, 5},
- {"icu-rules", required_argument, NULL, 6},
+ {"builtin-locale", required_argument, NULL, 5},
+ {"icu-locale", required_argument, NULL, 6},
+ {"icu-rules", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
@@ -67,6 +68,7 @@ main(int argc, char *argv[])
char *lc_ctype = NULL;
char *locale = NULL;
char *locale_provider = NULL;
+ char *builtin_locale = NULL;
char *icu_locale = NULL;
char *icu_rules = NULL;
@@ -134,9 +136,12 @@ main(int argc, char *argv[])
locale_provider = pg_strdup(optarg);
break;
case 5:
- icu_locale = pg_strdup(optarg);
+ builtin_locale = pg_strdup(optarg);
break;
case 6:
+ icu_locale = pg_strdup(optarg);
+ break;
+ case 7:
icu_rules = pg_strdup(optarg);
break;
default:
@@ -216,6 +221,11 @@ main(int argc, char *argv[])
appendPQExpBufferStr(&sql, " LOCALE ");
appendStringLiteralConn(&sql, locale, conn);
}
+ if (builtin_locale)
+ {
+ appendPQExpBufferStr(&sql, " BUILTIN_LOCALE ");
+ appendStringLiteralConn(&sql, builtin_locale, conn);
+ }
if (lc_collate)
{
appendPQExpBufferStr(&sql, " LC_COLLATE ");
@@ -296,7 +306,7 @@ help(const char *progname)
printf(_(" --lc-ctype=LOCALE LC_CTYPE setting for the database\n"));
printf(_(" --icu-locale=LOCALE ICU locale setting for the database\n"));
printf(_(" --icu-rules=RULES ICU rules setting for the database\n"));
- printf(_(" --locale-provider={libc|icu}\n"
+ printf(_(" --locale-provider={builtin|libc|icu}\n"
" locale provider for the database's default collation\n"));
printf(_(" -O, --owner=OWNER database user to own the new database\n"));
printf(_(" -S, --strategy=STRATEGY database creation strategy wal_log or file_copy\n"));
diff --git a/src/bin/scripts/t/020_createdb.pl b/src/bin/scripts/t/020_createdb.pl
index 37e47b0078..8da5fbd4f0 100644
--- a/src/bin/scripts/t/020_createdb.pl
+++ b/src/bin/scripts/t/020_createdb.pl
@@ -105,6 +105,78 @@ else
'create database with ICU fails since no ICU support');
}
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ 'tbuiltin1'
+ ],
+ 'create database with provider "builtin" fails without --locale'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', 'tbuiltin2'
+ ],
+ 'create database with provider "builtin" and locale "C"'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--lc-collate=C', 'tbuiltin3'
+ ],
+ 'create database with provider "builtin" and LC_COLLATE=C'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--lc-ctype=C', 'tbuiltin4'
+ ],
+ 'create database with provider "builtin" and LC_CTYPE=C'
+);
+
+$node->command_ok(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '-E UTF-8', '--builtin-locale=C.UTF8', 'tbuiltin5'
+ ],
+ 'create database with provider "builtin" and --builtin-locale C.UTF-8'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '-E LATIN1', '--builtin-locale=C.UTF-8', 'tbuiltin6'
+ ],
+ 'create database with provider "builtin" and --builtin-locale C.UTF-8'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--icu-locale=en', 'tbuiltin7'
+ ],
+ 'create database with provider "builtin" and ICU_LOCALE="en"'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template0', '--locale-provider=builtin',
+ '--locale=C', '--icu-rules=""', 'tbuiltin8'
+ ],
+ 'create database with provider "builtin" and ICU_RULES=""'
+);
+
+$node->command_fails(
+ [
+ 'createdb', '-T', 'template1', '--locale-provider=builtin',
+ '--locale=C', 'tbuiltin9'
+ ],
+ 'create database with provider "builtin" not matching template'
+);
+
$node->command_fails([ 'createdb', 'foobar1' ],
'fails if database already exists');
diff --git a/src/common/wchar.c b/src/common/wchar.c
index fd2927f942..a2b5661391 100644
--- a/src/common/wchar.c
+++ b/src/common/wchar.c
@@ -476,8 +476,8 @@ pg_utf2wchar_with_len(const unsigned char *from, pg_wchar *to, int len)
/*
- * Map a Unicode code point to UTF-8. utf8string must have 4 bytes of
- * space allocated.
+ * Map a Unicode code point to UTF-8. utf8string must have at least
+ * unicode_utf8len(c) bytes available.
*/
unsigned char *
unicode_to_utf8(pg_wchar c, unsigned char *utf8string)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 7396ff10c4..1e439e6975 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -23,12 +23,15 @@
descr => 'standard POSIX collation',
collname => 'POSIX', collprovider => 'c', collencoding => '-1',
collcollate => 'POSIX', collctype => 'POSIX' },
-{ oid => '962', descr => 'sorts by Unicode code point',
- collname => 'ucs_basic', collprovider => 'c', collencoding => '6',
- collcollate => 'C', collctype => 'C' },
+{ oid => '962', descr => 'sorts by Unicode code point, C character semantics',
+ collname => 'ucs_basic', collprovider => 'b', collencoding => '6',
+ colllocale => 'C' },
{ oid => '963',
descr => 'sorts using the Unicode Collation Algorithm with default settings',
collname => 'unicode', collprovider => 'i', collencoding => '-1',
colllocale => 'und' },
+{ oid => '970', descr => 'sorts by Unicode code point; Unicode & POSIX character semantics',
+ collname => 'pg_c_utf8', collprovider => 'b', collencoding => '6',
+ colllocale => 'C.UTF8' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 1ee2d264a1..b881b48e75 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -65,6 +65,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_collation_oid_index, 3085, CollationOidIndexId, pg_
#ifdef EXPOSE_TO_CLIENT_CODE
#define COLLPROVIDER_DEFAULT 'd'
+#define COLLPROVIDER_BUILTIN 'b'
#define COLLPROVIDER_ICU 'i'
#define COLLPROVIDER_LIBC 'c'
@@ -73,6 +74,8 @@ collprovider_name(char c)
{
switch (c)
{
+ case COLLPROVIDER_BUILTIN:
+ return "builtin";
case COLLPROVIDER_ICU:
return "icu";
case COLLPROVIDER_LIBC:
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 61acb6f567..6d4ef84c04 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -723,6 +723,22 @@ extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
#endif
+/*
+ * Number of bytes needed to represent the given char in UTF8.
+ */
+static inline int
+unicode_utf8len(pg_wchar c)
+{
+ if (c <= 0x7F)
+ return 1;
+ else if (c <= 0x7FF)
+ return 2;
+ else if (c <= 0xFFFF)
+ return 3;
+ else
+ return 4;
+}
+
/*
* Verify a chunk of bytes for valid ASCII.
*
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 28c925b5af..f7a4ef3829 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -76,6 +76,11 @@ struct pg_locale_struct
bool deterministic;
union
{
+ struct
+ {
+ const char *locale;
+ bool cclass_posix;
+ } builtin;
locale_t lt;
#ifdef USE_ICU
struct
@@ -112,7 +117,7 @@ extern size_t pg_strxfrm_prefix(char *dest, const char *src, size_t destsize,
pg_locale_t locale);
extern size_t pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
size_t srclen, pg_locale_t locale);
-
+extern const char *builtin_validate_locale(int encoding, const char *loc_str);
extern void icu_validate_locale(const char *loc_str);
extern char *icu_language_tag(const char *loc_str, int elevel);
diff --git a/src/test/icu/t/010_database.pl b/src/test/icu/t/010_database.pl
index 8a1fc12ec6..ccf7a9d643 100644
--- a/src/test/icu/t/010_database.pl
+++ b/src/test/icu/t/010_database.pl
@@ -63,14 +63,14 @@ is( $node1->psql(
0,
"C locale works for ICU");
-# Test that LOCALE works for ICU locales if LC_COLLATE and LC_CTYPE
-# are specified
-is( $node1->psql(
- 'postgres',
- q{CREATE DATABASE dbicu2 LOCALE_PROVIDER icu LOCALE '@colStrength=primary'
- LC_COLLATE='C' LC_CTYPE='C' TEMPLATE template0 ENCODING UTF8}
- ),
- 0,
- "LOCALE works for ICU locales if LC_COLLATE and LC_CTYPE are specified");
+my ($ret, $stdout, $stderr) = $node1->psql('postgres',
+ q{CREATE DATABASE dbicu LOCALE_PROVIDER builtin LOCALE 'C' TEMPLATE dbicu}
+);
+isnt($ret, 0,
+ "locale provider must match template: exit code not 0");
+like(
+ $stderr,
+ qr/ERROR: new locale provider \(builtin\) does not match locale provider of the template database \(icu\)/,
+ "locale provider must match template: error message");
done_testing();
diff --git a/src/test/regress/expected/collate.out b/src/test/regress/expected/collate.out
index 0649564485..ece4a8e99d 100644
--- a/src/test/regress/expected/collate.out
+++ b/src/test/regress/expected/collate.out
@@ -650,6 +650,26 @@ EXPLAIN (COSTS OFF)
(3 rows)
-- CREATE/DROP COLLATION
+CREATE COLLATION builtin_c ( PROVIDER = builtin, LOCALE = "C" );
+CREATE COLLATION builtin_posix ( PROVIDER = builtin, LOCALE = "POSIX" );
+SELECT b FROM collate_test1 ORDER BY b COLLATE builtin_c;
+ b
+-----
+ ABD
+ Abc
+ abc
+ bbc
+(4 rows)
+
+CREATE COLLATION builtin2 ( PROVIDER = builtin ); -- fails
+ERROR: parameter "locale" must be specified
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "en_US" ); -- fails
+ERROR: invalid locale name "en_US" for builtin provider
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LC_CTYPE = "C", LC_COLLATE = "C" ); -- fails
+ERROR: parameter "locale" must be specified
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "POSIX", LC_CTYPE = "POSIX" ); -- fails
+ERROR: conflicting or redundant options
+DETAIL: LOCALE cannot be specified together with LC_COLLATE or LC_CTYPE.
CREATE COLLATION mycoll1 FROM "C";
CREATE COLLATION mycoll2 ( LC_COLLATE = "POSIX", LC_CTYPE = "POSIX" );
CREATE COLLATION mycoll3 FROM "default"; -- intentionally unsupported
@@ -754,7 +774,7 @@ DETAIL: FROM cannot be specified together with any other options.
-- must get rid of them.
--
DROP SCHEMA collate_tests CASCADE;
-NOTICE: drop cascades to 19 other objects
+NOTICE: drop cascades to 21 other objects
DETAIL: drop cascades to table collate_test1
drop cascades to table collate_test_like
drop cascades to table collate_test2
@@ -771,6 +791,8 @@ drop cascades to function dup(anyelement)
drop cascades to table collate_test20
drop cascades to table collate_test21
drop cascades to table collate_test22
+drop cascades to collation builtin_c
+drop cascades to collation builtin_posix
drop cascades to collation mycoll2
drop cascades to table collate_test23
drop cascades to view collate_on_int
diff --git a/src/test/regress/expected/collate.utf8.out b/src/test/regress/expected/collate.utf8.out
new file mode 100644
index 0000000000..2ef1826d91
--- /dev/null
+++ b/src/test/regress/expected/collate.utf8.out
@@ -0,0 +1,109 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+SET client_encoding TO UTF8;
+--
+-- Test preinstalled PG_C_UTF8 collation.
+--
+CREATE TABLE builtin_test (
+ t TEXT COLLATE PG_C_UTF8
+);
+INSERT INTO builtin_test VALUES
+ ('abc DEF'),
+ ('ábc DÉF'),
+ ('DŽxxDŽ džxxDž Džxxdž'),
+ ('ȺȺȺ'),
+ ('ⱥⱥⱥ'),
+ ('ⱥȺ');
+SELECT
+ t, lower(t), initcap(t), upper(t),
+ length(convert_to(t, 'UTF8')) AS t_bytes,
+ length(convert_to(lower(t), 'UTF8')) AS lower_t_bytes,
+ length(convert_to(initcap(t), 'UTF8')) AS initcap_t_bytes,
+ length(convert_to(upper(t), 'UTF8')) AS upper_t_bytes
+ FROM builtin_test;
+ t | lower | initcap | upper | t_bytes | lower_t_bytes | initcap_t_bytes | upper_t_bytes
+----------------+----------------+----------------+----------------+---------+---------------+-----------------+---------------
+ abc DEF | abc def | Abc Def | ABC DEF | 7 | 7 | 7 | 7
+ ábc DÉF | ábc déf | Ábc Déf | ÁBC DÉF | 9 | 9 | 9 | 9
+ DŽxxDŽ džxxDž Džxxdž | džxxdž džxxdž džxxdž | DŽxxdž DŽxxdž DŽxxdž | DŽXXDŽ DŽXXDŽ DŽXXDŽ | 20 | 20 | 20 | 20
+ ȺȺȺ | ⱥⱥⱥ | Ⱥⱥⱥ | ȺȺȺ | 6 | 9 | 8 | 6
+ ⱥⱥⱥ | ⱥⱥⱥ | Ⱥⱥⱥ | ȺȺȺ | 9 | 9 | 8 | 6
+ ⱥȺ | ⱥⱥ | Ⱥⱥ | ȺȺ | 5 | 6 | 5 | 4
+(6 rows)
+
+DROP TABLE builtin_test;
+-- character classes
+SELECT 'xyz' ~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xyz' !~ '[[:upper:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '@' !~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '@' ~ '[[:punct:]]' COLLATE PG_C_UTF8; -- symbols are punctuation in posix
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'a8a' ~ '[[:digit:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '൧' !~ '\d' COLLATE PG_C_UTF8; -- only 0-9 considered digits in posix
+ ?column?
+----------
+ t
+(1 row)
+
+-- case mapping
+SELECT 'xYz' ~* 'XyZ' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xAb' ~* '[W-Y]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xAb' !~* '[c-d]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'Δ' ~* '[α-λ]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'δ' ~* '[Γ-Λ]' COLLATE PG_C_UTF8; -- same as above with cases reversed
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/collate.utf8_1.out b/src/test/regress/expected/collate.utf8_1.out
new file mode 100644
index 0000000000..e73fdf50c3
--- /dev/null
+++ b/src/test/regress/expected/collate.utf8_1.out
@@ -0,0 +1,8 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index f0987ff537..292bc54932 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -78,9 +78,9 @@ test: brin_bloom brin_multi
# psql depends on create_am
# amutils depends on geometry, create_index_spgist, hash_index, brin
# ----------
-test: create_table_like alter_generic alter_operator misc async dbsize merge misc_functions sysviews tsrf tid tidscan tidrangescan collate.icu.utf8 incremental_sort create_role
+test: create_table_like alter_generic alter_operator misc async dbsize merge misc_functions sysviews tsrf tid tidscan tidrangescan collate.utf8 collate.icu.utf8 incremental_sort create_role
-# collate.*.utf8 tests cannot be run in parallel with each other
+# collate.linux.utf8 and collate.icu.utf8 tests cannot be run in parallel with each other
test: rules psql psql_crosstab amutils stats_ext collate.linux.utf8 collate.windows.win1252
# ----------
diff --git a/src/test/regress/sql/collate.sql b/src/test/regress/sql/collate.sql
index c3d40fc195..01d5c69fe4 100644
--- a/src/test/regress/sql/collate.sql
+++ b/src/test/regress/sql/collate.sql
@@ -244,6 +244,16 @@ EXPLAIN (COSTS OFF)
-- CREATE/DROP COLLATION
+CREATE COLLATION builtin_c ( PROVIDER = builtin, LOCALE = "C" );
+CREATE COLLATION builtin_posix ( PROVIDER = builtin, LOCALE = "POSIX" );
+
+SELECT b FROM collate_test1 ORDER BY b COLLATE builtin_c;
+
+CREATE COLLATION builtin2 ( PROVIDER = builtin ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "en_US" ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LC_CTYPE = "C", LC_COLLATE = "C" ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "POSIX", LC_CTYPE = "POSIX" ); -- fails
+
CREATE COLLATION mycoll1 FROM "C";
CREATE COLLATION mycoll2 ( LC_COLLATE = "POSIX", LC_CTYPE = "POSIX" );
CREATE COLLATION mycoll3 FROM "default"; -- intentionally unsupported
diff --git a/src/test/regress/sql/collate.utf8.sql b/src/test/regress/sql/collate.utf8.sql
new file mode 100644
index 0000000000..584c50f915
--- /dev/null
+++ b/src/test/regress/sql/collate.utf8.sql
@@ -0,0 +1,54 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+SET client_encoding TO UTF8;
+
+--
+-- Test preinstalled PG_C_UTF8 collation.
+--
+
+CREATE TABLE builtin_test (
+ t TEXT COLLATE PG_C_UTF8
+);
+INSERT INTO builtin_test VALUES
+ ('abc DEF'),
+ ('ábc DÉF'),
+ ('DŽxxDŽ džxxDž Džxxdž'),
+ ('ȺȺȺ'),
+ ('ⱥⱥⱥ'),
+ ('ⱥȺ');
+
+SELECT
+ t, lower(t), initcap(t), upper(t),
+ length(convert_to(t, 'UTF8')) AS t_bytes,
+ length(convert_to(lower(t), 'UTF8')) AS lower_t_bytes,
+ length(convert_to(initcap(t), 'UTF8')) AS initcap_t_bytes,
+ length(convert_to(upper(t), 'UTF8')) AS upper_t_bytes
+ FROM builtin_test;
+
+DROP TABLE builtin_test;
+
+-- character classes
+
+SELECT 'xyz' ~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+SELECT 'xyz' !~ '[[:upper:]]' COLLATE PG_C_UTF8;
+SELECT '@' !~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+SELECT '@' ~ '[[:punct:]]' COLLATE PG_C_UTF8; -- symbols are punctuation in posix
+SELECT 'a8a' ~ '[[:digit:]]' COLLATE PG_C_UTF8;
+SELECT '൧' !~ '\d' COLLATE PG_C_UTF8; -- only 0-9 considered digits in posix
+
+-- case mapping
+
+SELECT 'xYz' ~* 'XyZ' COLLATE PG_C_UTF8;
+SELECT 'xAb' ~* '[W-Y]' COLLATE PG_C_UTF8;
+SELECT 'xAb' !~* '[c-d]' COLLATE PG_C_UTF8;
+SELECT 'Δ' ~* '[α-λ]' COLLATE PG_C_UTF8;
+SELECT 'δ' ~* '[Γ-Λ]' COLLATE PG_C_UTF8; -- same as above with cases reversed
--
2.34.1
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-12 16:58 Jeff Davis <[email protected]>
parent: Jeff Davis <[email protected]>
3 siblings, 0 replies; 12+ messages in thread
From: Jeff Davis @ 2024-01-12 16:58 UTC (permalink / raw)
To: Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On Thu, 2024-01-11 at 18:02 -0800, Jeff Davis wrote:
> Jeremy also raised a problem with old versions of psql connecting to
> a
> new server: the \l and \dO won't work. Not sure exactly what to do
> there, but I could work around it by adding a new field rather than
> renaming (though that's not ideal).
I did a bit of research for a precedent, and the closest I could find
is that \dp was broken between 14 and 15 by commit 07eee5a0dc.
That is some precedent, but it's more narrow. I think that might
justify breaking \dO in older clients, but \l is used frequently.
It seems safer to just introduce new columns "datbuiltinlocale" and
"collbuiltinlocale". They'll be nullable anyway.
If we want to clean this up we can do so as a separate commit. There
are other parts of the catalog representation related to collation that
we might want to clean up as well.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-15 14:30 Daniel Verite <[email protected]>
parent: Jeff Davis <[email protected]>
3 siblings, 0 replies; 12+ messages in thread
From: Daniel Verite @ 2024-01-15 14:30 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
Jeff Davis wrote:
> New version attached.
[v16]
Concerning the target category_test, it produces failures with
versions of ICU with Unicode < 15. The first one I see with Ubuntu
22.04 (ICU 70.1) is:
category_test: Postgres Unicode version: 15.1
category_test: ICU Unicode version: 14.0
category_test: FAILURE for codepoint 0x000c04
category_test: Postgres property
alphabetic/lowercase/uppercase/white_space/hex_digit/join_control:
1/0/0/0/0/0
category_test: ICU property
alphabetic/lowercase/uppercase/white_space/hex_digit/join_control:
0/0/0/0/0/0
U+0C04 is a codepoint added in Unicode 11.
https://en.wikipedia.org/wiki/Telugu_(Unicode_block)
In Unicode.txt:
0C04;TELUGU SIGN COMBINING ANUSVARA ABOVE;Mn;0;NSM;;;;;N;;;;;
In Unicode 15, it has been assigned Other_Alphabetic in PropList.txt
$ grep 0C04 PropList.txt
0C04 ; Other_Alphabetic # Mn TELUGU SIGN COMBINING ANUSVARA
ABOVE
But in Unicode 14 it was not there.
As a result its binary property UCHAR_ALPHABETIC has changed from
false to true in ICU 72 vs previous versions.
As I understand, the stability policy says that such things happen.
From https://www.unicode.org/policies/stability_policy.html
Once a character is encoded, its properties may still be changed,
but not in such a way as to change the fundamental identity of the
character.
The Consortium will endeavor to keep the values of the other
properties as stable as possible, but some circumstances may arise
that require changing them. Particularly in the situation where
the Unicode Standard first encodes less well-documented characters
and scripts, the exact character properties and behavior initially
may not be well known.
As more experience is gathered in implementing the characters,
adjustments in the properties may become necessary. Examples of
such properties include, but are not limited to, the following:
- General_Category
- Case mappings
- Bidirectional properties
[...]
I've commented the exit(1) in category_test to collect all errors, and
built it with versions of ICU from 74 down to 60 (that is Unicode 10.0).
Results are attached. As expected, the older the ICU version, the more
differences are found against Unicode 15.1.
I find these results interesting because they tell us what contents
can break regexp-based check constraints on upgrades.
But about category_test as a pass-or-fail kind of test, it can only be
used when the Unicode version in ICU is the same as in Postgres.
Best regards,
--
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite
Attachments:
[application/octet-stream] results-category-tests-multiple-icu-versions.tar.bz2 (2.1K, ../../[email protected]/2-results-category-tests-multiple-icu-versions.tar.bz2)
download
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-18 12:53 Peter Eisentraut <[email protected]>
parent: Jeff Davis <[email protected]>
3 siblings, 2 replies; 12+ messages in thread
From: Peter Eisentraut @ 2024-01-18 12:53 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On 12.01.24 03:02, Jeff Davis wrote:
> New version attached. Changes:
>
> * Named collation object PG_C_UTF8, which seems like a good idea to
> prevent name conflicts with existing collations. The locale name is
> still C.UTF-8, which still makes sense to me because it matches the
> behavior of the libc locale of the same name so closely.
I am catching up on this thread. The discussions have been very
complicated, so maybe I didn't get it all.
The patches look pretty sound, but I'm questioning how useful this
feature is and where you plan to take it.
Earlier in the thread, the aim was summarized as
> If the Postgres default was bytewise sorting+locale-agnostic
> ctype functions directly derived from Unicode data files,
> as opposed to libc/$LANG at initdb time, the main
> annoyance would be that "ORDER BY textcol" would no
> longer be the human-favored sort.
I think that would be a terrible direction to take, because it would
regress the default sort order from "correct" to "useless". Aside from
the overall message this sends about how PostgreSQL cares about locales
and Unicode and such.
Maybe you don't intend for this to be the default provider? But then
who would really use it? I mean, sure, some people would, but how would
you even explain, in practice, the particular niche of users or use cases?
Maybe if this new provider would be called "minimal", it might describe
the purpose better.
I could see a use for this builtin provider if it also included the
default UCA collation (what COLLATE UNICODE does now). Then it would
provide a "common" default behavior out of the box, and if you want more
fine-tuning, you can go to ICU. There would still be some questions
about making sure the builtin behavior and the ICU behavior are
consistent (different Unicode versions, stock UCA vs CLDR, etc.). But
for practical purposes, it might work.
There would still be a risk with that approach, since it would
permanently marginalize ICU functionality, in the sense that only some
locales would need ICU, and so we might not pay the same amount of
attention to the ICU functionality.
I would be curious what your overall vision is here? Is switching the
default to ICU still your goal? Or do you want the builtin provider to
be the default? Or something else?
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-18 19:42 Daniel Verite <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 0 replies; 12+ messages in thread
From: Daniel Verite @ 2024-01-18 19:42 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Jeff Davis <[email protected]>; Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
Peter Eisentraut wrote:
> > If the Postgres default was bytewise sorting+locale-agnostic
> > ctype functions directly derived from Unicode data files,
> > as opposed to libc/$LANG at initdb time, the main
> > annoyance would be that "ORDER BY textcol" would no
> > longer be the human-favored sort.
>
> I think that would be a terrible direction to take, because it would
> regress the default sort order from "correct" to "useless". Aside from
> the overall message this sends about how PostgreSQL cares about
> locales and Unicode and such.
Well, offering a viable solution to avoid as much as possible
the dreaded:
"WARNING: collation "xyz" has version mismatch
... HINT: Rebuild all objects affected by this collation..."
that doesn't sound like a bad message to send.
Currently, to have in codepoint order the indexes that don't need a
linguistic order, you're supposed to use collate "C", which then means
that upper(), lower() etc.. don't work beyond ASCII.
Here our Unicode support is not good enough, and the proposal
addresses that.
Best regards,
--
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-18 22:03 Jeff Davis <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 1 reply; 12+ messages in thread
From: Jeff Davis @ 2024-01-18 22:03 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On Thu, 2024-01-18 at 13:53 +0100, Peter Eisentraut wrote:
> I think that would be a terrible direction to take, because it would
> regress the default sort order from "correct" to "useless".
I don't agree that the current default is "correct". There are a lot of
ways it can be wrong:
* the environment variables at initdb time don't reflect what the
users of the database actually want
* there are so many different users using so many different
applications connected to the database that no one "correct" sort order
exists
* libc has some implementation quirks
* the version of Unicode that libc is based on is not what you expect
* the version of libc is not what you expect
> Aside from
> the overall message this sends about how PostgreSQL cares about
> locales
> and Unicode and such.
Unicode is primarily about the semantics of characters and their
relationships. The patches I propose here do a great job of that.
Collation (relationships between *strings*) is a part of Unicode, but
not the whole thing or even the main thing.
> Maybe you don't intend for this to be the default provider?
I am not proposing that this provider be the initdb-time default.
> But then
> who would really use it? I mean, sure, some people would, but how
> would
> you even explain, in practice, the particular niche of users or use
> cases?
It's for users who want to respect Unicode support text from
international sources in their database; but are not experts on the
subject and don't know precisely what they want or understand the
consequences. If and when such users do notice a problem with the sort
order, they'd handle it at that time (perhaps with a COLLATE clause, or
sorting in the application).
> Maybe if this new provider would be called "minimal", it might
> describe
> the purpose better.
"Builtin" communicates that it's available everywhere (not a
dependency), that specific behaviors can be documented and tested, and
that behavior doesn't change within a major version. I want to
communicate all of those things.
> I could see a use for this builtin provider if it also included the
> default UCA collation (what COLLATE UNICODE does now).
I won't rule that out, but I'm not proposing that right now and my
proposal is already offering useful functionality.
> There would still be a risk with that approach, since it would
> permanently marginalize ICU functionality
Yeah, ICU already does a good job offering the root collation. I don't
think the builtin provider needs to do so.
> I would be curious what your overall vision is here?
Vision:
* The builtin provider will offer Unicode character semantics, basic
collation, platform-independence, and high performance. It can be used
on its own or in combination with ICU via the COLLATE clause.
* ICU offers COLLATE UNICODE, locale tailoring, case-insensitive
matching, and customization with rules. It's the solution for
everything from "slightly more advanced" to "very advanced".
* libc would be for databases serving applications on the same machine
where a matching sort order is helpful, risks to indexes are
acceptable, and performance is not important.
> Is switching the
> default to ICU still your goal? Or do you want the builtin provider
> to
> be the default?
It's hard to answer this question while initdb chooses the database
default collation based on the environment. Neither ICU nor the builtin
provider can reasonably handle whatever those environment variables
might be set to.
Stepping back from the focus on what initdb does, we should be
providing the right encouragement in documentation and packaging to
guide users toward the right provider based their needs and the vision
outlined above.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-22 18:49 Peter Eisentraut <[email protected]>
parent: Jeff Davis <[email protected]>
0 siblings, 1 reply; 12+ messages in thread
From: Peter Eisentraut @ 2024-01-22 18:49 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On 18.01.24 23:03, Jeff Davis wrote:
> On Thu, 2024-01-18 at 13:53 +0100, Peter Eisentraut wrote:
>> I think that would be a terrible direction to take, because it would
>> regress the default sort order from "correct" to "useless".
>
> I don't agree that the current default is "correct". There are a lot of
> ways it can be wrong:
>
> * the environment variables at initdb time don't reflect what the
> users of the database actually want
> * there are so many different users using so many different
> applications connected to the database that no one "correct" sort order
> exists
> * libc has some implementation quirks
> * the version of Unicode that libc is based on is not what you expect
> * the version of libc is not what you expect
These are arguments why the current defaults are not universally
perfect, but I'd argue that they are still most often the right thing as
the default.
>> Aside from
>> the overall message this sends about how PostgreSQL cares about
>> locales
>> and Unicode and such.
>
> Unicode is primarily about the semantics of characters and their
> relationships. The patches I propose here do a great job of that.
>
> Collation (relationships between *strings*) is a part of Unicode, but
> not the whole thing or even the main thing.
I don't get this argument. Of course, people care about sorting and
sort order. Whether you consider this part of Unicode or adjacent to
it, people still want it.
>> Maybe you don't intend for this to be the default provider?
>
> I am not proposing that this provider be the initdb-time default.
ok
>> But then
>> who would really use it? I mean, sure, some people would, but how
>> would
>> you even explain, in practice, the particular niche of users or use
>> cases?
>
> It's for users who want to respect Unicode support text from
> international sources in their database; but are not experts on the
> subject and don't know precisely what they want or understand the
> consequences. If and when such users do notice a problem with the sort
> order, they'd handle it at that time (perhaps with a COLLATE clause, or
> sorting in the application).
> Vision:
> * ICU offers COLLATE UNICODE, locale tailoring, case-insensitive
> matching, and customization with rules. It's the solution for
> everything from "slightly more advanced" to "very advanced".
I am astonished by this. In your world, do users not want their text
data sorted? Do they not care what the sort order is? You consider UCA
sort order an "advanced" feature?
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-01-22 23:33 Jeff Davis <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 0 replies; 12+ messages in thread
From: Jeff Davis @ 2024-01-22 23:33 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On Mon, 2024-01-22 at 19:49 +0100, Peter Eisentraut wrote:
> >
> I don't get this argument. Of course, people care about sorting and
> sort order. Whether you consider this part of Unicode or adjacent to
> it, people still want it.
You said that my proposal sends a message that we somehow don't care
about Unicode, and I strongly disagree. The built-in provider I'm
proposing does implement Unicode semantics.
Surely a database that offers UCS_BASIC (a SQL spec feature) isn't
sending a message that it doesn't care about Unicode, and neither is my
proposal.
> >
> > * ICU offers COLLATE UNICODE, locale tailoring, case-insensitive
> > matching, and customization with rules. It's the solution for
> > everything from "slightly more advanced" to "very advanced".
>
> I am astonished by this. In your world, do users not want their text
> data sorted? Do they not care what the sort order is?
I obviously care about Unicode and collation. I've put a lot of effort
recently into contributions in this area, and I wouldn't have done that
if I thought users didn't care. You've made much greater contributions
and I thank you for that.
The logical conclusion of your line of argument would be that libc's
"C.UTF-8" locale and UCS_BASIC simply should not exist. But they do
exist, and for good reason.
One of those good reasons is that only *human* users care about the
human-friendliness of sort order. If Postgres is just feeding the
results to another system -- or an application layer that re-sorts the
data anyway -- then stability, performance, and interoperability matter
more than human-friendliness. (Though Unicode character semantics are
still useful even when the data is not going directly to a human.)
> You consider UCA
> sort order an "advanced" feature?
I said "slightly more advanced" compared with "basic". "Advanced" can
be taken in either a positive way ("more useful") or a negative way
("complex"). I'm sorry for the misunderstanding, but my point was this:
* The builtin provider is for people who are fine with code point order
and no tailoring, but want Unicode character semantics, collation
stability, and performance.
* ICU is the right solution for anyone who wants human-friendly
collation or tailoring, and is willing to put up with some collation
stability risk and lower collation performance.
Both have their place and the user is free to mix and match as needed,
thanks to the COLLATE clause for columns and queries.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-02-07 09:53 Peter Eisentraut <[email protected]>
parent: Jeff Davis <[email protected]>
3 siblings, 2 replies; 12+ messages in thread
From: Peter Eisentraut @ 2024-02-07 09:53 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
Review of the v16 patch set:
(Btw., I suppose you started this patch series with 0002 because some
0001 was committed earlier. But I have found this rather confusing. I
think it's ok to renumber from 0001 for each new version.)
* v16-0002-Add-Unicode-property-tables.patch
Various comments are updated to include the term "character class". I
don't recognize that as an official Unicode term. There are categories
and properties. Let's check this.
Some files need heavy pgindent and perltidy. You were surely going to
do this eventually, but maybe you want to do this sooner to check
whether you like the results.
- src/common/unicode/Makefile
This patch series adds some new post-update-unicode tests. Should we
have a separate target for each or just one common "unicode test"
target? Not sure.
- .../generate-unicode_category_table.pl
The trailing commas handling ($firsttime etc.) is not necessary with
C99. The code can be simplified.
For this kind of code:
+print $OT <<"HEADER";
let's use a common marker like EOS instead of a different one for each
block. That just introduces unnecessary variations.
- src/common/unicode_category.c
The mask stuff at the top could use more explanation. It's impossible
to figure out exactly what, say, PG_U_PC_MASK does.
Line breaks in the different pg_u_prop_* functions are gratuitously
different.
Is it potentially confusing that only some pg_u_prop_* have a posix
variant? Would it be better for a consistent interface to have a
"posix" argument for each and just ignore it if not used? Not sure.
Let's use size_t instead of Size for new code.
* v16-0003-Add-unicode-case-mapping-tables-and-functions.patch
Several of the above points apply here analogously.
* v16-0004-Catalog-changes-preparing-for-builtin-collation-.patch
This is mostly a straightforward renaming patch, but there are some
changes in initdb and pg_dump that pre-assume the changes in the next
patch, like which locale columns apply for which providers. I think it
would be better for the historical record to make this a straight
renaming patch and move those semantic changes to the next patch (or a
separate intermediate patch, if you prefer).
- src/bin/psql/describe.c
- src/test/regress/expected/psql.out
This would be a good opportunity to improve the output columns for
collations. The updated view is now:
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules |
Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
This is historically grown but suboptimal. Why is Locale after Collate
and Ctype, and why does it show both? I think we could have just the
Locale column, and if the libc provider is used with different
collate/ctype (very rare!), we somehow write that into the single locale
column.
(A change like this would be a separate patch.)
* v16-0005-Introduce-collation-provider-builtin-for-C-and-C.patch
About this initdb --builtin-locale option and analogous options
elsewhere: Maybe we should flip this around and provide a --libc-locale
option, and have all the other providers just use the --locale option.
This would be more consistent with the fact that it's libc that is
special in this context.
Do we even need the "C" locale? We have established that "C.UTF-8" is
useful, but if that is easily available, who would need "C"?
Some changes in this patch appear to be just straight renamings, like in
src/backend/utils/init/postinit.c and
src/bin/pg_upgrade/t/002_pg_upgrade.pl. Maybe those should be put into
the previous patch instead.
On the collation naming: My expectation would have been that the
"C.UTF-8" locale would be exposed as the UCS_BASIC collation. And the
"C" locale as some other name (or not at all, see above). You have this
the other way around.
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-02-13 02:01 Jeff Davis <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 0 replies; 12+ messages in thread
From: Jeff Davis @ 2024-02-13 02:01 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On Wed, 2024-02-07 at 10:53 +0100, Peter Eisentraut wrote:
> Various comments are updated to include the term "character class".
> I
> don't recognize that as an official Unicode term. There are
> categories
> and properties. Let's check this.
It's based on
https://www.unicode.org/reports/tr18/#Compatibility_Properties
so I suppose the right name is "properties".
> Is it potentially confusing that only some pg_u_prop_* have a posix
> variant? Would it be better for a consistent interface to have a
> "posix" argument for each and just ignore it if not used? Not sure.
I thought about it but didn't see a clear advantage one way or another.
> About this initdb --builtin-locale option and analogous options
> elsewhere: Maybe we should flip this around and provide a --libc-
> locale
> option, and have all the other providers just use the --locale
> option.
> This would be more consistent with the fact that it's libc that is
> special in this context.
Would --libc-locale affect all the environment variables or just
LC_CTYPE/LC_COLLATE? How do we avoid breakage?
I like the general direction here but we might need to phase in the
option or come up with a new name. Suggestions welcome.
> Do we even need the "C" locale? We have established that "C.UTF-8"
> is
> useful, but if that is easily available, who would need "C"?
I don't think we should encourage its use generally but I also don't
think it will disappear any time soon. Some people will want it on
simplicity grounds. I hope fewer people will use "C" when we have a
better builtin option.
> Some changes in this patch appear to be just straight renamings, like
> in
> src/backend/utils/init/postinit.c and
> src/bin/pg_upgrade/t/002_pg_upgrade.pl. Maybe those should be put
> into
> the previous patch instead.
>
> On the collation naming: My expectation would have been that the
> "C.UTF-8" locale would be exposed as the UCS_BASIC collation.
I'd like that. We have to sort out a couple things first, though:
1. The SQL spec mentions the capitalization of "ß" as "SS"
specifically. Should UCS_BASIC use the unconditional mappings in
SpecialCasing.txt? I already have some code to do that (not posted
yet).
2. Should UCS_BASIC use the "POSIX" or "Standard" variant of regex
properties? (The main difference seems to be whether symbols get
treated as punctuation or not.)
3. What do we do about potential breakage for existing users of
UCS_BASIC who might be expecting C-like behavior?
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 12+ messages in thread
* Re: Built-in CTYPE provider
@ 2024-02-16 00:13 Jeff Davis <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 0 replies; 12+ messages in thread
From: Jeff Davis @ 2024-02-16 00:13 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; Daniel Verite <[email protected]>; +Cc: Robert Haas <[email protected]>; Jeremy Schneider <[email protected]>; pgsql-hackers
On Wed, 2024-02-07 at 10:53 +0100, Peter Eisentraut wrote:
> Review of the v16 patch set:
>
> (Btw., I suppose you started this patch series with 0002 because some
> 0001 was committed earlier. But I have found this rather confusing.
> I
> think it's ok to renumber from 0001 for each new version.)
Fixed.
> Various comments are updated to include the term "character class".
> I
> don't recognize that as an official Unicode term. There are
> categories
> and properties. Let's check this.
Changed to "properties" or "compatibility properties", except for a
couple places in the test. The test compares against ICU, which does
use the term "character classes" when discussing regexes:
https://unicode-org.github.io/icu-docs/apidoc/dev/icu4c/uchar_8h.html#details
> Some files need heavy pgindent and perltidy.
Done.
> This patch series adds some new post-update-unicode tests. Should we
> have a separate target for each or just one common "unicode test"
> target? Not sure.
I didn't make a change here. I suspect anyone updating unicode would
want to run them all, but I don't have a strong opinion.
> - .../generate-unicode_category_table.pl
>
> The trailing commas handling ($firsttime etc.) is not necessary with
> C99. The code can be simplified.
Thank you, fixed.
> For this kind of code:
>
> +print $OT <<"HEADER";
Done. I used the <<"EOS" style which is more friendly to emacs, but I'm
not sure if that's right for the project style.
> Is it potentially confusing that only some pg_u_prop_* have a posix
> variant? Would it be better for a consistent interface to have a
> "posix" argument for each and just ignore it if not used? Not sure.
I don't have a strong opinion here, so I didn't make a change. I can if
you think it's cleaner.
> Let's use size_t instead of Size for new code.
Done.
> * v16-0003-Add-unicode-case-mapping-tables-and-functions.patch
>
> Several of the above points apply here analogously.
Fixed, I think.
> * v16-0004-Catalog-changes-preparing-for-builtin-collation-.patch
>
> This is mostly a straightforward renaming patch, but there are some
> changes in initdb and pg_dump that pre-assume the changes in the next
> patch, like which locale columns apply for which providers. I think
> it
> would be better for the historical record to make this a straight
> renaming patch and move those semantic changes to the next patch (or
> a
> separate intermediate patch, if you prefer).
Agreed, put those non-renaming changes in the next patch.
> - src/bin/psql/describe.c
> - src/test/regress/expected/psql.out
>
> This would be a good opportunity to improve the output columns for
> collations. The updated view is now:
>
> + Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules |
> Deterministic?
> +--------+------+----------+---------+-------+--------+-----------+--
> --------------
>
> This is historically grown but suboptimal. Why is Locale after
> Collate
> and Ctype, and why does it show both? I think we could have just the
> Locale column, and if the libc provider is used with different
> collate/ctype (very rare!), we somehow write that into the single
> locale
> column.
>
> (A change like this would be a separate patch.)
I didn't do this, yet.
> * v16-0005-Introduce-collation-provider-builtin-for-C-and-C.patch
>
> About this initdb --builtin-locale option and analogous options
> elsewhere: Maybe we should flip this around and provide a --libc-
> locale
> option, and have all the other providers just use the --locale
> option.
> This would be more consistent with the fact that it's libc that is
> special in this context.
I agree that libc is the odd one out. I'm not quite sure how we should
express that, though, because there are also the other environment
variables to worry about (e.g. LC_MESSAGES). Probably best as a
separate patch.
> Some changes in this patch appear to be just straight renamings, like
> in
> src/backend/utils/init/postinit.c and
> src/bin/pg_upgrade/t/002_pg_upgrade.pl. Maybe those should be put
> into
> the previous patch instead.
Moved renamings to the previous patch.
Regards,
Jeff Davis
Attachments:
[text/x-patch] v17-0001-Add-Unicode-property-tables.patch (92.8K, ../../[email protected]/2-v17-0001-Add-Unicode-property-tables.patch)
download | inline diff:
From 59500e0cece10c0e3a70545081cf163a01f94120 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Sat, 18 Nov 2023 15:34:24 -0800
Subject: [PATCH v17 1/4] Add Unicode property tables.
Provide functions to test for regex character classes (e.g. 'alpha',
'punct') using Unicode properties.
---
src/common/unicode/Makefile | 6 +-
src/common/unicode/category_test.c | 215 +-
.../generate-unicode_category_table.pl | 289 +-
src/common/unicode/meson.build | 4 +-
src/common/unicode_category.c | 271 +-
src/include/common/unicode_category.h | 25 +-
src/include/common/unicode_category_table.h | 2534 ++++++++++++++++-
7 files changed, 3250 insertions(+), 94 deletions(-)
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 04d81dd5cb..27f0408d8b 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -29,13 +29,13 @@ update-unicode: unicode_category_table.h unicode_east_asian_fw_table.h unicode_n
# These files are part of the Unicode Character Database. Download
# them on demand. The dependency on Makefile.global is for
# UNICODE_VERSION.
-CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt: $(top_builddir)/src/Makefile.global
+CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt: $(top_builddir)/src/Makefile.global
$(DOWNLOAD) https://www.unicode.org/Public/$(UNICODE_VERSION)/ucd/$(@F)
unicode_version.h: generate-unicode_version.pl
$(PERL) $< --version $(UNICODE_VERSION)
-unicode_category_table.h: generate-unicode_category_table.pl UnicodeData.txt
+unicode_category_table.h: generate-unicode_category_table.pl DerivedCoreProperties.txt PropList.txt UnicodeData.txt
$(PERL) $<
# Generation of conversion tables used for string normalization with
@@ -82,4 +82,4 @@ clean:
rm -f $(OBJS) category_test category_test.o norm_test norm_test.o
distclean: clean
- rm -f CompositionExclusions.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
+ rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
diff --git a/src/common/unicode/category_test.c b/src/common/unicode/category_test.c
index f1aaac0f61..63e664e3f2 100644
--- a/src/common/unicode/category_test.c
+++ b/src/common/unicode/category_test.c
@@ -1,6 +1,6 @@
/*-------------------------------------------------------------------------
* category_test.c
- * Program to test Unicode general category functions.
+ * Program to test Unicode general category and character properties.
*
* Portions Copyright (c) 2017-2024, PostgreSQL Global Development Group
*
@@ -14,17 +14,22 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
-
#ifdef USE_ICU
#include <unicode/uchar.h>
#endif
-#include "common/unicode_category.h"
+#include <wctype.h>
+
#include "common/unicode_version.h"
+#include "common/unicode_category.h"
+
+static int pg_unicode_version = 0;
+#ifdef USE_ICU
+static int icu_unicode_version = 0;
+#endif
/*
* Parse version into integer for easy comparison.
*/
-#ifdef USE_ICU
static int
parse_unicode_version(const char *version)
{
@@ -39,57 +44,165 @@ parse_unicode_version(const char *version)
return major * 100 + minor;
}
-#endif
+#ifdef USE_ICU
/*
- * Exhaustively test that the Unicode category for each codepoint matches that
- * returned by ICU.
+ * Test Postgres Unicode tables by comparing with ICU. Test the General
+ * Category, as well as the properties Alphabetic, Lowercase, Uppercase,
+ * White_Space, and Hex_Digit.
*/
-int
-main(int argc, char **argv)
+static void
+icu_test()
{
-#ifdef USE_ICU
- int pg_unicode_version = parse_unicode_version(PG_UNICODE_VERSION);
- int icu_unicode_version = parse_unicode_version(U_UNICODE_VERSION);
+ int successful = 0;
int pg_skipped_codepoints = 0;
int icu_skipped_codepoints = 0;
- printf("category_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
- printf("category_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
-
- for (UChar32 code = 0; code <= 0x10ffff; code++)
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
{
uint8_t pg_category = unicode_category(code);
uint8_t icu_category = u_charType(code);
+ /* Property tests */
+ bool prop_alphabetic = pg_u_prop_alphabetic(code);
+ bool prop_lowercase = pg_u_prop_lowercase(code);
+ bool prop_uppercase = pg_u_prop_uppercase(code);
+ bool prop_white_space = pg_u_prop_white_space(code);
+ bool prop_hex_digit = pg_u_prop_hex_digit(code);
+ bool prop_join_control = pg_u_prop_join_control(code);
+
+ bool icu_prop_alphabetic = u_hasBinaryProperty(
+ code, UCHAR_ALPHABETIC);
+ bool icu_prop_lowercase = u_hasBinaryProperty(
+ code, UCHAR_LOWERCASE);
+ bool icu_prop_uppercase = u_hasBinaryProperty(
+ code, UCHAR_UPPERCASE);
+ bool icu_prop_white_space = u_hasBinaryProperty(
+ code, UCHAR_WHITE_SPACE);
+ bool icu_prop_hex_digit = u_hasBinaryProperty(
+ code, UCHAR_HEX_DIGIT);
+ bool icu_prop_join_control = u_hasBinaryProperty(
+ code, UCHAR_JOIN_CONTROL);
+
+ /*
+ * Compare with ICU for character classes using:
+ *
+ * https://unicode-org.github.io/icu-docs/apidoc/dev/icu4c/uchar_8h.html#details
+ *
+ * which describes how to use ICU to test for membership in regex
+ * character classes.
+ *
+ * NB: the document suggests testing for some properties such as
+ * UCHAR_POSIX_ALNUM, but that doesn't mean that we're testing for the
+ * "POSIX Compatible" character classes.
+ */
+ bool isalpha = pg_u_isalpha(code);
+ bool islower = pg_u_islower(code);
+ bool isupper = pg_u_isupper(code);
+ bool ispunct = pg_u_ispunct(code, false);
+ bool isdigit = pg_u_isdigit(code, false);
+ bool isxdigit = pg_u_isxdigit(code, false);
+ bool isalnum = pg_u_isalnum(code, false);
+ bool isspace = pg_u_isspace(code);
+ bool isblank = pg_u_isblank(code);
+ bool iscntrl = pg_u_iscntrl(code);
+ bool isgraph = pg_u_isgraph(code);
+ bool isprint = pg_u_isprint(code);
+
+ bool icu_isalpha = u_isUAlphabetic(code);
+ bool icu_islower = u_isULowercase(code);
+ bool icu_isupper = u_isUUppercase(code);
+ bool icu_ispunct = u_ispunct(code);
+ bool icu_isdigit = u_isdigit(code);
+ bool icu_isxdigit = u_hasBinaryProperty(code,
+ UCHAR_POSIX_XDIGIT);
+ bool icu_isalnum = u_hasBinaryProperty(code,
+ UCHAR_POSIX_ALNUM);
+ bool icu_isspace = u_isUWhiteSpace(code);
+ bool icu_isblank = u_isblank(code);
+ bool icu_iscntrl = icu_category == PG_U_CONTROL;
+ bool icu_isgraph = u_hasBinaryProperty(code,
+ UCHAR_POSIX_GRAPH);
+ bool icu_isprint = u_hasBinaryProperty(code,
+ UCHAR_POSIX_PRINT);
+
+ /*
+ * A version mismatch means that some assigned codepoints in the newer
+ * version may be unassigned in the older version. That's OK, though
+ * the test will not cover those codepoints marked unassigned in the
+ * older version (that is, it will no longer be an exhaustive test).
+ */
+ if (pg_category == PG_U_UNASSIGNED &&
+ icu_category != PG_U_UNASSIGNED &&
+ pg_unicode_version < icu_unicode_version)
+ {
+ pg_skipped_codepoints++;
+ continue;
+ }
+
+ if (icu_category == PG_U_UNASSIGNED &&
+ pg_category != PG_U_UNASSIGNED &&
+ icu_unicode_version < pg_unicode_version)
+ {
+ icu_skipped_codepoints++;
+ continue;
+ }
+
if (pg_category != icu_category)
{
- /*
- * A version mismatch means that some assigned codepoints in the
- * newer version may be unassigned in the older version. That's
- * OK, though the test will not cover those codepoints marked
- * unassigned in the older version (that is, it will no longer be
- * an exhaustive test).
- */
- if (pg_category == PG_U_UNASSIGNED &&
- pg_unicode_version < icu_unicode_version)
- pg_skipped_codepoints++;
- else if (icu_category == PG_U_UNASSIGNED &&
- icu_unicode_version < pg_unicode_version)
- icu_skipped_codepoints++;
- else
- {
- printf("category_test: FAILURE for codepoint 0x%06x\n", code);
- printf("category_test: Postgres category: %02d %s %s\n", pg_category,
- unicode_category_abbrev(pg_category),
- unicode_category_string(pg_category));
- printf("category_test: ICU category: %02d %s %s\n", icu_category,
- unicode_category_abbrev(icu_category),
- unicode_category_string(icu_category));
- printf("\n");
- exit(1);
- }
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres category: %02d %s %s\n", pg_category,
+ unicode_category_abbrev(pg_category),
+ unicode_category_string(pg_category));
+ printf("category_test: ICU category: %02d %s %s\n", icu_category,
+ unicode_category_abbrev(icu_category),
+ unicode_category_string(icu_category));
+ printf("\n");
+ exit(1);
}
+
+ if (prop_alphabetic != icu_prop_alphabetic ||
+ prop_lowercase != icu_prop_lowercase ||
+ prop_uppercase != icu_prop_uppercase ||
+ prop_white_space != icu_prop_white_space ||
+ prop_hex_digit != icu_prop_hex_digit ||
+ prop_join_control != icu_prop_join_control)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres property alphabetic/lowercase/uppercase/white_space/hex_digit/join_control: %d/%d/%d/%d/%d/%d\n",
+ prop_alphabetic, prop_lowercase, prop_uppercase,
+ prop_white_space, prop_hex_digit, prop_join_control);
+ printf("category_test: ICU property alphabetic/lowercase/uppercase/white_space/hex_digit/join_control: %d/%d/%d/%d/%d/%d\n",
+ icu_prop_alphabetic, icu_prop_lowercase, icu_prop_uppercase,
+ icu_prop_white_space, icu_prop_hex_digit, icu_prop_join_control);
+ printf("\n");
+ exit(1);
+ }
+
+ if (isalpha != icu_isalpha ||
+ islower != icu_islower ||
+ isupper != icu_isupper ||
+ ispunct != icu_ispunct ||
+ isdigit != icu_isdigit ||
+ isxdigit != icu_isxdigit ||
+ isalnum != icu_isalnum ||
+ isspace != icu_isspace ||
+ isblank != icu_isblank ||
+ iscntrl != icu_iscntrl ||
+ isgraph != icu_isgraph ||
+ isprint != icu_isprint)
+ {
+ printf("category_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("category_test: Postgres class alpha/lower/upper/punct/digit/xdigit/alnum/space/blank/cntrl/graph/print: %d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d\n",
+ isalpha, islower, isupper, ispunct, isdigit, isxdigit, isalnum, isspace, isblank, iscntrl, isgraph, isprint);
+ printf("category_test: ICU class alpha/lower/upper/punct/digit/xdigit/alnum/space/blank/cntrl/graph/print: %d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d/%d\n",
+ icu_isalpha, icu_islower, icu_isupper, icu_ispunct, icu_isdigit, icu_isxdigit, icu_isalnum, icu_isspace, icu_isblank, icu_iscntrl, icu_isgraph, icu_isprint);
+ printf("\n");
+ exit(1);
+ }
+
+ if (pg_category != PG_U_UNASSIGNED)
+ successful++;
}
if (pg_skipped_codepoints > 0)
@@ -99,10 +212,22 @@ main(int argc, char **argv)
printf("category_test: skipped %d codepoints unassigned in ICU due to Unicode version mismatch\n",
icu_skipped_codepoints);
- printf("category_test: success\n");
- exit(0);
+ printf("category_test: ICU test: %d codepoints successful\n", successful);
+}
+#endif
+
+int
+main(int argc, char **argv)
+{
+ pg_unicode_version = parse_unicode_version(PG_UNICODE_VERSION);
+ printf("category_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
+
+#ifdef USE_ICU
+ icu_unicode_version = parse_unicode_version(U_UNICODE_VERSION);
+ printf("category_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
+
+ icu_test();
#else
- printf("category_test: ICU support required for test; skipping\n");
- exit(0);
+ printf("category_test: ICU not available; skipping\n");
#endif
}
diff --git a/src/common/unicode/generate-unicode_category_table.pl b/src/common/unicode/generate-unicode_category_table.pl
index a50c87b7e9..7376eedb62 100644
--- a/src/common/unicode/generate-unicode_category_table.pl
+++ b/src/common/unicode/generate-unicode_category_table.pl
@@ -48,21 +48,30 @@ while (my $line = <$FH>)
my $category = $elts[2];
die "codepoint out of range" if $code > 0x10FFFF;
- die "unassigned codepoint in UnicodeData.txt" if $category eq $CATEGORY_UNASSIGNED;
+ die "unassigned codepoint in UnicodeData.txt"
+ if $category eq $CATEGORY_UNASSIGNED;
- if (!defined($range_start)) {
+ if (!defined($range_start))
+ {
my $code_str = sprintf "0x%06x", $code;
- die if defined($range_end) || defined($range_category) || defined($gap_category);
+ die
+ if defined($range_end)
+ || defined($range_category)
+ || defined($gap_category);
die "unexpected first entry <..., Last>" if ($name =~ /Last>/);
- die "expected 0x000000 for first entry, got $code_str" if $code != 0x000000;
+ die "expected 0x000000 for first entry, got $code_str"
+ if $code != 0x000000;
# initialize
$range_start = $code;
$range_end = $code;
$range_category = $category;
- if ($name =~ /<.*, First>$/) {
+ if ($name =~ /<.*, First>$/)
+ {
$gap_category = $category;
- } else {
+ }
+ else
+ {
$gap_category = $CATEGORY_UNASSIGNED;
}
next;
@@ -71,10 +80,17 @@ while (my $line = <$FH>)
# Gap in codepoints detected. If it's a different category than
# the current range, emit the current range and initialize a new
# range representing the gap.
- if ($range_end + 1 != $code && $range_category ne $gap_category) {
- if ($range_category ne $CATEGORY_UNASSIGNED) {
- push(@category_ranges, {start => $range_start, end => $range_end,
- category => $range_category});
+ if ($range_end + 1 != $code && $range_category ne $gap_category)
+ {
+ if ($range_category ne $CATEGORY_UNASSIGNED)
+ {
+ push(
+ @category_ranges,
+ {
+ start => $range_start,
+ end => $range_end,
+ category => $range_category
+ });
}
$range_start = $range_end + 1;
$range_end = $code - 1;
@@ -82,27 +98,39 @@ while (my $line = <$FH>)
}
# different category; new range
- if ($range_category ne $category) {
- if ($range_category ne $CATEGORY_UNASSIGNED) {
- push(@category_ranges, {start => $range_start, end => $range_end,
- category => $range_category});
+ if ($range_category ne $category)
+ {
+ if ($range_category ne $CATEGORY_UNASSIGNED)
+ {
+ push(
+ @category_ranges,
+ {
+ start => $range_start,
+ end => $range_end,
+ category => $range_category
+ });
}
$range_start = $code;
$range_end = $code;
$range_category = $category;
}
- if ($name =~ /<.*, First>$/) {
- die "<..., First> entry unexpectedly follows another <..., First> entry"
+ if ($name =~ /<.*, First>$/)
+ {
+ die
+ "<..., First> entry unexpectedly follows another <..., First> entry"
if $gap_category ne $CATEGORY_UNASSIGNED;
$gap_category = $category;
}
- elsif ($name =~ /<.*, Last>$/) {
- die "<..., First> and <..., Last> entries have mismatching general category"
+ elsif ($name =~ /<.*, Last>$/)
+ {
+ die
+ "<..., First> and <..., Last> entries have mismatching general category"
if $gap_category ne $category;
$gap_category = $CATEGORY_UNASSIGNED;
}
- else {
+ else
+ {
die "unexpected entry found between <..., First> and <..., Last>"
if $gap_category ne $CATEGORY_UNASSIGNED;
}
@@ -115,13 +143,17 @@ die "<..., First> entry with no corresponding <..., Last> entry"
if $gap_category ne $CATEGORY_UNASSIGNED;
# emit final range
-if ($range_category ne $CATEGORY_UNASSIGNED) {
- push(@category_ranges, {start => $range_start, end => $range_end,
- category => $range_category});
+if ($range_category ne $CATEGORY_UNASSIGNED)
+{
+ push(
+ @category_ranges,
+ {
+ start => $range_start,
+ end => $range_end,
+ category => $range_category
+ });
}
-my $num_ranges = scalar @category_ranges;
-
# See: https://www.unicode.org/reports/tr44/#General_Category_Values
my $categories = {
Cn => 'PG_U_UNASSIGNED',
@@ -156,11 +188,110 @@ my $categories = {
Pf => 'PG_U_FINAL_PUNCTUATION'
};
-# Start writing out the output files
+# Find White_Space and Hex_Digit characters
+my @white_space = ();
+my @hex_digits = ();
+my @join_control = ();
+open($FH, '<', "$output_path/PropList.txt")
+ or die "Could not open $output_path/PropList.txt: $!.";
+while (my $line = <$FH>)
+{
+ my $pattern = qr/([0-9A-F\.]+)\s*;\s*(\w+)\s*#.*/s;
+ next unless $line =~ $pattern;
+
+ my $code = $line =~ s/$pattern/$1/rg;
+ my $property = $line =~ s/$pattern/$2/rg;
+ my $start;
+ my $end;
+
+ if ($code =~ /\.\./)
+ {
+ # code range
+ my @sp = split /\.\./, $code;
+ $start = hex($sp[0]);
+ $end = hex($sp[1]);
+ }
+ else
+ {
+ # single code point
+ $start = hex($code);
+ $end = hex($code);
+ }
+
+ if ($property eq "White_Space")
+ {
+ push @white_space, { start => $start, end => $end };
+ }
+ elsif ($property eq "Hex_Digit")
+ {
+ push @hex_digits, { start => $start, end => $end };
+ }
+ elsif ($property eq "Join_Control")
+ {
+ push @join_control, { start => $start, end => $end };
+ }
+}
+
+# Find Alphabetic, Lowercase, and Uppercase characters
+my @alphabetic = ();
+my @lowercase = ();
+my @uppercase = ();
+open($FH, '<', "$output_path/DerivedCoreProperties.txt")
+ or die "Could not open $output_path/DerivedCoreProperties.txt: $!.";
+while (my $line = <$FH>)
+{
+ my $pattern = qr/^([0-9A-F\.]+)\s*;\s*(\w+)\s*#.*$/s;
+ next unless $line =~ $pattern;
+
+ my $code = $line =~ s/$pattern/$1/rg;
+ my $property = $line =~ s/$pattern/$2/rg;
+ my $start;
+ my $end;
+
+ if ($code =~ /\.\./)
+ {
+ # code range
+ my @sp = split /\.\./, $code;
+ die "line: {$line} code: {$code} sp[0] {$sp[0]} sp[1] {$sp[1]}"
+ unless $sp[0] =~ /^[0-9A-F]+$/ && $sp[1] =~ /^[0-9A-F]+$/;
+ $start = hex($sp[0]);
+ $end = hex($sp[1]);
+ }
+ else
+ {
+ die "line: {$line} code: {$code}" unless $code =~ /^[0-9A-F]+$/;
+ # single code point
+ $start = hex($code);
+ $end = hex($code);
+ }
+
+ if ($property eq "Alphabetic")
+ {
+ push @alphabetic, { start => $start, end => $end };
+ }
+ elsif ($property eq "Lowercase")
+ {
+ push @lowercase, { start => $start, end => $end };
+ }
+ elsif ($property eq "Uppercase")
+ {
+ push @uppercase, { start => $start, end => $end };
+ }
+}
+
+my $num_category_ranges = scalar @category_ranges;
+my $num_alphabetic_ranges = scalar @alphabetic;
+my $num_lowercase_ranges = scalar @lowercase;
+my $num_uppercase_ranges = scalar @uppercase;
+my $num_white_space_ranges = scalar @white_space;
+my $num_hex_digit_ranges = scalar @hex_digits;
+my $num_join_control_ranges = scalar @join_control;
+
+# Start writing out the output file
open my $OT, '>', $output_table_file
or die "Could not open output file $output_table_file: $!\n";
-print $OT <<HEADER;
+print $OT <<"EOS";
/*-------------------------------------------------------------------------
*
* unicode_category_table.h
@@ -188,18 +319,104 @@ typedef struct
uint8 category; /* General Category */
} pg_category_range;
-/* table of Unicode codepoint ranges and their categories */
-static const pg_category_range unicode_categories[$num_ranges] =
+typedef struct
{
-HEADER
+ uint32 first; /* Unicode codepoint */
+ uint32 last; /* Unicode codepoint */
+} pg_unicode_range;
-my $firsttime = 1;
-foreach my $range (@category_ranges) {
- printf $OT ",\n" unless $firsttime;
- $firsttime = 0;
+EOS
- my $category = $categories->{$range->{category}};
+print $OT <<"EOS";
+/* table of Unicode codepoint ranges and their categories */
+static const pg_category_range unicode_categories[$num_category_ranges] =
+{
+EOS
+
+foreach my $range (@category_ranges)
+{
+ my $category = $categories->{ $range->{category} };
die "category missing: $range->{category}" unless $category;
- printf $OT "\t{0x%06x, 0x%06x, %s}", $range->{start}, $range->{end}, $category;
+ printf $OT "\t{0x%06x, 0x%06x, %s},\n", $range->{start}, $range->{end},
+ $category;
}
-print $OT "\n};\n";
+
+print $OT "};\n\n";
+
+print $OT <<"EOS";
+/* table of Unicode codepoint ranges of Alphabetic characters */
+static const pg_unicode_range unicode_alphabetic[$num_alphabetic_ranges] =
+{
+EOS
+
+foreach my $range (@alphabetic)
+{
+ printf $OT "\t{0x%06x, 0x%06x},\n", $range->{start}, $range->{end};
+}
+
+print $OT "};\n\n";
+
+print $OT <<"EOS";
+/* table of Unicode codepoint ranges of Lowercase characters */
+static const pg_unicode_range unicode_lowercase[$num_lowercase_ranges] =
+{
+EOS
+
+foreach my $range (@lowercase)
+{
+ printf $OT "\t{0x%06x, 0x%06x},\n", $range->{start}, $range->{end};
+}
+
+print $OT "};\n\n";
+
+print $OT <<"EOS";
+/* table of Unicode codepoint ranges of Uppercase characters */
+static const pg_unicode_range unicode_uppercase[$num_uppercase_ranges] =
+{
+EOS
+
+foreach my $range (@uppercase)
+{
+ printf $OT "\t{0x%06x, 0x%06x},\n", $range->{start}, $range->{end};
+}
+
+print $OT "};\n\n";
+
+print $OT <<"EOS";
+/* table of Unicode codepoint ranges of White_Space characters */
+static const pg_unicode_range unicode_white_space[$num_white_space_ranges] =
+{
+EOS
+
+foreach my $range (@white_space)
+{
+ printf $OT "\t{0x%06x, 0x%06x},\n", $range->{start}, $range->{end};
+}
+
+print $OT "};\n\n";
+
+print $OT <<"EOS";
+/* table of Unicode codepoint ranges of Hex_Digit characters */
+static const pg_unicode_range unicode_hex_digit[$num_hex_digit_ranges] =
+{
+EOS
+
+foreach my $range (@hex_digits)
+{
+ printf $OT "\t{0x%06x, 0x%06x},\n", $range->{start}, $range->{end};
+}
+
+print $OT "};\n\n";
+
+print $OT <<"EOS";
+/* table of Unicode codepoint ranges of Join_Control characters */
+static const pg_unicode_range unicode_join_control[$num_join_control_ranges] =
+{
+EOS
+
+foreach my $range (@join_control)
+{
+ printf $OT "\t{0x%06x, 0x%06x},\n", $range->{start}, $range->{end};
+}
+
+print $OT "};\n";
diff --git a/src/common/unicode/meson.build b/src/common/unicode/meson.build
index df4f3a4ed1..d7190bb8ca 100644
--- a/src/common/unicode/meson.build
+++ b/src/common/unicode/meson.build
@@ -11,7 +11,7 @@ endif
# These files are part of the Unicode Character Database. Download them on
# demand.
-foreach f : ['CompositionExclusions.txt', 'DerivedNormalizationProps.txt', 'EastAsianWidth.txt', 'NormalizationTest.txt', 'UnicodeData.txt']
+foreach f : ['CompositionExclusions.txt', 'DerivedCoreProperties.txt', 'DerivedNormalizationProps.txt', 'EastAsianWidth.txt', 'NormalizationTest.txt', 'PropList.txt', 'UnicodeData.txt']
url = unicode_baseurl.format(UNICODE_VERSION, f)
target = custom_target(f,
output: f,
@@ -26,7 +26,7 @@ update_unicode_targets = []
update_unicode_targets += \
custom_target('unicode_category_table.h',
- input: [unicode_data['UnicodeData.txt']],
+ input: [unicode_data['UnicodeData.txt'], unicode_data['DerivedCoreProperties.txt'], unicode_data['PropList.txt']],
output: ['unicode_category_table.h'],
command: [
perl, files('generate-unicode_category_table.pl'),
diff --git a/src/common/unicode_category.c b/src/common/unicode_category.c
index 668051b461..c3bed221de 100644
--- a/src/common/unicode_category.c
+++ b/src/common/unicode_category.c
@@ -1,6 +1,8 @@
/*-------------------------------------------------------------------------
* unicode_category.c
- * Determine general category of Unicode characters.
+ * Determine general category and character properties of Unicode
+ * characters. Encoding must be UTF8, where we assume that the pg_wchar
+ * representation is a code point.
*
* Portions Copyright (c) 2017-2024, PostgreSQL Global Development Group
*
@@ -18,24 +20,82 @@
#include "common/unicode_category.h"
#include "common/unicode_category_table.h"
+/*
+ * Create bitmasks from pg_unicode_category values for efficient comparison of
+ * multiple categories. For instance, PG_U_MN_MASK is a bitmask representing
+ * the general cateogry Mn; and PG_U_M_MASK represents general categories Mn,
+ * Me, and Mc.
+ *
+ * The number of Unicode General Categories should never grow, so a 32-bit
+ * mask is fine.
+ */
+#define PG_U_CATEGORY_MASK(X) ((uint32)(1 << (X)))
+
+#define PG_U_LU_MASK PG_U_CATEGORY_MASK(PG_U_UPPERCASE_LETTER)
+#define PG_U_LL_MASK PG_U_CATEGORY_MASK(PG_U_LOWERCASE_LETTER)
+#define PG_U_LT_MASK PG_U_CATEGORY_MASK(PG_U_TITLECASE_LETTER)
+#define PG_U_LC_MASK (PG_U_LU_MASK|PG_U_LL_MASK|PG_U_LT_MASK)
+#define PG_U_LM_MASK PG_U_CATEGORY_MASK(PG_U_MODIFIER_LETTER)
+#define PG_U_LO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_LETTER)
+#define PG_U_L_MASK (PG_U_LU_MASK|PG_U_LL_MASK|PG_U_LT_MASK|PG_U_LM_MASK|\
+ PG_U_LO_MASK)
+#define PG_U_MN_MASK PG_U_CATEGORY_MASK(PG_U_NONSPACING_MARK)
+#define PG_U_ME_MASK PG_U_CATEGORY_MASK(PG_U_ENCLOSING_MARK)
+#define PG_U_MC_MASK PG_U_CATEGORY_MASK(PG_U_SPACING_MARK)
+#define PG_U_M_MASK (PG_U_MN_MASK|PG_U_MC_MASK|PG_U_ME_MASK)
+#define PG_U_ND_MASK PG_U_CATEGORY_MASK(PG_U_DECIMAL_NUMBER)
+#define PG_U_NL_MASK PG_U_CATEGORY_MASK(PG_U_LETTER_NUMBER)
+#define PG_U_NO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_NUMBER)
+#define PG_U_N_MASK (PG_U_ND_MASK|PG_U_NL_MASK|PG_U_NO_MASK)
+#define PG_U_PC_MASK PG_U_CATEGORY_MASK(PG_U_CONNECTOR_PUNCTUATION)
+#define PG_U_PD_MASK PG_U_CATEGORY_MASK(PG_U_DASH_PUNCTUATION)
+#define PG_U_PS_MASK PG_U_CATEGORY_MASK(PG_U_OPEN_PUNCTUATION)
+#define PG_U_PE_MASK PG_U_CATEGORY_MASK(PG_U_CLOSE_PUNCTUATION)
+#define PG_U_PI_MASK PG_U_CATEGORY_MASK(PG_U_INITIAL_PUNCTUATION)
+#define PG_U_PF_MASK PG_U_CATEGORY_MASK(PG_U_FINAL_PUNCTUATION)
+#define PG_U_PO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_PUNCTUATION)
+#define PG_U_P_MASK (PG_U_PC_MASK|PG_U_PD_MASK|PG_U_PS_MASK|PG_U_PE_MASK|\
+ PG_U_PI_MASK|PG_U_PF_MASK|PG_U_PO_MASK)
+#define PG_U_SM_MASK PG_U_CATEGORY_MASK(PG_U_MATH_SYMBOL)
+#define PG_U_SC_MASK PG_U_CATEGORY_MASK(PG_U_CURRENCY_SYMBOL)
+#define PG_U_SK_MASK PG_U_CATEGORY_MASK(PG_U_MODIFIER_SYMBOL)
+#define PG_U_SO_MASK PG_U_CATEGORY_MASK(PG_U_OTHER_SYMBOL)
+#define PG_U_S_MASK (PG_U_SM_MASK|PG_U_SC_MASK|PG_U_SK_MASK|PG_U_SO_MASK)
+#define PG_U_ZS_MASK PG_U_CATEGORY_MASK(PG_U_SPACE_SEPARATOR)
+#define PG_U_ZL_MASK PG_U_CATEGORY_MASK(PG_U_LINE_SEPARATOR)
+#define PG_U_ZP_MASK PG_U_CATEGORY_MASK(PG_U_PARAGRAPH_SEPARATOR)
+#define PG_U_Z_MASK (PG_U_ZS_MASK|PG_U_ZL_MASK|PG_U_ZP_MASK)
+#define PG_U_CC_MASK PG_U_CATEGORY_MASK(PG_U_CONTROL)
+#define PG_U_CF_MASK PG_U_CATEGORY_MASK(PG_U_FORMAT)
+#define PG_U_CS_MASK PG_U_CATEGORY_MASK(PG_U_SURROGATE)
+#define PG_U_CO_MASK PG_U_CATEGORY_MASK(PG_U_PRIVATE_USE)
+#define PG_U_CN_MASK PG_U_CATEGORY_MASK(PG_U_UNASSIGNED)
+#define PG_U_C_MASK (PG_U_CC_MASK|PG_U_CF_MASK|PG_U_CS_MASK|PG_U_CO_MASK|\
+ PG_U_CN_MASK)
+
+#define PG_U_CHARACTER_TAB 0x09
+
+static bool range_search(const pg_unicode_range * tbl, size_t size,
+ pg_wchar code);
+
/*
* Unicode general category for the given codepoint.
*/
pg_unicode_category
-unicode_category(pg_wchar ucs)
+unicode_category(pg_wchar code)
{
int min = 0;
int mid;
int max = lengthof(unicode_categories) - 1;
- Assert(ucs <= 0x10ffff);
+ Assert(code <= 0x10ffff);
while (max >= min)
{
mid = (min + max) / 2;
- if (ucs > unicode_categories[mid].last)
+ if (code > unicode_categories[mid].last)
min = mid + 1;
- else if (ucs < unicode_categories[mid].first)
+ else if (code < unicode_categories[mid].first)
max = mid - 1;
else
return unicode_categories[mid].category;
@@ -44,6 +104,180 @@ unicode_category(pg_wchar ucs)
return PG_U_UNASSIGNED;
}
+bool
+pg_u_prop_alphabetic(pg_wchar code)
+{
+ return range_search(unicode_alphabetic,
+ lengthof(unicode_alphabetic),
+ code);
+}
+
+bool
+pg_u_prop_lowercase(pg_wchar code)
+{
+ return range_search(unicode_lowercase,
+ lengthof(unicode_lowercase),
+ code);
+}
+
+bool
+pg_u_prop_uppercase(pg_wchar code)
+{
+ return range_search(unicode_uppercase,
+ lengthof(unicode_uppercase),
+ code);
+}
+
+bool
+pg_u_prop_white_space(pg_wchar code)
+{
+ return range_search(unicode_white_space,
+ lengthof(unicode_white_space),
+ code);
+}
+
+bool
+pg_u_prop_hex_digit(pg_wchar code)
+{
+ return range_search(unicode_hex_digit,
+ lengthof(unicode_hex_digit),
+ code);
+}
+
+bool
+pg_u_prop_join_control(pg_wchar code)
+{
+ return range_search(unicode_join_control,
+ lengthof(unicode_join_control),
+ code);
+}
+
+/*
+ * The following functions implement the Compatibility Properties described
+ * at: http://www.unicode.org/reports/tr18/#Compatibility_Properties
+ *
+ * If 'posix' is true, implements the "POSIX Compatible" variant, otherwise
+ * the "Standard" variant.
+ */
+
+bool
+pg_u_isdigit(pg_wchar code, bool posix)
+{
+ if (posix)
+ return ('0' <= code && code <= '9');
+ else
+ return unicode_category(code) == PG_U_DECIMAL_NUMBER;
+}
+
+bool
+pg_u_isalpha(pg_wchar code)
+{
+ return pg_u_prop_alphabetic(code);
+}
+
+bool
+pg_u_isalnum(pg_wchar code, bool posix)
+{
+ return pg_u_isalpha(code) || pg_u_isdigit(code, posix);
+}
+
+bool
+pg_u_isword(pg_wchar code)
+{
+ uint32 category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ return
+ category_mask & (PG_U_M_MASK | PG_U_ND_MASK | PG_U_PC_MASK) ||
+ pg_u_isalpha(code) ||
+ pg_u_prop_join_control(code);
+}
+
+bool
+pg_u_isupper(pg_wchar code)
+{
+ return pg_u_prop_uppercase(code);
+}
+
+bool
+pg_u_islower(pg_wchar code)
+{
+ return pg_u_prop_lowercase(code);
+}
+
+bool
+pg_u_isblank(pg_wchar code)
+{
+ return code == PG_U_CHARACTER_TAB ||
+ unicode_category(code) == PG_U_SPACE_SEPARATOR;
+}
+
+bool
+pg_u_iscntrl(pg_wchar code)
+{
+ return unicode_category(code) == PG_U_CONTROL;
+}
+
+bool
+pg_u_isgraph(pg_wchar code)
+{
+ uint32 category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ if (category_mask & (PG_U_CC_MASK | PG_U_CS_MASK | PG_U_CN_MASK) ||
+ pg_u_isspace(code))
+ return false;
+ return true;
+}
+
+bool
+pg_u_isprint(pg_wchar code)
+{
+ pg_unicode_category category = unicode_category(code);
+
+ if (category == PG_U_CONTROL)
+ return false;
+
+ return pg_u_isgraph(code) || pg_u_isblank(code);
+}
+
+bool
+pg_u_ispunct(pg_wchar code, bool posix)
+{
+ uint32 category_mask;
+
+ if (posix)
+ {
+ if (pg_u_isalpha(code))
+ return false;
+
+ category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+ return category_mask & (PG_U_P_MASK | PG_U_S_MASK);
+ }
+ else
+ {
+ category_mask = PG_U_CATEGORY_MASK(unicode_category(code));
+
+ return category_mask & PG_U_P_MASK;
+ }
+}
+
+bool
+pg_u_isspace(pg_wchar code)
+{
+ return pg_u_prop_white_space(code);
+}
+
+bool
+pg_u_isxdigit(pg_wchar code, bool posix)
+{
+ if (posix)
+ return (('0' <= code && code <= '9') ||
+ ('A' <= code && code <= 'F') ||
+ ('a' <= code && code <= 'f'));
+ else
+ return unicode_category(code) == PG_U_DECIMAL_NUMBER ||
+ pg_u_prop_hex_digit(code);
+}
+
/*
* Description of Unicode general category.
*/
@@ -191,3 +425,30 @@ unicode_category_abbrev(pg_unicode_category category)
Assert(false);
return "??"; /* keep compiler quiet */
}
+
+/*
+ * Binary search to test if given codepoint exists in one of the ranges in the
+ * given table.
+ */
+static bool
+range_search(const pg_unicode_range * tbl, size_t size, pg_wchar code)
+{
+ int min = 0;
+ int mid;
+ int max = size - 1;
+
+ Assert(code <= 0x10ffff);
+
+ while (max >= min)
+ {
+ mid = (min + max) / 2;
+ if (code > tbl[mid].last)
+ min = mid + 1;
+ else if (code < tbl[mid].first)
+ max = mid - 1;
+ else
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/include/common/unicode_category.h b/src/include/common/unicode_category.h
index 5bad280615..dd79e44cb2 100644
--- a/src/include/common/unicode_category.h
+++ b/src/include/common/unicode_category.h
@@ -62,7 +62,28 @@ typedef enum pg_unicode_category
} pg_unicode_category;
extern pg_unicode_category unicode_category(pg_wchar ucs);
-const char *unicode_category_string(pg_unicode_category category);
-const char *unicode_category_abbrev(pg_unicode_category category);
+extern const char *unicode_category_string(pg_unicode_category category);
+extern const char *unicode_category_abbrev(pg_unicode_category category);
+
+extern bool pg_u_prop_alphabetic(pg_wchar c);
+extern bool pg_u_prop_lowercase(pg_wchar c);
+extern bool pg_u_prop_uppercase(pg_wchar c);
+extern bool pg_u_prop_white_space(pg_wchar c);
+extern bool pg_u_prop_hex_digit(pg_wchar c);
+extern bool pg_u_prop_join_control(pg_wchar c);
+
+extern bool pg_u_isdigit(pg_wchar c, bool posix);
+extern bool pg_u_isalpha(pg_wchar c);
+extern bool pg_u_isalnum(pg_wchar c, bool posix);
+extern bool pg_u_isword(pg_wchar c);
+extern bool pg_u_isupper(pg_wchar c);
+extern bool pg_u_islower(pg_wchar c);
+extern bool pg_u_isblank(pg_wchar c);
+extern bool pg_u_iscntrl(pg_wchar c);
+extern bool pg_u_isgraph(pg_wchar c);
+extern bool pg_u_isprint(pg_wchar c);
+extern bool pg_u_ispunct(pg_wchar c, bool posix);
+extern bool pg_u_isspace(pg_wchar c);
+extern bool pg_u_isxdigit(pg_wchar c, bool posix);
#endif /* UNICODE_CATEGORY_H */
diff --git a/src/include/common/unicode_category_table.h b/src/include/common/unicode_category_table.h
index d7ef996189..b1541c5f17 100644
--- a/src/include/common/unicode_category_table.h
+++ b/src/include/common/unicode_category_table.h
@@ -25,6 +25,12 @@ typedef struct
uint8 category; /* General Category */
} pg_category_range;
+typedef struct
+{
+ uint32 first; /* Unicode codepoint */
+ uint32 last; /* Unicode codepoint */
+} pg_unicode_range;
+
/* table of Unicode codepoint ranges and their categories */
static const pg_category_range unicode_categories[3302] =
{
@@ -3329,5 +3335,2531 @@ static const pg_category_range unicode_categories[3302] =
{0x0e0020, 0x0e007f, PG_U_FORMAT},
{0x0e0100, 0x0e01ef, PG_U_NONSPACING_MARK},
{0x0f0000, 0x0ffffd, PG_U_PRIVATE_USE},
- {0x100000, 0x10fffd, PG_U_PRIVATE_USE}
+ {0x100000, 0x10fffd, PG_U_PRIVATE_USE},
+};
+
+/* table of Unicode codepoint ranges of Alphabetic characters */
+static const pg_unicode_range unicode_alphabetic[1141] =
+{
+ {0x000041, 0x00005a},
+ {0x000061, 0x00007a},
+ {0x0000aa, 0x0000aa},
+ {0x0000b5, 0x0000b5},
+ {0x0000ba, 0x0000ba},
+ {0x0000c0, 0x0000d6},
+ {0x0000d8, 0x0000f6},
+ {0x0000f8, 0x0001ba},
+ {0x0001bb, 0x0001bb},
+ {0x0001bc, 0x0001bf},
+ {0x0001c0, 0x0001c3},
+ {0x0001c4, 0x000293},
+ {0x000294, 0x000294},
+ {0x000295, 0x0002af},
+ {0x0002b0, 0x0002c1},
+ {0x0002c6, 0x0002d1},
+ {0x0002e0, 0x0002e4},
+ {0x0002ec, 0x0002ec},
+ {0x0002ee, 0x0002ee},
+ {0x000345, 0x000345},
+ {0x000370, 0x000373},
+ {0x000374, 0x000374},
+ {0x000376, 0x000377},
+ {0x00037a, 0x00037a},
+ {0x00037b, 0x00037d},
+ {0x00037f, 0x00037f},
+ {0x000386, 0x000386},
+ {0x000388, 0x00038a},
+ {0x00038c, 0x00038c},
+ {0x00038e, 0x0003a1},
+ {0x0003a3, 0x0003f5},
+ {0x0003f7, 0x000481},
+ {0x00048a, 0x00052f},
+ {0x000531, 0x000556},
+ {0x000559, 0x000559},
+ {0x000560, 0x000588},
+ {0x0005b0, 0x0005bd},
+ {0x0005bf, 0x0005bf},
+ {0x0005c1, 0x0005c2},
+ {0x0005c4, 0x0005c5},
+ {0x0005c7, 0x0005c7},
+ {0x0005d0, 0x0005ea},
+ {0x0005ef, 0x0005f2},
+ {0x000610, 0x00061a},
+ {0x000620, 0x00063f},
+ {0x000640, 0x000640},
+ {0x000641, 0x00064a},
+ {0x00064b, 0x000657},
+ {0x000659, 0x00065f},
+ {0x00066e, 0x00066f},
+ {0x000670, 0x000670},
+ {0x000671, 0x0006d3},
+ {0x0006d5, 0x0006d5},
+ {0x0006d6, 0x0006dc},
+ {0x0006e1, 0x0006e4},
+ {0x0006e5, 0x0006e6},
+ {0x0006e7, 0x0006e8},
+ {0x0006ed, 0x0006ed},
+ {0x0006ee, 0x0006ef},
+ {0x0006fa, 0x0006fc},
+ {0x0006ff, 0x0006ff},
+ {0x000710, 0x000710},
+ {0x000711, 0x000711},
+ {0x000712, 0x00072f},
+ {0x000730, 0x00073f},
+ {0x00074d, 0x0007a5},
+ {0x0007a6, 0x0007b0},
+ {0x0007b1, 0x0007b1},
+ {0x0007ca, 0x0007ea},
+ {0x0007f4, 0x0007f5},
+ {0x0007fa, 0x0007fa},
+ {0x000800, 0x000815},
+ {0x000816, 0x000817},
+ {0x00081a, 0x00081a},
+ {0x00081b, 0x000823},
+ {0x000824, 0x000824},
+ {0x000825, 0x000827},
+ {0x000828, 0x000828},
+ {0x000829, 0x00082c},
+ {0x000840, 0x000858},
+ {0x000860, 0x00086a},
+ {0x000870, 0x000887},
+ {0x000889, 0x00088e},
+ {0x0008a0, 0x0008c8},
+ {0x0008c9, 0x0008c9},
+ {0x0008d4, 0x0008df},
+ {0x0008e3, 0x0008e9},
+ {0x0008f0, 0x000902},
+ {0x000903, 0x000903},
+ {0x000904, 0x000939},
+ {0x00093a, 0x00093a},
+ {0x00093b, 0x00093b},
+ {0x00093d, 0x00093d},
+ {0x00093e, 0x000940},
+ {0x000941, 0x000948},
+ {0x000949, 0x00094c},
+ {0x00094e, 0x00094f},
+ {0x000950, 0x000950},
+ {0x000955, 0x000957},
+ {0x000958, 0x000961},
+ {0x000962, 0x000963},
+ {0x000971, 0x000971},
+ {0x000972, 0x000980},
+ {0x000981, 0x000981},
+ {0x000982, 0x000983},
+ {0x000985, 0x00098c},
+ {0x00098f, 0x000990},
+ {0x000993, 0x0009a8},
+ {0x0009aa, 0x0009b0},
+ {0x0009b2, 0x0009b2},
+ {0x0009b6, 0x0009b9},
+ {0x0009bd, 0x0009bd},
+ {0x0009be, 0x0009c0},
+ {0x0009c1, 0x0009c4},
+ {0x0009c7, 0x0009c8},
+ {0x0009cb, 0x0009cc},
+ {0x0009ce, 0x0009ce},
+ {0x0009d7, 0x0009d7},
+ {0x0009dc, 0x0009dd},
+ {0x0009df, 0x0009e1},
+ {0x0009e2, 0x0009e3},
+ {0x0009f0, 0x0009f1},
+ {0x0009fc, 0x0009fc},
+ {0x000a01, 0x000a02},
+ {0x000a03, 0x000a03},
+ {0x000a05, 0x000a0a},
+ {0x000a0f, 0x000a10},
+ {0x000a13, 0x000a28},
+ {0x000a2a, 0x000a30},
+ {0x000a32, 0x000a33},
+ {0x000a35, 0x000a36},
+ {0x000a38, 0x000a39},
+ {0x000a3e, 0x000a40},
+ {0x000a41, 0x000a42},
+ {0x000a47, 0x000a48},
+ {0x000a4b, 0x000a4c},
+ {0x000a51, 0x000a51},
+ {0x000a59, 0x000a5c},
+ {0x000a5e, 0x000a5e},
+ {0x000a70, 0x000a71},
+ {0x000a72, 0x000a74},
+ {0x000a75, 0x000a75},
+ {0x000a81, 0x000a82},
+ {0x000a83, 0x000a83},
+ {0x000a85, 0x000a8d},
+ {0x000a8f, 0x000a91},
+ {0x000a93, 0x000aa8},
+ {0x000aaa, 0x000ab0},
+ {0x000ab2, 0x000ab3},
+ {0x000ab5, 0x000ab9},
+ {0x000abd, 0x000abd},
+ {0x000abe, 0x000ac0},
+ {0x000ac1, 0x000ac5},
+ {0x000ac7, 0x000ac8},
+ {0x000ac9, 0x000ac9},
+ {0x000acb, 0x000acc},
+ {0x000ad0, 0x000ad0},
+ {0x000ae0, 0x000ae1},
+ {0x000ae2, 0x000ae3},
+ {0x000af9, 0x000af9},
+ {0x000afa, 0x000afc},
+ {0x000b01, 0x000b01},
+ {0x000b02, 0x000b03},
+ {0x000b05, 0x000b0c},
+ {0x000b0f, 0x000b10},
+ {0x000b13, 0x000b28},
+ {0x000b2a, 0x000b30},
+ {0x000b32, 0x000b33},
+ {0x000b35, 0x000b39},
+ {0x000b3d, 0x000b3d},
+ {0x000b3e, 0x000b3e},
+ {0x000b3f, 0x000b3f},
+ {0x000b40, 0x000b40},
+ {0x000b41, 0x000b44},
+ {0x000b47, 0x000b48},
+ {0x000b4b, 0x000b4c},
+ {0x000b56, 0x000b56},
+ {0x000b57, 0x000b57},
+ {0x000b5c, 0x000b5d},
+ {0x000b5f, 0x000b61},
+ {0x000b62, 0x000b63},
+ {0x000b71, 0x000b71},
+ {0x000b82, 0x000b82},
+ {0x000b83, 0x000b83},
+ {0x000b85, 0x000b8a},
+ {0x000b8e, 0x000b90},
+ {0x000b92, 0x000b95},
+ {0x000b99, 0x000b9a},
+ {0x000b9c, 0x000b9c},
+ {0x000b9e, 0x000b9f},
+ {0x000ba3, 0x000ba4},
+ {0x000ba8, 0x000baa},
+ {0x000bae, 0x000bb9},
+ {0x000bbe, 0x000bbf},
+ {0x000bc0, 0x000bc0},
+ {0x000bc1, 0x000bc2},
+ {0x000bc6, 0x000bc8},
+ {0x000bca, 0x000bcc},
+ {0x000bd0, 0x000bd0},
+ {0x000bd7, 0x000bd7},
+ {0x000c00, 0x000c00},
+ {0x000c01, 0x000c03},
+ {0x000c04, 0x000c04},
+ {0x000c05, 0x000c0c},
+ {0x000c0e, 0x000c10},
+ {0x000c12, 0x000c28},
+ {0x000c2a, 0x000c39},
+ {0x000c3d, 0x000c3d},
+ {0x000c3e, 0x000c40},
+ {0x000c41, 0x000c44},
+ {0x000c46, 0x000c48},
+ {0x000c4a, 0x000c4c},
+ {0x000c55, 0x000c56},
+ {0x000c58, 0x000c5a},
+ {0x000c5d, 0x000c5d},
+ {0x000c60, 0x000c61},
+ {0x000c62, 0x000c63},
+ {0x000c80, 0x000c80},
+ {0x000c81, 0x000c81},
+ {0x000c82, 0x000c83},
+ {0x000c85, 0x000c8c},
+ {0x000c8e, 0x000c90},
+ {0x000c92, 0x000ca8},
+ {0x000caa, 0x000cb3},
+ {0x000cb5, 0x000cb9},
+ {0x000cbd, 0x000cbd},
+ {0x000cbe, 0x000cbe},
+ {0x000cbf, 0x000cbf},
+ {0x000cc0, 0x000cc4},
+ {0x000cc6, 0x000cc6},
+ {0x000cc7, 0x000cc8},
+ {0x000cca, 0x000ccb},
+ {0x000ccc, 0x000ccc},
+ {0x000cd5, 0x000cd6},
+ {0x000cdd, 0x000cde},
+ {0x000ce0, 0x000ce1},
+ {0x000ce2, 0x000ce3},
+ {0x000cf1, 0x000cf2},
+ {0x000cf3, 0x000cf3},
+ {0x000d00, 0x000d01},
+ {0x000d02, 0x000d03},
+ {0x000d04, 0x000d0c},
+ {0x000d0e, 0x000d10},
+ {0x000d12, 0x000d3a},
+ {0x000d3d, 0x000d3d},
+ {0x000d3e, 0x000d40},
+ {0x000d41, 0x000d44},
+ {0x000d46, 0x000d48},
+ {0x000d4a, 0x000d4c},
+ {0x000d4e, 0x000d4e},
+ {0x000d54, 0x000d56},
+ {0x000d57, 0x000d57},
+ {0x000d5f, 0x000d61},
+ {0x000d62, 0x000d63},
+ {0x000d7a, 0x000d7f},
+ {0x000d81, 0x000d81},
+ {0x000d82, 0x000d83},
+ {0x000d85, 0x000d96},
+ {0x000d9a, 0x000db1},
+ {0x000db3, 0x000dbb},
+ {0x000dbd, 0x000dbd},
+ {0x000dc0, 0x000dc6},
+ {0x000dcf, 0x000dd1},
+ {0x000dd2, 0x000dd4},
+ {0x000dd6, 0x000dd6},
+ {0x000dd8, 0x000ddf},
+ {0x000df2, 0x000df3},
+ {0x000e01, 0x000e30},
+ {0x000e31, 0x000e31},
+ {0x000e32, 0x000e33},
+ {0x000e34, 0x000e3a},
+ {0x000e40, 0x000e45},
+ {0x000e46, 0x000e46},
+ {0x000e4d, 0x000e4d},
+ {0x000e81, 0x000e82},
+ {0x000e84, 0x000e84},
+ {0x000e86, 0x000e8a},
+ {0x000e8c, 0x000ea3},
+ {0x000ea5, 0x000ea5},
+ {0x000ea7, 0x000eb0},
+ {0x000eb1, 0x000eb1},
+ {0x000eb2, 0x000eb3},
+ {0x000eb4, 0x000eb9},
+ {0x000ebb, 0x000ebc},
+ {0x000ebd, 0x000ebd},
+ {0x000ec0, 0x000ec4},
+ {0x000ec6, 0x000ec6},
+ {0x000ecd, 0x000ecd},
+ {0x000edc, 0x000edf},
+ {0x000f00, 0x000f00},
+ {0x000f40, 0x000f47},
+ {0x000f49, 0x000f6c},
+ {0x000f71, 0x000f7e},
+ {0x000f7f, 0x000f7f},
+ {0x000f80, 0x000f83},
+ {0x000f88, 0x000f8c},
+ {0x000f8d, 0x000f97},
+ {0x000f99, 0x000fbc},
+ {0x001000, 0x00102a},
+ {0x00102b, 0x00102c},
+ {0x00102d, 0x001030},
+ {0x001031, 0x001031},
+ {0x001032, 0x001036},
+ {0x001038, 0x001038},
+ {0x00103b, 0x00103c},
+ {0x00103d, 0x00103e},
+ {0x00103f, 0x00103f},
+ {0x001050, 0x001055},
+ {0x001056, 0x001057},
+ {0x001058, 0x001059},
+ {0x00105a, 0x00105d},
+ {0x00105e, 0x001060},
+ {0x001061, 0x001061},
+ {0x001062, 0x001064},
+ {0x001065, 0x001066},
+ {0x001067, 0x00106d},
+ {0x00106e, 0x001070},
+ {0x001071, 0x001074},
+ {0x001075, 0x001081},
+ {0x001082, 0x001082},
+ {0x001083, 0x001084},
+ {0x001085, 0x001086},
+ {0x001087, 0x00108c},
+ {0x00108d, 0x00108d},
+ {0x00108e, 0x00108e},
+ {0x00108f, 0x00108f},
+ {0x00109a, 0x00109c},
+ {0x00109d, 0x00109d},
+ {0x0010a0, 0x0010c5},
+ {0x0010c7, 0x0010c7},
+ {0x0010cd, 0x0010cd},
+ {0x0010d0, 0x0010fa},
+ {0x0010fc, 0x0010fc},
+ {0x0010fd, 0x0010ff},
+ {0x001100, 0x001248},
+ {0x00124a, 0x00124d},
+ {0x001250, 0x001256},
+ {0x001258, 0x001258},
+ {0x00125a, 0x00125d},
+ {0x001260, 0x001288},
+ {0x00128a, 0x00128d},
+ {0x001290, 0x0012b0},
+ {0x0012b2, 0x0012b5},
+ {0x0012b8, 0x0012be},
+ {0x0012c0, 0x0012c0},
+ {0x0012c2, 0x0012c5},
+ {0x0012c8, 0x0012d6},
+ {0x0012d8, 0x001310},
+ {0x001312, 0x001315},
+ {0x001318, 0x00135a},
+ {0x001380, 0x00138f},
+ {0x0013a0, 0x0013f5},
+ {0x0013f8, 0x0013fd},
+ {0x001401, 0x00166c},
+ {0x00166f, 0x00167f},
+ {0x001681, 0x00169a},
+ {0x0016a0, 0x0016ea},
+ {0x0016ee, 0x0016f0},
+ {0x0016f1, 0x0016f8},
+ {0x001700, 0x001711},
+ {0x001712, 0x001713},
+ {0x00171f, 0x001731},
+ {0x001732, 0x001733},
+ {0x001740, 0x001751},
+ {0x001752, 0x001753},
+ {0x001760, 0x00176c},
+ {0x00176e, 0x001770},
+ {0x001772, 0x001773},
+ {0x001780, 0x0017b3},
+ {0x0017b6, 0x0017b6},
+ {0x0017b7, 0x0017bd},
+ {0x0017be, 0x0017c5},
+ {0x0017c6, 0x0017c6},
+ {0x0017c7, 0x0017c8},
+ {0x0017d7, 0x0017d7},
+ {0x0017dc, 0x0017dc},
+ {0x001820, 0x001842},
+ {0x001843, 0x001843},
+ {0x001844, 0x001878},
+ {0x001880, 0x001884},
+ {0x001885, 0x001886},
+ {0x001887, 0x0018a8},
+ {0x0018a9, 0x0018a9},
+ {0x0018aa, 0x0018aa},
+ {0x0018b0, 0x0018f5},
+ {0x001900, 0x00191e},
+ {0x001920, 0x001922},
+ {0x001923, 0x001926},
+ {0x001927, 0x001928},
+ {0x001929, 0x00192b},
+ {0x001930, 0x001931},
+ {0x001932, 0x001932},
+ {0x001933, 0x001938},
+ {0x001950, 0x00196d},
+ {0x001970, 0x001974},
+ {0x001980, 0x0019ab},
+ {0x0019b0, 0x0019c9},
+ {0x001a00, 0x001a16},
+ {0x001a17, 0x001a18},
+ {0x001a19, 0x001a1a},
+ {0x001a1b, 0x001a1b},
+ {0x001a20, 0x001a54},
+ {0x001a55, 0x001a55},
+ {0x001a56, 0x001a56},
+ {0x001a57, 0x001a57},
+ {0x001a58, 0x001a5e},
+ {0x001a61, 0x001a61},
+ {0x001a62, 0x001a62},
+ {0x001a63, 0x001a64},
+ {0x001a65, 0x001a6c},
+ {0x001a6d, 0x001a72},
+ {0x001a73, 0x001a74},
+ {0x001aa7, 0x001aa7},
+ {0x001abf, 0x001ac0},
+ {0x001acc, 0x001ace},
+ {0x001b00, 0x001b03},
+ {0x001b04, 0x001b04},
+ {0x001b05, 0x001b33},
+ {0x001b35, 0x001b35},
+ {0x001b36, 0x001b3a},
+ {0x001b3b, 0x001b3b},
+ {0x001b3c, 0x001b3c},
+ {0x001b3d, 0x001b41},
+ {0x001b42, 0x001b42},
+ {0x001b43, 0x001b43},
+ {0x001b45, 0x001b4c},
+ {0x001b80, 0x001b81},
+ {0x001b82, 0x001b82},
+ {0x001b83, 0x001ba0},
+ {0x001ba1, 0x001ba1},
+ {0x001ba2, 0x001ba5},
+ {0x001ba6, 0x001ba7},
+ {0x001ba8, 0x001ba9},
+ {0x001bac, 0x001bad},
+ {0x001bae, 0x001baf},
+ {0x001bba, 0x001be5},
+ {0x001be7, 0x001be7},
+ {0x001be8, 0x001be9},
+ {0x001bea, 0x001bec},
+ {0x001bed, 0x001bed},
+ {0x001bee, 0x001bee},
+ {0x001bef, 0x001bf1},
+ {0x001c00, 0x001c23},
+ {0x001c24, 0x001c2b},
+ {0x001c2c, 0x001c33},
+ {0x001c34, 0x001c35},
+ {0x001c36, 0x001c36},
+ {0x001c4d, 0x001c4f},
+ {0x001c5a, 0x001c77},
+ {0x001c78, 0x001c7d},
+ {0x001c80, 0x001c88},
+ {0x001c90, 0x001cba},
+ {0x001cbd, 0x001cbf},
+ {0x001ce9, 0x001cec},
+ {0x001cee, 0x001cf3},
+ {0x001cf5, 0x001cf6},
+ {0x001cfa, 0x001cfa},
+ {0x001d00, 0x001d2b},
+ {0x001d2c, 0x001d6a},
+ {0x001d6b, 0x001d77},
+ {0x001d78, 0x001d78},
+ {0x001d79, 0x001d9a},
+ {0x001d9b, 0x001dbf},
+ {0x001de7, 0x001df4},
+ {0x001e00, 0x001f15},
+ {0x001f18, 0x001f1d},
+ {0x001f20, 0x001f45},
+ {0x001f48, 0x001f4d},
+ {0x001f50, 0x001f57},
+ {0x001f59, 0x001f59},
+ {0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f7d},
+ {0x001f80, 0x001fb4},
+ {0x001fb6, 0x001fbc},
+ {0x001fbe, 0x001fbe},
+ {0x001fc2, 0x001fc4},
+ {0x001fc6, 0x001fcc},
+ {0x001fd0, 0x001fd3},
+ {0x001fd6, 0x001fdb},
+ {0x001fe0, 0x001fec},
+ {0x001ff2, 0x001ff4},
+ {0x001ff6, 0x001ffc},
+ {0x002071, 0x002071},
+ {0x00207f, 0x00207f},
+ {0x002090, 0x00209c},
+ {0x002102, 0x002102},
+ {0x002107, 0x002107},
+ {0x00210a, 0x002113},
+ {0x002115, 0x002115},
+ {0x002119, 0x00211d},
+ {0x002124, 0x002124},
+ {0x002126, 0x002126},
+ {0x002128, 0x002128},
+ {0x00212a, 0x00212d},
+ {0x00212f, 0x002134},
+ {0x002135, 0x002138},
+ {0x002139, 0x002139},
+ {0x00213c, 0x00213f},
+ {0x002145, 0x002149},
+ {0x00214e, 0x00214e},
+ {0x002160, 0x002182},
+ {0x002183, 0x002184},
+ {0x002185, 0x002188},
+ {0x0024b6, 0x0024e9},
+ {0x002c00, 0x002c7b},
+ {0x002c7c, 0x002c7d},
+ {0x002c7e, 0x002ce4},
+ {0x002ceb, 0x002cee},
+ {0x002cf2, 0x002cf3},
+ {0x002d00, 0x002d25},
+ {0x002d27, 0x002d27},
+ {0x002d2d, 0x002d2d},
+ {0x002d30, 0x002d67},
+ {0x002d6f, 0x002d6f},
+ {0x002d80, 0x002d96},
+ {0x002da0, 0x002da6},
+ {0x002da8, 0x002dae},
+ {0x002db0, 0x002db6},
+ {0x002db8, 0x002dbe},
+ {0x002dc0, 0x002dc6},
+ {0x002dc8, 0x002dce},
+ {0x002dd0, 0x002dd6},
+ {0x002dd8, 0x002dde},
+ {0x002de0, 0x002dff},
+ {0x002e2f, 0x002e2f},
+ {0x003005, 0x003005},
+ {0x003006, 0x003006},
+ {0x003007, 0x003007},
+ {0x003021, 0x003029},
+ {0x003031, 0x003035},
+ {0x003038, 0x00303a},
+ {0x00303b, 0x00303b},
+ {0x00303c, 0x00303c},
+ {0x003041, 0x003096},
+ {0x00309d, 0x00309e},
+ {0x00309f, 0x00309f},
+ {0x0030a1, 0x0030fa},
+ {0x0030fc, 0x0030fe},
+ {0x0030ff, 0x0030ff},
+ {0x003105, 0x00312f},
+ {0x003131, 0x00318e},
+ {0x0031a0, 0x0031bf},
+ {0x0031f0, 0x0031ff},
+ {0x003400, 0x004dbf},
+ {0x004e00, 0x00a014},
+ {0x00a015, 0x00a015},
+ {0x00a016, 0x00a48c},
+ {0x00a4d0, 0x00a4f7},
+ {0x00a4f8, 0x00a4fd},
+ {0x00a500, 0x00a60b},
+ {0x00a60c, 0x00a60c},
+ {0x00a610, 0x00a61f},
+ {0x00a62a, 0x00a62b},
+ {0x00a640, 0x00a66d},
+ {0x00a66e, 0x00a66e},
+ {0x00a674, 0x00a67b},
+ {0x00a67f, 0x00a67f},
+ {0x00a680, 0x00a69b},
+ {0x00a69c, 0x00a69d},
+ {0x00a69e, 0x00a69f},
+ {0x00a6a0, 0x00a6e5},
+ {0x00a6e6, 0x00a6ef},
+ {0x00a717, 0x00a71f},
+ {0x00a722, 0x00a76f},
+ {0x00a770, 0x00a770},
+ {0x00a771, 0x00a787},
+ {0x00a788, 0x00a788},
+ {0x00a78b, 0x00a78e},
+ {0x00a78f, 0x00a78f},
+ {0x00a790, 0x00a7ca},
+ {0x00a7d0, 0x00a7d1},
+ {0x00a7d3, 0x00a7d3},
+ {0x00a7d5, 0x00a7d9},
+ {0x00a7f2, 0x00a7f4},
+ {0x00a7f5, 0x00a7f6},
+ {0x00a7f7, 0x00a7f7},
+ {0x00a7f8, 0x00a7f9},
+ {0x00a7fa, 0x00a7fa},
+ {0x00a7fb, 0x00a801},
+ {0x00a802, 0x00a802},
+ {0x00a803, 0x00a805},
+ {0x00a807, 0x00a80a},
+ {0x00a80b, 0x00a80b},
+ {0x00a80c, 0x00a822},
+ {0x00a823, 0x00a824},
+ {0x00a825, 0x00a826},
+ {0x00a827, 0x00a827},
+ {0x00a840, 0x00a873},
+ {0x00a880, 0x00a881},
+ {0x00a882, 0x00a8b3},
+ {0x00a8b4, 0x00a8c3},
+ {0x00a8c5, 0x00a8c5},
+ {0x00a8f2, 0x00a8f7},
+ {0x00a8fb, 0x00a8fb},
+ {0x00a8fd, 0x00a8fe},
+ {0x00a8ff, 0x00a8ff},
+ {0x00a90a, 0x00a925},
+ {0x00a926, 0x00a92a},
+ {0x00a930, 0x00a946},
+ {0x00a947, 0x00a951},
+ {0x00a952, 0x00a952},
+ {0x00a960, 0x00a97c},
+ {0x00a980, 0x00a982},
+ {0x00a983, 0x00a983},
+ {0x00a984, 0x00a9b2},
+ {0x00a9b4, 0x00a9b5},
+ {0x00a9b6, 0x00a9b9},
+ {0x00a9ba, 0x00a9bb},
+ {0x00a9bc, 0x00a9bd},
+ {0x00a9be, 0x00a9bf},
+ {0x00a9cf, 0x00a9cf},
+ {0x00a9e0, 0x00a9e4},
+ {0x00a9e5, 0x00a9e5},
+ {0x00a9e6, 0x00a9e6},
+ {0x00a9e7, 0x00a9ef},
+ {0x00a9fa, 0x00a9fe},
+ {0x00aa00, 0x00aa28},
+ {0x00aa29, 0x00aa2e},
+ {0x00aa2f, 0x00aa30},
+ {0x00aa31, 0x00aa32},
+ {0x00aa33, 0x00aa34},
+ {0x00aa35, 0x00aa36},
+ {0x00aa40, 0x00aa42},
+ {0x00aa43, 0x00aa43},
+ {0x00aa44, 0x00aa4b},
+ {0x00aa4c, 0x00aa4c},
+ {0x00aa4d, 0x00aa4d},
+ {0x00aa60, 0x00aa6f},
+ {0x00aa70, 0x00aa70},
+ {0x00aa71, 0x00aa76},
+ {0x00aa7a, 0x00aa7a},
+ {0x00aa7b, 0x00aa7b},
+ {0x00aa7c, 0x00aa7c},
+ {0x00aa7d, 0x00aa7d},
+ {0x00aa7e, 0x00aaaf},
+ {0x00aab0, 0x00aab0},
+ {0x00aab1, 0x00aab1},
+ {0x00aab2, 0x00aab4},
+ {0x00aab5, 0x00aab6},
+ {0x00aab7, 0x00aab8},
+ {0x00aab9, 0x00aabd},
+ {0x00aabe, 0x00aabe},
+ {0x00aac0, 0x00aac0},
+ {0x00aac2, 0x00aac2},
+ {0x00aadb, 0x00aadc},
+ {0x00aadd, 0x00aadd},
+ {0x00aae0, 0x00aaea},
+ {0x00aaeb, 0x00aaeb},
+ {0x00aaec, 0x00aaed},
+ {0x00aaee, 0x00aaef},
+ {0x00aaf2, 0x00aaf2},
+ {0x00aaf3, 0x00aaf4},
+ {0x00aaf5, 0x00aaf5},
+ {0x00ab01, 0x00ab06},
+ {0x00ab09, 0x00ab0e},
+ {0x00ab11, 0x00ab16},
+ {0x00ab20, 0x00ab26},
+ {0x00ab28, 0x00ab2e},
+ {0x00ab30, 0x00ab5a},
+ {0x00ab5c, 0x00ab5f},
+ {0x00ab60, 0x00ab68},
+ {0x00ab69, 0x00ab69},
+ {0x00ab70, 0x00abbf},
+ {0x00abc0, 0x00abe2},
+ {0x00abe3, 0x00abe4},
+ {0x00abe5, 0x00abe5},
+ {0x00abe6, 0x00abe7},
+ {0x00abe8, 0x00abe8},
+ {0x00abe9, 0x00abea},
+ {0x00ac00, 0x00d7a3},
+ {0x00d7b0, 0x00d7c6},
+ {0x00d7cb, 0x00d7fb},
+ {0x00f900, 0x00fa6d},
+ {0x00fa70, 0x00fad9},
+ {0x00fb00, 0x00fb06},
+ {0x00fb13, 0x00fb17},
+ {0x00fb1d, 0x00fb1d},
+ {0x00fb1e, 0x00fb1e},
+ {0x00fb1f, 0x00fb28},
+ {0x00fb2a, 0x00fb36},
+ {0x00fb38, 0x00fb3c},
+ {0x00fb3e, 0x00fb3e},
+ {0x00fb40, 0x00fb41},
+ {0x00fb43, 0x00fb44},
+ {0x00fb46, 0x00fbb1},
+ {0x00fbd3, 0x00fd3d},
+ {0x00fd50, 0x00fd8f},
+ {0x00fd92, 0x00fdc7},
+ {0x00fdf0, 0x00fdfb},
+ {0x00fe70, 0x00fe74},
+ {0x00fe76, 0x00fefc},
+ {0x00ff21, 0x00ff3a},
+ {0x00ff41, 0x00ff5a},
+ {0x00ff66, 0x00ff6f},
+ {0x00ff70, 0x00ff70},
+ {0x00ff71, 0x00ff9d},
+ {0x00ff9e, 0x00ff9f},
+ {0x00ffa0, 0x00ffbe},
+ {0x00ffc2, 0x00ffc7},
+ {0x00ffca, 0x00ffcf},
+ {0x00ffd2, 0x00ffd7},
+ {0x00ffda, 0x00ffdc},
+ {0x010000, 0x01000b},
+ {0x01000d, 0x010026},
+ {0x010028, 0x01003a},
+ {0x01003c, 0x01003d},
+ {0x01003f, 0x01004d},
+ {0x010050, 0x01005d},
+ {0x010080, 0x0100fa},
+ {0x010140, 0x010174},
+ {0x010280, 0x01029c},
+ {0x0102a0, 0x0102d0},
+ {0x010300, 0x01031f},
+ {0x01032d, 0x010340},
+ {0x010341, 0x010341},
+ {0x010342, 0x010349},
+ {0x01034a, 0x01034a},
+ {0x010350, 0x010375},
+ {0x010376, 0x01037a},
+ {0x010380, 0x01039d},
+ {0x0103a0, 0x0103c3},
+ {0x0103c8, 0x0103cf},
+ {0x0103d1, 0x0103d5},
+ {0x010400, 0x01044f},
+ {0x010450, 0x01049d},
+ {0x0104b0, 0x0104d3},
+ {0x0104d8, 0x0104fb},
+ {0x010500, 0x010527},
+ {0x010530, 0x010563},
+ {0x010570, 0x01057a},
+ {0x01057c, 0x01058a},
+ {0x01058c, 0x010592},
+ {0x010594, 0x010595},
+ {0x010597, 0x0105a1},
+ {0x0105a3, 0x0105b1},
+ {0x0105b3, 0x0105b9},
+ {0x0105bb, 0x0105bc},
+ {0x010600, 0x010736},
+ {0x010740, 0x010755},
+ {0x010760, 0x010767},
+ {0x010780, 0x010785},
+ {0x010787, 0x0107b0},
+ {0x0107b2, 0x0107ba},
+ {0x010800, 0x010805},
+ {0x010808, 0x010808},
+ {0x01080a, 0x010835},
+ {0x010837, 0x010838},
+ {0x01083c, 0x01083c},
+ {0x01083f, 0x010855},
+ {0x010860, 0x010876},
+ {0x010880, 0x01089e},
+ {0x0108e0, 0x0108f2},
+ {0x0108f4, 0x0108f5},
+ {0x010900, 0x010915},
+ {0x010920, 0x010939},
+ {0x010980, 0x0109b7},
+ {0x0109be, 0x0109bf},
+ {0x010a00, 0x010a00},
+ {0x010a01, 0x010a03},
+ {0x010a05, 0x010a06},
+ {0x010a0c, 0x010a0f},
+ {0x010a10, 0x010a13},
+ {0x010a15, 0x010a17},
+ {0x010a19, 0x010a35},
+ {0x010a60, 0x010a7c},
+ {0x010a80, 0x010a9c},
+ {0x010ac0, 0x010ac7},
+ {0x010ac9, 0x010ae4},
+ {0x010b00, 0x010b35},
+ {0x010b40, 0x010b55},
+ {0x010b60, 0x010b72},
+ {0x010b80, 0x010b91},
+ {0x010c00, 0x010c48},
+ {0x010c80, 0x010cb2},
+ {0x010cc0, 0x010cf2},
+ {0x010d00, 0x010d23},
+ {0x010d24, 0x010d27},
+ {0x010e80, 0x010ea9},
+ {0x010eab, 0x010eac},
+ {0x010eb0, 0x010eb1},
+ {0x010f00, 0x010f1c},
+ {0x010f27, 0x010f27},
+ {0x010f30, 0x010f45},
+ {0x010f70, 0x010f81},
+ {0x010fb0, 0x010fc4},
+ {0x010fe0, 0x010ff6},
+ {0x011000, 0x011000},
+ {0x011001, 0x011001},
+ {0x011002, 0x011002},
+ {0x011003, 0x011037},
+ {0x011038, 0x011045},
+ {0x011071, 0x011072},
+ {0x011073, 0x011074},
+ {0x011075, 0x011075},
+ {0x011080, 0x011081},
+ {0x011082, 0x011082},
+ {0x011083, 0x0110af},
+ {0x0110b0, 0x0110b2},
+ {0x0110b3, 0x0110b6},
+ {0x0110b7, 0x0110b8},
+ {0x0110c2, 0x0110c2},
+ {0x0110d0, 0x0110e8},
+ {0x011100, 0x011102},
+ {0x011103, 0x011126},
+ {0x011127, 0x01112b},
+ {0x01112c, 0x01112c},
+ {0x01112d, 0x011132},
+ {0x011144, 0x011144},
+ {0x011145, 0x011146},
+ {0x011147, 0x011147},
+ {0x011150, 0x011172},
+ {0x011176, 0x011176},
+ {0x011180, 0x011181},
+ {0x011182, 0x011182},
+ {0x011183, 0x0111b2},
+ {0x0111b3, 0x0111b5},
+ {0x0111b6, 0x0111be},
+ {0x0111bf, 0x0111bf},
+ {0x0111c1, 0x0111c4},
+ {0x0111ce, 0x0111ce},
+ {0x0111cf, 0x0111cf},
+ {0x0111da, 0x0111da},
+ {0x0111dc, 0x0111dc},
+ {0x011200, 0x011211},
+ {0x011213, 0x01122b},
+ {0x01122c, 0x01122e},
+ {0x01122f, 0x011231},
+ {0x011232, 0x011233},
+ {0x011234, 0x011234},
+ {0x011237, 0x011237},
+ {0x01123e, 0x01123e},
+ {0x01123f, 0x011240},
+ {0x011241, 0x011241},
+ {0x011280, 0x011286},
+ {0x011288, 0x011288},
+ {0x01128a, 0x01128d},
+ {0x01128f, 0x01129d},
+ {0x01129f, 0x0112a8},
+ {0x0112b0, 0x0112de},
+ {0x0112df, 0x0112df},
+ {0x0112e0, 0x0112e2},
+ {0x0112e3, 0x0112e8},
+ {0x011300, 0x011301},
+ {0x011302, 0x011303},
+ {0x011305, 0x01130c},
+ {0x01130f, 0x011310},
+ {0x011313, 0x011328},
+ {0x01132a, 0x011330},
+ {0x011332, 0x011333},
+ {0x011335, 0x011339},
+ {0x01133d, 0x01133d},
+ {0x01133e, 0x01133f},
+ {0x011340, 0x011340},
+ {0x011341, 0x011344},
+ {0x011347, 0x011348},
+ {0x01134b, 0x01134c},
+ {0x011350, 0x011350},
+ {0x011357, 0x011357},
+ {0x01135d, 0x011361},
+ {0x011362, 0x011363},
+ {0x011400, 0x011434},
+ {0x011435, 0x011437},
+ {0x011438, 0x01143f},
+ {0x011440, 0x011441},
+ {0x011443, 0x011444},
+ {0x011445, 0x011445},
+ {0x011447, 0x01144a},
+ {0x01145f, 0x011461},
+ {0x011480, 0x0114af},
+ {0x0114b0, 0x0114b2},
+ {0x0114b3, 0x0114b8},
+ {0x0114b9, 0x0114b9},
+ {0x0114ba, 0x0114ba},
+ {0x0114bb, 0x0114be},
+ {0x0114bf, 0x0114c0},
+ {0x0114c1, 0x0114c1},
+ {0x0114c4, 0x0114c5},
+ {0x0114c7, 0x0114c7},
+ {0x011580, 0x0115ae},
+ {0x0115af, 0x0115b1},
+ {0x0115b2, 0x0115b5},
+ {0x0115b8, 0x0115bb},
+ {0x0115bc, 0x0115bd},
+ {0x0115be, 0x0115be},
+ {0x0115d8, 0x0115db},
+ {0x0115dc, 0x0115dd},
+ {0x011600, 0x01162f},
+ {0x011630, 0x011632},
+ {0x011633, 0x01163a},
+ {0x01163b, 0x01163c},
+ {0x01163d, 0x01163d},
+ {0x01163e, 0x01163e},
+ {0x011640, 0x011640},
+ {0x011644, 0x011644},
+ {0x011680, 0x0116aa},
+ {0x0116ab, 0x0116ab},
+ {0x0116ac, 0x0116ac},
+ {0x0116ad, 0x0116ad},
+ {0x0116ae, 0x0116af},
+ {0x0116b0, 0x0116b5},
+ {0x0116b8, 0x0116b8},
+ {0x011700, 0x01171a},
+ {0x01171d, 0x01171f},
+ {0x011720, 0x011721},
+ {0x011722, 0x011725},
+ {0x011726, 0x011726},
+ {0x011727, 0x01172a},
+ {0x011740, 0x011746},
+ {0x011800, 0x01182b},
+ {0x01182c, 0x01182e},
+ {0x01182f, 0x011837},
+ {0x011838, 0x011838},
+ {0x0118a0, 0x0118df},
+ {0x0118ff, 0x011906},
+ {0x011909, 0x011909},
+ {0x01190c, 0x011913},
+ {0x011915, 0x011916},
+ {0x011918, 0x01192f},
+ {0x011930, 0x011935},
+ {0x011937, 0x011938},
+ {0x01193b, 0x01193c},
+ {0x01193f, 0x01193f},
+ {0x011940, 0x011940},
+ {0x011941, 0x011941},
+ {0x011942, 0x011942},
+ {0x0119a0, 0x0119a7},
+ {0x0119aa, 0x0119d0},
+ {0x0119d1, 0x0119d3},
+ {0x0119d4, 0x0119d7},
+ {0x0119da, 0x0119db},
+ {0x0119dc, 0x0119df},
+ {0x0119e1, 0x0119e1},
+ {0x0119e3, 0x0119e3},
+ {0x0119e4, 0x0119e4},
+ {0x011a00, 0x011a00},
+ {0x011a01, 0x011a0a},
+ {0x011a0b, 0x011a32},
+ {0x011a35, 0x011a38},
+ {0x011a39, 0x011a39},
+ {0x011a3a, 0x011a3a},
+ {0x011a3b, 0x011a3e},
+ {0x011a50, 0x011a50},
+ {0x011a51, 0x011a56},
+ {0x011a57, 0x011a58},
+ {0x011a59, 0x011a5b},
+ {0x011a5c, 0x011a89},
+ {0x011a8a, 0x011a96},
+ {0x011a97, 0x011a97},
+ {0x011a9d, 0x011a9d},
+ {0x011ab0, 0x011af8},
+ {0x011c00, 0x011c08},
+ {0x011c0a, 0x011c2e},
+ {0x011c2f, 0x011c2f},
+ {0x011c30, 0x011c36},
+ {0x011c38, 0x011c3d},
+ {0x011c3e, 0x011c3e},
+ {0x011c40, 0x011c40},
+ {0x011c72, 0x011c8f},
+ {0x011c92, 0x011ca7},
+ {0x011ca9, 0x011ca9},
+ {0x011caa, 0x011cb0},
+ {0x011cb1, 0x011cb1},
+ {0x011cb2, 0x011cb3},
+ {0x011cb4, 0x011cb4},
+ {0x011cb5, 0x011cb6},
+ {0x011d00, 0x011d06},
+ {0x011d08, 0x011d09},
+ {0x011d0b, 0x011d30},
+ {0x011d31, 0x011d36},
+ {0x011d3a, 0x011d3a},
+ {0x011d3c, 0x011d3d},
+ {0x011d3f, 0x011d41},
+ {0x011d43, 0x011d43},
+ {0x011d46, 0x011d46},
+ {0x011d47, 0x011d47},
+ {0x011d60, 0x011d65},
+ {0x011d67, 0x011d68},
+ {0x011d6a, 0x011d89},
+ {0x011d8a, 0x011d8e},
+ {0x011d90, 0x011d91},
+ {0x011d93, 0x011d94},
+ {0x011d95, 0x011d95},
+ {0x011d96, 0x011d96},
+ {0x011d98, 0x011d98},
+ {0x011ee0, 0x011ef2},
+ {0x011ef3, 0x011ef4},
+ {0x011ef5, 0x011ef6},
+ {0x011f00, 0x011f01},
+ {0x011f02, 0x011f02},
+ {0x011f03, 0x011f03},
+ {0x011f04, 0x011f10},
+ {0x011f12, 0x011f33},
+ {0x011f34, 0x011f35},
+ {0x011f36, 0x011f3a},
+ {0x011f3e, 0x011f3f},
+ {0x011f40, 0x011f40},
+ {0x011fb0, 0x011fb0},
+ {0x012000, 0x012399},
+ {0x012400, 0x01246e},
+ {0x012480, 0x012543},
+ {0x012f90, 0x012ff0},
+ {0x013000, 0x01342f},
+ {0x013441, 0x013446},
+ {0x014400, 0x014646},
+ {0x016800, 0x016a38},
+ {0x016a40, 0x016a5e},
+ {0x016a70, 0x016abe},
+ {0x016ad0, 0x016aed},
+ {0x016b00, 0x016b2f},
+ {0x016b40, 0x016b43},
+ {0x016b63, 0x016b77},
+ {0x016b7d, 0x016b8f},
+ {0x016e40, 0x016e7f},
+ {0x016f00, 0x016f4a},
+ {0x016f4f, 0x016f4f},
+ {0x016f50, 0x016f50},
+ {0x016f51, 0x016f87},
+ {0x016f8f, 0x016f92},
+ {0x016f93, 0x016f9f},
+ {0x016fe0, 0x016fe1},
+ {0x016fe3, 0x016fe3},
+ {0x016ff0, 0x016ff1},
+ {0x017000, 0x0187f7},
+ {0x018800, 0x018cd5},
+ {0x018d00, 0x018d08},
+ {0x01aff0, 0x01aff3},
+ {0x01aff5, 0x01affb},
+ {0x01affd, 0x01affe},
+ {0x01b000, 0x01b122},
+ {0x01b132, 0x01b132},
+ {0x01b150, 0x01b152},
+ {0x01b155, 0x01b155},
+ {0x01b164, 0x01b167},
+ {0x01b170, 0x01b2fb},
+ {0x01bc00, 0x01bc6a},
+ {0x01bc70, 0x01bc7c},
+ {0x01bc80, 0x01bc88},
+ {0x01bc90, 0x01bc99},
+ {0x01bc9e, 0x01bc9e},
+ {0x01d400, 0x01d454},
+ {0x01d456, 0x01d49c},
+ {0x01d49e, 0x01d49f},
+ {0x01d4a2, 0x01d4a2},
+ {0x01d4a5, 0x01d4a6},
+ {0x01d4a9, 0x01d4ac},
+ {0x01d4ae, 0x01d4b9},
+ {0x01d4bb, 0x01d4bb},
+ {0x01d4bd, 0x01d4c3},
+ {0x01d4c5, 0x01d505},
+ {0x01d507, 0x01d50a},
+ {0x01d50d, 0x01d514},
+ {0x01d516, 0x01d51c},
+ {0x01d51e, 0x01d539},
+ {0x01d53b, 0x01d53e},
+ {0x01d540, 0x01d544},
+ {0x01d546, 0x01d546},
+ {0x01d54a, 0x01d550},
+ {0x01d552, 0x01d6a5},
+ {0x01d6a8, 0x01d6c0},
+ {0x01d6c2, 0x01d6da},
+ {0x01d6dc, 0x01d6fa},
+ {0x01d6fc, 0x01d714},
+ {0x01d716, 0x01d734},
+ {0x01d736, 0x01d74e},
+ {0x01d750, 0x01d76e},
+ {0x01d770, 0x01d788},
+ {0x01d78a, 0x01d7a8},
+ {0x01d7aa, 0x01d7c2},
+ {0x01d7c4, 0x01d7cb},
+ {0x01df00, 0x01df09},
+ {0x01df0a, 0x01df0a},
+ {0x01df0b, 0x01df1e},
+ {0x01df25, 0x01df2a},
+ {0x01e000, 0x01e006},
+ {0x01e008, 0x01e018},
+ {0x01e01b, 0x01e021},
+ {0x01e023, 0x01e024},
+ {0x01e026, 0x01e02a},
+ {0x01e030, 0x01e06d},
+ {0x01e08f, 0x01e08f},
+ {0x01e100, 0x01e12c},
+ {0x01e137, 0x01e13d},
+ {0x01e14e, 0x01e14e},
+ {0x01e290, 0x01e2ad},
+ {0x01e2c0, 0x01e2eb},
+ {0x01e4d0, 0x01e4ea},
+ {0x01e4eb, 0x01e4eb},
+ {0x01e7e0, 0x01e7e6},
+ {0x01e7e8, 0x01e7eb},
+ {0x01e7ed, 0x01e7ee},
+ {0x01e7f0, 0x01e7fe},
+ {0x01e800, 0x01e8c4},
+ {0x01e900, 0x01e943},
+ {0x01e947, 0x01e947},
+ {0x01e94b, 0x01e94b},
+ {0x01ee00, 0x01ee03},
+ {0x01ee05, 0x01ee1f},
+ {0x01ee21, 0x01ee22},
+ {0x01ee24, 0x01ee24},
+ {0x01ee27, 0x01ee27},
+ {0x01ee29, 0x01ee32},
+ {0x01ee34, 0x01ee37},
+ {0x01ee39, 0x01ee39},
+ {0x01ee3b, 0x01ee3b},
+ {0x01ee42, 0x01ee42},
+ {0x01ee47, 0x01ee47},
+ {0x01ee49, 0x01ee49},
+ {0x01ee4b, 0x01ee4b},
+ {0x01ee4d, 0x01ee4f},
+ {0x01ee51, 0x01ee52},
+ {0x01ee54, 0x01ee54},
+ {0x01ee57, 0x01ee57},
+ {0x01ee59, 0x01ee59},
+ {0x01ee5b, 0x01ee5b},
+ {0x01ee5d, 0x01ee5d},
+ {0x01ee5f, 0x01ee5f},
+ {0x01ee61, 0x01ee62},
+ {0x01ee64, 0x01ee64},
+ {0x01ee67, 0x01ee6a},
+ {0x01ee6c, 0x01ee72},
+ {0x01ee74, 0x01ee77},
+ {0x01ee79, 0x01ee7c},
+ {0x01ee7e, 0x01ee7e},
+ {0x01ee80, 0x01ee89},
+ {0x01ee8b, 0x01ee9b},
+ {0x01eea1, 0x01eea3},
+ {0x01eea5, 0x01eea9},
+ {0x01eeab, 0x01eebb},
+ {0x01f130, 0x01f149},
+ {0x01f150, 0x01f169},
+ {0x01f170, 0x01f189},
+ {0x020000, 0x02a6df},
+ {0x02a700, 0x02b739},
+ {0x02b740, 0x02b81d},
+ {0x02b820, 0x02cea1},
+ {0x02ceb0, 0x02ebe0},
+ {0x02ebf0, 0x02ee5d},
+ {0x02f800, 0x02fa1d},
+ {0x030000, 0x03134a},
+ {0x031350, 0x0323af},
+};
+
+/* table of Unicode codepoint ranges of Lowercase characters */
+static const pg_unicode_range unicode_lowercase[686] =
+{
+ {0x000061, 0x00007a},
+ {0x0000aa, 0x0000aa},
+ {0x0000b5, 0x0000b5},
+ {0x0000ba, 0x0000ba},
+ {0x0000df, 0x0000f6},
+ {0x0000f8, 0x0000ff},
+ {0x000101, 0x000101},
+ {0x000103, 0x000103},
+ {0x000105, 0x000105},
+ {0x000107, 0x000107},
+ {0x000109, 0x000109},
+ {0x00010b, 0x00010b},
+ {0x00010d, 0x00010d},
+ {0x00010f, 0x00010f},
+ {0x000111, 0x000111},
+ {0x000113, 0x000113},
+ {0x000115, 0x000115},
+ {0x000117, 0x000117},
+ {0x000119, 0x000119},
+ {0x00011b, 0x00011b},
+ {0x00011d, 0x00011d},
+ {0x00011f, 0x00011f},
+ {0x000121, 0x000121},
+ {0x000123, 0x000123},
+ {0x000125, 0x000125},
+ {0x000127, 0x000127},
+ {0x000129, 0x000129},
+ {0x00012b, 0x00012b},
+ {0x00012d, 0x00012d},
+ {0x00012f, 0x00012f},
+ {0x000131, 0x000131},
+ {0x000133, 0x000133},
+ {0x000135, 0x000135},
+ {0x000137, 0x000138},
+ {0x00013a, 0x00013a},
+ {0x00013c, 0x00013c},
+ {0x00013e, 0x00013e},
+ {0x000140, 0x000140},
+ {0x000142, 0x000142},
+ {0x000144, 0x000144},
+ {0x000146, 0x000146},
+ {0x000148, 0x000149},
+ {0x00014b, 0x00014b},
+ {0x00014d, 0x00014d},
+ {0x00014f, 0x00014f},
+ {0x000151, 0x000151},
+ {0x000153, 0x000153},
+ {0x000155, 0x000155},
+ {0x000157, 0x000157},
+ {0x000159, 0x000159},
+ {0x00015b, 0x00015b},
+ {0x00015d, 0x00015d},
+ {0x00015f, 0x00015f},
+ {0x000161, 0x000161},
+ {0x000163, 0x000163},
+ {0x000165, 0x000165},
+ {0x000167, 0x000167},
+ {0x000169, 0x000169},
+ {0x00016b, 0x00016b},
+ {0x00016d, 0x00016d},
+ {0x00016f, 0x00016f},
+ {0x000171, 0x000171},
+ {0x000173, 0x000173},
+ {0x000175, 0x000175},
+ {0x000177, 0x000177},
+ {0x00017a, 0x00017a},
+ {0x00017c, 0x00017c},
+ {0x00017e, 0x000180},
+ {0x000183, 0x000183},
+ {0x000185, 0x000185},
+ {0x000188, 0x000188},
+ {0x00018c, 0x00018d},
+ {0x000192, 0x000192},
+ {0x000195, 0x000195},
+ {0x000199, 0x00019b},
+ {0x00019e, 0x00019e},
+ {0x0001a1, 0x0001a1},
+ {0x0001a3, 0x0001a3},
+ {0x0001a5, 0x0001a5},
+ {0x0001a8, 0x0001a8},
+ {0x0001aa, 0x0001ab},
+ {0x0001ad, 0x0001ad},
+ {0x0001b0, 0x0001b0},
+ {0x0001b4, 0x0001b4},
+ {0x0001b6, 0x0001b6},
+ {0x0001b9, 0x0001ba},
+ {0x0001bd, 0x0001bf},
+ {0x0001c6, 0x0001c6},
+ {0x0001c9, 0x0001c9},
+ {0x0001cc, 0x0001cc},
+ {0x0001ce, 0x0001ce},
+ {0x0001d0, 0x0001d0},
+ {0x0001d2, 0x0001d2},
+ {0x0001d4, 0x0001d4},
+ {0x0001d6, 0x0001d6},
+ {0x0001d8, 0x0001d8},
+ {0x0001da, 0x0001da},
+ {0x0001dc, 0x0001dd},
+ {0x0001df, 0x0001df},
+ {0x0001e1, 0x0001e1},
+ {0x0001e3, 0x0001e3},
+ {0x0001e5, 0x0001e5},
+ {0x0001e7, 0x0001e7},
+ {0x0001e9, 0x0001e9},
+ {0x0001eb, 0x0001eb},
+ {0x0001ed, 0x0001ed},
+ {0x0001ef, 0x0001f0},
+ {0x0001f3, 0x0001f3},
+ {0x0001f5, 0x0001f5},
+ {0x0001f9, 0x0001f9},
+ {0x0001fb, 0x0001fb},
+ {0x0001fd, 0x0001fd},
+ {0x0001ff, 0x0001ff},
+ {0x000201, 0x000201},
+ {0x000203, 0x000203},
+ {0x000205, 0x000205},
+ {0x000207, 0x000207},
+ {0x000209, 0x000209},
+ {0x00020b, 0x00020b},
+ {0x00020d, 0x00020d},
+ {0x00020f, 0x00020f},
+ {0x000211, 0x000211},
+ {0x000213, 0x000213},
+ {0x000215, 0x000215},
+ {0x000217, 0x000217},
+ {0x000219, 0x000219},
+ {0x00021b, 0x00021b},
+ {0x00021d, 0x00021d},
+ {0x00021f, 0x00021f},
+ {0x000221, 0x000221},
+ {0x000223, 0x000223},
+ {0x000225, 0x000225},
+ {0x000227, 0x000227},
+ {0x000229, 0x000229},
+ {0x00022b, 0x00022b},
+ {0x00022d, 0x00022d},
+ {0x00022f, 0x00022f},
+ {0x000231, 0x000231},
+ {0x000233, 0x000239},
+ {0x00023c, 0x00023c},
+ {0x00023f, 0x000240},
+ {0x000242, 0x000242},
+ {0x000247, 0x000247},
+ {0x000249, 0x000249},
+ {0x00024b, 0x00024b},
+ {0x00024d, 0x00024d},
+ {0x00024f, 0x000293},
+ {0x000295, 0x0002af},
+ {0x0002b0, 0x0002b8},
+ {0x0002c0, 0x0002c1},
+ {0x0002e0, 0x0002e4},
+ {0x000345, 0x000345},
+ {0x000371, 0x000371},
+ {0x000373, 0x000373},
+ {0x000377, 0x000377},
+ {0x00037a, 0x00037a},
+ {0x00037b, 0x00037d},
+ {0x000390, 0x000390},
+ {0x0003ac, 0x0003ce},
+ {0x0003d0, 0x0003d1},
+ {0x0003d5, 0x0003d7},
+ {0x0003d9, 0x0003d9},
+ {0x0003db, 0x0003db},
+ {0x0003dd, 0x0003dd},
+ {0x0003df, 0x0003df},
+ {0x0003e1, 0x0003e1},
+ {0x0003e3, 0x0003e3},
+ {0x0003e5, 0x0003e5},
+ {0x0003e7, 0x0003e7},
+ {0x0003e9, 0x0003e9},
+ {0x0003eb, 0x0003eb},
+ {0x0003ed, 0x0003ed},
+ {0x0003ef, 0x0003f3},
+ {0x0003f5, 0x0003f5},
+ {0x0003f8, 0x0003f8},
+ {0x0003fb, 0x0003fc},
+ {0x000430, 0x00045f},
+ {0x000461, 0x000461},
+ {0x000463, 0x000463},
+ {0x000465, 0x000465},
+ {0x000467, 0x000467},
+ {0x000469, 0x000469},
+ {0x00046b, 0x00046b},
+ {0x00046d, 0x00046d},
+ {0x00046f, 0x00046f},
+ {0x000471, 0x000471},
+ {0x000473, 0x000473},
+ {0x000475, 0x000475},
+ {0x000477, 0x000477},
+ {0x000479, 0x000479},
+ {0x00047b, 0x00047b},
+ {0x00047d, 0x00047d},
+ {0x00047f, 0x00047f},
+ {0x000481, 0x000481},
+ {0x00048b, 0x00048b},
+ {0x00048d, 0x00048d},
+ {0x00048f, 0x00048f},
+ {0x000491, 0x000491},
+ {0x000493, 0x000493},
+ {0x000495, 0x000495},
+ {0x000497, 0x000497},
+ {0x000499, 0x000499},
+ {0x00049b, 0x00049b},
+ {0x00049d, 0x00049d},
+ {0x00049f, 0x00049f},
+ {0x0004a1, 0x0004a1},
+ {0x0004a3, 0x0004a3},
+ {0x0004a5, 0x0004a5},
+ {0x0004a7, 0x0004a7},
+ {0x0004a9, 0x0004a9},
+ {0x0004ab, 0x0004ab},
+ {0x0004ad, 0x0004ad},
+ {0x0004af, 0x0004af},
+ {0x0004b1, 0x0004b1},
+ {0x0004b3, 0x0004b3},
+ {0x0004b5, 0x0004b5},
+ {0x0004b7, 0x0004b7},
+ {0x0004b9, 0x0004b9},
+ {0x0004bb, 0x0004bb},
+ {0x0004bd, 0x0004bd},
+ {0x0004bf, 0x0004bf},
+ {0x0004c2, 0x0004c2},
+ {0x0004c4, 0x0004c4},
+ {0x0004c6, 0x0004c6},
+ {0x0004c8, 0x0004c8},
+ {0x0004ca, 0x0004ca},
+ {0x0004cc, 0x0004cc},
+ {0x0004ce, 0x0004cf},
+ {0x0004d1, 0x0004d1},
+ {0x0004d3, 0x0004d3},
+ {0x0004d5, 0x0004d5},
+ {0x0004d7, 0x0004d7},
+ {0x0004d9, 0x0004d9},
+ {0x0004db, 0x0004db},
+ {0x0004dd, 0x0004dd},
+ {0x0004df, 0x0004df},
+ {0x0004e1, 0x0004e1},
+ {0x0004e3, 0x0004e3},
+ {0x0004e5, 0x0004e5},
+ {0x0004e7, 0x0004e7},
+ {0x0004e9, 0x0004e9},
+ {0x0004eb, 0x0004eb},
+ {0x0004ed, 0x0004ed},
+ {0x0004ef, 0x0004ef},
+ {0x0004f1, 0x0004f1},
+ {0x0004f3, 0x0004f3},
+ {0x0004f5, 0x0004f5},
+ {0x0004f7, 0x0004f7},
+ {0x0004f9, 0x0004f9},
+ {0x0004fb, 0x0004fb},
+ {0x0004fd, 0x0004fd},
+ {0x0004ff, 0x0004ff},
+ {0x000501, 0x000501},
+ {0x000503, 0x000503},
+ {0x000505, 0x000505},
+ {0x000507, 0x000507},
+ {0x000509, 0x000509},
+ {0x00050b, 0x00050b},
+ {0x00050d, 0x00050d},
+ {0x00050f, 0x00050f},
+ {0x000511, 0x000511},
+ {0x000513, 0x000513},
+ {0x000515, 0x000515},
+ {0x000517, 0x000517},
+ {0x000519, 0x000519},
+ {0x00051b, 0x00051b},
+ {0x00051d, 0x00051d},
+ {0x00051f, 0x00051f},
+ {0x000521, 0x000521},
+ {0x000523, 0x000523},
+ {0x000525, 0x000525},
+ {0x000527, 0x000527},
+ {0x000529, 0x000529},
+ {0x00052b, 0x00052b},
+ {0x00052d, 0x00052d},
+ {0x00052f, 0x00052f},
+ {0x000560, 0x000588},
+ {0x0010d0, 0x0010fa},
+ {0x0010fc, 0x0010fc},
+ {0x0010fd, 0x0010ff},
+ {0x0013f8, 0x0013fd},
+ {0x001c80, 0x001c88},
+ {0x001d00, 0x001d2b},
+ {0x001d2c, 0x001d6a},
+ {0x001d6b, 0x001d77},
+ {0x001d78, 0x001d78},
+ {0x001d79, 0x001d9a},
+ {0x001d9b, 0x001dbf},
+ {0x001e01, 0x001e01},
+ {0x001e03, 0x001e03},
+ {0x001e05, 0x001e05},
+ {0x001e07, 0x001e07},
+ {0x001e09, 0x001e09},
+ {0x001e0b, 0x001e0b},
+ {0x001e0d, 0x001e0d},
+ {0x001e0f, 0x001e0f},
+ {0x001e11, 0x001e11},
+ {0x001e13, 0x001e13},
+ {0x001e15, 0x001e15},
+ {0x001e17, 0x001e17},
+ {0x001e19, 0x001e19},
+ {0x001e1b, 0x001e1b},
+ {0x001e1d, 0x001e1d},
+ {0x001e1f, 0x001e1f},
+ {0x001e21, 0x001e21},
+ {0x001e23, 0x001e23},
+ {0x001e25, 0x001e25},
+ {0x001e27, 0x001e27},
+ {0x001e29, 0x001e29},
+ {0x001e2b, 0x001e2b},
+ {0x001e2d, 0x001e2d},
+ {0x001e2f, 0x001e2f},
+ {0x001e31, 0x001e31},
+ {0x001e33, 0x001e33},
+ {0x001e35, 0x001e35},
+ {0x001e37, 0x001e37},
+ {0x001e39, 0x001e39},
+ {0x001e3b, 0x001e3b},
+ {0x001e3d, 0x001e3d},
+ {0x001e3f, 0x001e3f},
+ {0x001e41, 0x001e41},
+ {0x001e43, 0x001e43},
+ {0x001e45, 0x001e45},
+ {0x001e47, 0x001e47},
+ {0x001e49, 0x001e49},
+ {0x001e4b, 0x001e4b},
+ {0x001e4d, 0x001e4d},
+ {0x001e4f, 0x001e4f},
+ {0x001e51, 0x001e51},
+ {0x001e53, 0x001e53},
+ {0x001e55, 0x001e55},
+ {0x001e57, 0x001e57},
+ {0x001e59, 0x001e59},
+ {0x001e5b, 0x001e5b},
+ {0x001e5d, 0x001e5d},
+ {0x001e5f, 0x001e5f},
+ {0x001e61, 0x001e61},
+ {0x001e63, 0x001e63},
+ {0x001e65, 0x001e65},
+ {0x001e67, 0x001e67},
+ {0x001e69, 0x001e69},
+ {0x001e6b, 0x001e6b},
+ {0x001e6d, 0x001e6d},
+ {0x001e6f, 0x001e6f},
+ {0x001e71, 0x001e71},
+ {0x001e73, 0x001e73},
+ {0x001e75, 0x001e75},
+ {0x001e77, 0x001e77},
+ {0x001e79, 0x001e79},
+ {0x001e7b, 0x001e7b},
+ {0x001e7d, 0x001e7d},
+ {0x001e7f, 0x001e7f},
+ {0x001e81, 0x001e81},
+ {0x001e83, 0x001e83},
+ {0x001e85, 0x001e85},
+ {0x001e87, 0x001e87},
+ {0x001e89, 0x001e89},
+ {0x001e8b, 0x001e8b},
+ {0x001e8d, 0x001e8d},
+ {0x001e8f, 0x001e8f},
+ {0x001e91, 0x001e91},
+ {0x001e93, 0x001e93},
+ {0x001e95, 0x001e9d},
+ {0x001e9f, 0x001e9f},
+ {0x001ea1, 0x001ea1},
+ {0x001ea3, 0x001ea3},
+ {0x001ea5, 0x001ea5},
+ {0x001ea7, 0x001ea7},
+ {0x001ea9, 0x001ea9},
+ {0x001eab, 0x001eab},
+ {0x001ead, 0x001ead},
+ {0x001eaf, 0x001eaf},
+ {0x001eb1, 0x001eb1},
+ {0x001eb3, 0x001eb3},
+ {0x001eb5, 0x001eb5},
+ {0x001eb7, 0x001eb7},
+ {0x001eb9, 0x001eb9},
+ {0x001ebb, 0x001ebb},
+ {0x001ebd, 0x001ebd},
+ {0x001ebf, 0x001ebf},
+ {0x001ec1, 0x001ec1},
+ {0x001ec3, 0x001ec3},
+ {0x001ec5, 0x001ec5},
+ {0x001ec7, 0x001ec7},
+ {0x001ec9, 0x001ec9},
+ {0x001ecb, 0x001ecb},
+ {0x001ecd, 0x001ecd},
+ {0x001ecf, 0x001ecf},
+ {0x001ed1, 0x001ed1},
+ {0x001ed3, 0x001ed3},
+ {0x001ed5, 0x001ed5},
+ {0x001ed7, 0x001ed7},
+ {0x001ed9, 0x001ed9},
+ {0x001edb, 0x001edb},
+ {0x001edd, 0x001edd},
+ {0x001edf, 0x001edf},
+ {0x001ee1, 0x001ee1},
+ {0x001ee3, 0x001ee3},
+ {0x001ee5, 0x001ee5},
+ {0x001ee7, 0x001ee7},
+ {0x001ee9, 0x001ee9},
+ {0x001eeb, 0x001eeb},
+ {0x001eed, 0x001eed},
+ {0x001eef, 0x001eef},
+ {0x001ef1, 0x001ef1},
+ {0x001ef3, 0x001ef3},
+ {0x001ef5, 0x001ef5},
+ {0x001ef7, 0x001ef7},
+ {0x001ef9, 0x001ef9},
+ {0x001efb, 0x001efb},
+ {0x001efd, 0x001efd},
+ {0x001eff, 0x001f07},
+ {0x001f10, 0x001f15},
+ {0x001f20, 0x001f27},
+ {0x001f30, 0x001f37},
+ {0x001f40, 0x001f45},
+ {0x001f50, 0x001f57},
+ {0x001f60, 0x001f67},
+ {0x001f70, 0x001f7d},
+ {0x001f80, 0x001f87},
+ {0x001f90, 0x001f97},
+ {0x001fa0, 0x001fa7},
+ {0x001fb0, 0x001fb4},
+ {0x001fb6, 0x001fb7},
+ {0x001fbe, 0x001fbe},
+ {0x001fc2, 0x001fc4},
+ {0x001fc6, 0x001fc7},
+ {0x001fd0, 0x001fd3},
+ {0x001fd6, 0x001fd7},
+ {0x001fe0, 0x001fe7},
+ {0x001ff2, 0x001ff4},
+ {0x001ff6, 0x001ff7},
+ {0x002071, 0x002071},
+ {0x00207f, 0x00207f},
+ {0x002090, 0x00209c},
+ {0x00210a, 0x00210a},
+ {0x00210e, 0x00210f},
+ {0x002113, 0x002113},
+ {0x00212f, 0x00212f},
+ {0x002134, 0x002134},
+ {0x002139, 0x002139},
+ {0x00213c, 0x00213d},
+ {0x002146, 0x002149},
+ {0x00214e, 0x00214e},
+ {0x002170, 0x00217f},
+ {0x002184, 0x002184},
+ {0x0024d0, 0x0024e9},
+ {0x002c30, 0x002c5f},
+ {0x002c61, 0x002c61},
+ {0x002c65, 0x002c66},
+ {0x002c68, 0x002c68},
+ {0x002c6a, 0x002c6a},
+ {0x002c6c, 0x002c6c},
+ {0x002c71, 0x002c71},
+ {0x002c73, 0x002c74},
+ {0x002c76, 0x002c7b},
+ {0x002c7c, 0x002c7d},
+ {0x002c81, 0x002c81},
+ {0x002c83, 0x002c83},
+ {0x002c85, 0x002c85},
+ {0x002c87, 0x002c87},
+ {0x002c89, 0x002c89},
+ {0x002c8b, 0x002c8b},
+ {0x002c8d, 0x002c8d},
+ {0x002c8f, 0x002c8f},
+ {0x002c91, 0x002c91},
+ {0x002c93, 0x002c93},
+ {0x002c95, 0x002c95},
+ {0x002c97, 0x002c97},
+ {0x002c99, 0x002c99},
+ {0x002c9b, 0x002c9b},
+ {0x002c9d, 0x002c9d},
+ {0x002c9f, 0x002c9f},
+ {0x002ca1, 0x002ca1},
+ {0x002ca3, 0x002ca3},
+ {0x002ca5, 0x002ca5},
+ {0x002ca7, 0x002ca7},
+ {0x002ca9, 0x002ca9},
+ {0x002cab, 0x002cab},
+ {0x002cad, 0x002cad},
+ {0x002caf, 0x002caf},
+ {0x002cb1, 0x002cb1},
+ {0x002cb3, 0x002cb3},
+ {0x002cb5, 0x002cb5},
+ {0x002cb7, 0x002cb7},
+ {0x002cb9, 0x002cb9},
+ {0x002cbb, 0x002cbb},
+ {0x002cbd, 0x002cbd},
+ {0x002cbf, 0x002cbf},
+ {0x002cc1, 0x002cc1},
+ {0x002cc3, 0x002cc3},
+ {0x002cc5, 0x002cc5},
+ {0x002cc7, 0x002cc7},
+ {0x002cc9, 0x002cc9},
+ {0x002ccb, 0x002ccb},
+ {0x002ccd, 0x002ccd},
+ {0x002ccf, 0x002ccf},
+ {0x002cd1, 0x002cd1},
+ {0x002cd3, 0x002cd3},
+ {0x002cd5, 0x002cd5},
+ {0x002cd7, 0x002cd7},
+ {0x002cd9, 0x002cd9},
+ {0x002cdb, 0x002cdb},
+ {0x002cdd, 0x002cdd},
+ {0x002cdf, 0x002cdf},
+ {0x002ce1, 0x002ce1},
+ {0x002ce3, 0x002ce4},
+ {0x002cec, 0x002cec},
+ {0x002cee, 0x002cee},
+ {0x002cf3, 0x002cf3},
+ {0x002d00, 0x002d25},
+ {0x002d27, 0x002d27},
+ {0x002d2d, 0x002d2d},
+ {0x00a641, 0x00a641},
+ {0x00a643, 0x00a643},
+ {0x00a645, 0x00a645},
+ {0x00a647, 0x00a647},
+ {0x00a649, 0x00a649},
+ {0x00a64b, 0x00a64b},
+ {0x00a64d, 0x00a64d},
+ {0x00a64f, 0x00a64f},
+ {0x00a651, 0x00a651},
+ {0x00a653, 0x00a653},
+ {0x00a655, 0x00a655},
+ {0x00a657, 0x00a657},
+ {0x00a659, 0x00a659},
+ {0x00a65b, 0x00a65b},
+ {0x00a65d, 0x00a65d},
+ {0x00a65f, 0x00a65f},
+ {0x00a661, 0x00a661},
+ {0x00a663, 0x00a663},
+ {0x00a665, 0x00a665},
+ {0x00a667, 0x00a667},
+ {0x00a669, 0x00a669},
+ {0x00a66b, 0x00a66b},
+ {0x00a66d, 0x00a66d},
+ {0x00a681, 0x00a681},
+ {0x00a683, 0x00a683},
+ {0x00a685, 0x00a685},
+ {0x00a687, 0x00a687},
+ {0x00a689, 0x00a689},
+ {0x00a68b, 0x00a68b},
+ {0x00a68d, 0x00a68d},
+ {0x00a68f, 0x00a68f},
+ {0x00a691, 0x00a691},
+ {0x00a693, 0x00a693},
+ {0x00a695, 0x00a695},
+ {0x00a697, 0x00a697},
+ {0x00a699, 0x00a699},
+ {0x00a69b, 0x00a69b},
+ {0x00a69c, 0x00a69d},
+ {0x00a723, 0x00a723},
+ {0x00a725, 0x00a725},
+ {0x00a727, 0x00a727},
+ {0x00a729, 0x00a729},
+ {0x00a72b, 0x00a72b},
+ {0x00a72d, 0x00a72d},
+ {0x00a72f, 0x00a731},
+ {0x00a733, 0x00a733},
+ {0x00a735, 0x00a735},
+ {0x00a737, 0x00a737},
+ {0x00a739, 0x00a739},
+ {0x00a73b, 0x00a73b},
+ {0x00a73d, 0x00a73d},
+ {0x00a73f, 0x00a73f},
+ {0x00a741, 0x00a741},
+ {0x00a743, 0x00a743},
+ {0x00a745, 0x00a745},
+ {0x00a747, 0x00a747},
+ {0x00a749, 0x00a749},
+ {0x00a74b, 0x00a74b},
+ {0x00a74d, 0x00a74d},
+ {0x00a74f, 0x00a74f},
+ {0x00a751, 0x00a751},
+ {0x00a753, 0x00a753},
+ {0x00a755, 0x00a755},
+ {0x00a757, 0x00a757},
+ {0x00a759, 0x00a759},
+ {0x00a75b, 0x00a75b},
+ {0x00a75d, 0x00a75d},
+ {0x00a75f, 0x00a75f},
+ {0x00a761, 0x00a761},
+ {0x00a763, 0x00a763},
+ {0x00a765, 0x00a765},
+ {0x00a767, 0x00a767},
+ {0x00a769, 0x00a769},
+ {0x00a76b, 0x00a76b},
+ {0x00a76d, 0x00a76d},
+ {0x00a76f, 0x00a76f},
+ {0x00a770, 0x00a770},
+ {0x00a771, 0x00a778},
+ {0x00a77a, 0x00a77a},
+ {0x00a77c, 0x00a77c},
+ {0x00a77f, 0x00a77f},
+ {0x00a781, 0x00a781},
+ {0x00a783, 0x00a783},
+ {0x00a785, 0x00a785},
+ {0x00a787, 0x00a787},
+ {0x00a78c, 0x00a78c},
+ {0x00a78e, 0x00a78e},
+ {0x00a791, 0x00a791},
+ {0x00a793, 0x00a795},
+ {0x00a797, 0x00a797},
+ {0x00a799, 0x00a799},
+ {0x00a79b, 0x00a79b},
+ {0x00a79d, 0x00a79d},
+ {0x00a79f, 0x00a79f},
+ {0x00a7a1, 0x00a7a1},
+ {0x00a7a3, 0x00a7a3},
+ {0x00a7a5, 0x00a7a5},
+ {0x00a7a7, 0x00a7a7},
+ {0x00a7a9, 0x00a7a9},
+ {0x00a7af, 0x00a7af},
+ {0x00a7b5, 0x00a7b5},
+ {0x00a7b7, 0x00a7b7},
+ {0x00a7b9, 0x00a7b9},
+ {0x00a7bb, 0x00a7bb},
+ {0x00a7bd, 0x00a7bd},
+ {0x00a7bf, 0x00a7bf},
+ {0x00a7c1, 0x00a7c1},
+ {0x00a7c3, 0x00a7c3},
+ {0x00a7c8, 0x00a7c8},
+ {0x00a7ca, 0x00a7ca},
+ {0x00a7d1, 0x00a7d1},
+ {0x00a7d3, 0x00a7d3},
+ {0x00a7d5, 0x00a7d5},
+ {0x00a7d7, 0x00a7d7},
+ {0x00a7d9, 0x00a7d9},
+ {0x00a7f2, 0x00a7f4},
+ {0x00a7f6, 0x00a7f6},
+ {0x00a7f8, 0x00a7f9},
+ {0x00a7fa, 0x00a7fa},
+ {0x00ab30, 0x00ab5a},
+ {0x00ab5c, 0x00ab5f},
+ {0x00ab60, 0x00ab68},
+ {0x00ab69, 0x00ab69},
+ {0x00ab70, 0x00abbf},
+ {0x00fb00, 0x00fb06},
+ {0x00fb13, 0x00fb17},
+ {0x00ff41, 0x00ff5a},
+ {0x010428, 0x01044f},
+ {0x0104d8, 0x0104fb},
+ {0x010597, 0x0105a1},
+ {0x0105a3, 0x0105b1},
+ {0x0105b3, 0x0105b9},
+ {0x0105bb, 0x0105bc},
+ {0x010780, 0x010780},
+ {0x010783, 0x010785},
+ {0x010787, 0x0107b0},
+ {0x0107b2, 0x0107ba},
+ {0x010cc0, 0x010cf2},
+ {0x0118c0, 0x0118df},
+ {0x016e60, 0x016e7f},
+ {0x01d41a, 0x01d433},
+ {0x01d44e, 0x01d454},
+ {0x01d456, 0x01d467},
+ {0x01d482, 0x01d49b},
+ {0x01d4b6, 0x01d4b9},
+ {0x01d4bb, 0x01d4bb},
+ {0x01d4bd, 0x01d4c3},
+ {0x01d4c5, 0x01d4cf},
+ {0x01d4ea, 0x01d503},
+ {0x01d51e, 0x01d537},
+ {0x01d552, 0x01d56b},
+ {0x01d586, 0x01d59f},
+ {0x01d5ba, 0x01d5d3},
+ {0x01d5ee, 0x01d607},
+ {0x01d622, 0x01d63b},
+ {0x01d656, 0x01d66f},
+ {0x01d68a, 0x01d6a5},
+ {0x01d6c2, 0x01d6da},
+ {0x01d6dc, 0x01d6e1},
+ {0x01d6fc, 0x01d714},
+ {0x01d716, 0x01d71b},
+ {0x01d736, 0x01d74e},
+ {0x01d750, 0x01d755},
+ {0x01d770, 0x01d788},
+ {0x01d78a, 0x01d78f},
+ {0x01d7aa, 0x01d7c2},
+ {0x01d7c4, 0x01d7c9},
+ {0x01d7cb, 0x01d7cb},
+ {0x01df00, 0x01df09},
+ {0x01df0b, 0x01df1e},
+ {0x01df25, 0x01df2a},
+ {0x01e030, 0x01e06d},
+ {0x01e922, 0x01e943},
+};
+
+/* table of Unicode codepoint ranges of Uppercase characters */
+static const pg_unicode_range unicode_uppercase[651] =
+{
+ {0x000041, 0x00005a},
+ {0x0000c0, 0x0000d6},
+ {0x0000d8, 0x0000de},
+ {0x000100, 0x000100},
+ {0x000102, 0x000102},
+ {0x000104, 0x000104},
+ {0x000106, 0x000106},
+ {0x000108, 0x000108},
+ {0x00010a, 0x00010a},
+ {0x00010c, 0x00010c},
+ {0x00010e, 0x00010e},
+ {0x000110, 0x000110},
+ {0x000112, 0x000112},
+ {0x000114, 0x000114},
+ {0x000116, 0x000116},
+ {0x000118, 0x000118},
+ {0x00011a, 0x00011a},
+ {0x00011c, 0x00011c},
+ {0x00011e, 0x00011e},
+ {0x000120, 0x000120},
+ {0x000122, 0x000122},
+ {0x000124, 0x000124},
+ {0x000126, 0x000126},
+ {0x000128, 0x000128},
+ {0x00012a, 0x00012a},
+ {0x00012c, 0x00012c},
+ {0x00012e, 0x00012e},
+ {0x000130, 0x000130},
+ {0x000132, 0x000132},
+ {0x000134, 0x000134},
+ {0x000136, 0x000136},
+ {0x000139, 0x000139},
+ {0x00013b, 0x00013b},
+ {0x00013d, 0x00013d},
+ {0x00013f, 0x00013f},
+ {0x000141, 0x000141},
+ {0x000143, 0x000143},
+ {0x000145, 0x000145},
+ {0x000147, 0x000147},
+ {0x00014a, 0x00014a},
+ {0x00014c, 0x00014c},
+ {0x00014e, 0x00014e},
+ {0x000150, 0x000150},
+ {0x000152, 0x000152},
+ {0x000154, 0x000154},
+ {0x000156, 0x000156},
+ {0x000158, 0x000158},
+ {0x00015a, 0x00015a},
+ {0x00015c, 0x00015c},
+ {0x00015e, 0x00015e},
+ {0x000160, 0x000160},
+ {0x000162, 0x000162},
+ {0x000164, 0x000164},
+ {0x000166, 0x000166},
+ {0x000168, 0x000168},
+ {0x00016a, 0x00016a},
+ {0x00016c, 0x00016c},
+ {0x00016e, 0x00016e},
+ {0x000170, 0x000170},
+ {0x000172, 0x000172},
+ {0x000174, 0x000174},
+ {0x000176, 0x000176},
+ {0x000178, 0x000179},
+ {0x00017b, 0x00017b},
+ {0x00017d, 0x00017d},
+ {0x000181, 0x000182},
+ {0x000184, 0x000184},
+ {0x000186, 0x000187},
+ {0x000189, 0x00018b},
+ {0x00018e, 0x000191},
+ {0x000193, 0x000194},
+ {0x000196, 0x000198},
+ {0x00019c, 0x00019d},
+ {0x00019f, 0x0001a0},
+ {0x0001a2, 0x0001a2},
+ {0x0001a4, 0x0001a4},
+ {0x0001a6, 0x0001a7},
+ {0x0001a9, 0x0001a9},
+ {0x0001ac, 0x0001ac},
+ {0x0001ae, 0x0001af},
+ {0x0001b1, 0x0001b3},
+ {0x0001b5, 0x0001b5},
+ {0x0001b7, 0x0001b8},
+ {0x0001bc, 0x0001bc},
+ {0x0001c4, 0x0001c4},
+ {0x0001c7, 0x0001c7},
+ {0x0001ca, 0x0001ca},
+ {0x0001cd, 0x0001cd},
+ {0x0001cf, 0x0001cf},
+ {0x0001d1, 0x0001d1},
+ {0x0001d3, 0x0001d3},
+ {0x0001d5, 0x0001d5},
+ {0x0001d7, 0x0001d7},
+ {0x0001d9, 0x0001d9},
+ {0x0001db, 0x0001db},
+ {0x0001de, 0x0001de},
+ {0x0001e0, 0x0001e0},
+ {0x0001e2, 0x0001e2},
+ {0x0001e4, 0x0001e4},
+ {0x0001e6, 0x0001e6},
+ {0x0001e8, 0x0001e8},
+ {0x0001ea, 0x0001ea},
+ {0x0001ec, 0x0001ec},
+ {0x0001ee, 0x0001ee},
+ {0x0001f1, 0x0001f1},
+ {0x0001f4, 0x0001f4},
+ {0x0001f6, 0x0001f8},
+ {0x0001fa, 0x0001fa},
+ {0x0001fc, 0x0001fc},
+ {0x0001fe, 0x0001fe},
+ {0x000200, 0x000200},
+ {0x000202, 0x000202},
+ {0x000204, 0x000204},
+ {0x000206, 0x000206},
+ {0x000208, 0x000208},
+ {0x00020a, 0x00020a},
+ {0x00020c, 0x00020c},
+ {0x00020e, 0x00020e},
+ {0x000210, 0x000210},
+ {0x000212, 0x000212},
+ {0x000214, 0x000214},
+ {0x000216, 0x000216},
+ {0x000218, 0x000218},
+ {0x00021a, 0x00021a},
+ {0x00021c, 0x00021c},
+ {0x00021e, 0x00021e},
+ {0x000220, 0x000220},
+ {0x000222, 0x000222},
+ {0x000224, 0x000224},
+ {0x000226, 0x000226},
+ {0x000228, 0x000228},
+ {0x00022a, 0x00022a},
+ {0x00022c, 0x00022c},
+ {0x00022e, 0x00022e},
+ {0x000230, 0x000230},
+ {0x000232, 0x000232},
+ {0x00023a, 0x00023b},
+ {0x00023d, 0x00023e},
+ {0x000241, 0x000241},
+ {0x000243, 0x000246},
+ {0x000248, 0x000248},
+ {0x00024a, 0x00024a},
+ {0x00024c, 0x00024c},
+ {0x00024e, 0x00024e},
+ {0x000370, 0x000370},
+ {0x000372, 0x000372},
+ {0x000376, 0x000376},
+ {0x00037f, 0x00037f},
+ {0x000386, 0x000386},
+ {0x000388, 0x00038a},
+ {0x00038c, 0x00038c},
+ {0x00038e, 0x00038f},
+ {0x000391, 0x0003a1},
+ {0x0003a3, 0x0003ab},
+ {0x0003cf, 0x0003cf},
+ {0x0003d2, 0x0003d4},
+ {0x0003d8, 0x0003d8},
+ {0x0003da, 0x0003da},
+ {0x0003dc, 0x0003dc},
+ {0x0003de, 0x0003de},
+ {0x0003e0, 0x0003e0},
+ {0x0003e2, 0x0003e2},
+ {0x0003e4, 0x0003e4},
+ {0x0003e6, 0x0003e6},
+ {0x0003e8, 0x0003e8},
+ {0x0003ea, 0x0003ea},
+ {0x0003ec, 0x0003ec},
+ {0x0003ee, 0x0003ee},
+ {0x0003f4, 0x0003f4},
+ {0x0003f7, 0x0003f7},
+ {0x0003f9, 0x0003fa},
+ {0x0003fd, 0x00042f},
+ {0x000460, 0x000460},
+ {0x000462, 0x000462},
+ {0x000464, 0x000464},
+ {0x000466, 0x000466},
+ {0x000468, 0x000468},
+ {0x00046a, 0x00046a},
+ {0x00046c, 0x00046c},
+ {0x00046e, 0x00046e},
+ {0x000470, 0x000470},
+ {0x000472, 0x000472},
+ {0x000474, 0x000474},
+ {0x000476, 0x000476},
+ {0x000478, 0x000478},
+ {0x00047a, 0x00047a},
+ {0x00047c, 0x00047c},
+ {0x00047e, 0x00047e},
+ {0x000480, 0x000480},
+ {0x00048a, 0x00048a},
+ {0x00048c, 0x00048c},
+ {0x00048e, 0x00048e},
+ {0x000490, 0x000490},
+ {0x000492, 0x000492},
+ {0x000494, 0x000494},
+ {0x000496, 0x000496},
+ {0x000498, 0x000498},
+ {0x00049a, 0x00049a},
+ {0x00049c, 0x00049c},
+ {0x00049e, 0x00049e},
+ {0x0004a0, 0x0004a0},
+ {0x0004a2, 0x0004a2},
+ {0x0004a4, 0x0004a4},
+ {0x0004a6, 0x0004a6},
+ {0x0004a8, 0x0004a8},
+ {0x0004aa, 0x0004aa},
+ {0x0004ac, 0x0004ac},
+ {0x0004ae, 0x0004ae},
+ {0x0004b0, 0x0004b0},
+ {0x0004b2, 0x0004b2},
+ {0x0004b4, 0x0004b4},
+ {0x0004b6, 0x0004b6},
+ {0x0004b8, 0x0004b8},
+ {0x0004ba, 0x0004ba},
+ {0x0004bc, 0x0004bc},
+ {0x0004be, 0x0004be},
+ {0x0004c0, 0x0004c1},
+ {0x0004c3, 0x0004c3},
+ {0x0004c5, 0x0004c5},
+ {0x0004c7, 0x0004c7},
+ {0x0004c9, 0x0004c9},
+ {0x0004cb, 0x0004cb},
+ {0x0004cd, 0x0004cd},
+ {0x0004d0, 0x0004d0},
+ {0x0004d2, 0x0004d2},
+ {0x0004d4, 0x0004d4},
+ {0x0004d6, 0x0004d6},
+ {0x0004d8, 0x0004d8},
+ {0x0004da, 0x0004da},
+ {0x0004dc, 0x0004dc},
+ {0x0004de, 0x0004de},
+ {0x0004e0, 0x0004e0},
+ {0x0004e2, 0x0004e2},
+ {0x0004e4, 0x0004e4},
+ {0x0004e6, 0x0004e6},
+ {0x0004e8, 0x0004e8},
+ {0x0004ea, 0x0004ea},
+ {0x0004ec, 0x0004ec},
+ {0x0004ee, 0x0004ee},
+ {0x0004f0, 0x0004f0},
+ {0x0004f2, 0x0004f2},
+ {0x0004f4, 0x0004f4},
+ {0x0004f6, 0x0004f6},
+ {0x0004f8, 0x0004f8},
+ {0x0004fa, 0x0004fa},
+ {0x0004fc, 0x0004fc},
+ {0x0004fe, 0x0004fe},
+ {0x000500, 0x000500},
+ {0x000502, 0x000502},
+ {0x000504, 0x000504},
+ {0x000506, 0x000506},
+ {0x000508, 0x000508},
+ {0x00050a, 0x00050a},
+ {0x00050c, 0x00050c},
+ {0x00050e, 0x00050e},
+ {0x000510, 0x000510},
+ {0x000512, 0x000512},
+ {0x000514, 0x000514},
+ {0x000516, 0x000516},
+ {0x000518, 0x000518},
+ {0x00051a, 0x00051a},
+ {0x00051c, 0x00051c},
+ {0x00051e, 0x00051e},
+ {0x000520, 0x000520},
+ {0x000522, 0x000522},
+ {0x000524, 0x000524},
+ {0x000526, 0x000526},
+ {0x000528, 0x000528},
+ {0x00052a, 0x00052a},
+ {0x00052c, 0x00052c},
+ {0x00052e, 0x00052e},
+ {0x000531, 0x000556},
+ {0x0010a0, 0x0010c5},
+ {0x0010c7, 0x0010c7},
+ {0x0010cd, 0x0010cd},
+ {0x0013a0, 0x0013f5},
+ {0x001c90, 0x001cba},
+ {0x001cbd, 0x001cbf},
+ {0x001e00, 0x001e00},
+ {0x001e02, 0x001e02},
+ {0x001e04, 0x001e04},
+ {0x001e06, 0x001e06},
+ {0x001e08, 0x001e08},
+ {0x001e0a, 0x001e0a},
+ {0x001e0c, 0x001e0c},
+ {0x001e0e, 0x001e0e},
+ {0x001e10, 0x001e10},
+ {0x001e12, 0x001e12},
+ {0x001e14, 0x001e14},
+ {0x001e16, 0x001e16},
+ {0x001e18, 0x001e18},
+ {0x001e1a, 0x001e1a},
+ {0x001e1c, 0x001e1c},
+ {0x001e1e, 0x001e1e},
+ {0x001e20, 0x001e20},
+ {0x001e22, 0x001e22},
+ {0x001e24, 0x001e24},
+ {0x001e26, 0x001e26},
+ {0x001e28, 0x001e28},
+ {0x001e2a, 0x001e2a},
+ {0x001e2c, 0x001e2c},
+ {0x001e2e, 0x001e2e},
+ {0x001e30, 0x001e30},
+ {0x001e32, 0x001e32},
+ {0x001e34, 0x001e34},
+ {0x001e36, 0x001e36},
+ {0x001e38, 0x001e38},
+ {0x001e3a, 0x001e3a},
+ {0x001e3c, 0x001e3c},
+ {0x001e3e, 0x001e3e},
+ {0x001e40, 0x001e40},
+ {0x001e42, 0x001e42},
+ {0x001e44, 0x001e44},
+ {0x001e46, 0x001e46},
+ {0x001e48, 0x001e48},
+ {0x001e4a, 0x001e4a},
+ {0x001e4c, 0x001e4c},
+ {0x001e4e, 0x001e4e},
+ {0x001e50, 0x001e50},
+ {0x001e52, 0x001e52},
+ {0x001e54, 0x001e54},
+ {0x001e56, 0x001e56},
+ {0x001e58, 0x001e58},
+ {0x001e5a, 0x001e5a},
+ {0x001e5c, 0x001e5c},
+ {0x001e5e, 0x001e5e},
+ {0x001e60, 0x001e60},
+ {0x001e62, 0x001e62},
+ {0x001e64, 0x001e64},
+ {0x001e66, 0x001e66},
+ {0x001e68, 0x001e68},
+ {0x001e6a, 0x001e6a},
+ {0x001e6c, 0x001e6c},
+ {0x001e6e, 0x001e6e},
+ {0x001e70, 0x001e70},
+ {0x001e72, 0x001e72},
+ {0x001e74, 0x001e74},
+ {0x001e76, 0x001e76},
+ {0x001e78, 0x001e78},
+ {0x001e7a, 0x001e7a},
+ {0x001e7c, 0x001e7c},
+ {0x001e7e, 0x001e7e},
+ {0x001e80, 0x001e80},
+ {0x001e82, 0x001e82},
+ {0x001e84, 0x001e84},
+ {0x001e86, 0x001e86},
+ {0x001e88, 0x001e88},
+ {0x001e8a, 0x001e8a},
+ {0x001e8c, 0x001e8c},
+ {0x001e8e, 0x001e8e},
+ {0x001e90, 0x001e90},
+ {0x001e92, 0x001e92},
+ {0x001e94, 0x001e94},
+ {0x001e9e, 0x001e9e},
+ {0x001ea0, 0x001ea0},
+ {0x001ea2, 0x001ea2},
+ {0x001ea4, 0x001ea4},
+ {0x001ea6, 0x001ea6},
+ {0x001ea8, 0x001ea8},
+ {0x001eaa, 0x001eaa},
+ {0x001eac, 0x001eac},
+ {0x001eae, 0x001eae},
+ {0x001eb0, 0x001eb0},
+ {0x001eb2, 0x001eb2},
+ {0x001eb4, 0x001eb4},
+ {0x001eb6, 0x001eb6},
+ {0x001eb8, 0x001eb8},
+ {0x001eba, 0x001eba},
+ {0x001ebc, 0x001ebc},
+ {0x001ebe, 0x001ebe},
+ {0x001ec0, 0x001ec0},
+ {0x001ec2, 0x001ec2},
+ {0x001ec4, 0x001ec4},
+ {0x001ec6, 0x001ec6},
+ {0x001ec8, 0x001ec8},
+ {0x001eca, 0x001eca},
+ {0x001ecc, 0x001ecc},
+ {0x001ece, 0x001ece},
+ {0x001ed0, 0x001ed0},
+ {0x001ed2, 0x001ed2},
+ {0x001ed4, 0x001ed4},
+ {0x001ed6, 0x001ed6},
+ {0x001ed8, 0x001ed8},
+ {0x001eda, 0x001eda},
+ {0x001edc, 0x001edc},
+ {0x001ede, 0x001ede},
+ {0x001ee0, 0x001ee0},
+ {0x001ee2, 0x001ee2},
+ {0x001ee4, 0x001ee4},
+ {0x001ee6, 0x001ee6},
+ {0x001ee8, 0x001ee8},
+ {0x001eea, 0x001eea},
+ {0x001eec, 0x001eec},
+ {0x001eee, 0x001eee},
+ {0x001ef0, 0x001ef0},
+ {0x001ef2, 0x001ef2},
+ {0x001ef4, 0x001ef4},
+ {0x001ef6, 0x001ef6},
+ {0x001ef8, 0x001ef8},
+ {0x001efa, 0x001efa},
+ {0x001efc, 0x001efc},
+ {0x001efe, 0x001efe},
+ {0x001f08, 0x001f0f},
+ {0x001f18, 0x001f1d},
+ {0x001f28, 0x001f2f},
+ {0x001f38, 0x001f3f},
+ {0x001f48, 0x001f4d},
+ {0x001f59, 0x001f59},
+ {0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f5f},
+ {0x001f68, 0x001f6f},
+ {0x001fb8, 0x001fbb},
+ {0x001fc8, 0x001fcb},
+ {0x001fd8, 0x001fdb},
+ {0x001fe8, 0x001fec},
+ {0x001ff8, 0x001ffb},
+ {0x002102, 0x002102},
+ {0x002107, 0x002107},
+ {0x00210b, 0x00210d},
+ {0x002110, 0x002112},
+ {0x002115, 0x002115},
+ {0x002119, 0x00211d},
+ {0x002124, 0x002124},
+ {0x002126, 0x002126},
+ {0x002128, 0x002128},
+ {0x00212a, 0x00212d},
+ {0x002130, 0x002133},
+ {0x00213e, 0x00213f},
+ {0x002145, 0x002145},
+ {0x002160, 0x00216f},
+ {0x002183, 0x002183},
+ {0x0024b6, 0x0024cf},
+ {0x002c00, 0x002c2f},
+ {0x002c60, 0x002c60},
+ {0x002c62, 0x002c64},
+ {0x002c67, 0x002c67},
+ {0x002c69, 0x002c69},
+ {0x002c6b, 0x002c6b},
+ {0x002c6d, 0x002c70},
+ {0x002c72, 0x002c72},
+ {0x002c75, 0x002c75},
+ {0x002c7e, 0x002c80},
+ {0x002c82, 0x002c82},
+ {0x002c84, 0x002c84},
+ {0x002c86, 0x002c86},
+ {0x002c88, 0x002c88},
+ {0x002c8a, 0x002c8a},
+ {0x002c8c, 0x002c8c},
+ {0x002c8e, 0x002c8e},
+ {0x002c90, 0x002c90},
+ {0x002c92, 0x002c92},
+ {0x002c94, 0x002c94},
+ {0x002c96, 0x002c96},
+ {0x002c98, 0x002c98},
+ {0x002c9a, 0x002c9a},
+ {0x002c9c, 0x002c9c},
+ {0x002c9e, 0x002c9e},
+ {0x002ca0, 0x002ca0},
+ {0x002ca2, 0x002ca2},
+ {0x002ca4, 0x002ca4},
+ {0x002ca6, 0x002ca6},
+ {0x002ca8, 0x002ca8},
+ {0x002caa, 0x002caa},
+ {0x002cac, 0x002cac},
+ {0x002cae, 0x002cae},
+ {0x002cb0, 0x002cb0},
+ {0x002cb2, 0x002cb2},
+ {0x002cb4, 0x002cb4},
+ {0x002cb6, 0x002cb6},
+ {0x002cb8, 0x002cb8},
+ {0x002cba, 0x002cba},
+ {0x002cbc, 0x002cbc},
+ {0x002cbe, 0x002cbe},
+ {0x002cc0, 0x002cc0},
+ {0x002cc2, 0x002cc2},
+ {0x002cc4, 0x002cc4},
+ {0x002cc6, 0x002cc6},
+ {0x002cc8, 0x002cc8},
+ {0x002cca, 0x002cca},
+ {0x002ccc, 0x002ccc},
+ {0x002cce, 0x002cce},
+ {0x002cd0, 0x002cd0},
+ {0x002cd2, 0x002cd2},
+ {0x002cd4, 0x002cd4},
+ {0x002cd6, 0x002cd6},
+ {0x002cd8, 0x002cd8},
+ {0x002cda, 0x002cda},
+ {0x002cdc, 0x002cdc},
+ {0x002cde, 0x002cde},
+ {0x002ce0, 0x002ce0},
+ {0x002ce2, 0x002ce2},
+ {0x002ceb, 0x002ceb},
+ {0x002ced, 0x002ced},
+ {0x002cf2, 0x002cf2},
+ {0x00a640, 0x00a640},
+ {0x00a642, 0x00a642},
+ {0x00a644, 0x00a644},
+ {0x00a646, 0x00a646},
+ {0x00a648, 0x00a648},
+ {0x00a64a, 0x00a64a},
+ {0x00a64c, 0x00a64c},
+ {0x00a64e, 0x00a64e},
+ {0x00a650, 0x00a650},
+ {0x00a652, 0x00a652},
+ {0x00a654, 0x00a654},
+ {0x00a656, 0x00a656},
+ {0x00a658, 0x00a658},
+ {0x00a65a, 0x00a65a},
+ {0x00a65c, 0x00a65c},
+ {0x00a65e, 0x00a65e},
+ {0x00a660, 0x00a660},
+ {0x00a662, 0x00a662},
+ {0x00a664, 0x00a664},
+ {0x00a666, 0x00a666},
+ {0x00a668, 0x00a668},
+ {0x00a66a, 0x00a66a},
+ {0x00a66c, 0x00a66c},
+ {0x00a680, 0x00a680},
+ {0x00a682, 0x00a682},
+ {0x00a684, 0x00a684},
+ {0x00a686, 0x00a686},
+ {0x00a688, 0x00a688},
+ {0x00a68a, 0x00a68a},
+ {0x00a68c, 0x00a68c},
+ {0x00a68e, 0x00a68e},
+ {0x00a690, 0x00a690},
+ {0x00a692, 0x00a692},
+ {0x00a694, 0x00a694},
+ {0x00a696, 0x00a696},
+ {0x00a698, 0x00a698},
+ {0x00a69a, 0x00a69a},
+ {0x00a722, 0x00a722},
+ {0x00a724, 0x00a724},
+ {0x00a726, 0x00a726},
+ {0x00a728, 0x00a728},
+ {0x00a72a, 0x00a72a},
+ {0x00a72c, 0x00a72c},
+ {0x00a72e, 0x00a72e},
+ {0x00a732, 0x00a732},
+ {0x00a734, 0x00a734},
+ {0x00a736, 0x00a736},
+ {0x00a738, 0x00a738},
+ {0x00a73a, 0x00a73a},
+ {0x00a73c, 0x00a73c},
+ {0x00a73e, 0x00a73e},
+ {0x00a740, 0x00a740},
+ {0x00a742, 0x00a742},
+ {0x00a744, 0x00a744},
+ {0x00a746, 0x00a746},
+ {0x00a748, 0x00a748},
+ {0x00a74a, 0x00a74a},
+ {0x00a74c, 0x00a74c},
+ {0x00a74e, 0x00a74e},
+ {0x00a750, 0x00a750},
+ {0x00a752, 0x00a752},
+ {0x00a754, 0x00a754},
+ {0x00a756, 0x00a756},
+ {0x00a758, 0x00a758},
+ {0x00a75a, 0x00a75a},
+ {0x00a75c, 0x00a75c},
+ {0x00a75e, 0x00a75e},
+ {0x00a760, 0x00a760},
+ {0x00a762, 0x00a762},
+ {0x00a764, 0x00a764},
+ {0x00a766, 0x00a766},
+ {0x00a768, 0x00a768},
+ {0x00a76a, 0x00a76a},
+ {0x00a76c, 0x00a76c},
+ {0x00a76e, 0x00a76e},
+ {0x00a779, 0x00a779},
+ {0x00a77b, 0x00a77b},
+ {0x00a77d, 0x00a77e},
+ {0x00a780, 0x00a780},
+ {0x00a782, 0x00a782},
+ {0x00a784, 0x00a784},
+ {0x00a786, 0x00a786},
+ {0x00a78b, 0x00a78b},
+ {0x00a78d, 0x00a78d},
+ {0x00a790, 0x00a790},
+ {0x00a792, 0x00a792},
+ {0x00a796, 0x00a796},
+ {0x00a798, 0x00a798},
+ {0x00a79a, 0x00a79a},
+ {0x00a79c, 0x00a79c},
+ {0x00a79e, 0x00a79e},
+ {0x00a7a0, 0x00a7a0},
+ {0x00a7a2, 0x00a7a2},
+ {0x00a7a4, 0x00a7a4},
+ {0x00a7a6, 0x00a7a6},
+ {0x00a7a8, 0x00a7a8},
+ {0x00a7aa, 0x00a7ae},
+ {0x00a7b0, 0x00a7b4},
+ {0x00a7b6, 0x00a7b6},
+ {0x00a7b8, 0x00a7b8},
+ {0x00a7ba, 0x00a7ba},
+ {0x00a7bc, 0x00a7bc},
+ {0x00a7be, 0x00a7be},
+ {0x00a7c0, 0x00a7c0},
+ {0x00a7c2, 0x00a7c2},
+ {0x00a7c4, 0x00a7c7},
+ {0x00a7c9, 0x00a7c9},
+ {0x00a7d0, 0x00a7d0},
+ {0x00a7d6, 0x00a7d6},
+ {0x00a7d8, 0x00a7d8},
+ {0x00a7f5, 0x00a7f5},
+ {0x00ff21, 0x00ff3a},
+ {0x010400, 0x010427},
+ {0x0104b0, 0x0104d3},
+ {0x010570, 0x01057a},
+ {0x01057c, 0x01058a},
+ {0x01058c, 0x010592},
+ {0x010594, 0x010595},
+ {0x010c80, 0x010cb2},
+ {0x0118a0, 0x0118bf},
+ {0x016e40, 0x016e5f},
+ {0x01d400, 0x01d419},
+ {0x01d434, 0x01d44d},
+ {0x01d468, 0x01d481},
+ {0x01d49c, 0x01d49c},
+ {0x01d49e, 0x01d49f},
+ {0x01d4a2, 0x01d4a2},
+ {0x01d4a5, 0x01d4a6},
+ {0x01d4a9, 0x01d4ac},
+ {0x01d4ae, 0x01d4b5},
+ {0x01d4d0, 0x01d4e9},
+ {0x01d504, 0x01d505},
+ {0x01d507, 0x01d50a},
+ {0x01d50d, 0x01d514},
+ {0x01d516, 0x01d51c},
+ {0x01d538, 0x01d539},
+ {0x01d53b, 0x01d53e},
+ {0x01d540, 0x01d544},
+ {0x01d546, 0x01d546},
+ {0x01d54a, 0x01d550},
+ {0x01d56c, 0x01d585},
+ {0x01d5a0, 0x01d5b9},
+ {0x01d5d4, 0x01d5ed},
+ {0x01d608, 0x01d621},
+ {0x01d63c, 0x01d655},
+ {0x01d670, 0x01d689},
+ {0x01d6a8, 0x01d6c0},
+ {0x01d6e2, 0x01d6fa},
+ {0x01d71c, 0x01d734},
+ {0x01d756, 0x01d76e},
+ {0x01d790, 0x01d7a8},
+ {0x01d7ca, 0x01d7ca},
+ {0x01e900, 0x01e921},
+ {0x01f130, 0x01f149},
+ {0x01f150, 0x01f169},
+ {0x01f170, 0x01f189},
+};
+
+/* table of Unicode codepoint ranges of White_Space characters */
+static const pg_unicode_range unicode_white_space[11] =
+{
+ {0x000009, 0x00000d},
+ {0x000020, 0x000020},
+ {0x000085, 0x000085},
+ {0x0000a0, 0x0000a0},
+ {0x001680, 0x001680},
+ {0x002000, 0x00200a},
+ {0x002028, 0x002028},
+ {0x002029, 0x002029},
+ {0x00202f, 0x00202f},
+ {0x00205f, 0x00205f},
+ {0x003000, 0x003000},
+};
+
+/* table of Unicode codepoint ranges of Hex_Digit characters */
+static const pg_unicode_range unicode_hex_digit[6] =
+{
+ {0x000030, 0x000039},
+ {0x000041, 0x000046},
+ {0x000061, 0x000066},
+ {0x00ff10, 0x00ff19},
+ {0x00ff21, 0x00ff26},
+ {0x00ff41, 0x00ff46},
+};
+
+/* table of Unicode codepoint ranges of Join_Control characters */
+static const pg_unicode_range unicode_join_control[1] =
+{
+ {0x00200c, 0x00200d},
};
--
2.34.1
[text/x-patch] v17-0002-Add-unicode-case-mapping-tables-and-functions.patch (143.6K, ../../[email protected]/3-v17-0002-Add-unicode-case-mapping-tables-and-functions.patch)
download | inline diff:
From 0004e44a99e12703543bc61ee8494abd9ee88706 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Mon, 30 Oct 2023 17:38:54 -0700
Subject: [PATCH v17 2/4] Add unicode case mapping tables and functions.
---
src/common/Makefile | 1 +
src/common/meson.build | 1 +
src/common/unicode/Makefile | 13 +-
src/common/unicode/case_test.c | 100 +
.../unicode/generate-unicode_case_table.pl | 128 +
src/common/unicode/meson.build | 31 +
src/common/unicode_case.c | 88 +
src/include/common/unicode_case.h | 23 +
src/include/common/unicode_case_table.h | 2994 +++++++++++++++++
9 files changed, 3377 insertions(+), 2 deletions(-)
create mode 100644 src/common/unicode/case_test.c
create mode 100644 src/common/unicode/generate-unicode_case_table.pl
create mode 100644 src/common/unicode_case.c
create mode 100644 src/include/common/unicode_case.h
create mode 100644 src/include/common/unicode_case_table.h
diff --git a/src/common/Makefile b/src/common/Makefile
index 2ba5069dca..3d83299432 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -78,6 +78,7 @@ OBJS_COMMON = \
scram-common.o \
string.o \
stringinfo.o \
+ unicode_case.o \
unicode_category.o \
unicode_norm.o \
username.o \
diff --git a/src/common/meson.build b/src/common/meson.build
index 4eb16024cb..de68e408fa 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -32,6 +32,7 @@ common_sources = files(
'scram-common.c',
'string.c',
'stringinfo.c',
+ 'unicode_case.c',
'unicode_category.c',
'unicode_norm.c',
'username.c',
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 27f0408d8b..8223d02375 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -21,8 +21,9 @@ CPPFLAGS += $(ICU_CFLAGS)
# By default, do nothing.
all:
-update-unicode: unicode_category_table.h unicode_east_asian_fw_table.h unicode_nonspacing_table.h unicode_norm_hashfunc.h unicode_norm_table.h unicode_normprops_table.h unicode_version.h
+update-unicode: unicode_case_table.h unicode_category_table.h unicode_east_asian_fw_table.h unicode_nonspacing_table.h unicode_norm_hashfunc.h unicode_norm_table.h unicode_normprops_table.h unicode_version.h
mv $^ $(top_srcdir)/src/include/common/
+ $(MAKE) case-check
$(MAKE) category-check
$(MAKE) normalization-check
@@ -35,6 +36,9 @@ CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.tx
unicode_version.h: generate-unicode_version.pl
$(PERL) $< --version $(UNICODE_VERSION)
+unicode_case_table.h: generate-unicode_case_table.pl UnicodeData.txt
+ $(PERL) $<
+
unicode_category_table.h: generate-unicode_category_table.pl DerivedCoreProperties.txt PropList.txt UnicodeData.txt
$(PERL) $<
@@ -55,12 +59,17 @@ unicode_normprops_table.h: generate-unicode_normprops_table.pl DerivedNormalizat
$(PERL) $^ >$@
# Test suite
+case-check: case_test
+ ./case_test
+
category-check: category_test
./category_test
normalization-check: norm_test
./norm_test
+case_test: case_test.o ../unicode_case.o | submake-common
+
category_test: category_test.o ../unicode_category.o | submake-common
norm_test: norm_test.o ../unicode_norm.o | submake-common
@@ -82,4 +91,4 @@ clean:
rm -f $(OBJS) category_test category_test.o norm_test norm_test.o
distclean: clean
- rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_category_table.h unicode_norm_table.h
+ rm -f CompositionExclusions.txt DerivedCoreProperties.txt DerivedNormalizationProps.txt EastAsianWidth.txt NormalizationTest.txt PropList.txt UnicodeData.txt norm_test_table.h unicode_case_table.h unicode_category_table.h unicode_norm_table.h
diff --git a/src/common/unicode/case_test.c b/src/common/unicode/case_test.c
new file mode 100644
index 0000000000..7b82d5e0aa
--- /dev/null
+++ b/src/common/unicode/case_test.c
@@ -0,0 +1,100 @@
+/*-------------------------------------------------------------------------
+ * case_test.c
+ * Program to test Unicode case mapping functions.
+ *
+ * Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/unicode/case_test.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres_fe.h"
+
+#include <locale.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <wctype.h>
+
+#ifdef USE_ICU
+#include <unicode/uchar.h>
+#endif
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
+#include "common/unicode_version.h"
+
+#ifdef USE_ICU
+
+static void
+icu_test_simple(pg_wchar code)
+{
+ pg_wchar lower = unicode_lowercase_simple(code);
+ pg_wchar title = unicode_titlecase_simple(code);
+ pg_wchar upper = unicode_uppercase_simple(code);
+ pg_wchar iculower = u_tolower(code);
+ pg_wchar icutitle = u_totitle(code);
+ pg_wchar icuupper = u_toupper(code);
+
+ if (lower != iculower || title != icutitle || upper != icuupper)
+ {
+ printf("case_test: FAILURE for codepoint 0x%06x\n", code);
+ printf("case_test: Postgres lower/title/upper: 0x%06x/0x%06x/0x%06x\n",
+ lower, title, upper);
+ printf("case_test: ICU lower/title/upper: 0x%06x/0x%06x/0x%06x\n",
+ iculower, icutitle, icuupper);
+ printf("\n");
+ exit(1);
+ }
+}
+
+static void
+test_icu(void)
+{
+ int successful = 0;
+ int skipped_mismatch = 0;
+
+ for (pg_wchar code = 0; code <= 0x10ffff; code++)
+ {
+ pg_unicode_category category = unicode_category(code);
+
+ if (category != PG_U_UNASSIGNED)
+ {
+ uint8_t icu_category = u_charType(code);
+
+ if (icu_category == PG_U_UNASSIGNED)
+ {
+ skipped_mismatch++;
+ continue;
+ }
+
+ icu_test_simple(code);
+ successful++;
+ }
+ }
+
+ if (skipped_mismatch > 0)
+ printf("case_test: skipped %d codepoints unassigned in ICU due to Unicode version mismatch\n",
+ skipped_mismatch);
+
+ printf("case_test: ICU simple mapping test: %d codepoints successful\n",
+ successful);
+}
+#endif
+
+/*
+ * Exhaustively compare case mappings with the results from libc and ICU.
+ */
+int
+main(int argc, char **argv)
+{
+ printf("case_test: Postgres Unicode version:\t%s\n", PG_UNICODE_VERSION);
+#ifdef USE_ICU
+ printf("case_test: ICU Unicode version:\t\t%s\n", U_UNICODE_VERSION);
+ test_icu();
+#else
+ printf("case_test: ICU not available; skipping\n");
+#endif
+
+ exit(0);
+}
diff --git a/src/common/unicode/generate-unicode_case_table.pl b/src/common/unicode/generate-unicode_case_table.pl
new file mode 100644
index 0000000000..e8854de944
--- /dev/null
+++ b/src/common/unicode/generate-unicode_case_table.pl
@@ -0,0 +1,128 @@
+#!/usr/bin/perl
+#
+# Generate Unicode character case mappings. Does not include tailoring
+# or locale-specific mappings.
+#
+# Input: UnicodeData.txt
+# Output: unicode_case_table.h
+#
+# Copyright (c) 2000-2023, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+use Getopt::Long;
+
+use FindBin;
+use lib "$FindBin::RealBin/../../tools/";
+
+my $output_path = '.';
+
+GetOptions('outdir:s' => \$output_path);
+
+my $output_table_file = "$output_path/unicode_case_table.h";
+
+# The maximum number of codepoints that can result from case mapping
+# of a single character. See Unicode section 5.18 "Case Mappings".
+my $MAX_CASE_EXPANSION = 3;
+
+my $FH;
+
+my %simple = ();
+
+open($FH, '<', "$output_path/UnicodeData.txt")
+ or die "Could not open $output_path/UnicodeData.txt: $!.";
+while (my $line = <$FH>)
+{
+ my @elts = split(';', $line);
+ my $code = hex($elts[0]);
+ my $simple_uppercase = hex($elts[12] =~ s/^\s+|\s+$//rg);
+ my $simple_lowercase = hex($elts[13] =~ s/^\s+|\s+$//rg);
+ my $simple_titlecase = hex($elts[14] =~ s/^\s+|\s+$//rg);
+
+ die "codepoint $code out of range" if $code > 0x10FFFF;
+ die "Simple_Lowercase $code out of range" if $simple_lowercase > 0x10FFFF;
+ die "Simple_Titlecase $code out of range" if $simple_titlecase > 0x10FFFF;
+ die "Simple_Uppercase $code out of range" if $simple_uppercase > 0x10FFFF;
+
+ if ($simple_lowercase || $simple_titlecase || $simple_uppercase)
+ {
+ $simple{$code} = {
+ Simple_Lowercase => ($simple_lowercase || $code),
+ Simple_Titlecase => ($simple_titlecase || $code),
+ Simple_Uppercase => ($simple_uppercase || $code)
+ };
+ }
+}
+close $FH;
+
+# Start writing out the output files
+open my $OT, '>', $output_table_file
+ or die "Could not open output file $output_table_file: $!\n";
+
+# determine size of array given that codepoints <= 0x80 are dense and
+# the rest of the entries are sparse
+my $num_simple = 0x80;
+foreach my $code (sort { $a <=> $b } (keys %simple))
+{
+ $num_simple++ unless $code < 0x80;
+}
+
+print $OT <<"EOS";
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case_table.h
+ * Case mapping and information table.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/unicode_case_table.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+/*
+ * File auto-generated by src/common/unicode/generate-unicode_case_table.pl,
+ * do not edit. There is deliberately not an #ifndef PG_UNICODE_CASE_TABLE_H
+ * here.
+ */
+
+#include "mb/pg_wchar.h"
+
+typedef struct
+{
+ pg_wchar codepoint; /* Unicode codepoint */
+ pg_wchar simple_lowercase;
+ pg_wchar simple_titlecase;
+ pg_wchar simple_uppercase;
+} pg_simple_case_map;
+
+/*
+ * Case mapping table. Dense for codepoints < 0x80 (enabling fast lookup),
+ * sparse for higher codepoints (requiring scan or binary search).
+ */
+static const pg_simple_case_map simple_case_map[$num_simple] =
+{
+EOS
+
+printf $OT "\t/* begin dense entries for codepoints < 0x80 */\n";
+for (my $code = 0; $code < 0x80; $code++)
+{
+ my $lc = ($simple{$code}{Simple_Lowercase} || $code);
+ my $tc = ($simple{$code}{Simple_Titlecase} || $code);
+ my $uc = ($simple{$code}{Simple_Uppercase} || $code);
+ printf $OT "\t{0x%06x, 0x%06x, 0x%06x, 0x%06x},\n", $code, $lc, $tc, $uc;
+}
+printf $OT "\n";
+
+printf $OT "\t/* begin sparse entries for codepoints >= 0x80 */\n";
+foreach my $code (sort { $a <=> $b } (keys %simple))
+{
+ next unless $code >= 0x80; # already output above
+
+ my $map = $simple{$code};
+ printf $OT "\t{0x%06x, 0x%06x, 0x%06x, 0x%06x},\n",
+ $code, $map->{Simple_Lowercase}, $map->{Simple_Titlecase},
+ $map->{Simple_Uppercase};
+}
+print $OT "};\n";
diff --git a/src/common/unicode/meson.build b/src/common/unicode/meson.build
index d7190bb8ca..b9a4181c32 100644
--- a/src/common/unicode/meson.build
+++ b/src/common/unicode/meson.build
@@ -24,6 +24,16 @@ endforeach
update_unicode_targets = []
+update_unicode_targets += \
+ custom_target('unicode_case_table.h',
+ input: [unicode_data['UnicodeData.txt']],
+ output: ['unicode_case_table.h'],
+ command: [
+ perl, files('generate-unicode_case_table.pl'),
+ '--outdir', '@OUTDIR@', '@INPUT@'],
+ build_by_default: false,
+ )
+
update_unicode_targets += \
custom_target('unicode_category_table.h',
input: [unicode_data['UnicodeData.txt'], unicode_data['DerivedCoreProperties.txt'], unicode_data['PropList.txt']],
@@ -92,6 +102,17 @@ norm_test_table = custom_target('norm_test_table.h',
inc = include_directories('.')
+case_test = executable('case_test',
+ ['case_test.c'],
+ dependencies: [frontend_port_code, icu],
+ include_directories: inc,
+ link_with: [common_static, pgport_static],
+ build_by_default: false,
+ kwargs: default_bin_args + {
+ 'install': false,
+ }
+)
+
category_test = executable('category_test',
['category_test.c'],
dependencies: [frontend_port_code, icu],
@@ -116,6 +137,16 @@ norm_test = executable('norm_test',
update_unicode_dep = []
+if not meson.is_cross_build()
+ update_unicode_dep += custom_target('case_test.run',
+ output: 'case_test.run',
+ input: update_unicode_targets,
+ command: [case_test, UNICODE_VERSION],
+ build_by_default: false,
+ build_always_stale: true,
+ )
+endif
+
if not meson.is_cross_build()
update_unicode_dep += custom_target('category_test.run',
output: 'category_test.run',
diff --git a/src/common/unicode_case.c b/src/common/unicode_case.c
new file mode 100644
index 0000000000..043e063a53
--- /dev/null
+++ b/src/common/unicode_case.c
@@ -0,0 +1,88 @@
+/*-------------------------------------------------------------------------
+ * unicode_case.c
+ * Conversion to upper or lower case.
+ *
+ * Portions Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/unicode_case.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/unicode_case.h"
+#include "common/unicode_case_table.h"
+
+/* find entry in simple case map, if any */
+static inline const pg_simple_case_map *
+find_case_map(pg_wchar ucs)
+{
+ int min = 0;
+ int mid;
+ int max = lengthof(simple_case_map) - 1;
+
+ /* all chars <= 0x80 are stored in array for fast lookup */
+ Assert(max >= 0x7f);
+ if (ucs < 0x80)
+ {
+ const pg_simple_case_map *map = &simple_case_map[ucs];
+
+ Assert(map->codepoint == ucs);
+ return map;
+ }
+
+ /* otherwise, binary search */
+ while (max >= min)
+ {
+ mid = (min + max) / 2;
+ if (ucs > simple_case_map[mid].codepoint)
+ min = mid + 1;
+ else if (ucs < simple_case_map[mid].codepoint)
+ max = mid - 1;
+ else
+ return &simple_case_map[mid];
+ }
+
+ return NULL;
+}
+
+/*
+ * Returns simple lowercase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_lowercase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_lowercase : ucs;
+}
+
+/*
+ * Returns simple titlecase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_titlecase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_titlecase : ucs;
+}
+
+/*
+ * Returns simple uppercase mapping for the given character, or the original
+ * character if none. Sets *special to the special case mapping, if any.
+ */
+pg_wchar
+unicode_uppercase_simple(pg_wchar ucs)
+{
+ const pg_simple_case_map *map = find_case_map(ucs);
+
+ return map ? map->simple_uppercase : ucs;
+}
diff --git a/src/include/common/unicode_case.h b/src/include/common/unicode_case.h
new file mode 100644
index 0000000000..fe5fe6af63
--- /dev/null
+++ b/src/include/common/unicode_case.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case.h
+ * Routines for converting character case.
+ *
+ * These definitions can be used by both frontend and backend code.
+ *
+ * Copyright (c) 2017-2023, PostgreSQL Global Development Group
+ *
+ * src/include/common/unicode_case.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef UNICODE_CASE_H
+#define UNICODE_CASE_H
+
+#include "mb/pg_wchar.h"
+
+pg_wchar unicode_lowercase_simple(pg_wchar ucs);
+pg_wchar unicode_titlecase_simple(pg_wchar ucs);
+pg_wchar unicode_uppercase_simple(pg_wchar ucs);
+
+#endif /* UNICODE_CASE_H */
diff --git a/src/include/common/unicode_case_table.h b/src/include/common/unicode_case_table.h
new file mode 100644
index 0000000000..7f8ec3c23f
--- /dev/null
+++ b/src/include/common/unicode_case_table.h
@@ -0,0 +1,2994 @@
+/*-------------------------------------------------------------------------
+ *
+ * unicode_case_table.h
+ * Case mapping and information table.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/unicode_case_table.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+/*
+ * File auto-generated by src/common/unicode/generate-unicode_case_table.pl,
+ * do not edit. There is deliberately not an #ifndef PG_UNICODE_CASE_TABLE_H
+ * here.
+ */
+
+#include "mb/pg_wchar.h"
+
+typedef struct
+{
+ pg_wchar codepoint; /* Unicode codepoint */
+ pg_wchar simple_lowercase;
+ pg_wchar simple_titlecase;
+ pg_wchar simple_uppercase;
+} pg_simple_case_map;
+
+/*
+ * Case mapping table. Dense for codepoints < 0x80 (enabling fast lookup),
+ * sparse for higher codepoints (requiring scan or binary search).
+ */
+static const pg_simple_case_map simple_case_map[2955] =
+{
+ /* begin dense entries for codepoints < 0x80 */
+ {0x000000, 0x000000, 0x000000, 0x000000},
+ {0x000001, 0x000001, 0x000001, 0x000001},
+ {0x000002, 0x000002, 0x000002, 0x000002},
+ {0x000003, 0x000003, 0x000003, 0x000003},
+ {0x000004, 0x000004, 0x000004, 0x000004},
+ {0x000005, 0x000005, 0x000005, 0x000005},
+ {0x000006, 0x000006, 0x000006, 0x000006},
+ {0x000007, 0x000007, 0x000007, 0x000007},
+ {0x000008, 0x000008, 0x000008, 0x000008},
+ {0x000009, 0x000009, 0x000009, 0x000009},
+ {0x00000a, 0x00000a, 0x00000a, 0x00000a},
+ {0x00000b, 0x00000b, 0x00000b, 0x00000b},
+ {0x00000c, 0x00000c, 0x00000c, 0x00000c},
+ {0x00000d, 0x00000d, 0x00000d, 0x00000d},
+ {0x00000e, 0x00000e, 0x00000e, 0x00000e},
+ {0x00000f, 0x00000f, 0x00000f, 0x00000f},
+ {0x000010, 0x000010, 0x000010, 0x000010},
+ {0x000011, 0x000011, 0x000011, 0x000011},
+ {0x000012, 0x000012, 0x000012, 0x000012},
+ {0x000013, 0x000013, 0x000013, 0x000013},
+ {0x000014, 0x000014, 0x000014, 0x000014},
+ {0x000015, 0x000015, 0x000015, 0x000015},
+ {0x000016, 0x000016, 0x000016, 0x000016},
+ {0x000017, 0x000017, 0x000017, 0x000017},
+ {0x000018, 0x000018, 0x000018, 0x000018},
+ {0x000019, 0x000019, 0x000019, 0x000019},
+ {0x00001a, 0x00001a, 0x00001a, 0x00001a},
+ {0x00001b, 0x00001b, 0x00001b, 0x00001b},
+ {0x00001c, 0x00001c, 0x00001c, 0x00001c},
+ {0x00001d, 0x00001d, 0x00001d, 0x00001d},
+ {0x00001e, 0x00001e, 0x00001e, 0x00001e},
+ {0x00001f, 0x00001f, 0x00001f, 0x00001f},
+ {0x000020, 0x000020, 0x000020, 0x000020},
+ {0x000021, 0x000021, 0x000021, 0x000021},
+ {0x000022, 0x000022, 0x000022, 0x000022},
+ {0x000023, 0x000023, 0x000023, 0x000023},
+ {0x000024, 0x000024, 0x000024, 0x000024},
+ {0x000025, 0x000025, 0x000025, 0x000025},
+ {0x000026, 0x000026, 0x000026, 0x000026},
+ {0x000027, 0x000027, 0x000027, 0x000027},
+ {0x000028, 0x000028, 0x000028, 0x000028},
+ {0x000029, 0x000029, 0x000029, 0x000029},
+ {0x00002a, 0x00002a, 0x00002a, 0x00002a},
+ {0x00002b, 0x00002b, 0x00002b, 0x00002b},
+ {0x00002c, 0x00002c, 0x00002c, 0x00002c},
+ {0x00002d, 0x00002d, 0x00002d, 0x00002d},
+ {0x00002e, 0x00002e, 0x00002e, 0x00002e},
+ {0x00002f, 0x00002f, 0x00002f, 0x00002f},
+ {0x000030, 0x000030, 0x000030, 0x000030},
+ {0x000031, 0x000031, 0x000031, 0x000031},
+ {0x000032, 0x000032, 0x000032, 0x000032},
+ {0x000033, 0x000033, 0x000033, 0x000033},
+ {0x000034, 0x000034, 0x000034, 0x000034},
+ {0x000035, 0x000035, 0x000035, 0x000035},
+ {0x000036, 0x000036, 0x000036, 0x000036},
+ {0x000037, 0x000037, 0x000037, 0x000037},
+ {0x000038, 0x000038, 0x000038, 0x000038},
+ {0x000039, 0x000039, 0x000039, 0x000039},
+ {0x00003a, 0x00003a, 0x00003a, 0x00003a},
+ {0x00003b, 0x00003b, 0x00003b, 0x00003b},
+ {0x00003c, 0x00003c, 0x00003c, 0x00003c},
+ {0x00003d, 0x00003d, 0x00003d, 0x00003d},
+ {0x00003e, 0x00003e, 0x00003e, 0x00003e},
+ {0x00003f, 0x00003f, 0x00003f, 0x00003f},
+ {0x000040, 0x000040, 0x000040, 0x000040},
+ {0x000041, 0x000061, 0x000041, 0x000041},
+ {0x000042, 0x000062, 0x000042, 0x000042},
+ {0x000043, 0x000063, 0x000043, 0x000043},
+ {0x000044, 0x000064, 0x000044, 0x000044},
+ {0x000045, 0x000065, 0x000045, 0x000045},
+ {0x000046, 0x000066, 0x000046, 0x000046},
+ {0x000047, 0x000067, 0x000047, 0x000047},
+ {0x000048, 0x000068, 0x000048, 0x000048},
+ {0x000049, 0x000069, 0x000049, 0x000049},
+ {0x00004a, 0x00006a, 0x00004a, 0x00004a},
+ {0x00004b, 0x00006b, 0x00004b, 0x00004b},
+ {0x00004c, 0x00006c, 0x00004c, 0x00004c},
+ {0x00004d, 0x00006d, 0x00004d, 0x00004d},
+ {0x00004e, 0x00006e, 0x00004e, 0x00004e},
+ {0x00004f, 0x00006f, 0x00004f, 0x00004f},
+ {0x000050, 0x000070, 0x000050, 0x000050},
+ {0x000051, 0x000071, 0x000051, 0x000051},
+ {0x000052, 0x000072, 0x000052, 0x000052},
+ {0x000053, 0x000073, 0x000053, 0x000053},
+ {0x000054, 0x000074, 0x000054, 0x000054},
+ {0x000055, 0x000075, 0x000055, 0x000055},
+ {0x000056, 0x000076, 0x000056, 0x000056},
+ {0x000057, 0x000077, 0x000057, 0x000057},
+ {0x000058, 0x000078, 0x000058, 0x000058},
+ {0x000059, 0x000079, 0x000059, 0x000059},
+ {0x00005a, 0x00007a, 0x00005a, 0x00005a},
+ {0x00005b, 0x00005b, 0x00005b, 0x00005b},
+ {0x00005c, 0x00005c, 0x00005c, 0x00005c},
+ {0x00005d, 0x00005d, 0x00005d, 0x00005d},
+ {0x00005e, 0x00005e, 0x00005e, 0x00005e},
+ {0x00005f, 0x00005f, 0x00005f, 0x00005f},
+ {0x000060, 0x000060, 0x000060, 0x000060},
+ {0x000061, 0x000061, 0x000041, 0x000041},
+ {0x000062, 0x000062, 0x000042, 0x000042},
+ {0x000063, 0x000063, 0x000043, 0x000043},
+ {0x000064, 0x000064, 0x000044, 0x000044},
+ {0x000065, 0x000065, 0x000045, 0x000045},
+ {0x000066, 0x000066, 0x000046, 0x000046},
+ {0x000067, 0x000067, 0x000047, 0x000047},
+ {0x000068, 0x000068, 0x000048, 0x000048},
+ {0x000069, 0x000069, 0x000049, 0x000049},
+ {0x00006a, 0x00006a, 0x00004a, 0x00004a},
+ {0x00006b, 0x00006b, 0x00004b, 0x00004b},
+ {0x00006c, 0x00006c, 0x00004c, 0x00004c},
+ {0x00006d, 0x00006d, 0x00004d, 0x00004d},
+ {0x00006e, 0x00006e, 0x00004e, 0x00004e},
+ {0x00006f, 0x00006f, 0x00004f, 0x00004f},
+ {0x000070, 0x000070, 0x000050, 0x000050},
+ {0x000071, 0x000071, 0x000051, 0x000051},
+ {0x000072, 0x000072, 0x000052, 0x000052},
+ {0x000073, 0x000073, 0x000053, 0x000053},
+ {0x000074, 0x000074, 0x000054, 0x000054},
+ {0x000075, 0x000075, 0x000055, 0x000055},
+ {0x000076, 0x000076, 0x000056, 0x000056},
+ {0x000077, 0x000077, 0x000057, 0x000057},
+ {0x000078, 0x000078, 0x000058, 0x000058},
+ {0x000079, 0x000079, 0x000059, 0x000059},
+ {0x00007a, 0x00007a, 0x00005a, 0x00005a},
+ {0x00007b, 0x00007b, 0x00007b, 0x00007b},
+ {0x00007c, 0x00007c, 0x00007c, 0x00007c},
+ {0x00007d, 0x00007d, 0x00007d, 0x00007d},
+ {0x00007e, 0x00007e, 0x00007e, 0x00007e},
+ {0x00007f, 0x00007f, 0x00007f, 0x00007f},
+
+ /* begin sparse entries for codepoints >= 0x80 */
+ {0x0000b5, 0x0000b5, 0x00039c, 0x00039c},
+ {0x0000c0, 0x0000e0, 0x0000c0, 0x0000c0},
+ {0x0000c1, 0x0000e1, 0x0000c1, 0x0000c1},
+ {0x0000c2, 0x0000e2, 0x0000c2, 0x0000c2},
+ {0x0000c3, 0x0000e3, 0x0000c3, 0x0000c3},
+ {0x0000c4, 0x0000e4, 0x0000c4, 0x0000c4},
+ {0x0000c5, 0x0000e5, 0x0000c5, 0x0000c5},
+ {0x0000c6, 0x0000e6, 0x0000c6, 0x0000c6},
+ {0x0000c7, 0x0000e7, 0x0000c7, 0x0000c7},
+ {0x0000c8, 0x0000e8, 0x0000c8, 0x0000c8},
+ {0x0000c9, 0x0000e9, 0x0000c9, 0x0000c9},
+ {0x0000ca, 0x0000ea, 0x0000ca, 0x0000ca},
+ {0x0000cb, 0x0000eb, 0x0000cb, 0x0000cb},
+ {0x0000cc, 0x0000ec, 0x0000cc, 0x0000cc},
+ {0x0000cd, 0x0000ed, 0x0000cd, 0x0000cd},
+ {0x0000ce, 0x0000ee, 0x0000ce, 0x0000ce},
+ {0x0000cf, 0x0000ef, 0x0000cf, 0x0000cf},
+ {0x0000d0, 0x0000f0, 0x0000d0, 0x0000d0},
+ {0x0000d1, 0x0000f1, 0x0000d1, 0x0000d1},
+ {0x0000d2, 0x0000f2, 0x0000d2, 0x0000d2},
+ {0x0000d3, 0x0000f3, 0x0000d3, 0x0000d3},
+ {0x0000d4, 0x0000f4, 0x0000d4, 0x0000d4},
+ {0x0000d5, 0x0000f5, 0x0000d5, 0x0000d5},
+ {0x0000d6, 0x0000f6, 0x0000d6, 0x0000d6},
+ {0x0000d8, 0x0000f8, 0x0000d8, 0x0000d8},
+ {0x0000d9, 0x0000f9, 0x0000d9, 0x0000d9},
+ {0x0000da, 0x0000fa, 0x0000da, 0x0000da},
+ {0x0000db, 0x0000fb, 0x0000db, 0x0000db},
+ {0x0000dc, 0x0000fc, 0x0000dc, 0x0000dc},
+ {0x0000dd, 0x0000fd, 0x0000dd, 0x0000dd},
+ {0x0000de, 0x0000fe, 0x0000de, 0x0000de},
+ {0x0000e0, 0x0000e0, 0x0000c0, 0x0000c0},
+ {0x0000e1, 0x0000e1, 0x0000c1, 0x0000c1},
+ {0x0000e2, 0x0000e2, 0x0000c2, 0x0000c2},
+ {0x0000e3, 0x0000e3, 0x0000c3, 0x0000c3},
+ {0x0000e4, 0x0000e4, 0x0000c4, 0x0000c4},
+ {0x0000e5, 0x0000e5, 0x0000c5, 0x0000c5},
+ {0x0000e6, 0x0000e6, 0x0000c6, 0x0000c6},
+ {0x0000e7, 0x0000e7, 0x0000c7, 0x0000c7},
+ {0x0000e8, 0x0000e8, 0x0000c8, 0x0000c8},
+ {0x0000e9, 0x0000e9, 0x0000c9, 0x0000c9},
+ {0x0000ea, 0x0000ea, 0x0000ca, 0x0000ca},
+ {0x0000eb, 0x0000eb, 0x0000cb, 0x0000cb},
+ {0x0000ec, 0x0000ec, 0x0000cc, 0x0000cc},
+ {0x0000ed, 0x0000ed, 0x0000cd, 0x0000cd},
+ {0x0000ee, 0x0000ee, 0x0000ce, 0x0000ce},
+ {0x0000ef, 0x0000ef, 0x0000cf, 0x0000cf},
+ {0x0000f0, 0x0000f0, 0x0000d0, 0x0000d0},
+ {0x0000f1, 0x0000f1, 0x0000d1, 0x0000d1},
+ {0x0000f2, 0x0000f2, 0x0000d2, 0x0000d2},
+ {0x0000f3, 0x0000f3, 0x0000d3, 0x0000d3},
+ {0x0000f4, 0x0000f4, 0x0000d4, 0x0000d4},
+ {0x0000f5, 0x0000f5, 0x0000d5, 0x0000d5},
+ {0x0000f6, 0x0000f6, 0x0000d6, 0x0000d6},
+ {0x0000f8, 0x0000f8, 0x0000d8, 0x0000d8},
+ {0x0000f9, 0x0000f9, 0x0000d9, 0x0000d9},
+ {0x0000fa, 0x0000fa, 0x0000da, 0x0000da},
+ {0x0000fb, 0x0000fb, 0x0000db, 0x0000db},
+ {0x0000fc, 0x0000fc, 0x0000dc, 0x0000dc},
+ {0x0000fd, 0x0000fd, 0x0000dd, 0x0000dd},
+ {0x0000fe, 0x0000fe, 0x0000de, 0x0000de},
+ {0x0000ff, 0x0000ff, 0x000178, 0x000178},
+ {0x000100, 0x000101, 0x000100, 0x000100},
+ {0x000101, 0x000101, 0x000100, 0x000100},
+ {0x000102, 0x000103, 0x000102, 0x000102},
+ {0x000103, 0x000103, 0x000102, 0x000102},
+ {0x000104, 0x000105, 0x000104, 0x000104},
+ {0x000105, 0x000105, 0x000104, 0x000104},
+ {0x000106, 0x000107, 0x000106, 0x000106},
+ {0x000107, 0x000107, 0x000106, 0x000106},
+ {0x000108, 0x000109, 0x000108, 0x000108},
+ {0x000109, 0x000109, 0x000108, 0x000108},
+ {0x00010a, 0x00010b, 0x00010a, 0x00010a},
+ {0x00010b, 0x00010b, 0x00010a, 0x00010a},
+ {0x00010c, 0x00010d, 0x00010c, 0x00010c},
+ {0x00010d, 0x00010d, 0x00010c, 0x00010c},
+ {0x00010e, 0x00010f, 0x00010e, 0x00010e},
+ {0x00010f, 0x00010f, 0x00010e, 0x00010e},
+ {0x000110, 0x000111, 0x000110, 0x000110},
+ {0x000111, 0x000111, 0x000110, 0x000110},
+ {0x000112, 0x000113, 0x000112, 0x000112},
+ {0x000113, 0x000113, 0x000112, 0x000112},
+ {0x000114, 0x000115, 0x000114, 0x000114},
+ {0x000115, 0x000115, 0x000114, 0x000114},
+ {0x000116, 0x000117, 0x000116, 0x000116},
+ {0x000117, 0x000117, 0x000116, 0x000116},
+ {0x000118, 0x000119, 0x000118, 0x000118},
+ {0x000119, 0x000119, 0x000118, 0x000118},
+ {0x00011a, 0x00011b, 0x00011a, 0x00011a},
+ {0x00011b, 0x00011b, 0x00011a, 0x00011a},
+ {0x00011c, 0x00011d, 0x00011c, 0x00011c},
+ {0x00011d, 0x00011d, 0x00011c, 0x00011c},
+ {0x00011e, 0x00011f, 0x00011e, 0x00011e},
+ {0x00011f, 0x00011f, 0x00011e, 0x00011e},
+ {0x000120, 0x000121, 0x000120, 0x000120},
+ {0x000121, 0x000121, 0x000120, 0x000120},
+ {0x000122, 0x000123, 0x000122, 0x000122},
+ {0x000123, 0x000123, 0x000122, 0x000122},
+ {0x000124, 0x000125, 0x000124, 0x000124},
+ {0x000125, 0x000125, 0x000124, 0x000124},
+ {0x000126, 0x000127, 0x000126, 0x000126},
+ {0x000127, 0x000127, 0x000126, 0x000126},
+ {0x000128, 0x000129, 0x000128, 0x000128},
+ {0x000129, 0x000129, 0x000128, 0x000128},
+ {0x00012a, 0x00012b, 0x00012a, 0x00012a},
+ {0x00012b, 0x00012b, 0x00012a, 0x00012a},
+ {0x00012c, 0x00012d, 0x00012c, 0x00012c},
+ {0x00012d, 0x00012d, 0x00012c, 0x00012c},
+ {0x00012e, 0x00012f, 0x00012e, 0x00012e},
+ {0x00012f, 0x00012f, 0x00012e, 0x00012e},
+ {0x000130, 0x000069, 0x000130, 0x000130},
+ {0x000131, 0x000131, 0x000049, 0x000049},
+ {0x000132, 0x000133, 0x000132, 0x000132},
+ {0x000133, 0x000133, 0x000132, 0x000132},
+ {0x000134, 0x000135, 0x000134, 0x000134},
+ {0x000135, 0x000135, 0x000134, 0x000134},
+ {0x000136, 0x000137, 0x000136, 0x000136},
+ {0x000137, 0x000137, 0x000136, 0x000136},
+ {0x000139, 0x00013a, 0x000139, 0x000139},
+ {0x00013a, 0x00013a, 0x000139, 0x000139},
+ {0x00013b, 0x00013c, 0x00013b, 0x00013b},
+ {0x00013c, 0x00013c, 0x00013b, 0x00013b},
+ {0x00013d, 0x00013e, 0x00013d, 0x00013d},
+ {0x00013e, 0x00013e, 0x00013d, 0x00013d},
+ {0x00013f, 0x000140, 0x00013f, 0x00013f},
+ {0x000140, 0x000140, 0x00013f, 0x00013f},
+ {0x000141, 0x000142, 0x000141, 0x000141},
+ {0x000142, 0x000142, 0x000141, 0x000141},
+ {0x000143, 0x000144, 0x000143, 0x000143},
+ {0x000144, 0x000144, 0x000143, 0x000143},
+ {0x000145, 0x000146, 0x000145, 0x000145},
+ {0x000146, 0x000146, 0x000145, 0x000145},
+ {0x000147, 0x000148, 0x000147, 0x000147},
+ {0x000148, 0x000148, 0x000147, 0x000147},
+ {0x00014a, 0x00014b, 0x00014a, 0x00014a},
+ {0x00014b, 0x00014b, 0x00014a, 0x00014a},
+ {0x00014c, 0x00014d, 0x00014c, 0x00014c},
+ {0x00014d, 0x00014d, 0x00014c, 0x00014c},
+ {0x00014e, 0x00014f, 0x00014e, 0x00014e},
+ {0x00014f, 0x00014f, 0x00014e, 0x00014e},
+ {0x000150, 0x000151, 0x000150, 0x000150},
+ {0x000151, 0x000151, 0x000150, 0x000150},
+ {0x000152, 0x000153, 0x000152, 0x000152},
+ {0x000153, 0x000153, 0x000152, 0x000152},
+ {0x000154, 0x000155, 0x000154, 0x000154},
+ {0x000155, 0x000155, 0x000154, 0x000154},
+ {0x000156, 0x000157, 0x000156, 0x000156},
+ {0x000157, 0x000157, 0x000156, 0x000156},
+ {0x000158, 0x000159, 0x000158, 0x000158},
+ {0x000159, 0x000159, 0x000158, 0x000158},
+ {0x00015a, 0x00015b, 0x00015a, 0x00015a},
+ {0x00015b, 0x00015b, 0x00015a, 0x00015a},
+ {0x00015c, 0x00015d, 0x00015c, 0x00015c},
+ {0x00015d, 0x00015d, 0x00015c, 0x00015c},
+ {0x00015e, 0x00015f, 0x00015e, 0x00015e},
+ {0x00015f, 0x00015f, 0x00015e, 0x00015e},
+ {0x000160, 0x000161, 0x000160, 0x000160},
+ {0x000161, 0x000161, 0x000160, 0x000160},
+ {0x000162, 0x000163, 0x000162, 0x000162},
+ {0x000163, 0x000163, 0x000162, 0x000162},
+ {0x000164, 0x000165, 0x000164, 0x000164},
+ {0x000165, 0x000165, 0x000164, 0x000164},
+ {0x000166, 0x000167, 0x000166, 0x000166},
+ {0x000167, 0x000167, 0x000166, 0x000166},
+ {0x000168, 0x000169, 0x000168, 0x000168},
+ {0x000169, 0x000169, 0x000168, 0x000168},
+ {0x00016a, 0x00016b, 0x00016a, 0x00016a},
+ {0x00016b, 0x00016b, 0x00016a, 0x00016a},
+ {0x00016c, 0x00016d, 0x00016c, 0x00016c},
+ {0x00016d, 0x00016d, 0x00016c, 0x00016c},
+ {0x00016e, 0x00016f, 0x00016e, 0x00016e},
+ {0x00016f, 0x00016f, 0x00016e, 0x00016e},
+ {0x000170, 0x000171, 0x000170, 0x000170},
+ {0x000171, 0x000171, 0x000170, 0x000170},
+ {0x000172, 0x000173, 0x000172, 0x000172},
+ {0x000173, 0x000173, 0x000172, 0x000172},
+ {0x000174, 0x000175, 0x000174, 0x000174},
+ {0x000175, 0x000175, 0x000174, 0x000174},
+ {0x000176, 0x000177, 0x000176, 0x000176},
+ {0x000177, 0x000177, 0x000176, 0x000176},
+ {0x000178, 0x0000ff, 0x000178, 0x000178},
+ {0x000179, 0x00017a, 0x000179, 0x000179},
+ {0x00017a, 0x00017a, 0x000179, 0x000179},
+ {0x00017b, 0x00017c, 0x00017b, 0x00017b},
+ {0x00017c, 0x00017c, 0x00017b, 0x00017b},
+ {0x00017d, 0x00017e, 0x00017d, 0x00017d},
+ {0x00017e, 0x00017e, 0x00017d, 0x00017d},
+ {0x00017f, 0x00017f, 0x000053, 0x000053},
+ {0x000180, 0x000180, 0x000243, 0x000243},
+ {0x000181, 0x000253, 0x000181, 0x000181},
+ {0x000182, 0x000183, 0x000182, 0x000182},
+ {0x000183, 0x000183, 0x000182, 0x000182},
+ {0x000184, 0x000185, 0x000184, 0x000184},
+ {0x000185, 0x000185, 0x000184, 0x000184},
+ {0x000186, 0x000254, 0x000186, 0x000186},
+ {0x000187, 0x000188, 0x000187, 0x000187},
+ {0x000188, 0x000188, 0x000187, 0x000187},
+ {0x000189, 0x000256, 0x000189, 0x000189},
+ {0x00018a, 0x000257, 0x00018a, 0x00018a},
+ {0x00018b, 0x00018c, 0x00018b, 0x00018b},
+ {0x00018c, 0x00018c, 0x00018b, 0x00018b},
+ {0x00018e, 0x0001dd, 0x00018e, 0x00018e},
+ {0x00018f, 0x000259, 0x00018f, 0x00018f},
+ {0x000190, 0x00025b, 0x000190, 0x000190},
+ {0x000191, 0x000192, 0x000191, 0x000191},
+ {0x000192, 0x000192, 0x000191, 0x000191},
+ {0x000193, 0x000260, 0x000193, 0x000193},
+ {0x000194, 0x000263, 0x000194, 0x000194},
+ {0x000195, 0x000195, 0x0001f6, 0x0001f6},
+ {0x000196, 0x000269, 0x000196, 0x000196},
+ {0x000197, 0x000268, 0x000197, 0x000197},
+ {0x000198, 0x000199, 0x000198, 0x000198},
+ {0x000199, 0x000199, 0x000198, 0x000198},
+ {0x00019a, 0x00019a, 0x00023d, 0x00023d},
+ {0x00019c, 0x00026f, 0x00019c, 0x00019c},
+ {0x00019d, 0x000272, 0x00019d, 0x00019d},
+ {0x00019e, 0x00019e, 0x000220, 0x000220},
+ {0x00019f, 0x000275, 0x00019f, 0x00019f},
+ {0x0001a0, 0x0001a1, 0x0001a0, 0x0001a0},
+ {0x0001a1, 0x0001a1, 0x0001a0, 0x0001a0},
+ {0x0001a2, 0x0001a3, 0x0001a2, 0x0001a2},
+ {0x0001a3, 0x0001a3, 0x0001a2, 0x0001a2},
+ {0x0001a4, 0x0001a5, 0x0001a4, 0x0001a4},
+ {0x0001a5, 0x0001a5, 0x0001a4, 0x0001a4},
+ {0x0001a6, 0x000280, 0x0001a6, 0x0001a6},
+ {0x0001a7, 0x0001a8, 0x0001a7, 0x0001a7},
+ {0x0001a8, 0x0001a8, 0x0001a7, 0x0001a7},
+ {0x0001a9, 0x000283, 0x0001a9, 0x0001a9},
+ {0x0001ac, 0x0001ad, 0x0001ac, 0x0001ac},
+ {0x0001ad, 0x0001ad, 0x0001ac, 0x0001ac},
+ {0x0001ae, 0x000288, 0x0001ae, 0x0001ae},
+ {0x0001af, 0x0001b0, 0x0001af, 0x0001af},
+ {0x0001b0, 0x0001b0, 0x0001af, 0x0001af},
+ {0x0001b1, 0x00028a, 0x0001b1, 0x0001b1},
+ {0x0001b2, 0x00028b, 0x0001b2, 0x0001b2},
+ {0x0001b3, 0x0001b4, 0x0001b3, 0x0001b3},
+ {0x0001b4, 0x0001b4, 0x0001b3, 0x0001b3},
+ {0x0001b5, 0x0001b6, 0x0001b5, 0x0001b5},
+ {0x0001b6, 0x0001b6, 0x0001b5, 0x0001b5},
+ {0x0001b7, 0x000292, 0x0001b7, 0x0001b7},
+ {0x0001b8, 0x0001b9, 0x0001b8, 0x0001b8},
+ {0x0001b9, 0x0001b9, 0x0001b8, 0x0001b8},
+ {0x0001bc, 0x0001bd, 0x0001bc, 0x0001bc},
+ {0x0001bd, 0x0001bd, 0x0001bc, 0x0001bc},
+ {0x0001bf, 0x0001bf, 0x0001f7, 0x0001f7},
+ {0x0001c4, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c5, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c6, 0x0001c6, 0x0001c5, 0x0001c4},
+ {0x0001c7, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001c8, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001c9, 0x0001c9, 0x0001c8, 0x0001c7},
+ {0x0001ca, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cb, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cc, 0x0001cc, 0x0001cb, 0x0001ca},
+ {0x0001cd, 0x0001ce, 0x0001cd, 0x0001cd},
+ {0x0001ce, 0x0001ce, 0x0001cd, 0x0001cd},
+ {0x0001cf, 0x0001d0, 0x0001cf, 0x0001cf},
+ {0x0001d0, 0x0001d0, 0x0001cf, 0x0001cf},
+ {0x0001d1, 0x0001d2, 0x0001d1, 0x0001d1},
+ {0x0001d2, 0x0001d2, 0x0001d1, 0x0001d1},
+ {0x0001d3, 0x0001d4, 0x0001d3, 0x0001d3},
+ {0x0001d4, 0x0001d4, 0x0001d3, 0x0001d3},
+ {0x0001d5, 0x0001d6, 0x0001d5, 0x0001d5},
+ {0x0001d6, 0x0001d6, 0x0001d5, 0x0001d5},
+ {0x0001d7, 0x0001d8, 0x0001d7, 0x0001d7},
+ {0x0001d8, 0x0001d8, 0x0001d7, 0x0001d7},
+ {0x0001d9, 0x0001da, 0x0001d9, 0x0001d9},
+ {0x0001da, 0x0001da, 0x0001d9, 0x0001d9},
+ {0x0001db, 0x0001dc, 0x0001db, 0x0001db},
+ {0x0001dc, 0x0001dc, 0x0001db, 0x0001db},
+ {0x0001dd, 0x0001dd, 0x00018e, 0x00018e},
+ {0x0001de, 0x0001df, 0x0001de, 0x0001de},
+ {0x0001df, 0x0001df, 0x0001de, 0x0001de},
+ {0x0001e0, 0x0001e1, 0x0001e0, 0x0001e0},
+ {0x0001e1, 0x0001e1, 0x0001e0, 0x0001e0},
+ {0x0001e2, 0x0001e3, 0x0001e2, 0x0001e2},
+ {0x0001e3, 0x0001e3, 0x0001e2, 0x0001e2},
+ {0x0001e4, 0x0001e5, 0x0001e4, 0x0001e4},
+ {0x0001e5, 0x0001e5, 0x0001e4, 0x0001e4},
+ {0x0001e6, 0x0001e7, 0x0001e6, 0x0001e6},
+ {0x0001e7, 0x0001e7, 0x0001e6, 0x0001e6},
+ {0x0001e8, 0x0001e9, 0x0001e8, 0x0001e8},
+ {0x0001e9, 0x0001e9, 0x0001e8, 0x0001e8},
+ {0x0001ea, 0x0001eb, 0x0001ea, 0x0001ea},
+ {0x0001eb, 0x0001eb, 0x0001ea, 0x0001ea},
+ {0x0001ec, 0x0001ed, 0x0001ec, 0x0001ec},
+ {0x0001ed, 0x0001ed, 0x0001ec, 0x0001ec},
+ {0x0001ee, 0x0001ef, 0x0001ee, 0x0001ee},
+ {0x0001ef, 0x0001ef, 0x0001ee, 0x0001ee},
+ {0x0001f1, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f2, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f3, 0x0001f3, 0x0001f2, 0x0001f1},
+ {0x0001f4, 0x0001f5, 0x0001f4, 0x0001f4},
+ {0x0001f5, 0x0001f5, 0x0001f4, 0x0001f4},
+ {0x0001f6, 0x000195, 0x0001f6, 0x0001f6},
+ {0x0001f7, 0x0001bf, 0x0001f7, 0x0001f7},
+ {0x0001f8, 0x0001f9, 0x0001f8, 0x0001f8},
+ {0x0001f9, 0x0001f9, 0x0001f8, 0x0001f8},
+ {0x0001fa, 0x0001fb, 0x0001fa, 0x0001fa},
+ {0x0001fb, 0x0001fb, 0x0001fa, 0x0001fa},
+ {0x0001fc, 0x0001fd, 0x0001fc, 0x0001fc},
+ {0x0001fd, 0x0001fd, 0x0001fc, 0x0001fc},
+ {0x0001fe, 0x0001ff, 0x0001fe, 0x0001fe},
+ {0x0001ff, 0x0001ff, 0x0001fe, 0x0001fe},
+ {0x000200, 0x000201, 0x000200, 0x000200},
+ {0x000201, 0x000201, 0x000200, 0x000200},
+ {0x000202, 0x000203, 0x000202, 0x000202},
+ {0x000203, 0x000203, 0x000202, 0x000202},
+ {0x000204, 0x000205, 0x000204, 0x000204},
+ {0x000205, 0x000205, 0x000204, 0x000204},
+ {0x000206, 0x000207, 0x000206, 0x000206},
+ {0x000207, 0x000207, 0x000206, 0x000206},
+ {0x000208, 0x000209, 0x000208, 0x000208},
+ {0x000209, 0x000209, 0x000208, 0x000208},
+ {0x00020a, 0x00020b, 0x00020a, 0x00020a},
+ {0x00020b, 0x00020b, 0x00020a, 0x00020a},
+ {0x00020c, 0x00020d, 0x00020c, 0x00020c},
+ {0x00020d, 0x00020d, 0x00020c, 0x00020c},
+ {0x00020e, 0x00020f, 0x00020e, 0x00020e},
+ {0x00020f, 0x00020f, 0x00020e, 0x00020e},
+ {0x000210, 0x000211, 0x000210, 0x000210},
+ {0x000211, 0x000211, 0x000210, 0x000210},
+ {0x000212, 0x000213, 0x000212, 0x000212},
+ {0x000213, 0x000213, 0x000212, 0x000212},
+ {0x000214, 0x000215, 0x000214, 0x000214},
+ {0x000215, 0x000215, 0x000214, 0x000214},
+ {0x000216, 0x000217, 0x000216, 0x000216},
+ {0x000217, 0x000217, 0x000216, 0x000216},
+ {0x000218, 0x000219, 0x000218, 0x000218},
+ {0x000219, 0x000219, 0x000218, 0x000218},
+ {0x00021a, 0x00021b, 0x00021a, 0x00021a},
+ {0x00021b, 0x00021b, 0x00021a, 0x00021a},
+ {0x00021c, 0x00021d, 0x00021c, 0x00021c},
+ {0x00021d, 0x00021d, 0x00021c, 0x00021c},
+ {0x00021e, 0x00021f, 0x00021e, 0x00021e},
+ {0x00021f, 0x00021f, 0x00021e, 0x00021e},
+ {0x000220, 0x00019e, 0x000220, 0x000220},
+ {0x000222, 0x000223, 0x000222, 0x000222},
+ {0x000223, 0x000223, 0x000222, 0x000222},
+ {0x000224, 0x000225, 0x000224, 0x000224},
+ {0x000225, 0x000225, 0x000224, 0x000224},
+ {0x000226, 0x000227, 0x000226, 0x000226},
+ {0x000227, 0x000227, 0x000226, 0x000226},
+ {0x000228, 0x000229, 0x000228, 0x000228},
+ {0x000229, 0x000229, 0x000228, 0x000228},
+ {0x00022a, 0x00022b, 0x00022a, 0x00022a},
+ {0x00022b, 0x00022b, 0x00022a, 0x00022a},
+ {0x00022c, 0x00022d, 0x00022c, 0x00022c},
+ {0x00022d, 0x00022d, 0x00022c, 0x00022c},
+ {0x00022e, 0x00022f, 0x00022e, 0x00022e},
+ {0x00022f, 0x00022f, 0x00022e, 0x00022e},
+ {0x000230, 0x000231, 0x000230, 0x000230},
+ {0x000231, 0x000231, 0x000230, 0x000230},
+ {0x000232, 0x000233, 0x000232, 0x000232},
+ {0x000233, 0x000233, 0x000232, 0x000232},
+ {0x00023a, 0x002c65, 0x00023a, 0x00023a},
+ {0x00023b, 0x00023c, 0x00023b, 0x00023b},
+ {0x00023c, 0x00023c, 0x00023b, 0x00023b},
+ {0x00023d, 0x00019a, 0x00023d, 0x00023d},
+ {0x00023e, 0x002c66, 0x00023e, 0x00023e},
+ {0x00023f, 0x00023f, 0x002c7e, 0x002c7e},
+ {0x000240, 0x000240, 0x002c7f, 0x002c7f},
+ {0x000241, 0x000242, 0x000241, 0x000241},
+ {0x000242, 0x000242, 0x000241, 0x000241},
+ {0x000243, 0x000180, 0x000243, 0x000243},
+ {0x000244, 0x000289, 0x000244, 0x000244},
+ {0x000245, 0x00028c, 0x000245, 0x000245},
+ {0x000246, 0x000247, 0x000246, 0x000246},
+ {0x000247, 0x000247, 0x000246, 0x000246},
+ {0x000248, 0x000249, 0x000248, 0x000248},
+ {0x000249, 0x000249, 0x000248, 0x000248},
+ {0x00024a, 0x00024b, 0x00024a, 0x00024a},
+ {0x00024b, 0x00024b, 0x00024a, 0x00024a},
+ {0x00024c, 0x00024d, 0x00024c, 0x00024c},
+ {0x00024d, 0x00024d, 0x00024c, 0x00024c},
+ {0x00024e, 0x00024f, 0x00024e, 0x00024e},
+ {0x00024f, 0x00024f, 0x00024e, 0x00024e},
+ {0x000250, 0x000250, 0x002c6f, 0x002c6f},
+ {0x000251, 0x000251, 0x002c6d, 0x002c6d},
+ {0x000252, 0x000252, 0x002c70, 0x002c70},
+ {0x000253, 0x000253, 0x000181, 0x000181},
+ {0x000254, 0x000254, 0x000186, 0x000186},
+ {0x000256, 0x000256, 0x000189, 0x000189},
+ {0x000257, 0x000257, 0x00018a, 0x00018a},
+ {0x000259, 0x000259, 0x00018f, 0x00018f},
+ {0x00025b, 0x00025b, 0x000190, 0x000190},
+ {0x00025c, 0x00025c, 0x00a7ab, 0x00a7ab},
+ {0x000260, 0x000260, 0x000193, 0x000193},
+ {0x000261, 0x000261, 0x00a7ac, 0x00a7ac},
+ {0x000263, 0x000263, 0x000194, 0x000194},
+ {0x000265, 0x000265, 0x00a78d, 0x00a78d},
+ {0x000266, 0x000266, 0x00a7aa, 0x00a7aa},
+ {0x000268, 0x000268, 0x000197, 0x000197},
+ {0x000269, 0x000269, 0x000196, 0x000196},
+ {0x00026a, 0x00026a, 0x00a7ae, 0x00a7ae},
+ {0x00026b, 0x00026b, 0x002c62, 0x002c62},
+ {0x00026c, 0x00026c, 0x00a7ad, 0x00a7ad},
+ {0x00026f, 0x00026f, 0x00019c, 0x00019c},
+ {0x000271, 0x000271, 0x002c6e, 0x002c6e},
+ {0x000272, 0x000272, 0x00019d, 0x00019d},
+ {0x000275, 0x000275, 0x00019f, 0x00019f},
+ {0x00027d, 0x00027d, 0x002c64, 0x002c64},
+ {0x000280, 0x000280, 0x0001a6, 0x0001a6},
+ {0x000282, 0x000282, 0x00a7c5, 0x00a7c5},
+ {0x000283, 0x000283, 0x0001a9, 0x0001a9},
+ {0x000287, 0x000287, 0x00a7b1, 0x00a7b1},
+ {0x000288, 0x000288, 0x0001ae, 0x0001ae},
+ {0x000289, 0x000289, 0x000244, 0x000244},
+ {0x00028a, 0x00028a, 0x0001b1, 0x0001b1},
+ {0x00028b, 0x00028b, 0x0001b2, 0x0001b2},
+ {0x00028c, 0x00028c, 0x000245, 0x000245},
+ {0x000292, 0x000292, 0x0001b7, 0x0001b7},
+ {0x00029d, 0x00029d, 0x00a7b2, 0x00a7b2},
+ {0x00029e, 0x00029e, 0x00a7b0, 0x00a7b0},
+ {0x000345, 0x000345, 0x000399, 0x000399},
+ {0x000370, 0x000371, 0x000370, 0x000370},
+ {0x000371, 0x000371, 0x000370, 0x000370},
+ {0x000372, 0x000373, 0x000372, 0x000372},
+ {0x000373, 0x000373, 0x000372, 0x000372},
+ {0x000376, 0x000377, 0x000376, 0x000376},
+ {0x000377, 0x000377, 0x000376, 0x000376},
+ {0x00037b, 0x00037b, 0x0003fd, 0x0003fd},
+ {0x00037c, 0x00037c, 0x0003fe, 0x0003fe},
+ {0x00037d, 0x00037d, 0x0003ff, 0x0003ff},
+ {0x00037f, 0x0003f3, 0x00037f, 0x00037f},
+ {0x000386, 0x0003ac, 0x000386, 0x000386},
+ {0x000388, 0x0003ad, 0x000388, 0x000388},
+ {0x000389, 0x0003ae, 0x000389, 0x000389},
+ {0x00038a, 0x0003af, 0x00038a, 0x00038a},
+ {0x00038c, 0x0003cc, 0x00038c, 0x00038c},
+ {0x00038e, 0x0003cd, 0x00038e, 0x00038e},
+ {0x00038f, 0x0003ce, 0x00038f, 0x00038f},
+ {0x000391, 0x0003b1, 0x000391, 0x000391},
+ {0x000392, 0x0003b2, 0x000392, 0x000392},
+ {0x000393, 0x0003b3, 0x000393, 0x000393},
+ {0x000394, 0x0003b4, 0x000394, 0x000394},
+ {0x000395, 0x0003b5, 0x000395, 0x000395},
+ {0x000396, 0x0003b6, 0x000396, 0x000396},
+ {0x000397, 0x0003b7, 0x000397, 0x000397},
+ {0x000398, 0x0003b8, 0x000398, 0x000398},
+ {0x000399, 0x0003b9, 0x000399, 0x000399},
+ {0x00039a, 0x0003ba, 0x00039a, 0x00039a},
+ {0x00039b, 0x0003bb, 0x00039b, 0x00039b},
+ {0x00039c, 0x0003bc, 0x00039c, 0x00039c},
+ {0x00039d, 0x0003bd, 0x00039d, 0x00039d},
+ {0x00039e, 0x0003be, 0x00039e, 0x00039e},
+ {0x00039f, 0x0003bf, 0x00039f, 0x00039f},
+ {0x0003a0, 0x0003c0, 0x0003a0, 0x0003a0},
+ {0x0003a1, 0x0003c1, 0x0003a1, 0x0003a1},
+ {0x0003a3, 0x0003c3, 0x0003a3, 0x0003a3},
+ {0x0003a4, 0x0003c4, 0x0003a4, 0x0003a4},
+ {0x0003a5, 0x0003c5, 0x0003a5, 0x0003a5},
+ {0x0003a6, 0x0003c6, 0x0003a6, 0x0003a6},
+ {0x0003a7, 0x0003c7, 0x0003a7, 0x0003a7},
+ {0x0003a8, 0x0003c8, 0x0003a8, 0x0003a8},
+ {0x0003a9, 0x0003c9, 0x0003a9, 0x0003a9},
+ {0x0003aa, 0x0003ca, 0x0003aa, 0x0003aa},
+ {0x0003ab, 0x0003cb, 0x0003ab, 0x0003ab},
+ {0x0003ac, 0x0003ac, 0x000386, 0x000386},
+ {0x0003ad, 0x0003ad, 0x000388, 0x000388},
+ {0x0003ae, 0x0003ae, 0x000389, 0x000389},
+ {0x0003af, 0x0003af, 0x00038a, 0x00038a},
+ {0x0003b1, 0x0003b1, 0x000391, 0x000391},
+ {0x0003b2, 0x0003b2, 0x000392, 0x000392},
+ {0x0003b3, 0x0003b3, 0x000393, 0x000393},
+ {0x0003b4, 0x0003b4, 0x000394, 0x000394},
+ {0x0003b5, 0x0003b5, 0x000395, 0x000395},
+ {0x0003b6, 0x0003b6, 0x000396, 0x000396},
+ {0x0003b7, 0x0003b7, 0x000397, 0x000397},
+ {0x0003b8, 0x0003b8, 0x000398, 0x000398},
+ {0x0003b9, 0x0003b9, 0x000399, 0x000399},
+ {0x0003ba, 0x0003ba, 0x00039a, 0x00039a},
+ {0x0003bb, 0x0003bb, 0x00039b, 0x00039b},
+ {0x0003bc, 0x0003bc, 0x00039c, 0x00039c},
+ {0x0003bd, 0x0003bd, 0x00039d, 0x00039d},
+ {0x0003be, 0x0003be, 0x00039e, 0x00039e},
+ {0x0003bf, 0x0003bf, 0x00039f, 0x00039f},
+ {0x0003c0, 0x0003c0, 0x0003a0, 0x0003a0},
+ {0x0003c1, 0x0003c1, 0x0003a1, 0x0003a1},
+ {0x0003c2, 0x0003c2, 0x0003a3, 0x0003a3},
+ {0x0003c3, 0x0003c3, 0x0003a3, 0x0003a3},
+ {0x0003c4, 0x0003c4, 0x0003a4, 0x0003a4},
+ {0x0003c5, 0x0003c5, 0x0003a5, 0x0003a5},
+ {0x0003c6, 0x0003c6, 0x0003a6, 0x0003a6},
+ {0x0003c7, 0x0003c7, 0x0003a7, 0x0003a7},
+ {0x0003c8, 0x0003c8, 0x0003a8, 0x0003a8},
+ {0x0003c9, 0x0003c9, 0x0003a9, 0x0003a9},
+ {0x0003ca, 0x0003ca, 0x0003aa, 0x0003aa},
+ {0x0003cb, 0x0003cb, 0x0003ab, 0x0003ab},
+ {0x0003cc, 0x0003cc, 0x00038c, 0x00038c},
+ {0x0003cd, 0x0003cd, 0x00038e, 0x00038e},
+ {0x0003ce, 0x0003ce, 0x00038f, 0x00038f},
+ {0x0003cf, 0x0003d7, 0x0003cf, 0x0003cf},
+ {0x0003d0, 0x0003d0, 0x000392, 0x000392},
+ {0x0003d1, 0x0003d1, 0x000398, 0x000398},
+ {0x0003d5, 0x0003d5, 0x0003a6, 0x0003a6},
+ {0x0003d6, 0x0003d6, 0x0003a0, 0x0003a0},
+ {0x0003d7, 0x0003d7, 0x0003cf, 0x0003cf},
+ {0x0003d8, 0x0003d9, 0x0003d8, 0x0003d8},
+ {0x0003d9, 0x0003d9, 0x0003d8, 0x0003d8},
+ {0x0003da, 0x0003db, 0x0003da, 0x0003da},
+ {0x0003db, 0x0003db, 0x0003da, 0x0003da},
+ {0x0003dc, 0x0003dd, 0x0003dc, 0x0003dc},
+ {0x0003dd, 0x0003dd, 0x0003dc, 0x0003dc},
+ {0x0003de, 0x0003df, 0x0003de, 0x0003de},
+ {0x0003df, 0x0003df, 0x0003de, 0x0003de},
+ {0x0003e0, 0x0003e1, 0x0003e0, 0x0003e0},
+ {0x0003e1, 0x0003e1, 0x0003e0, 0x0003e0},
+ {0x0003e2, 0x0003e3, 0x0003e2, 0x0003e2},
+ {0x0003e3, 0x0003e3, 0x0003e2, 0x0003e2},
+ {0x0003e4, 0x0003e5, 0x0003e4, 0x0003e4},
+ {0x0003e5, 0x0003e5, 0x0003e4, 0x0003e4},
+ {0x0003e6, 0x0003e7, 0x0003e6, 0x0003e6},
+ {0x0003e7, 0x0003e7, 0x0003e6, 0x0003e6},
+ {0x0003e8, 0x0003e9, 0x0003e8, 0x0003e8},
+ {0x0003e9, 0x0003e9, 0x0003e8, 0x0003e8},
+ {0x0003ea, 0x0003eb, 0x0003ea, 0x0003ea},
+ {0x0003eb, 0x0003eb, 0x0003ea, 0x0003ea},
+ {0x0003ec, 0x0003ed, 0x0003ec, 0x0003ec},
+ {0x0003ed, 0x0003ed, 0x0003ec, 0x0003ec},
+ {0x0003ee, 0x0003ef, 0x0003ee, 0x0003ee},
+ {0x0003ef, 0x0003ef, 0x0003ee, 0x0003ee},
+ {0x0003f0, 0x0003f0, 0x00039a, 0x00039a},
+ {0x0003f1, 0x0003f1, 0x0003a1, 0x0003a1},
+ {0x0003f2, 0x0003f2, 0x0003f9, 0x0003f9},
+ {0x0003f3, 0x0003f3, 0x00037f, 0x00037f},
+ {0x0003f4, 0x0003b8, 0x0003f4, 0x0003f4},
+ {0x0003f5, 0x0003f5, 0x000395, 0x000395},
+ {0x0003f7, 0x0003f8, 0x0003f7, 0x0003f7},
+ {0x0003f8, 0x0003f8, 0x0003f7, 0x0003f7},
+ {0x0003f9, 0x0003f2, 0x0003f9, 0x0003f9},
+ {0x0003fa, 0x0003fb, 0x0003fa, 0x0003fa},
+ {0x0003fb, 0x0003fb, 0x0003fa, 0x0003fa},
+ {0x0003fd, 0x00037b, 0x0003fd, 0x0003fd},
+ {0x0003fe, 0x00037c, 0x0003fe, 0x0003fe},
+ {0x0003ff, 0x00037d, 0x0003ff, 0x0003ff},
+ {0x000400, 0x000450, 0x000400, 0x000400},
+ {0x000401, 0x000451, 0x000401, 0x000401},
+ {0x000402, 0x000452, 0x000402, 0x000402},
+ {0x000403, 0x000453, 0x000403, 0x000403},
+ {0x000404, 0x000454, 0x000404, 0x000404},
+ {0x000405, 0x000455, 0x000405, 0x000405},
+ {0x000406, 0x000456, 0x000406, 0x000406},
+ {0x000407, 0x000457, 0x000407, 0x000407},
+ {0x000408, 0x000458, 0x000408, 0x000408},
+ {0x000409, 0x000459, 0x000409, 0x000409},
+ {0x00040a, 0x00045a, 0x00040a, 0x00040a},
+ {0x00040b, 0x00045b, 0x00040b, 0x00040b},
+ {0x00040c, 0x00045c, 0x00040c, 0x00040c},
+ {0x00040d, 0x00045d, 0x00040d, 0x00040d},
+ {0x00040e, 0x00045e, 0x00040e, 0x00040e},
+ {0x00040f, 0x00045f, 0x00040f, 0x00040f},
+ {0x000410, 0x000430, 0x000410, 0x000410},
+ {0x000411, 0x000431, 0x000411, 0x000411},
+ {0x000412, 0x000432, 0x000412, 0x000412},
+ {0x000413, 0x000433, 0x000413, 0x000413},
+ {0x000414, 0x000434, 0x000414, 0x000414},
+ {0x000415, 0x000435, 0x000415, 0x000415},
+ {0x000416, 0x000436, 0x000416, 0x000416},
+ {0x000417, 0x000437, 0x000417, 0x000417},
+ {0x000418, 0x000438, 0x000418, 0x000418},
+ {0x000419, 0x000439, 0x000419, 0x000419},
+ {0x00041a, 0x00043a, 0x00041a, 0x00041a},
+ {0x00041b, 0x00043b, 0x00041b, 0x00041b},
+ {0x00041c, 0x00043c, 0x00041c, 0x00041c},
+ {0x00041d, 0x00043d, 0x00041d, 0x00041d},
+ {0x00041e, 0x00043e, 0x00041e, 0x00041e},
+ {0x00041f, 0x00043f, 0x00041f, 0x00041f},
+ {0x000420, 0x000440, 0x000420, 0x000420},
+ {0x000421, 0x000441, 0x000421, 0x000421},
+ {0x000422, 0x000442, 0x000422, 0x000422},
+ {0x000423, 0x000443, 0x000423, 0x000423},
+ {0x000424, 0x000444, 0x000424, 0x000424},
+ {0x000425, 0x000445, 0x000425, 0x000425},
+ {0x000426, 0x000446, 0x000426, 0x000426},
+ {0x000427, 0x000447, 0x000427, 0x000427},
+ {0x000428, 0x000448, 0x000428, 0x000428},
+ {0x000429, 0x000449, 0x000429, 0x000429},
+ {0x00042a, 0x00044a, 0x00042a, 0x00042a},
+ {0x00042b, 0x00044b, 0x00042b, 0x00042b},
+ {0x00042c, 0x00044c, 0x00042c, 0x00042c},
+ {0x00042d, 0x00044d, 0x00042d, 0x00042d},
+ {0x00042e, 0x00044e, 0x00042e, 0x00042e},
+ {0x00042f, 0x00044f, 0x00042f, 0x00042f},
+ {0x000430, 0x000430, 0x000410, 0x000410},
+ {0x000431, 0x000431, 0x000411, 0x000411},
+ {0x000432, 0x000432, 0x000412, 0x000412},
+ {0x000433, 0x000433, 0x000413, 0x000413},
+ {0x000434, 0x000434, 0x000414, 0x000414},
+ {0x000435, 0x000435, 0x000415, 0x000415},
+ {0x000436, 0x000436, 0x000416, 0x000416},
+ {0x000437, 0x000437, 0x000417, 0x000417},
+ {0x000438, 0x000438, 0x000418, 0x000418},
+ {0x000439, 0x000439, 0x000419, 0x000419},
+ {0x00043a, 0x00043a, 0x00041a, 0x00041a},
+ {0x00043b, 0x00043b, 0x00041b, 0x00041b},
+ {0x00043c, 0x00043c, 0x00041c, 0x00041c},
+ {0x00043d, 0x00043d, 0x00041d, 0x00041d},
+ {0x00043e, 0x00043e, 0x00041e, 0x00041e},
+ {0x00043f, 0x00043f, 0x00041f, 0x00041f},
+ {0x000440, 0x000440, 0x000420, 0x000420},
+ {0x000441, 0x000441, 0x000421, 0x000421},
+ {0x000442, 0x000442, 0x000422, 0x000422},
+ {0x000443, 0x000443, 0x000423, 0x000423},
+ {0x000444, 0x000444, 0x000424, 0x000424},
+ {0x000445, 0x000445, 0x000425, 0x000425},
+ {0x000446, 0x000446, 0x000426, 0x000426},
+ {0x000447, 0x000447, 0x000427, 0x000427},
+ {0x000448, 0x000448, 0x000428, 0x000428},
+ {0x000449, 0x000449, 0x000429, 0x000429},
+ {0x00044a, 0x00044a, 0x00042a, 0x00042a},
+ {0x00044b, 0x00044b, 0x00042b, 0x00042b},
+ {0x00044c, 0x00044c, 0x00042c, 0x00042c},
+ {0x00044d, 0x00044d, 0x00042d, 0x00042d},
+ {0x00044e, 0x00044e, 0x00042e, 0x00042e},
+ {0x00044f, 0x00044f, 0x00042f, 0x00042f},
+ {0x000450, 0x000450, 0x000400, 0x000400},
+ {0x000451, 0x000451, 0x000401, 0x000401},
+ {0x000452, 0x000452, 0x000402, 0x000402},
+ {0x000453, 0x000453, 0x000403, 0x000403},
+ {0x000454, 0x000454, 0x000404, 0x000404},
+ {0x000455, 0x000455, 0x000405, 0x000405},
+ {0x000456, 0x000456, 0x000406, 0x000406},
+ {0x000457, 0x000457, 0x000407, 0x000407},
+ {0x000458, 0x000458, 0x000408, 0x000408},
+ {0x000459, 0x000459, 0x000409, 0x000409},
+ {0x00045a, 0x00045a, 0x00040a, 0x00040a},
+ {0x00045b, 0x00045b, 0x00040b, 0x00040b},
+ {0x00045c, 0x00045c, 0x00040c, 0x00040c},
+ {0x00045d, 0x00045d, 0x00040d, 0x00040d},
+ {0x00045e, 0x00045e, 0x00040e, 0x00040e},
+ {0x00045f, 0x00045f, 0x00040f, 0x00040f},
+ {0x000460, 0x000461, 0x000460, 0x000460},
+ {0x000461, 0x000461, 0x000460, 0x000460},
+ {0x000462, 0x000463, 0x000462, 0x000462},
+ {0x000463, 0x000463, 0x000462, 0x000462},
+ {0x000464, 0x000465, 0x000464, 0x000464},
+ {0x000465, 0x000465, 0x000464, 0x000464},
+ {0x000466, 0x000467, 0x000466, 0x000466},
+ {0x000467, 0x000467, 0x000466, 0x000466},
+ {0x000468, 0x000469, 0x000468, 0x000468},
+ {0x000469, 0x000469, 0x000468, 0x000468},
+ {0x00046a, 0x00046b, 0x00046a, 0x00046a},
+ {0x00046b, 0x00046b, 0x00046a, 0x00046a},
+ {0x00046c, 0x00046d, 0x00046c, 0x00046c},
+ {0x00046d, 0x00046d, 0x00046c, 0x00046c},
+ {0x00046e, 0x00046f, 0x00046e, 0x00046e},
+ {0x00046f, 0x00046f, 0x00046e, 0x00046e},
+ {0x000470, 0x000471, 0x000470, 0x000470},
+ {0x000471, 0x000471, 0x000470, 0x000470},
+ {0x000472, 0x000473, 0x000472, 0x000472},
+ {0x000473, 0x000473, 0x000472, 0x000472},
+ {0x000474, 0x000475, 0x000474, 0x000474},
+ {0x000475, 0x000475, 0x000474, 0x000474},
+ {0x000476, 0x000477, 0x000476, 0x000476},
+ {0x000477, 0x000477, 0x000476, 0x000476},
+ {0x000478, 0x000479, 0x000478, 0x000478},
+ {0x000479, 0x000479, 0x000478, 0x000478},
+ {0x00047a, 0x00047b, 0x00047a, 0x00047a},
+ {0x00047b, 0x00047b, 0x00047a, 0x00047a},
+ {0x00047c, 0x00047d, 0x00047c, 0x00047c},
+ {0x00047d, 0x00047d, 0x00047c, 0x00047c},
+ {0x00047e, 0x00047f, 0x00047e, 0x00047e},
+ {0x00047f, 0x00047f, 0x00047e, 0x00047e},
+ {0x000480, 0x000481, 0x000480, 0x000480},
+ {0x000481, 0x000481, 0x000480, 0x000480},
+ {0x00048a, 0x00048b, 0x00048a, 0x00048a},
+ {0x00048b, 0x00048b, 0x00048a, 0x00048a},
+ {0x00048c, 0x00048d, 0x00048c, 0x00048c},
+ {0x00048d, 0x00048d, 0x00048c, 0x00048c},
+ {0x00048e, 0x00048f, 0x00048e, 0x00048e},
+ {0x00048f, 0x00048f, 0x00048e, 0x00048e},
+ {0x000490, 0x000491, 0x000490, 0x000490},
+ {0x000491, 0x000491, 0x000490, 0x000490},
+ {0x000492, 0x000493, 0x000492, 0x000492},
+ {0x000493, 0x000493, 0x000492, 0x000492},
+ {0x000494, 0x000495, 0x000494, 0x000494},
+ {0x000495, 0x000495, 0x000494, 0x000494},
+ {0x000496, 0x000497, 0x000496, 0x000496},
+ {0x000497, 0x000497, 0x000496, 0x000496},
+ {0x000498, 0x000499, 0x000498, 0x000498},
+ {0x000499, 0x000499, 0x000498, 0x000498},
+ {0x00049a, 0x00049b, 0x00049a, 0x00049a},
+ {0x00049b, 0x00049b, 0x00049a, 0x00049a},
+ {0x00049c, 0x00049d, 0x00049c, 0x00049c},
+ {0x00049d, 0x00049d, 0x00049c, 0x00049c},
+ {0x00049e, 0x00049f, 0x00049e, 0x00049e},
+ {0x00049f, 0x00049f, 0x00049e, 0x00049e},
+ {0x0004a0, 0x0004a1, 0x0004a0, 0x0004a0},
+ {0x0004a1, 0x0004a1, 0x0004a0, 0x0004a0},
+ {0x0004a2, 0x0004a3, 0x0004a2, 0x0004a2},
+ {0x0004a3, 0x0004a3, 0x0004a2, 0x0004a2},
+ {0x0004a4, 0x0004a5, 0x0004a4, 0x0004a4},
+ {0x0004a5, 0x0004a5, 0x0004a4, 0x0004a4},
+ {0x0004a6, 0x0004a7, 0x0004a6, 0x0004a6},
+ {0x0004a7, 0x0004a7, 0x0004a6, 0x0004a6},
+ {0x0004a8, 0x0004a9, 0x0004a8, 0x0004a8},
+ {0x0004a9, 0x0004a9, 0x0004a8, 0x0004a8},
+ {0x0004aa, 0x0004ab, 0x0004aa, 0x0004aa},
+ {0x0004ab, 0x0004ab, 0x0004aa, 0x0004aa},
+ {0x0004ac, 0x0004ad, 0x0004ac, 0x0004ac},
+ {0x0004ad, 0x0004ad, 0x0004ac, 0x0004ac},
+ {0x0004ae, 0x0004af, 0x0004ae, 0x0004ae},
+ {0x0004af, 0x0004af, 0x0004ae, 0x0004ae},
+ {0x0004b0, 0x0004b1, 0x0004b0, 0x0004b0},
+ {0x0004b1, 0x0004b1, 0x0004b0, 0x0004b0},
+ {0x0004b2, 0x0004b3, 0x0004b2, 0x0004b2},
+ {0x0004b3, 0x0004b3, 0x0004b2, 0x0004b2},
+ {0x0004b4, 0x0004b5, 0x0004b4, 0x0004b4},
+ {0x0004b5, 0x0004b5, 0x0004b4, 0x0004b4},
+ {0x0004b6, 0x0004b7, 0x0004b6, 0x0004b6},
+ {0x0004b7, 0x0004b7, 0x0004b6, 0x0004b6},
+ {0x0004b8, 0x0004b9, 0x0004b8, 0x0004b8},
+ {0x0004b9, 0x0004b9, 0x0004b8, 0x0004b8},
+ {0x0004ba, 0x0004bb, 0x0004ba, 0x0004ba},
+ {0x0004bb, 0x0004bb, 0x0004ba, 0x0004ba},
+ {0x0004bc, 0x0004bd, 0x0004bc, 0x0004bc},
+ {0x0004bd, 0x0004bd, 0x0004bc, 0x0004bc},
+ {0x0004be, 0x0004bf, 0x0004be, 0x0004be},
+ {0x0004bf, 0x0004bf, 0x0004be, 0x0004be},
+ {0x0004c0, 0x0004cf, 0x0004c0, 0x0004c0},
+ {0x0004c1, 0x0004c2, 0x0004c1, 0x0004c1},
+ {0x0004c2, 0x0004c2, 0x0004c1, 0x0004c1},
+ {0x0004c3, 0x0004c4, 0x0004c3, 0x0004c3},
+ {0x0004c4, 0x0004c4, 0x0004c3, 0x0004c3},
+ {0x0004c5, 0x0004c6, 0x0004c5, 0x0004c5},
+ {0x0004c6, 0x0004c6, 0x0004c5, 0x0004c5},
+ {0x0004c7, 0x0004c8, 0x0004c7, 0x0004c7},
+ {0x0004c8, 0x0004c8, 0x0004c7, 0x0004c7},
+ {0x0004c9, 0x0004ca, 0x0004c9, 0x0004c9},
+ {0x0004ca, 0x0004ca, 0x0004c9, 0x0004c9},
+ {0x0004cb, 0x0004cc, 0x0004cb, 0x0004cb},
+ {0x0004cc, 0x0004cc, 0x0004cb, 0x0004cb},
+ {0x0004cd, 0x0004ce, 0x0004cd, 0x0004cd},
+ {0x0004ce, 0x0004ce, 0x0004cd, 0x0004cd},
+ {0x0004cf, 0x0004cf, 0x0004c0, 0x0004c0},
+ {0x0004d0, 0x0004d1, 0x0004d0, 0x0004d0},
+ {0x0004d1, 0x0004d1, 0x0004d0, 0x0004d0},
+ {0x0004d2, 0x0004d3, 0x0004d2, 0x0004d2},
+ {0x0004d3, 0x0004d3, 0x0004d2, 0x0004d2},
+ {0x0004d4, 0x0004d5, 0x0004d4, 0x0004d4},
+ {0x0004d5, 0x0004d5, 0x0004d4, 0x0004d4},
+ {0x0004d6, 0x0004d7, 0x0004d6, 0x0004d6},
+ {0x0004d7, 0x0004d7, 0x0004d6, 0x0004d6},
+ {0x0004d8, 0x0004d9, 0x0004d8, 0x0004d8},
+ {0x0004d9, 0x0004d9, 0x0004d8, 0x0004d8},
+ {0x0004da, 0x0004db, 0x0004da, 0x0004da},
+ {0x0004db, 0x0004db, 0x0004da, 0x0004da},
+ {0x0004dc, 0x0004dd, 0x0004dc, 0x0004dc},
+ {0x0004dd, 0x0004dd, 0x0004dc, 0x0004dc},
+ {0x0004de, 0x0004df, 0x0004de, 0x0004de},
+ {0x0004df, 0x0004df, 0x0004de, 0x0004de},
+ {0x0004e0, 0x0004e1, 0x0004e0, 0x0004e0},
+ {0x0004e1, 0x0004e1, 0x0004e0, 0x0004e0},
+ {0x0004e2, 0x0004e3, 0x0004e2, 0x0004e2},
+ {0x0004e3, 0x0004e3, 0x0004e2, 0x0004e2},
+ {0x0004e4, 0x0004e5, 0x0004e4, 0x0004e4},
+ {0x0004e5, 0x0004e5, 0x0004e4, 0x0004e4},
+ {0x0004e6, 0x0004e7, 0x0004e6, 0x0004e6},
+ {0x0004e7, 0x0004e7, 0x0004e6, 0x0004e6},
+ {0x0004e8, 0x0004e9, 0x0004e8, 0x0004e8},
+ {0x0004e9, 0x0004e9, 0x0004e8, 0x0004e8},
+ {0x0004ea, 0x0004eb, 0x0004ea, 0x0004ea},
+ {0x0004eb, 0x0004eb, 0x0004ea, 0x0004ea},
+ {0x0004ec, 0x0004ed, 0x0004ec, 0x0004ec},
+ {0x0004ed, 0x0004ed, 0x0004ec, 0x0004ec},
+ {0x0004ee, 0x0004ef, 0x0004ee, 0x0004ee},
+ {0x0004ef, 0x0004ef, 0x0004ee, 0x0004ee},
+ {0x0004f0, 0x0004f1, 0x0004f0, 0x0004f0},
+ {0x0004f1, 0x0004f1, 0x0004f0, 0x0004f0},
+ {0x0004f2, 0x0004f3, 0x0004f2, 0x0004f2},
+ {0x0004f3, 0x0004f3, 0x0004f2, 0x0004f2},
+ {0x0004f4, 0x0004f5, 0x0004f4, 0x0004f4},
+ {0x0004f5, 0x0004f5, 0x0004f4, 0x0004f4},
+ {0x0004f6, 0x0004f7, 0x0004f6, 0x0004f6},
+ {0x0004f7, 0x0004f7, 0x0004f6, 0x0004f6},
+ {0x0004f8, 0x0004f9, 0x0004f8, 0x0004f8},
+ {0x0004f9, 0x0004f9, 0x0004f8, 0x0004f8},
+ {0x0004fa, 0x0004fb, 0x0004fa, 0x0004fa},
+ {0x0004fb, 0x0004fb, 0x0004fa, 0x0004fa},
+ {0x0004fc, 0x0004fd, 0x0004fc, 0x0004fc},
+ {0x0004fd, 0x0004fd, 0x0004fc, 0x0004fc},
+ {0x0004fe, 0x0004ff, 0x0004fe, 0x0004fe},
+ {0x0004ff, 0x0004ff, 0x0004fe, 0x0004fe},
+ {0x000500, 0x000501, 0x000500, 0x000500},
+ {0x000501, 0x000501, 0x000500, 0x000500},
+ {0x000502, 0x000503, 0x000502, 0x000502},
+ {0x000503, 0x000503, 0x000502, 0x000502},
+ {0x000504, 0x000505, 0x000504, 0x000504},
+ {0x000505, 0x000505, 0x000504, 0x000504},
+ {0x000506, 0x000507, 0x000506, 0x000506},
+ {0x000507, 0x000507, 0x000506, 0x000506},
+ {0x000508, 0x000509, 0x000508, 0x000508},
+ {0x000509, 0x000509, 0x000508, 0x000508},
+ {0x00050a, 0x00050b, 0x00050a, 0x00050a},
+ {0x00050b, 0x00050b, 0x00050a, 0x00050a},
+ {0x00050c, 0x00050d, 0x00050c, 0x00050c},
+ {0x00050d, 0x00050d, 0x00050c, 0x00050c},
+ {0x00050e, 0x00050f, 0x00050e, 0x00050e},
+ {0x00050f, 0x00050f, 0x00050e, 0x00050e},
+ {0x000510, 0x000511, 0x000510, 0x000510},
+ {0x000511, 0x000511, 0x000510, 0x000510},
+ {0x000512, 0x000513, 0x000512, 0x000512},
+ {0x000513, 0x000513, 0x000512, 0x000512},
+ {0x000514, 0x000515, 0x000514, 0x000514},
+ {0x000515, 0x000515, 0x000514, 0x000514},
+ {0x000516, 0x000517, 0x000516, 0x000516},
+ {0x000517, 0x000517, 0x000516, 0x000516},
+ {0x000518, 0x000519, 0x000518, 0x000518},
+ {0x000519, 0x000519, 0x000518, 0x000518},
+ {0x00051a, 0x00051b, 0x00051a, 0x00051a},
+ {0x00051b, 0x00051b, 0x00051a, 0x00051a},
+ {0x00051c, 0x00051d, 0x00051c, 0x00051c},
+ {0x00051d, 0x00051d, 0x00051c, 0x00051c},
+ {0x00051e, 0x00051f, 0x00051e, 0x00051e},
+ {0x00051f, 0x00051f, 0x00051e, 0x00051e},
+ {0x000520, 0x000521, 0x000520, 0x000520},
+ {0x000521, 0x000521, 0x000520, 0x000520},
+ {0x000522, 0x000523, 0x000522, 0x000522},
+ {0x000523, 0x000523, 0x000522, 0x000522},
+ {0x000524, 0x000525, 0x000524, 0x000524},
+ {0x000525, 0x000525, 0x000524, 0x000524},
+ {0x000526, 0x000527, 0x000526, 0x000526},
+ {0x000527, 0x000527, 0x000526, 0x000526},
+ {0x000528, 0x000529, 0x000528, 0x000528},
+ {0x000529, 0x000529, 0x000528, 0x000528},
+ {0x00052a, 0x00052b, 0x00052a, 0x00052a},
+ {0x00052b, 0x00052b, 0x00052a, 0x00052a},
+ {0x00052c, 0x00052d, 0x00052c, 0x00052c},
+ {0x00052d, 0x00052d, 0x00052c, 0x00052c},
+ {0x00052e, 0x00052f, 0x00052e, 0x00052e},
+ {0x00052f, 0x00052f, 0x00052e, 0x00052e},
+ {0x000531, 0x000561, 0x000531, 0x000531},
+ {0x000532, 0x000562, 0x000532, 0x000532},
+ {0x000533, 0x000563, 0x000533, 0x000533},
+ {0x000534, 0x000564, 0x000534, 0x000534},
+ {0x000535, 0x000565, 0x000535, 0x000535},
+ {0x000536, 0x000566, 0x000536, 0x000536},
+ {0x000537, 0x000567, 0x000537, 0x000537},
+ {0x000538, 0x000568, 0x000538, 0x000538},
+ {0x000539, 0x000569, 0x000539, 0x000539},
+ {0x00053a, 0x00056a, 0x00053a, 0x00053a},
+ {0x00053b, 0x00056b, 0x00053b, 0x00053b},
+ {0x00053c, 0x00056c, 0x00053c, 0x00053c},
+ {0x00053d, 0x00056d, 0x00053d, 0x00053d},
+ {0x00053e, 0x00056e, 0x00053e, 0x00053e},
+ {0x00053f, 0x00056f, 0x00053f, 0x00053f},
+ {0x000540, 0x000570, 0x000540, 0x000540},
+ {0x000541, 0x000571, 0x000541, 0x000541},
+ {0x000542, 0x000572, 0x000542, 0x000542},
+ {0x000543, 0x000573, 0x000543, 0x000543},
+ {0x000544, 0x000574, 0x000544, 0x000544},
+ {0x000545, 0x000575, 0x000545, 0x000545},
+ {0x000546, 0x000576, 0x000546, 0x000546},
+ {0x000547, 0x000577, 0x000547, 0x000547},
+ {0x000548, 0x000578, 0x000548, 0x000548},
+ {0x000549, 0x000579, 0x000549, 0x000549},
+ {0x00054a, 0x00057a, 0x00054a, 0x00054a},
+ {0x00054b, 0x00057b, 0x00054b, 0x00054b},
+ {0x00054c, 0x00057c, 0x00054c, 0x00054c},
+ {0x00054d, 0x00057d, 0x00054d, 0x00054d},
+ {0x00054e, 0x00057e, 0x00054e, 0x00054e},
+ {0x00054f, 0x00057f, 0x00054f, 0x00054f},
+ {0x000550, 0x000580, 0x000550, 0x000550},
+ {0x000551, 0x000581, 0x000551, 0x000551},
+ {0x000552, 0x000582, 0x000552, 0x000552},
+ {0x000553, 0x000583, 0x000553, 0x000553},
+ {0x000554, 0x000584, 0x000554, 0x000554},
+ {0x000555, 0x000585, 0x000555, 0x000555},
+ {0x000556, 0x000586, 0x000556, 0x000556},
+ {0x000561, 0x000561, 0x000531, 0x000531},
+ {0x000562, 0x000562, 0x000532, 0x000532},
+ {0x000563, 0x000563, 0x000533, 0x000533},
+ {0x000564, 0x000564, 0x000534, 0x000534},
+ {0x000565, 0x000565, 0x000535, 0x000535},
+ {0x000566, 0x000566, 0x000536, 0x000536},
+ {0x000567, 0x000567, 0x000537, 0x000537},
+ {0x000568, 0x000568, 0x000538, 0x000538},
+ {0x000569, 0x000569, 0x000539, 0x000539},
+ {0x00056a, 0x00056a, 0x00053a, 0x00053a},
+ {0x00056b, 0x00056b, 0x00053b, 0x00053b},
+ {0x00056c, 0x00056c, 0x00053c, 0x00053c},
+ {0x00056d, 0x00056d, 0x00053d, 0x00053d},
+ {0x00056e, 0x00056e, 0x00053e, 0x00053e},
+ {0x00056f, 0x00056f, 0x00053f, 0x00053f},
+ {0x000570, 0x000570, 0x000540, 0x000540},
+ {0x000571, 0x000571, 0x000541, 0x000541},
+ {0x000572, 0x000572, 0x000542, 0x000542},
+ {0x000573, 0x000573, 0x000543, 0x000543},
+ {0x000574, 0x000574, 0x000544, 0x000544},
+ {0x000575, 0x000575, 0x000545, 0x000545},
+ {0x000576, 0x000576, 0x000546, 0x000546},
+ {0x000577, 0x000577, 0x000547, 0x000547},
+ {0x000578, 0x000578, 0x000548, 0x000548},
+ {0x000579, 0x000579, 0x000549, 0x000549},
+ {0x00057a, 0x00057a, 0x00054a, 0x00054a},
+ {0x00057b, 0x00057b, 0x00054b, 0x00054b},
+ {0x00057c, 0x00057c, 0x00054c, 0x00054c},
+ {0x00057d, 0x00057d, 0x00054d, 0x00054d},
+ {0x00057e, 0x00057e, 0x00054e, 0x00054e},
+ {0x00057f, 0x00057f, 0x00054f, 0x00054f},
+ {0x000580, 0x000580, 0x000550, 0x000550},
+ {0x000581, 0x000581, 0x000551, 0x000551},
+ {0x000582, 0x000582, 0x000552, 0x000552},
+ {0x000583, 0x000583, 0x000553, 0x000553},
+ {0x000584, 0x000584, 0x000554, 0x000554},
+ {0x000585, 0x000585, 0x000555, 0x000555},
+ {0x000586, 0x000586, 0x000556, 0x000556},
+ {0x0010a0, 0x002d00, 0x0010a0, 0x0010a0},
+ {0x0010a1, 0x002d01, 0x0010a1, 0x0010a1},
+ {0x0010a2, 0x002d02, 0x0010a2, 0x0010a2},
+ {0x0010a3, 0x002d03, 0x0010a3, 0x0010a3},
+ {0x0010a4, 0x002d04, 0x0010a4, 0x0010a4},
+ {0x0010a5, 0x002d05, 0x0010a5, 0x0010a5},
+ {0x0010a6, 0x002d06, 0x0010a6, 0x0010a6},
+ {0x0010a7, 0x002d07, 0x0010a7, 0x0010a7},
+ {0x0010a8, 0x002d08, 0x0010a8, 0x0010a8},
+ {0x0010a9, 0x002d09, 0x0010a9, 0x0010a9},
+ {0x0010aa, 0x002d0a, 0x0010aa, 0x0010aa},
+ {0x0010ab, 0x002d0b, 0x0010ab, 0x0010ab},
+ {0x0010ac, 0x002d0c, 0x0010ac, 0x0010ac},
+ {0x0010ad, 0x002d0d, 0x0010ad, 0x0010ad},
+ {0x0010ae, 0x002d0e, 0x0010ae, 0x0010ae},
+ {0x0010af, 0x002d0f, 0x0010af, 0x0010af},
+ {0x0010b0, 0x002d10, 0x0010b0, 0x0010b0},
+ {0x0010b1, 0x002d11, 0x0010b1, 0x0010b1},
+ {0x0010b2, 0x002d12, 0x0010b2, 0x0010b2},
+ {0x0010b3, 0x002d13, 0x0010b3, 0x0010b3},
+ {0x0010b4, 0x002d14, 0x0010b4, 0x0010b4},
+ {0x0010b5, 0x002d15, 0x0010b5, 0x0010b5},
+ {0x0010b6, 0x002d16, 0x0010b6, 0x0010b6},
+ {0x0010b7, 0x002d17, 0x0010b7, 0x0010b7},
+ {0x0010b8, 0x002d18, 0x0010b8, 0x0010b8},
+ {0x0010b9, 0x002d19, 0x0010b9, 0x0010b9},
+ {0x0010ba, 0x002d1a, 0x0010ba, 0x0010ba},
+ {0x0010bb, 0x002d1b, 0x0010bb, 0x0010bb},
+ {0x0010bc, 0x002d1c, 0x0010bc, 0x0010bc},
+ {0x0010bd, 0x002d1d, 0x0010bd, 0x0010bd},
+ {0x0010be, 0x002d1e, 0x0010be, 0x0010be},
+ {0x0010bf, 0x002d1f, 0x0010bf, 0x0010bf},
+ {0x0010c0, 0x002d20, 0x0010c0, 0x0010c0},
+ {0x0010c1, 0x002d21, 0x0010c1, 0x0010c1},
+ {0x0010c2, 0x002d22, 0x0010c2, 0x0010c2},
+ {0x0010c3, 0x002d23, 0x0010c3, 0x0010c3},
+ {0x0010c4, 0x002d24, 0x0010c4, 0x0010c4},
+ {0x0010c5, 0x002d25, 0x0010c5, 0x0010c5},
+ {0x0010c7, 0x002d27, 0x0010c7, 0x0010c7},
+ {0x0010cd, 0x002d2d, 0x0010cd, 0x0010cd},
+ {0x0010d0, 0x0010d0, 0x0010d0, 0x001c90},
+ {0x0010d1, 0x0010d1, 0x0010d1, 0x001c91},
+ {0x0010d2, 0x0010d2, 0x0010d2, 0x001c92},
+ {0x0010d3, 0x0010d3, 0x0010d3, 0x001c93},
+ {0x0010d4, 0x0010d4, 0x0010d4, 0x001c94},
+ {0x0010d5, 0x0010d5, 0x0010d5, 0x001c95},
+ {0x0010d6, 0x0010d6, 0x0010d6, 0x001c96},
+ {0x0010d7, 0x0010d7, 0x0010d7, 0x001c97},
+ {0x0010d8, 0x0010d8, 0x0010d8, 0x001c98},
+ {0x0010d9, 0x0010d9, 0x0010d9, 0x001c99},
+ {0x0010da, 0x0010da, 0x0010da, 0x001c9a},
+ {0x0010db, 0x0010db, 0x0010db, 0x001c9b},
+ {0x0010dc, 0x0010dc, 0x0010dc, 0x001c9c},
+ {0x0010dd, 0x0010dd, 0x0010dd, 0x001c9d},
+ {0x0010de, 0x0010de, 0x0010de, 0x001c9e},
+ {0x0010df, 0x0010df, 0x0010df, 0x001c9f},
+ {0x0010e0, 0x0010e0, 0x0010e0, 0x001ca0},
+ {0x0010e1, 0x0010e1, 0x0010e1, 0x001ca1},
+ {0x0010e2, 0x0010e2, 0x0010e2, 0x001ca2},
+ {0x0010e3, 0x0010e3, 0x0010e3, 0x001ca3},
+ {0x0010e4, 0x0010e4, 0x0010e4, 0x001ca4},
+ {0x0010e5, 0x0010e5, 0x0010e5, 0x001ca5},
+ {0x0010e6, 0x0010e6, 0x0010e6, 0x001ca6},
+ {0x0010e7, 0x0010e7, 0x0010e7, 0x001ca7},
+ {0x0010e8, 0x0010e8, 0x0010e8, 0x001ca8},
+ {0x0010e9, 0x0010e9, 0x0010e9, 0x001ca9},
+ {0x0010ea, 0x0010ea, 0x0010ea, 0x001caa},
+ {0x0010eb, 0x0010eb, 0x0010eb, 0x001cab},
+ {0x0010ec, 0x0010ec, 0x0010ec, 0x001cac},
+ {0x0010ed, 0x0010ed, 0x0010ed, 0x001cad},
+ {0x0010ee, 0x0010ee, 0x0010ee, 0x001cae},
+ {0x0010ef, 0x0010ef, 0x0010ef, 0x001caf},
+ {0x0010f0, 0x0010f0, 0x0010f0, 0x001cb0},
+ {0x0010f1, 0x0010f1, 0x0010f1, 0x001cb1},
+ {0x0010f2, 0x0010f2, 0x0010f2, 0x001cb2},
+ {0x0010f3, 0x0010f3, 0x0010f3, 0x001cb3},
+ {0x0010f4, 0x0010f4, 0x0010f4, 0x001cb4},
+ {0x0010f5, 0x0010f5, 0x0010f5, 0x001cb5},
+ {0x0010f6, 0x0010f6, 0x0010f6, 0x001cb6},
+ {0x0010f7, 0x0010f7, 0x0010f7, 0x001cb7},
+ {0x0010f8, 0x0010f8, 0x0010f8, 0x001cb8},
+ {0x0010f9, 0x0010f9, 0x0010f9, 0x001cb9},
+ {0x0010fa, 0x0010fa, 0x0010fa, 0x001cba},
+ {0x0010fd, 0x0010fd, 0x0010fd, 0x001cbd},
+ {0x0010fe, 0x0010fe, 0x0010fe, 0x001cbe},
+ {0x0010ff, 0x0010ff, 0x0010ff, 0x001cbf},
+ {0x0013a0, 0x00ab70, 0x0013a0, 0x0013a0},
+ {0x0013a1, 0x00ab71, 0x0013a1, 0x0013a1},
+ {0x0013a2, 0x00ab72, 0x0013a2, 0x0013a2},
+ {0x0013a3, 0x00ab73, 0x0013a3, 0x0013a3},
+ {0x0013a4, 0x00ab74, 0x0013a4, 0x0013a4},
+ {0x0013a5, 0x00ab75, 0x0013a5, 0x0013a5},
+ {0x0013a6, 0x00ab76, 0x0013a6, 0x0013a6},
+ {0x0013a7, 0x00ab77, 0x0013a7, 0x0013a7},
+ {0x0013a8, 0x00ab78, 0x0013a8, 0x0013a8},
+ {0x0013a9, 0x00ab79, 0x0013a9, 0x0013a9},
+ {0x0013aa, 0x00ab7a, 0x0013aa, 0x0013aa},
+ {0x0013ab, 0x00ab7b, 0x0013ab, 0x0013ab},
+ {0x0013ac, 0x00ab7c, 0x0013ac, 0x0013ac},
+ {0x0013ad, 0x00ab7d, 0x0013ad, 0x0013ad},
+ {0x0013ae, 0x00ab7e, 0x0013ae, 0x0013ae},
+ {0x0013af, 0x00ab7f, 0x0013af, 0x0013af},
+ {0x0013b0, 0x00ab80, 0x0013b0, 0x0013b0},
+ {0x0013b1, 0x00ab81, 0x0013b1, 0x0013b1},
+ {0x0013b2, 0x00ab82, 0x0013b2, 0x0013b2},
+ {0x0013b3, 0x00ab83, 0x0013b3, 0x0013b3},
+ {0x0013b4, 0x00ab84, 0x0013b4, 0x0013b4},
+ {0x0013b5, 0x00ab85, 0x0013b5, 0x0013b5},
+ {0x0013b6, 0x00ab86, 0x0013b6, 0x0013b6},
+ {0x0013b7, 0x00ab87, 0x0013b7, 0x0013b7},
+ {0x0013b8, 0x00ab88, 0x0013b8, 0x0013b8},
+ {0x0013b9, 0x00ab89, 0x0013b9, 0x0013b9},
+ {0x0013ba, 0x00ab8a, 0x0013ba, 0x0013ba},
+ {0x0013bb, 0x00ab8b, 0x0013bb, 0x0013bb},
+ {0x0013bc, 0x00ab8c, 0x0013bc, 0x0013bc},
+ {0x0013bd, 0x00ab8d, 0x0013bd, 0x0013bd},
+ {0x0013be, 0x00ab8e, 0x0013be, 0x0013be},
+ {0x0013bf, 0x00ab8f, 0x0013bf, 0x0013bf},
+ {0x0013c0, 0x00ab90, 0x0013c0, 0x0013c0},
+ {0x0013c1, 0x00ab91, 0x0013c1, 0x0013c1},
+ {0x0013c2, 0x00ab92, 0x0013c2, 0x0013c2},
+ {0x0013c3, 0x00ab93, 0x0013c3, 0x0013c3},
+ {0x0013c4, 0x00ab94, 0x0013c4, 0x0013c4},
+ {0x0013c5, 0x00ab95, 0x0013c5, 0x0013c5},
+ {0x0013c6, 0x00ab96, 0x0013c6, 0x0013c6},
+ {0x0013c7, 0x00ab97, 0x0013c7, 0x0013c7},
+ {0x0013c8, 0x00ab98, 0x0013c8, 0x0013c8},
+ {0x0013c9, 0x00ab99, 0x0013c9, 0x0013c9},
+ {0x0013ca, 0x00ab9a, 0x0013ca, 0x0013ca},
+ {0x0013cb, 0x00ab9b, 0x0013cb, 0x0013cb},
+ {0x0013cc, 0x00ab9c, 0x0013cc, 0x0013cc},
+ {0x0013cd, 0x00ab9d, 0x0013cd, 0x0013cd},
+ {0x0013ce, 0x00ab9e, 0x0013ce, 0x0013ce},
+ {0x0013cf, 0x00ab9f, 0x0013cf, 0x0013cf},
+ {0x0013d0, 0x00aba0, 0x0013d0, 0x0013d0},
+ {0x0013d1, 0x00aba1, 0x0013d1, 0x0013d1},
+ {0x0013d2, 0x00aba2, 0x0013d2, 0x0013d2},
+ {0x0013d3, 0x00aba3, 0x0013d3, 0x0013d3},
+ {0x0013d4, 0x00aba4, 0x0013d4, 0x0013d4},
+ {0x0013d5, 0x00aba5, 0x0013d5, 0x0013d5},
+ {0x0013d6, 0x00aba6, 0x0013d6, 0x0013d6},
+ {0x0013d7, 0x00aba7, 0x0013d7, 0x0013d7},
+ {0x0013d8, 0x00aba8, 0x0013d8, 0x0013d8},
+ {0x0013d9, 0x00aba9, 0x0013d9, 0x0013d9},
+ {0x0013da, 0x00abaa, 0x0013da, 0x0013da},
+ {0x0013db, 0x00abab, 0x0013db, 0x0013db},
+ {0x0013dc, 0x00abac, 0x0013dc, 0x0013dc},
+ {0x0013dd, 0x00abad, 0x0013dd, 0x0013dd},
+ {0x0013de, 0x00abae, 0x0013de, 0x0013de},
+ {0x0013df, 0x00abaf, 0x0013df, 0x0013df},
+ {0x0013e0, 0x00abb0, 0x0013e0, 0x0013e0},
+ {0x0013e1, 0x00abb1, 0x0013e1, 0x0013e1},
+ {0x0013e2, 0x00abb2, 0x0013e2, 0x0013e2},
+ {0x0013e3, 0x00abb3, 0x0013e3, 0x0013e3},
+ {0x0013e4, 0x00abb4, 0x0013e4, 0x0013e4},
+ {0x0013e5, 0x00abb5, 0x0013e5, 0x0013e5},
+ {0x0013e6, 0x00abb6, 0x0013e6, 0x0013e6},
+ {0x0013e7, 0x00abb7, 0x0013e7, 0x0013e7},
+ {0x0013e8, 0x00abb8, 0x0013e8, 0x0013e8},
+ {0x0013e9, 0x00abb9, 0x0013e9, 0x0013e9},
+ {0x0013ea, 0x00abba, 0x0013ea, 0x0013ea},
+ {0x0013eb, 0x00abbb, 0x0013eb, 0x0013eb},
+ {0x0013ec, 0x00abbc, 0x0013ec, 0x0013ec},
+ {0x0013ed, 0x00abbd, 0x0013ed, 0x0013ed},
+ {0x0013ee, 0x00abbe, 0x0013ee, 0x0013ee},
+ {0x0013ef, 0x00abbf, 0x0013ef, 0x0013ef},
+ {0x0013f0, 0x0013f8, 0x0013f0, 0x0013f0},
+ {0x0013f1, 0x0013f9, 0x0013f1, 0x0013f1},
+ {0x0013f2, 0x0013fa, 0x0013f2, 0x0013f2},
+ {0x0013f3, 0x0013fb, 0x0013f3, 0x0013f3},
+ {0x0013f4, 0x0013fc, 0x0013f4, 0x0013f4},
+ {0x0013f5, 0x0013fd, 0x0013f5, 0x0013f5},
+ {0x0013f8, 0x0013f8, 0x0013f0, 0x0013f0},
+ {0x0013f9, 0x0013f9, 0x0013f1, 0x0013f1},
+ {0x0013fa, 0x0013fa, 0x0013f2, 0x0013f2},
+ {0x0013fb, 0x0013fb, 0x0013f3, 0x0013f3},
+ {0x0013fc, 0x0013fc, 0x0013f4, 0x0013f4},
+ {0x0013fd, 0x0013fd, 0x0013f5, 0x0013f5},
+ {0x001c80, 0x001c80, 0x000412, 0x000412},
+ {0x001c81, 0x001c81, 0x000414, 0x000414},
+ {0x001c82, 0x001c82, 0x00041e, 0x00041e},
+ {0x001c83, 0x001c83, 0x000421, 0x000421},
+ {0x001c84, 0x001c84, 0x000422, 0x000422},
+ {0x001c85, 0x001c85, 0x000422, 0x000422},
+ {0x001c86, 0x001c86, 0x00042a, 0x00042a},
+ {0x001c87, 0x001c87, 0x000462, 0x000462},
+ {0x001c88, 0x001c88, 0x00a64a, 0x00a64a},
+ {0x001c90, 0x0010d0, 0x001c90, 0x001c90},
+ {0x001c91, 0x0010d1, 0x001c91, 0x001c91},
+ {0x001c92, 0x0010d2, 0x001c92, 0x001c92},
+ {0x001c93, 0x0010d3, 0x001c93, 0x001c93},
+ {0x001c94, 0x0010d4, 0x001c94, 0x001c94},
+ {0x001c95, 0x0010d5, 0x001c95, 0x001c95},
+ {0x001c96, 0x0010d6, 0x001c96, 0x001c96},
+ {0x001c97, 0x0010d7, 0x001c97, 0x001c97},
+ {0x001c98, 0x0010d8, 0x001c98, 0x001c98},
+ {0x001c99, 0x0010d9, 0x001c99, 0x001c99},
+ {0x001c9a, 0x0010da, 0x001c9a, 0x001c9a},
+ {0x001c9b, 0x0010db, 0x001c9b, 0x001c9b},
+ {0x001c9c, 0x0010dc, 0x001c9c, 0x001c9c},
+ {0x001c9d, 0x0010dd, 0x001c9d, 0x001c9d},
+ {0x001c9e, 0x0010de, 0x001c9e, 0x001c9e},
+ {0x001c9f, 0x0010df, 0x001c9f, 0x001c9f},
+ {0x001ca0, 0x0010e0, 0x001ca0, 0x001ca0},
+ {0x001ca1, 0x0010e1, 0x001ca1, 0x001ca1},
+ {0x001ca2, 0x0010e2, 0x001ca2, 0x001ca2},
+ {0x001ca3, 0x0010e3, 0x001ca3, 0x001ca3},
+ {0x001ca4, 0x0010e4, 0x001ca4, 0x001ca4},
+ {0x001ca5, 0x0010e5, 0x001ca5, 0x001ca5},
+ {0x001ca6, 0x0010e6, 0x001ca6, 0x001ca6},
+ {0x001ca7, 0x0010e7, 0x001ca7, 0x001ca7},
+ {0x001ca8, 0x0010e8, 0x001ca8, 0x001ca8},
+ {0x001ca9, 0x0010e9, 0x001ca9, 0x001ca9},
+ {0x001caa, 0x0010ea, 0x001caa, 0x001caa},
+ {0x001cab, 0x0010eb, 0x001cab, 0x001cab},
+ {0x001cac, 0x0010ec, 0x001cac, 0x001cac},
+ {0x001cad, 0x0010ed, 0x001cad, 0x001cad},
+ {0x001cae, 0x0010ee, 0x001cae, 0x001cae},
+ {0x001caf, 0x0010ef, 0x001caf, 0x001caf},
+ {0x001cb0, 0x0010f0, 0x001cb0, 0x001cb0},
+ {0x001cb1, 0x0010f1, 0x001cb1, 0x001cb1},
+ {0x001cb2, 0x0010f2, 0x001cb2, 0x001cb2},
+ {0x001cb3, 0x0010f3, 0x001cb3, 0x001cb3},
+ {0x001cb4, 0x0010f4, 0x001cb4, 0x001cb4},
+ {0x001cb5, 0x0010f5, 0x001cb5, 0x001cb5},
+ {0x001cb6, 0x0010f6, 0x001cb6, 0x001cb6},
+ {0x001cb7, 0x0010f7, 0x001cb7, 0x001cb7},
+ {0x001cb8, 0x0010f8, 0x001cb8, 0x001cb8},
+ {0x001cb9, 0x0010f9, 0x001cb9, 0x001cb9},
+ {0x001cba, 0x0010fa, 0x001cba, 0x001cba},
+ {0x001cbd, 0x0010fd, 0x001cbd, 0x001cbd},
+ {0x001cbe, 0x0010fe, 0x001cbe, 0x001cbe},
+ {0x001cbf, 0x0010ff, 0x001cbf, 0x001cbf},
+ {0x001d79, 0x001d79, 0x00a77d, 0x00a77d},
+ {0x001d7d, 0x001d7d, 0x002c63, 0x002c63},
+ {0x001d8e, 0x001d8e, 0x00a7c6, 0x00a7c6},
+ {0x001e00, 0x001e01, 0x001e00, 0x001e00},
+ {0x001e01, 0x001e01, 0x001e00, 0x001e00},
+ {0x001e02, 0x001e03, 0x001e02, 0x001e02},
+ {0x001e03, 0x001e03, 0x001e02, 0x001e02},
+ {0x001e04, 0x001e05, 0x001e04, 0x001e04},
+ {0x001e05, 0x001e05, 0x001e04, 0x001e04},
+ {0x001e06, 0x001e07, 0x001e06, 0x001e06},
+ {0x001e07, 0x001e07, 0x001e06, 0x001e06},
+ {0x001e08, 0x001e09, 0x001e08, 0x001e08},
+ {0x001e09, 0x001e09, 0x001e08, 0x001e08},
+ {0x001e0a, 0x001e0b, 0x001e0a, 0x001e0a},
+ {0x001e0b, 0x001e0b, 0x001e0a, 0x001e0a},
+ {0x001e0c, 0x001e0d, 0x001e0c, 0x001e0c},
+ {0x001e0d, 0x001e0d, 0x001e0c, 0x001e0c},
+ {0x001e0e, 0x001e0f, 0x001e0e, 0x001e0e},
+ {0x001e0f, 0x001e0f, 0x001e0e, 0x001e0e},
+ {0x001e10, 0x001e11, 0x001e10, 0x001e10},
+ {0x001e11, 0x001e11, 0x001e10, 0x001e10},
+ {0x001e12, 0x001e13, 0x001e12, 0x001e12},
+ {0x001e13, 0x001e13, 0x001e12, 0x001e12},
+ {0x001e14, 0x001e15, 0x001e14, 0x001e14},
+ {0x001e15, 0x001e15, 0x001e14, 0x001e14},
+ {0x001e16, 0x001e17, 0x001e16, 0x001e16},
+ {0x001e17, 0x001e17, 0x001e16, 0x001e16},
+ {0x001e18, 0x001e19, 0x001e18, 0x001e18},
+ {0x001e19, 0x001e19, 0x001e18, 0x001e18},
+ {0x001e1a, 0x001e1b, 0x001e1a, 0x001e1a},
+ {0x001e1b, 0x001e1b, 0x001e1a, 0x001e1a},
+ {0x001e1c, 0x001e1d, 0x001e1c, 0x001e1c},
+ {0x001e1d, 0x001e1d, 0x001e1c, 0x001e1c},
+ {0x001e1e, 0x001e1f, 0x001e1e, 0x001e1e},
+ {0x001e1f, 0x001e1f, 0x001e1e, 0x001e1e},
+ {0x001e20, 0x001e21, 0x001e20, 0x001e20},
+ {0x001e21, 0x001e21, 0x001e20, 0x001e20},
+ {0x001e22, 0x001e23, 0x001e22, 0x001e22},
+ {0x001e23, 0x001e23, 0x001e22, 0x001e22},
+ {0x001e24, 0x001e25, 0x001e24, 0x001e24},
+ {0x001e25, 0x001e25, 0x001e24, 0x001e24},
+ {0x001e26, 0x001e27, 0x001e26, 0x001e26},
+ {0x001e27, 0x001e27, 0x001e26, 0x001e26},
+ {0x001e28, 0x001e29, 0x001e28, 0x001e28},
+ {0x001e29, 0x001e29, 0x001e28, 0x001e28},
+ {0x001e2a, 0x001e2b, 0x001e2a, 0x001e2a},
+ {0x001e2b, 0x001e2b, 0x001e2a, 0x001e2a},
+ {0x001e2c, 0x001e2d, 0x001e2c, 0x001e2c},
+ {0x001e2d, 0x001e2d, 0x001e2c, 0x001e2c},
+ {0x001e2e, 0x001e2f, 0x001e2e, 0x001e2e},
+ {0x001e2f, 0x001e2f, 0x001e2e, 0x001e2e},
+ {0x001e30, 0x001e31, 0x001e30, 0x001e30},
+ {0x001e31, 0x001e31, 0x001e30, 0x001e30},
+ {0x001e32, 0x001e33, 0x001e32, 0x001e32},
+ {0x001e33, 0x001e33, 0x001e32, 0x001e32},
+ {0x001e34, 0x001e35, 0x001e34, 0x001e34},
+ {0x001e35, 0x001e35, 0x001e34, 0x001e34},
+ {0x001e36, 0x001e37, 0x001e36, 0x001e36},
+ {0x001e37, 0x001e37, 0x001e36, 0x001e36},
+ {0x001e38, 0x001e39, 0x001e38, 0x001e38},
+ {0x001e39, 0x001e39, 0x001e38, 0x001e38},
+ {0x001e3a, 0x001e3b, 0x001e3a, 0x001e3a},
+ {0x001e3b, 0x001e3b, 0x001e3a, 0x001e3a},
+ {0x001e3c, 0x001e3d, 0x001e3c, 0x001e3c},
+ {0x001e3d, 0x001e3d, 0x001e3c, 0x001e3c},
+ {0x001e3e, 0x001e3f, 0x001e3e, 0x001e3e},
+ {0x001e3f, 0x001e3f, 0x001e3e, 0x001e3e},
+ {0x001e40, 0x001e41, 0x001e40, 0x001e40},
+ {0x001e41, 0x001e41, 0x001e40, 0x001e40},
+ {0x001e42, 0x001e43, 0x001e42, 0x001e42},
+ {0x001e43, 0x001e43, 0x001e42, 0x001e42},
+ {0x001e44, 0x001e45, 0x001e44, 0x001e44},
+ {0x001e45, 0x001e45, 0x001e44, 0x001e44},
+ {0x001e46, 0x001e47, 0x001e46, 0x001e46},
+ {0x001e47, 0x001e47, 0x001e46, 0x001e46},
+ {0x001e48, 0x001e49, 0x001e48, 0x001e48},
+ {0x001e49, 0x001e49, 0x001e48, 0x001e48},
+ {0x001e4a, 0x001e4b, 0x001e4a, 0x001e4a},
+ {0x001e4b, 0x001e4b, 0x001e4a, 0x001e4a},
+ {0x001e4c, 0x001e4d, 0x001e4c, 0x001e4c},
+ {0x001e4d, 0x001e4d, 0x001e4c, 0x001e4c},
+ {0x001e4e, 0x001e4f, 0x001e4e, 0x001e4e},
+ {0x001e4f, 0x001e4f, 0x001e4e, 0x001e4e},
+ {0x001e50, 0x001e51, 0x001e50, 0x001e50},
+ {0x001e51, 0x001e51, 0x001e50, 0x001e50},
+ {0x001e52, 0x001e53, 0x001e52, 0x001e52},
+ {0x001e53, 0x001e53, 0x001e52, 0x001e52},
+ {0x001e54, 0x001e55, 0x001e54, 0x001e54},
+ {0x001e55, 0x001e55, 0x001e54, 0x001e54},
+ {0x001e56, 0x001e57, 0x001e56, 0x001e56},
+ {0x001e57, 0x001e57, 0x001e56, 0x001e56},
+ {0x001e58, 0x001e59, 0x001e58, 0x001e58},
+ {0x001e59, 0x001e59, 0x001e58, 0x001e58},
+ {0x001e5a, 0x001e5b, 0x001e5a, 0x001e5a},
+ {0x001e5b, 0x001e5b, 0x001e5a, 0x001e5a},
+ {0x001e5c, 0x001e5d, 0x001e5c, 0x001e5c},
+ {0x001e5d, 0x001e5d, 0x001e5c, 0x001e5c},
+ {0x001e5e, 0x001e5f, 0x001e5e, 0x001e5e},
+ {0x001e5f, 0x001e5f, 0x001e5e, 0x001e5e},
+ {0x001e60, 0x001e61, 0x001e60, 0x001e60},
+ {0x001e61, 0x001e61, 0x001e60, 0x001e60},
+ {0x001e62, 0x001e63, 0x001e62, 0x001e62},
+ {0x001e63, 0x001e63, 0x001e62, 0x001e62},
+ {0x001e64, 0x001e65, 0x001e64, 0x001e64},
+ {0x001e65, 0x001e65, 0x001e64, 0x001e64},
+ {0x001e66, 0x001e67, 0x001e66, 0x001e66},
+ {0x001e67, 0x001e67, 0x001e66, 0x001e66},
+ {0x001e68, 0x001e69, 0x001e68, 0x001e68},
+ {0x001e69, 0x001e69, 0x001e68, 0x001e68},
+ {0x001e6a, 0x001e6b, 0x001e6a, 0x001e6a},
+ {0x001e6b, 0x001e6b, 0x001e6a, 0x001e6a},
+ {0x001e6c, 0x001e6d, 0x001e6c, 0x001e6c},
+ {0x001e6d, 0x001e6d, 0x001e6c, 0x001e6c},
+ {0x001e6e, 0x001e6f, 0x001e6e, 0x001e6e},
+ {0x001e6f, 0x001e6f, 0x001e6e, 0x001e6e},
+ {0x001e70, 0x001e71, 0x001e70, 0x001e70},
+ {0x001e71, 0x001e71, 0x001e70, 0x001e70},
+ {0x001e72, 0x001e73, 0x001e72, 0x001e72},
+ {0x001e73, 0x001e73, 0x001e72, 0x001e72},
+ {0x001e74, 0x001e75, 0x001e74, 0x001e74},
+ {0x001e75, 0x001e75, 0x001e74, 0x001e74},
+ {0x001e76, 0x001e77, 0x001e76, 0x001e76},
+ {0x001e77, 0x001e77, 0x001e76, 0x001e76},
+ {0x001e78, 0x001e79, 0x001e78, 0x001e78},
+ {0x001e79, 0x001e79, 0x001e78, 0x001e78},
+ {0x001e7a, 0x001e7b, 0x001e7a, 0x001e7a},
+ {0x001e7b, 0x001e7b, 0x001e7a, 0x001e7a},
+ {0x001e7c, 0x001e7d, 0x001e7c, 0x001e7c},
+ {0x001e7d, 0x001e7d, 0x001e7c, 0x001e7c},
+ {0x001e7e, 0x001e7f, 0x001e7e, 0x001e7e},
+ {0x001e7f, 0x001e7f, 0x001e7e, 0x001e7e},
+ {0x001e80, 0x001e81, 0x001e80, 0x001e80},
+ {0x001e81, 0x001e81, 0x001e80, 0x001e80},
+ {0x001e82, 0x001e83, 0x001e82, 0x001e82},
+ {0x001e83, 0x001e83, 0x001e82, 0x001e82},
+ {0x001e84, 0x001e85, 0x001e84, 0x001e84},
+ {0x001e85, 0x001e85, 0x001e84, 0x001e84},
+ {0x001e86, 0x001e87, 0x001e86, 0x001e86},
+ {0x001e87, 0x001e87, 0x001e86, 0x001e86},
+ {0x001e88, 0x001e89, 0x001e88, 0x001e88},
+ {0x001e89, 0x001e89, 0x001e88, 0x001e88},
+ {0x001e8a, 0x001e8b, 0x001e8a, 0x001e8a},
+ {0x001e8b, 0x001e8b, 0x001e8a, 0x001e8a},
+ {0x001e8c, 0x001e8d, 0x001e8c, 0x001e8c},
+ {0x001e8d, 0x001e8d, 0x001e8c, 0x001e8c},
+ {0x001e8e, 0x001e8f, 0x001e8e, 0x001e8e},
+ {0x001e8f, 0x001e8f, 0x001e8e, 0x001e8e},
+ {0x001e90, 0x001e91, 0x001e90, 0x001e90},
+ {0x001e91, 0x001e91, 0x001e90, 0x001e90},
+ {0x001e92, 0x001e93, 0x001e92, 0x001e92},
+ {0x001e93, 0x001e93, 0x001e92, 0x001e92},
+ {0x001e94, 0x001e95, 0x001e94, 0x001e94},
+ {0x001e95, 0x001e95, 0x001e94, 0x001e94},
+ {0x001e9b, 0x001e9b, 0x001e60, 0x001e60},
+ {0x001e9e, 0x0000df, 0x001e9e, 0x001e9e},
+ {0x001ea0, 0x001ea1, 0x001ea0, 0x001ea0},
+ {0x001ea1, 0x001ea1, 0x001ea0, 0x001ea0},
+ {0x001ea2, 0x001ea3, 0x001ea2, 0x001ea2},
+ {0x001ea3, 0x001ea3, 0x001ea2, 0x001ea2},
+ {0x001ea4, 0x001ea5, 0x001ea4, 0x001ea4},
+ {0x001ea5, 0x001ea5, 0x001ea4, 0x001ea4},
+ {0x001ea6, 0x001ea7, 0x001ea6, 0x001ea6},
+ {0x001ea7, 0x001ea7, 0x001ea6, 0x001ea6},
+ {0x001ea8, 0x001ea9, 0x001ea8, 0x001ea8},
+ {0x001ea9, 0x001ea9, 0x001ea8, 0x001ea8},
+ {0x001eaa, 0x001eab, 0x001eaa, 0x001eaa},
+ {0x001eab, 0x001eab, 0x001eaa, 0x001eaa},
+ {0x001eac, 0x001ead, 0x001eac, 0x001eac},
+ {0x001ead, 0x001ead, 0x001eac, 0x001eac},
+ {0x001eae, 0x001eaf, 0x001eae, 0x001eae},
+ {0x001eaf, 0x001eaf, 0x001eae, 0x001eae},
+ {0x001eb0, 0x001eb1, 0x001eb0, 0x001eb0},
+ {0x001eb1, 0x001eb1, 0x001eb0, 0x001eb0},
+ {0x001eb2, 0x001eb3, 0x001eb2, 0x001eb2},
+ {0x001eb3, 0x001eb3, 0x001eb2, 0x001eb2},
+ {0x001eb4, 0x001eb5, 0x001eb4, 0x001eb4},
+ {0x001eb5, 0x001eb5, 0x001eb4, 0x001eb4},
+ {0x001eb6, 0x001eb7, 0x001eb6, 0x001eb6},
+ {0x001eb7, 0x001eb7, 0x001eb6, 0x001eb6},
+ {0x001eb8, 0x001eb9, 0x001eb8, 0x001eb8},
+ {0x001eb9, 0x001eb9, 0x001eb8, 0x001eb8},
+ {0x001eba, 0x001ebb, 0x001eba, 0x001eba},
+ {0x001ebb, 0x001ebb, 0x001eba, 0x001eba},
+ {0x001ebc, 0x001ebd, 0x001ebc, 0x001ebc},
+ {0x001ebd, 0x001ebd, 0x001ebc, 0x001ebc},
+ {0x001ebe, 0x001ebf, 0x001ebe, 0x001ebe},
+ {0x001ebf, 0x001ebf, 0x001ebe, 0x001ebe},
+ {0x001ec0, 0x001ec1, 0x001ec0, 0x001ec0},
+ {0x001ec1, 0x001ec1, 0x001ec0, 0x001ec0},
+ {0x001ec2, 0x001ec3, 0x001ec2, 0x001ec2},
+ {0x001ec3, 0x001ec3, 0x001ec2, 0x001ec2},
+ {0x001ec4, 0x001ec5, 0x001ec4, 0x001ec4},
+ {0x001ec5, 0x001ec5, 0x001ec4, 0x001ec4},
+ {0x001ec6, 0x001ec7, 0x001ec6, 0x001ec6},
+ {0x001ec7, 0x001ec7, 0x001ec6, 0x001ec6},
+ {0x001ec8, 0x001ec9, 0x001ec8, 0x001ec8},
+ {0x001ec9, 0x001ec9, 0x001ec8, 0x001ec8},
+ {0x001eca, 0x001ecb, 0x001eca, 0x001eca},
+ {0x001ecb, 0x001ecb, 0x001eca, 0x001eca},
+ {0x001ecc, 0x001ecd, 0x001ecc, 0x001ecc},
+ {0x001ecd, 0x001ecd, 0x001ecc, 0x001ecc},
+ {0x001ece, 0x001ecf, 0x001ece, 0x001ece},
+ {0x001ecf, 0x001ecf, 0x001ece, 0x001ece},
+ {0x001ed0, 0x001ed1, 0x001ed0, 0x001ed0},
+ {0x001ed1, 0x001ed1, 0x001ed0, 0x001ed0},
+ {0x001ed2, 0x001ed3, 0x001ed2, 0x001ed2},
+ {0x001ed3, 0x001ed3, 0x001ed2, 0x001ed2},
+ {0x001ed4, 0x001ed5, 0x001ed4, 0x001ed4},
+ {0x001ed5, 0x001ed5, 0x001ed4, 0x001ed4},
+ {0x001ed6, 0x001ed7, 0x001ed6, 0x001ed6},
+ {0x001ed7, 0x001ed7, 0x001ed6, 0x001ed6},
+ {0x001ed8, 0x001ed9, 0x001ed8, 0x001ed8},
+ {0x001ed9, 0x001ed9, 0x001ed8, 0x001ed8},
+ {0x001eda, 0x001edb, 0x001eda, 0x001eda},
+ {0x001edb, 0x001edb, 0x001eda, 0x001eda},
+ {0x001edc, 0x001edd, 0x001edc, 0x001edc},
+ {0x001edd, 0x001edd, 0x001edc, 0x001edc},
+ {0x001ede, 0x001edf, 0x001ede, 0x001ede},
+ {0x001edf, 0x001edf, 0x001ede, 0x001ede},
+ {0x001ee0, 0x001ee1, 0x001ee0, 0x001ee0},
+ {0x001ee1, 0x001ee1, 0x001ee0, 0x001ee0},
+ {0x001ee2, 0x001ee3, 0x001ee2, 0x001ee2},
+ {0x001ee3, 0x001ee3, 0x001ee2, 0x001ee2},
+ {0x001ee4, 0x001ee5, 0x001ee4, 0x001ee4},
+ {0x001ee5, 0x001ee5, 0x001ee4, 0x001ee4},
+ {0x001ee6, 0x001ee7, 0x001ee6, 0x001ee6},
+ {0x001ee7, 0x001ee7, 0x001ee6, 0x001ee6},
+ {0x001ee8, 0x001ee9, 0x001ee8, 0x001ee8},
+ {0x001ee9, 0x001ee9, 0x001ee8, 0x001ee8},
+ {0x001eea, 0x001eeb, 0x001eea, 0x001eea},
+ {0x001eeb, 0x001eeb, 0x001eea, 0x001eea},
+ {0x001eec, 0x001eed, 0x001eec, 0x001eec},
+ {0x001eed, 0x001eed, 0x001eec, 0x001eec},
+ {0x001eee, 0x001eef, 0x001eee, 0x001eee},
+ {0x001eef, 0x001eef, 0x001eee, 0x001eee},
+ {0x001ef0, 0x001ef1, 0x001ef0, 0x001ef0},
+ {0x001ef1, 0x001ef1, 0x001ef0, 0x001ef0},
+ {0x001ef2, 0x001ef3, 0x001ef2, 0x001ef2},
+ {0x001ef3, 0x001ef3, 0x001ef2, 0x001ef2},
+ {0x001ef4, 0x001ef5, 0x001ef4, 0x001ef4},
+ {0x001ef5, 0x001ef5, 0x001ef4, 0x001ef4},
+ {0x001ef6, 0x001ef7, 0x001ef6, 0x001ef6},
+ {0x001ef7, 0x001ef7, 0x001ef6, 0x001ef6},
+ {0x001ef8, 0x001ef9, 0x001ef8, 0x001ef8},
+ {0x001ef9, 0x001ef9, 0x001ef8, 0x001ef8},
+ {0x001efa, 0x001efb, 0x001efa, 0x001efa},
+ {0x001efb, 0x001efb, 0x001efa, 0x001efa},
+ {0x001efc, 0x001efd, 0x001efc, 0x001efc},
+ {0x001efd, 0x001efd, 0x001efc, 0x001efc},
+ {0x001efe, 0x001eff, 0x001efe, 0x001efe},
+ {0x001eff, 0x001eff, 0x001efe, 0x001efe},
+ {0x001f00, 0x001f00, 0x001f08, 0x001f08},
+ {0x001f01, 0x001f01, 0x001f09, 0x001f09},
+ {0x001f02, 0x001f02, 0x001f0a, 0x001f0a},
+ {0x001f03, 0x001f03, 0x001f0b, 0x001f0b},
+ {0x001f04, 0x001f04, 0x001f0c, 0x001f0c},
+ {0x001f05, 0x001f05, 0x001f0d, 0x001f0d},
+ {0x001f06, 0x001f06, 0x001f0e, 0x001f0e},
+ {0x001f07, 0x001f07, 0x001f0f, 0x001f0f},
+ {0x001f08, 0x001f00, 0x001f08, 0x001f08},
+ {0x001f09, 0x001f01, 0x001f09, 0x001f09},
+ {0x001f0a, 0x001f02, 0x001f0a, 0x001f0a},
+ {0x001f0b, 0x001f03, 0x001f0b, 0x001f0b},
+ {0x001f0c, 0x001f04, 0x001f0c, 0x001f0c},
+ {0x001f0d, 0x001f05, 0x001f0d, 0x001f0d},
+ {0x001f0e, 0x001f06, 0x001f0e, 0x001f0e},
+ {0x001f0f, 0x001f07, 0x001f0f, 0x001f0f},
+ {0x001f10, 0x001f10, 0x001f18, 0x001f18},
+ {0x001f11, 0x001f11, 0x001f19, 0x001f19},
+ {0x001f12, 0x001f12, 0x001f1a, 0x001f1a},
+ {0x001f13, 0x001f13, 0x001f1b, 0x001f1b},
+ {0x001f14, 0x001f14, 0x001f1c, 0x001f1c},
+ {0x001f15, 0x001f15, 0x001f1d, 0x001f1d},
+ {0x001f18, 0x001f10, 0x001f18, 0x001f18},
+ {0x001f19, 0x001f11, 0x001f19, 0x001f19},
+ {0x001f1a, 0x001f12, 0x001f1a, 0x001f1a},
+ {0x001f1b, 0x001f13, 0x001f1b, 0x001f1b},
+ {0x001f1c, 0x001f14, 0x001f1c, 0x001f1c},
+ {0x001f1d, 0x001f15, 0x001f1d, 0x001f1d},
+ {0x001f20, 0x001f20, 0x001f28, 0x001f28},
+ {0x001f21, 0x001f21, 0x001f29, 0x001f29},
+ {0x001f22, 0x001f22, 0x001f2a, 0x001f2a},
+ {0x001f23, 0x001f23, 0x001f2b, 0x001f2b},
+ {0x001f24, 0x001f24, 0x001f2c, 0x001f2c},
+ {0x001f25, 0x001f25, 0x001f2d, 0x001f2d},
+ {0x001f26, 0x001f26, 0x001f2e, 0x001f2e},
+ {0x001f27, 0x001f27, 0x001f2f, 0x001f2f},
+ {0x001f28, 0x001f20, 0x001f28, 0x001f28},
+ {0x001f29, 0x001f21, 0x001f29, 0x001f29},
+ {0x001f2a, 0x001f22, 0x001f2a, 0x001f2a},
+ {0x001f2b, 0x001f23, 0x001f2b, 0x001f2b},
+ {0x001f2c, 0x001f24, 0x001f2c, 0x001f2c},
+ {0x001f2d, 0x001f25, 0x001f2d, 0x001f2d},
+ {0x001f2e, 0x001f26, 0x001f2e, 0x001f2e},
+ {0x001f2f, 0x001f27, 0x001f2f, 0x001f2f},
+ {0x001f30, 0x001f30, 0x001f38, 0x001f38},
+ {0x001f31, 0x001f31, 0x001f39, 0x001f39},
+ {0x001f32, 0x001f32, 0x001f3a, 0x001f3a},
+ {0x001f33, 0x001f33, 0x001f3b, 0x001f3b},
+ {0x001f34, 0x001f34, 0x001f3c, 0x001f3c},
+ {0x001f35, 0x001f35, 0x001f3d, 0x001f3d},
+ {0x001f36, 0x001f36, 0x001f3e, 0x001f3e},
+ {0x001f37, 0x001f37, 0x001f3f, 0x001f3f},
+ {0x001f38, 0x001f30, 0x001f38, 0x001f38},
+ {0x001f39, 0x001f31, 0x001f39, 0x001f39},
+ {0x001f3a, 0x001f32, 0x001f3a, 0x001f3a},
+ {0x001f3b, 0x001f33, 0x001f3b, 0x001f3b},
+ {0x001f3c, 0x001f34, 0x001f3c, 0x001f3c},
+ {0x001f3d, 0x001f35, 0x001f3d, 0x001f3d},
+ {0x001f3e, 0x001f36, 0x001f3e, 0x001f3e},
+ {0x001f3f, 0x001f37, 0x001f3f, 0x001f3f},
+ {0x001f40, 0x001f40, 0x001f48, 0x001f48},
+ {0x001f41, 0x001f41, 0x001f49, 0x001f49},
+ {0x001f42, 0x001f42, 0x001f4a, 0x001f4a},
+ {0x001f43, 0x001f43, 0x001f4b, 0x001f4b},
+ {0x001f44, 0x001f44, 0x001f4c, 0x001f4c},
+ {0x001f45, 0x001f45, 0x001f4d, 0x001f4d},
+ {0x001f48, 0x001f40, 0x001f48, 0x001f48},
+ {0x001f49, 0x001f41, 0x001f49, 0x001f49},
+ {0x001f4a, 0x001f42, 0x001f4a, 0x001f4a},
+ {0x001f4b, 0x001f43, 0x001f4b, 0x001f4b},
+ {0x001f4c, 0x001f44, 0x001f4c, 0x001f4c},
+ {0x001f4d, 0x001f45, 0x001f4d, 0x001f4d},
+ {0x001f51, 0x001f51, 0x001f59, 0x001f59},
+ {0x001f53, 0x001f53, 0x001f5b, 0x001f5b},
+ {0x001f55, 0x001f55, 0x001f5d, 0x001f5d},
+ {0x001f57, 0x001f57, 0x001f5f, 0x001f5f},
+ {0x001f59, 0x001f51, 0x001f59, 0x001f59},
+ {0x001f5b, 0x001f53, 0x001f5b, 0x001f5b},
+ {0x001f5d, 0x001f55, 0x001f5d, 0x001f5d},
+ {0x001f5f, 0x001f57, 0x001f5f, 0x001f5f},
+ {0x001f60, 0x001f60, 0x001f68, 0x001f68},
+ {0x001f61, 0x001f61, 0x001f69, 0x001f69},
+ {0x001f62, 0x001f62, 0x001f6a, 0x001f6a},
+ {0x001f63, 0x001f63, 0x001f6b, 0x001f6b},
+ {0x001f64, 0x001f64, 0x001f6c, 0x001f6c},
+ {0x001f65, 0x001f65, 0x001f6d, 0x001f6d},
+ {0x001f66, 0x001f66, 0x001f6e, 0x001f6e},
+ {0x001f67, 0x001f67, 0x001f6f, 0x001f6f},
+ {0x001f68, 0x001f60, 0x001f68, 0x001f68},
+ {0x001f69, 0x001f61, 0x001f69, 0x001f69},
+ {0x001f6a, 0x001f62, 0x001f6a, 0x001f6a},
+ {0x001f6b, 0x001f63, 0x001f6b, 0x001f6b},
+ {0x001f6c, 0x001f64, 0x001f6c, 0x001f6c},
+ {0x001f6d, 0x001f65, 0x001f6d, 0x001f6d},
+ {0x001f6e, 0x001f66, 0x001f6e, 0x001f6e},
+ {0x001f6f, 0x001f67, 0x001f6f, 0x001f6f},
+ {0x001f70, 0x001f70, 0x001fba, 0x001fba},
+ {0x001f71, 0x001f71, 0x001fbb, 0x001fbb},
+ {0x001f72, 0x001f72, 0x001fc8, 0x001fc8},
+ {0x001f73, 0x001f73, 0x001fc9, 0x001fc9},
+ {0x001f74, 0x001f74, 0x001fca, 0x001fca},
+ {0x001f75, 0x001f75, 0x001fcb, 0x001fcb},
+ {0x001f76, 0x001f76, 0x001fda, 0x001fda},
+ {0x001f77, 0x001f77, 0x001fdb, 0x001fdb},
+ {0x001f78, 0x001f78, 0x001ff8, 0x001ff8},
+ {0x001f79, 0x001f79, 0x001ff9, 0x001ff9},
+ {0x001f7a, 0x001f7a, 0x001fea, 0x001fea},
+ {0x001f7b, 0x001f7b, 0x001feb, 0x001feb},
+ {0x001f7c, 0x001f7c, 0x001ffa, 0x001ffa},
+ {0x001f7d, 0x001f7d, 0x001ffb, 0x001ffb},
+ {0x001f80, 0x001f80, 0x001f88, 0x001f88},
+ {0x001f81, 0x001f81, 0x001f89, 0x001f89},
+ {0x001f82, 0x001f82, 0x001f8a, 0x001f8a},
+ {0x001f83, 0x001f83, 0x001f8b, 0x001f8b},
+ {0x001f84, 0x001f84, 0x001f8c, 0x001f8c},
+ {0x001f85, 0x001f85, 0x001f8d, 0x001f8d},
+ {0x001f86, 0x001f86, 0x001f8e, 0x001f8e},
+ {0x001f87, 0x001f87, 0x001f8f, 0x001f8f},
+ {0x001f88, 0x001f80, 0x001f88, 0x001f88},
+ {0x001f89, 0x001f81, 0x001f89, 0x001f89},
+ {0x001f8a, 0x001f82, 0x001f8a, 0x001f8a},
+ {0x001f8b, 0x001f83, 0x001f8b, 0x001f8b},
+ {0x001f8c, 0x001f84, 0x001f8c, 0x001f8c},
+ {0x001f8d, 0x001f85, 0x001f8d, 0x001f8d},
+ {0x001f8e, 0x001f86, 0x001f8e, 0x001f8e},
+ {0x001f8f, 0x001f87, 0x001f8f, 0x001f8f},
+ {0x001f90, 0x001f90, 0x001f98, 0x001f98},
+ {0x001f91, 0x001f91, 0x001f99, 0x001f99},
+ {0x001f92, 0x001f92, 0x001f9a, 0x001f9a},
+ {0x001f93, 0x001f93, 0x001f9b, 0x001f9b},
+ {0x001f94, 0x001f94, 0x001f9c, 0x001f9c},
+ {0x001f95, 0x001f95, 0x001f9d, 0x001f9d},
+ {0x001f96, 0x001f96, 0x001f9e, 0x001f9e},
+ {0x001f97, 0x001f97, 0x001f9f, 0x001f9f},
+ {0x001f98, 0x001f90, 0x001f98, 0x001f98},
+ {0x001f99, 0x001f91, 0x001f99, 0x001f99},
+ {0x001f9a, 0x001f92, 0x001f9a, 0x001f9a},
+ {0x001f9b, 0x001f93, 0x001f9b, 0x001f9b},
+ {0x001f9c, 0x001f94, 0x001f9c, 0x001f9c},
+ {0x001f9d, 0x001f95, 0x001f9d, 0x001f9d},
+ {0x001f9e, 0x001f96, 0x001f9e, 0x001f9e},
+ {0x001f9f, 0x001f97, 0x001f9f, 0x001f9f},
+ {0x001fa0, 0x001fa0, 0x001fa8, 0x001fa8},
+ {0x001fa1, 0x001fa1, 0x001fa9, 0x001fa9},
+ {0x001fa2, 0x001fa2, 0x001faa, 0x001faa},
+ {0x001fa3, 0x001fa3, 0x001fab, 0x001fab},
+ {0x001fa4, 0x001fa4, 0x001fac, 0x001fac},
+ {0x001fa5, 0x001fa5, 0x001fad, 0x001fad},
+ {0x001fa6, 0x001fa6, 0x001fae, 0x001fae},
+ {0x001fa7, 0x001fa7, 0x001faf, 0x001faf},
+ {0x001fa8, 0x001fa0, 0x001fa8, 0x001fa8},
+ {0x001fa9, 0x001fa1, 0x001fa9, 0x001fa9},
+ {0x001faa, 0x001fa2, 0x001faa, 0x001faa},
+ {0x001fab, 0x001fa3, 0x001fab, 0x001fab},
+ {0x001fac, 0x001fa4, 0x001fac, 0x001fac},
+ {0x001fad, 0x001fa5, 0x001fad, 0x001fad},
+ {0x001fae, 0x001fa6, 0x001fae, 0x001fae},
+ {0x001faf, 0x001fa7, 0x001faf, 0x001faf},
+ {0x001fb0, 0x001fb0, 0x001fb8, 0x001fb8},
+ {0x001fb1, 0x001fb1, 0x001fb9, 0x001fb9},
+ {0x001fb3, 0x001fb3, 0x001fbc, 0x001fbc},
+ {0x001fb8, 0x001fb0, 0x001fb8, 0x001fb8},
+ {0x001fb9, 0x001fb1, 0x001fb9, 0x001fb9},
+ {0x001fba, 0x001f70, 0x001fba, 0x001fba},
+ {0x001fbb, 0x001f71, 0x001fbb, 0x001fbb},
+ {0x001fbc, 0x001fb3, 0x001fbc, 0x001fbc},
+ {0x001fbe, 0x001fbe, 0x000399, 0x000399},
+ {0x001fc3, 0x001fc3, 0x001fcc, 0x001fcc},
+ {0x001fc8, 0x001f72, 0x001fc8, 0x001fc8},
+ {0x001fc9, 0x001f73, 0x001fc9, 0x001fc9},
+ {0x001fca, 0x001f74, 0x001fca, 0x001fca},
+ {0x001fcb, 0x001f75, 0x001fcb, 0x001fcb},
+ {0x001fcc, 0x001fc3, 0x001fcc, 0x001fcc},
+ {0x001fd0, 0x001fd0, 0x001fd8, 0x001fd8},
+ {0x001fd1, 0x001fd1, 0x001fd9, 0x001fd9},
+ {0x001fd8, 0x001fd0, 0x001fd8, 0x001fd8},
+ {0x001fd9, 0x001fd1, 0x001fd9, 0x001fd9},
+ {0x001fda, 0x001f76, 0x001fda, 0x001fda},
+ {0x001fdb, 0x001f77, 0x001fdb, 0x001fdb},
+ {0x001fe0, 0x001fe0, 0x001fe8, 0x001fe8},
+ {0x001fe1, 0x001fe1, 0x001fe9, 0x001fe9},
+ {0x001fe5, 0x001fe5, 0x001fec, 0x001fec},
+ {0x001fe8, 0x001fe0, 0x001fe8, 0x001fe8},
+ {0x001fe9, 0x001fe1, 0x001fe9, 0x001fe9},
+ {0x001fea, 0x001f7a, 0x001fea, 0x001fea},
+ {0x001feb, 0x001f7b, 0x001feb, 0x001feb},
+ {0x001fec, 0x001fe5, 0x001fec, 0x001fec},
+ {0x001ff3, 0x001ff3, 0x001ffc, 0x001ffc},
+ {0x001ff8, 0x001f78, 0x001ff8, 0x001ff8},
+ {0x001ff9, 0x001f79, 0x001ff9, 0x001ff9},
+ {0x001ffa, 0x001f7c, 0x001ffa, 0x001ffa},
+ {0x001ffb, 0x001f7d, 0x001ffb, 0x001ffb},
+ {0x001ffc, 0x001ff3, 0x001ffc, 0x001ffc},
+ {0x002126, 0x0003c9, 0x002126, 0x002126},
+ {0x00212a, 0x00006b, 0x00212a, 0x00212a},
+ {0x00212b, 0x0000e5, 0x00212b, 0x00212b},
+ {0x002132, 0x00214e, 0x002132, 0x002132},
+ {0x00214e, 0x00214e, 0x002132, 0x002132},
+ {0x002160, 0x002170, 0x002160, 0x002160},
+ {0x002161, 0x002171, 0x002161, 0x002161},
+ {0x002162, 0x002172, 0x002162, 0x002162},
+ {0x002163, 0x002173, 0x002163, 0x002163},
+ {0x002164, 0x002174, 0x002164, 0x002164},
+ {0x002165, 0x002175, 0x002165, 0x002165},
+ {0x002166, 0x002176, 0x002166, 0x002166},
+ {0x002167, 0x002177, 0x002167, 0x002167},
+ {0x002168, 0x002178, 0x002168, 0x002168},
+ {0x002169, 0x002179, 0x002169, 0x002169},
+ {0x00216a, 0x00217a, 0x00216a, 0x00216a},
+ {0x00216b, 0x00217b, 0x00216b, 0x00216b},
+ {0x00216c, 0x00217c, 0x00216c, 0x00216c},
+ {0x00216d, 0x00217d, 0x00216d, 0x00216d},
+ {0x00216e, 0x00217e, 0x00216e, 0x00216e},
+ {0x00216f, 0x00217f, 0x00216f, 0x00216f},
+ {0x002170, 0x002170, 0x002160, 0x002160},
+ {0x002171, 0x002171, 0x002161, 0x002161},
+ {0x002172, 0x002172, 0x002162, 0x002162},
+ {0x002173, 0x002173, 0x002163, 0x002163},
+ {0x002174, 0x002174, 0x002164, 0x002164},
+ {0x002175, 0x002175, 0x002165, 0x002165},
+ {0x002176, 0x002176, 0x002166, 0x002166},
+ {0x002177, 0x002177, 0x002167, 0x002167},
+ {0x002178, 0x002178, 0x002168, 0x002168},
+ {0x002179, 0x002179, 0x002169, 0x002169},
+ {0x00217a, 0x00217a, 0x00216a, 0x00216a},
+ {0x00217b, 0x00217b, 0x00216b, 0x00216b},
+ {0x00217c, 0x00217c, 0x00216c, 0x00216c},
+ {0x00217d, 0x00217d, 0x00216d, 0x00216d},
+ {0x00217e, 0x00217e, 0x00216e, 0x00216e},
+ {0x00217f, 0x00217f, 0x00216f, 0x00216f},
+ {0x002183, 0x002184, 0x002183, 0x002183},
+ {0x002184, 0x002184, 0x002183, 0x002183},
+ {0x0024b6, 0x0024d0, 0x0024b6, 0x0024b6},
+ {0x0024b7, 0x0024d1, 0x0024b7, 0x0024b7},
+ {0x0024b8, 0x0024d2, 0x0024b8, 0x0024b8},
+ {0x0024b9, 0x0024d3, 0x0024b9, 0x0024b9},
+ {0x0024ba, 0x0024d4, 0x0024ba, 0x0024ba},
+ {0x0024bb, 0x0024d5, 0x0024bb, 0x0024bb},
+ {0x0024bc, 0x0024d6, 0x0024bc, 0x0024bc},
+ {0x0024bd, 0x0024d7, 0x0024bd, 0x0024bd},
+ {0x0024be, 0x0024d8, 0x0024be, 0x0024be},
+ {0x0024bf, 0x0024d9, 0x0024bf, 0x0024bf},
+ {0x0024c0, 0x0024da, 0x0024c0, 0x0024c0},
+ {0x0024c1, 0x0024db, 0x0024c1, 0x0024c1},
+ {0x0024c2, 0x0024dc, 0x0024c2, 0x0024c2},
+ {0x0024c3, 0x0024dd, 0x0024c3, 0x0024c3},
+ {0x0024c4, 0x0024de, 0x0024c4, 0x0024c4},
+ {0x0024c5, 0x0024df, 0x0024c5, 0x0024c5},
+ {0x0024c6, 0x0024e0, 0x0024c6, 0x0024c6},
+ {0x0024c7, 0x0024e1, 0x0024c7, 0x0024c7},
+ {0x0024c8, 0x0024e2, 0x0024c8, 0x0024c8},
+ {0x0024c9, 0x0024e3, 0x0024c9, 0x0024c9},
+ {0x0024ca, 0x0024e4, 0x0024ca, 0x0024ca},
+ {0x0024cb, 0x0024e5, 0x0024cb, 0x0024cb},
+ {0x0024cc, 0x0024e6, 0x0024cc, 0x0024cc},
+ {0x0024cd, 0x0024e7, 0x0024cd, 0x0024cd},
+ {0x0024ce, 0x0024e8, 0x0024ce, 0x0024ce},
+ {0x0024cf, 0x0024e9, 0x0024cf, 0x0024cf},
+ {0x0024d0, 0x0024d0, 0x0024b6, 0x0024b6},
+ {0x0024d1, 0x0024d1, 0x0024b7, 0x0024b7},
+ {0x0024d2, 0x0024d2, 0x0024b8, 0x0024b8},
+ {0x0024d3, 0x0024d3, 0x0024b9, 0x0024b9},
+ {0x0024d4, 0x0024d4, 0x0024ba, 0x0024ba},
+ {0x0024d5, 0x0024d5, 0x0024bb, 0x0024bb},
+ {0x0024d6, 0x0024d6, 0x0024bc, 0x0024bc},
+ {0x0024d7, 0x0024d7, 0x0024bd, 0x0024bd},
+ {0x0024d8, 0x0024d8, 0x0024be, 0x0024be},
+ {0x0024d9, 0x0024d9, 0x0024bf, 0x0024bf},
+ {0x0024da, 0x0024da, 0x0024c0, 0x0024c0},
+ {0x0024db, 0x0024db, 0x0024c1, 0x0024c1},
+ {0x0024dc, 0x0024dc, 0x0024c2, 0x0024c2},
+ {0x0024dd, 0x0024dd, 0x0024c3, 0x0024c3},
+ {0x0024de, 0x0024de, 0x0024c4, 0x0024c4},
+ {0x0024df, 0x0024df, 0x0024c5, 0x0024c5},
+ {0x0024e0, 0x0024e0, 0x0024c6, 0x0024c6},
+ {0x0024e1, 0x0024e1, 0x0024c7, 0x0024c7},
+ {0x0024e2, 0x0024e2, 0x0024c8, 0x0024c8},
+ {0x0024e3, 0x0024e3, 0x0024c9, 0x0024c9},
+ {0x0024e4, 0x0024e4, 0x0024ca, 0x0024ca},
+ {0x0024e5, 0x0024e5, 0x0024cb, 0x0024cb},
+ {0x0024e6, 0x0024e6, 0x0024cc, 0x0024cc},
+ {0x0024e7, 0x0024e7, 0x0024cd, 0x0024cd},
+ {0x0024e8, 0x0024e8, 0x0024ce, 0x0024ce},
+ {0x0024e9, 0x0024e9, 0x0024cf, 0x0024cf},
+ {0x002c00, 0x002c30, 0x002c00, 0x002c00},
+ {0x002c01, 0x002c31, 0x002c01, 0x002c01},
+ {0x002c02, 0x002c32, 0x002c02, 0x002c02},
+ {0x002c03, 0x002c33, 0x002c03, 0x002c03},
+ {0x002c04, 0x002c34, 0x002c04, 0x002c04},
+ {0x002c05, 0x002c35, 0x002c05, 0x002c05},
+ {0x002c06, 0x002c36, 0x002c06, 0x002c06},
+ {0x002c07, 0x002c37, 0x002c07, 0x002c07},
+ {0x002c08, 0x002c38, 0x002c08, 0x002c08},
+ {0x002c09, 0x002c39, 0x002c09, 0x002c09},
+ {0x002c0a, 0x002c3a, 0x002c0a, 0x002c0a},
+ {0x002c0b, 0x002c3b, 0x002c0b, 0x002c0b},
+ {0x002c0c, 0x002c3c, 0x002c0c, 0x002c0c},
+ {0x002c0d, 0x002c3d, 0x002c0d, 0x002c0d},
+ {0x002c0e, 0x002c3e, 0x002c0e, 0x002c0e},
+ {0x002c0f, 0x002c3f, 0x002c0f, 0x002c0f},
+ {0x002c10, 0x002c40, 0x002c10, 0x002c10},
+ {0x002c11, 0x002c41, 0x002c11, 0x002c11},
+ {0x002c12, 0x002c42, 0x002c12, 0x002c12},
+ {0x002c13, 0x002c43, 0x002c13, 0x002c13},
+ {0x002c14, 0x002c44, 0x002c14, 0x002c14},
+ {0x002c15, 0x002c45, 0x002c15, 0x002c15},
+ {0x002c16, 0x002c46, 0x002c16, 0x002c16},
+ {0x002c17, 0x002c47, 0x002c17, 0x002c17},
+ {0x002c18, 0x002c48, 0x002c18, 0x002c18},
+ {0x002c19, 0x002c49, 0x002c19, 0x002c19},
+ {0x002c1a, 0x002c4a, 0x002c1a, 0x002c1a},
+ {0x002c1b, 0x002c4b, 0x002c1b, 0x002c1b},
+ {0x002c1c, 0x002c4c, 0x002c1c, 0x002c1c},
+ {0x002c1d, 0x002c4d, 0x002c1d, 0x002c1d},
+ {0x002c1e, 0x002c4e, 0x002c1e, 0x002c1e},
+ {0x002c1f, 0x002c4f, 0x002c1f, 0x002c1f},
+ {0x002c20, 0x002c50, 0x002c20, 0x002c20},
+ {0x002c21, 0x002c51, 0x002c21, 0x002c21},
+ {0x002c22, 0x002c52, 0x002c22, 0x002c22},
+ {0x002c23, 0x002c53, 0x002c23, 0x002c23},
+ {0x002c24, 0x002c54, 0x002c24, 0x002c24},
+ {0x002c25, 0x002c55, 0x002c25, 0x002c25},
+ {0x002c26, 0x002c56, 0x002c26, 0x002c26},
+ {0x002c27, 0x002c57, 0x002c27, 0x002c27},
+ {0x002c28, 0x002c58, 0x002c28, 0x002c28},
+ {0x002c29, 0x002c59, 0x002c29, 0x002c29},
+ {0x002c2a, 0x002c5a, 0x002c2a, 0x002c2a},
+ {0x002c2b, 0x002c5b, 0x002c2b, 0x002c2b},
+ {0x002c2c, 0x002c5c, 0x002c2c, 0x002c2c},
+ {0x002c2d, 0x002c5d, 0x002c2d, 0x002c2d},
+ {0x002c2e, 0x002c5e, 0x002c2e, 0x002c2e},
+ {0x002c2f, 0x002c5f, 0x002c2f, 0x002c2f},
+ {0x002c30, 0x002c30, 0x002c00, 0x002c00},
+ {0x002c31, 0x002c31, 0x002c01, 0x002c01},
+ {0x002c32, 0x002c32, 0x002c02, 0x002c02},
+ {0x002c33, 0x002c33, 0x002c03, 0x002c03},
+ {0x002c34, 0x002c34, 0x002c04, 0x002c04},
+ {0x002c35, 0x002c35, 0x002c05, 0x002c05},
+ {0x002c36, 0x002c36, 0x002c06, 0x002c06},
+ {0x002c37, 0x002c37, 0x002c07, 0x002c07},
+ {0x002c38, 0x002c38, 0x002c08, 0x002c08},
+ {0x002c39, 0x002c39, 0x002c09, 0x002c09},
+ {0x002c3a, 0x002c3a, 0x002c0a, 0x002c0a},
+ {0x002c3b, 0x002c3b, 0x002c0b, 0x002c0b},
+ {0x002c3c, 0x002c3c, 0x002c0c, 0x002c0c},
+ {0x002c3d, 0x002c3d, 0x002c0d, 0x002c0d},
+ {0x002c3e, 0x002c3e, 0x002c0e, 0x002c0e},
+ {0x002c3f, 0x002c3f, 0x002c0f, 0x002c0f},
+ {0x002c40, 0x002c40, 0x002c10, 0x002c10},
+ {0x002c41, 0x002c41, 0x002c11, 0x002c11},
+ {0x002c42, 0x002c42, 0x002c12, 0x002c12},
+ {0x002c43, 0x002c43, 0x002c13, 0x002c13},
+ {0x002c44, 0x002c44, 0x002c14, 0x002c14},
+ {0x002c45, 0x002c45, 0x002c15, 0x002c15},
+ {0x002c46, 0x002c46, 0x002c16, 0x002c16},
+ {0x002c47, 0x002c47, 0x002c17, 0x002c17},
+ {0x002c48, 0x002c48, 0x002c18, 0x002c18},
+ {0x002c49, 0x002c49, 0x002c19, 0x002c19},
+ {0x002c4a, 0x002c4a, 0x002c1a, 0x002c1a},
+ {0x002c4b, 0x002c4b, 0x002c1b, 0x002c1b},
+ {0x002c4c, 0x002c4c, 0x002c1c, 0x002c1c},
+ {0x002c4d, 0x002c4d, 0x002c1d, 0x002c1d},
+ {0x002c4e, 0x002c4e, 0x002c1e, 0x002c1e},
+ {0x002c4f, 0x002c4f, 0x002c1f, 0x002c1f},
+ {0x002c50, 0x002c50, 0x002c20, 0x002c20},
+ {0x002c51, 0x002c51, 0x002c21, 0x002c21},
+ {0x002c52, 0x002c52, 0x002c22, 0x002c22},
+ {0x002c53, 0x002c53, 0x002c23, 0x002c23},
+ {0x002c54, 0x002c54, 0x002c24, 0x002c24},
+ {0x002c55, 0x002c55, 0x002c25, 0x002c25},
+ {0x002c56, 0x002c56, 0x002c26, 0x002c26},
+ {0x002c57, 0x002c57, 0x002c27, 0x002c27},
+ {0x002c58, 0x002c58, 0x002c28, 0x002c28},
+ {0x002c59, 0x002c59, 0x002c29, 0x002c29},
+ {0x002c5a, 0x002c5a, 0x002c2a, 0x002c2a},
+ {0x002c5b, 0x002c5b, 0x002c2b, 0x002c2b},
+ {0x002c5c, 0x002c5c, 0x002c2c, 0x002c2c},
+ {0x002c5d, 0x002c5d, 0x002c2d, 0x002c2d},
+ {0x002c5e, 0x002c5e, 0x002c2e, 0x002c2e},
+ {0x002c5f, 0x002c5f, 0x002c2f, 0x002c2f},
+ {0x002c60, 0x002c61, 0x002c60, 0x002c60},
+ {0x002c61, 0x002c61, 0x002c60, 0x002c60},
+ {0x002c62, 0x00026b, 0x002c62, 0x002c62},
+ {0x002c63, 0x001d7d, 0x002c63, 0x002c63},
+ {0x002c64, 0x00027d, 0x002c64, 0x002c64},
+ {0x002c65, 0x002c65, 0x00023a, 0x00023a},
+ {0x002c66, 0x002c66, 0x00023e, 0x00023e},
+ {0x002c67, 0x002c68, 0x002c67, 0x002c67},
+ {0x002c68, 0x002c68, 0x002c67, 0x002c67},
+ {0x002c69, 0x002c6a, 0x002c69, 0x002c69},
+ {0x002c6a, 0x002c6a, 0x002c69, 0x002c69},
+ {0x002c6b, 0x002c6c, 0x002c6b, 0x002c6b},
+ {0x002c6c, 0x002c6c, 0x002c6b, 0x002c6b},
+ {0x002c6d, 0x000251, 0x002c6d, 0x002c6d},
+ {0x002c6e, 0x000271, 0x002c6e, 0x002c6e},
+ {0x002c6f, 0x000250, 0x002c6f, 0x002c6f},
+ {0x002c70, 0x000252, 0x002c70, 0x002c70},
+ {0x002c72, 0x002c73, 0x002c72, 0x002c72},
+ {0x002c73, 0x002c73, 0x002c72, 0x002c72},
+ {0x002c75, 0x002c76, 0x002c75, 0x002c75},
+ {0x002c76, 0x002c76, 0x002c75, 0x002c75},
+ {0x002c7e, 0x00023f, 0x002c7e, 0x002c7e},
+ {0x002c7f, 0x000240, 0x002c7f, 0x002c7f},
+ {0x002c80, 0x002c81, 0x002c80, 0x002c80},
+ {0x002c81, 0x002c81, 0x002c80, 0x002c80},
+ {0x002c82, 0x002c83, 0x002c82, 0x002c82},
+ {0x002c83, 0x002c83, 0x002c82, 0x002c82},
+ {0x002c84, 0x002c85, 0x002c84, 0x002c84},
+ {0x002c85, 0x002c85, 0x002c84, 0x002c84},
+ {0x002c86, 0x002c87, 0x002c86, 0x002c86},
+ {0x002c87, 0x002c87, 0x002c86, 0x002c86},
+ {0x002c88, 0x002c89, 0x002c88, 0x002c88},
+ {0x002c89, 0x002c89, 0x002c88, 0x002c88},
+ {0x002c8a, 0x002c8b, 0x002c8a, 0x002c8a},
+ {0x002c8b, 0x002c8b, 0x002c8a, 0x002c8a},
+ {0x002c8c, 0x002c8d, 0x002c8c, 0x002c8c},
+ {0x002c8d, 0x002c8d, 0x002c8c, 0x002c8c},
+ {0x002c8e, 0x002c8f, 0x002c8e, 0x002c8e},
+ {0x002c8f, 0x002c8f, 0x002c8e, 0x002c8e},
+ {0x002c90, 0x002c91, 0x002c90, 0x002c90},
+ {0x002c91, 0x002c91, 0x002c90, 0x002c90},
+ {0x002c92, 0x002c93, 0x002c92, 0x002c92},
+ {0x002c93, 0x002c93, 0x002c92, 0x002c92},
+ {0x002c94, 0x002c95, 0x002c94, 0x002c94},
+ {0x002c95, 0x002c95, 0x002c94, 0x002c94},
+ {0x002c96, 0x002c97, 0x002c96, 0x002c96},
+ {0x002c97, 0x002c97, 0x002c96, 0x002c96},
+ {0x002c98, 0x002c99, 0x002c98, 0x002c98},
+ {0x002c99, 0x002c99, 0x002c98, 0x002c98},
+ {0x002c9a, 0x002c9b, 0x002c9a, 0x002c9a},
+ {0x002c9b, 0x002c9b, 0x002c9a, 0x002c9a},
+ {0x002c9c, 0x002c9d, 0x002c9c, 0x002c9c},
+ {0x002c9d, 0x002c9d, 0x002c9c, 0x002c9c},
+ {0x002c9e, 0x002c9f, 0x002c9e, 0x002c9e},
+ {0x002c9f, 0x002c9f, 0x002c9e, 0x002c9e},
+ {0x002ca0, 0x002ca1, 0x002ca0, 0x002ca0},
+ {0x002ca1, 0x002ca1, 0x002ca0, 0x002ca0},
+ {0x002ca2, 0x002ca3, 0x002ca2, 0x002ca2},
+ {0x002ca3, 0x002ca3, 0x002ca2, 0x002ca2},
+ {0x002ca4, 0x002ca5, 0x002ca4, 0x002ca4},
+ {0x002ca5, 0x002ca5, 0x002ca4, 0x002ca4},
+ {0x002ca6, 0x002ca7, 0x002ca6, 0x002ca6},
+ {0x002ca7, 0x002ca7, 0x002ca6, 0x002ca6},
+ {0x002ca8, 0x002ca9, 0x002ca8, 0x002ca8},
+ {0x002ca9, 0x002ca9, 0x002ca8, 0x002ca8},
+ {0x002caa, 0x002cab, 0x002caa, 0x002caa},
+ {0x002cab, 0x002cab, 0x002caa, 0x002caa},
+ {0x002cac, 0x002cad, 0x002cac, 0x002cac},
+ {0x002cad, 0x002cad, 0x002cac, 0x002cac},
+ {0x002cae, 0x002caf, 0x002cae, 0x002cae},
+ {0x002caf, 0x002caf, 0x002cae, 0x002cae},
+ {0x002cb0, 0x002cb1, 0x002cb0, 0x002cb0},
+ {0x002cb1, 0x002cb1, 0x002cb0, 0x002cb0},
+ {0x002cb2, 0x002cb3, 0x002cb2, 0x002cb2},
+ {0x002cb3, 0x002cb3, 0x002cb2, 0x002cb2},
+ {0x002cb4, 0x002cb5, 0x002cb4, 0x002cb4},
+ {0x002cb5, 0x002cb5, 0x002cb4, 0x002cb4},
+ {0x002cb6, 0x002cb7, 0x002cb6, 0x002cb6},
+ {0x002cb7, 0x002cb7, 0x002cb6, 0x002cb6},
+ {0x002cb8, 0x002cb9, 0x002cb8, 0x002cb8},
+ {0x002cb9, 0x002cb9, 0x002cb8, 0x002cb8},
+ {0x002cba, 0x002cbb, 0x002cba, 0x002cba},
+ {0x002cbb, 0x002cbb, 0x002cba, 0x002cba},
+ {0x002cbc, 0x002cbd, 0x002cbc, 0x002cbc},
+ {0x002cbd, 0x002cbd, 0x002cbc, 0x002cbc},
+ {0x002cbe, 0x002cbf, 0x002cbe, 0x002cbe},
+ {0x002cbf, 0x002cbf, 0x002cbe, 0x002cbe},
+ {0x002cc0, 0x002cc1, 0x002cc0, 0x002cc0},
+ {0x002cc1, 0x002cc1, 0x002cc0, 0x002cc0},
+ {0x002cc2, 0x002cc3, 0x002cc2, 0x002cc2},
+ {0x002cc3, 0x002cc3, 0x002cc2, 0x002cc2},
+ {0x002cc4, 0x002cc5, 0x002cc4, 0x002cc4},
+ {0x002cc5, 0x002cc5, 0x002cc4, 0x002cc4},
+ {0x002cc6, 0x002cc7, 0x002cc6, 0x002cc6},
+ {0x002cc7, 0x002cc7, 0x002cc6, 0x002cc6},
+ {0x002cc8, 0x002cc9, 0x002cc8, 0x002cc8},
+ {0x002cc9, 0x002cc9, 0x002cc8, 0x002cc8},
+ {0x002cca, 0x002ccb, 0x002cca, 0x002cca},
+ {0x002ccb, 0x002ccb, 0x002cca, 0x002cca},
+ {0x002ccc, 0x002ccd, 0x002ccc, 0x002ccc},
+ {0x002ccd, 0x002ccd, 0x002ccc, 0x002ccc},
+ {0x002cce, 0x002ccf, 0x002cce, 0x002cce},
+ {0x002ccf, 0x002ccf, 0x002cce, 0x002cce},
+ {0x002cd0, 0x002cd1, 0x002cd0, 0x002cd0},
+ {0x002cd1, 0x002cd1, 0x002cd0, 0x002cd0},
+ {0x002cd2, 0x002cd3, 0x002cd2, 0x002cd2},
+ {0x002cd3, 0x002cd3, 0x002cd2, 0x002cd2},
+ {0x002cd4, 0x002cd5, 0x002cd4, 0x002cd4},
+ {0x002cd5, 0x002cd5, 0x002cd4, 0x002cd4},
+ {0x002cd6, 0x002cd7, 0x002cd6, 0x002cd6},
+ {0x002cd7, 0x002cd7, 0x002cd6, 0x002cd6},
+ {0x002cd8, 0x002cd9, 0x002cd8, 0x002cd8},
+ {0x002cd9, 0x002cd9, 0x002cd8, 0x002cd8},
+ {0x002cda, 0x002cdb, 0x002cda, 0x002cda},
+ {0x002cdb, 0x002cdb, 0x002cda, 0x002cda},
+ {0x002cdc, 0x002cdd, 0x002cdc, 0x002cdc},
+ {0x002cdd, 0x002cdd, 0x002cdc, 0x002cdc},
+ {0x002cde, 0x002cdf, 0x002cde, 0x002cde},
+ {0x002cdf, 0x002cdf, 0x002cde, 0x002cde},
+ {0x002ce0, 0x002ce1, 0x002ce0, 0x002ce0},
+ {0x002ce1, 0x002ce1, 0x002ce0, 0x002ce0},
+ {0x002ce2, 0x002ce3, 0x002ce2, 0x002ce2},
+ {0x002ce3, 0x002ce3, 0x002ce2, 0x002ce2},
+ {0x002ceb, 0x002cec, 0x002ceb, 0x002ceb},
+ {0x002cec, 0x002cec, 0x002ceb, 0x002ceb},
+ {0x002ced, 0x002cee, 0x002ced, 0x002ced},
+ {0x002cee, 0x002cee, 0x002ced, 0x002ced},
+ {0x002cf2, 0x002cf3, 0x002cf2, 0x002cf2},
+ {0x002cf3, 0x002cf3, 0x002cf2, 0x002cf2},
+ {0x002d00, 0x002d00, 0x0010a0, 0x0010a0},
+ {0x002d01, 0x002d01, 0x0010a1, 0x0010a1},
+ {0x002d02, 0x002d02, 0x0010a2, 0x0010a2},
+ {0x002d03, 0x002d03, 0x0010a3, 0x0010a3},
+ {0x002d04, 0x002d04, 0x0010a4, 0x0010a4},
+ {0x002d05, 0x002d05, 0x0010a5, 0x0010a5},
+ {0x002d06, 0x002d06, 0x0010a6, 0x0010a6},
+ {0x002d07, 0x002d07, 0x0010a7, 0x0010a7},
+ {0x002d08, 0x002d08, 0x0010a8, 0x0010a8},
+ {0x002d09, 0x002d09, 0x0010a9, 0x0010a9},
+ {0x002d0a, 0x002d0a, 0x0010aa, 0x0010aa},
+ {0x002d0b, 0x002d0b, 0x0010ab, 0x0010ab},
+ {0x002d0c, 0x002d0c, 0x0010ac, 0x0010ac},
+ {0x002d0d, 0x002d0d, 0x0010ad, 0x0010ad},
+ {0x002d0e, 0x002d0e, 0x0010ae, 0x0010ae},
+ {0x002d0f, 0x002d0f, 0x0010af, 0x0010af},
+ {0x002d10, 0x002d10, 0x0010b0, 0x0010b0},
+ {0x002d11, 0x002d11, 0x0010b1, 0x0010b1},
+ {0x002d12, 0x002d12, 0x0010b2, 0x0010b2},
+ {0x002d13, 0x002d13, 0x0010b3, 0x0010b3},
+ {0x002d14, 0x002d14, 0x0010b4, 0x0010b4},
+ {0x002d15, 0x002d15, 0x0010b5, 0x0010b5},
+ {0x002d16, 0x002d16, 0x0010b6, 0x0010b6},
+ {0x002d17, 0x002d17, 0x0010b7, 0x0010b7},
+ {0x002d18, 0x002d18, 0x0010b8, 0x0010b8},
+ {0x002d19, 0x002d19, 0x0010b9, 0x0010b9},
+ {0x002d1a, 0x002d1a, 0x0010ba, 0x0010ba},
+ {0x002d1b, 0x002d1b, 0x0010bb, 0x0010bb},
+ {0x002d1c, 0x002d1c, 0x0010bc, 0x0010bc},
+ {0x002d1d, 0x002d1d, 0x0010bd, 0x0010bd},
+ {0x002d1e, 0x002d1e, 0x0010be, 0x0010be},
+ {0x002d1f, 0x002d1f, 0x0010bf, 0x0010bf},
+ {0x002d20, 0x002d20, 0x0010c0, 0x0010c0},
+ {0x002d21, 0x002d21, 0x0010c1, 0x0010c1},
+ {0x002d22, 0x002d22, 0x0010c2, 0x0010c2},
+ {0x002d23, 0x002d23, 0x0010c3, 0x0010c3},
+ {0x002d24, 0x002d24, 0x0010c4, 0x0010c4},
+ {0x002d25, 0x002d25, 0x0010c5, 0x0010c5},
+ {0x002d27, 0x002d27, 0x0010c7, 0x0010c7},
+ {0x002d2d, 0x002d2d, 0x0010cd, 0x0010cd},
+ {0x00a640, 0x00a641, 0x00a640, 0x00a640},
+ {0x00a641, 0x00a641, 0x00a640, 0x00a640},
+ {0x00a642, 0x00a643, 0x00a642, 0x00a642},
+ {0x00a643, 0x00a643, 0x00a642, 0x00a642},
+ {0x00a644, 0x00a645, 0x00a644, 0x00a644},
+ {0x00a645, 0x00a645, 0x00a644, 0x00a644},
+ {0x00a646, 0x00a647, 0x00a646, 0x00a646},
+ {0x00a647, 0x00a647, 0x00a646, 0x00a646},
+ {0x00a648, 0x00a649, 0x00a648, 0x00a648},
+ {0x00a649, 0x00a649, 0x00a648, 0x00a648},
+ {0x00a64a, 0x00a64b, 0x00a64a, 0x00a64a},
+ {0x00a64b, 0x00a64b, 0x00a64a, 0x00a64a},
+ {0x00a64c, 0x00a64d, 0x00a64c, 0x00a64c},
+ {0x00a64d, 0x00a64d, 0x00a64c, 0x00a64c},
+ {0x00a64e, 0x00a64f, 0x00a64e, 0x00a64e},
+ {0x00a64f, 0x00a64f, 0x00a64e, 0x00a64e},
+ {0x00a650, 0x00a651, 0x00a650, 0x00a650},
+ {0x00a651, 0x00a651, 0x00a650, 0x00a650},
+ {0x00a652, 0x00a653, 0x00a652, 0x00a652},
+ {0x00a653, 0x00a653, 0x00a652, 0x00a652},
+ {0x00a654, 0x00a655, 0x00a654, 0x00a654},
+ {0x00a655, 0x00a655, 0x00a654, 0x00a654},
+ {0x00a656, 0x00a657, 0x00a656, 0x00a656},
+ {0x00a657, 0x00a657, 0x00a656, 0x00a656},
+ {0x00a658, 0x00a659, 0x00a658, 0x00a658},
+ {0x00a659, 0x00a659, 0x00a658, 0x00a658},
+ {0x00a65a, 0x00a65b, 0x00a65a, 0x00a65a},
+ {0x00a65b, 0x00a65b, 0x00a65a, 0x00a65a},
+ {0x00a65c, 0x00a65d, 0x00a65c, 0x00a65c},
+ {0x00a65d, 0x00a65d, 0x00a65c, 0x00a65c},
+ {0x00a65e, 0x00a65f, 0x00a65e, 0x00a65e},
+ {0x00a65f, 0x00a65f, 0x00a65e, 0x00a65e},
+ {0x00a660, 0x00a661, 0x00a660, 0x00a660},
+ {0x00a661, 0x00a661, 0x00a660, 0x00a660},
+ {0x00a662, 0x00a663, 0x00a662, 0x00a662},
+ {0x00a663, 0x00a663, 0x00a662, 0x00a662},
+ {0x00a664, 0x00a665, 0x00a664, 0x00a664},
+ {0x00a665, 0x00a665, 0x00a664, 0x00a664},
+ {0x00a666, 0x00a667, 0x00a666, 0x00a666},
+ {0x00a667, 0x00a667, 0x00a666, 0x00a666},
+ {0x00a668, 0x00a669, 0x00a668, 0x00a668},
+ {0x00a669, 0x00a669, 0x00a668, 0x00a668},
+ {0x00a66a, 0x00a66b, 0x00a66a, 0x00a66a},
+ {0x00a66b, 0x00a66b, 0x00a66a, 0x00a66a},
+ {0x00a66c, 0x00a66d, 0x00a66c, 0x00a66c},
+ {0x00a66d, 0x00a66d, 0x00a66c, 0x00a66c},
+ {0x00a680, 0x00a681, 0x00a680, 0x00a680},
+ {0x00a681, 0x00a681, 0x00a680, 0x00a680},
+ {0x00a682, 0x00a683, 0x00a682, 0x00a682},
+ {0x00a683, 0x00a683, 0x00a682, 0x00a682},
+ {0x00a684, 0x00a685, 0x00a684, 0x00a684},
+ {0x00a685, 0x00a685, 0x00a684, 0x00a684},
+ {0x00a686, 0x00a687, 0x00a686, 0x00a686},
+ {0x00a687, 0x00a687, 0x00a686, 0x00a686},
+ {0x00a688, 0x00a689, 0x00a688, 0x00a688},
+ {0x00a689, 0x00a689, 0x00a688, 0x00a688},
+ {0x00a68a, 0x00a68b, 0x00a68a, 0x00a68a},
+ {0x00a68b, 0x00a68b, 0x00a68a, 0x00a68a},
+ {0x00a68c, 0x00a68d, 0x00a68c, 0x00a68c},
+ {0x00a68d, 0x00a68d, 0x00a68c, 0x00a68c},
+ {0x00a68e, 0x00a68f, 0x00a68e, 0x00a68e},
+ {0x00a68f, 0x00a68f, 0x00a68e, 0x00a68e},
+ {0x00a690, 0x00a691, 0x00a690, 0x00a690},
+ {0x00a691, 0x00a691, 0x00a690, 0x00a690},
+ {0x00a692, 0x00a693, 0x00a692, 0x00a692},
+ {0x00a693, 0x00a693, 0x00a692, 0x00a692},
+ {0x00a694, 0x00a695, 0x00a694, 0x00a694},
+ {0x00a695, 0x00a695, 0x00a694, 0x00a694},
+ {0x00a696, 0x00a697, 0x00a696, 0x00a696},
+ {0x00a697, 0x00a697, 0x00a696, 0x00a696},
+ {0x00a698, 0x00a699, 0x00a698, 0x00a698},
+ {0x00a699, 0x00a699, 0x00a698, 0x00a698},
+ {0x00a69a, 0x00a69b, 0x00a69a, 0x00a69a},
+ {0x00a69b, 0x00a69b, 0x00a69a, 0x00a69a},
+ {0x00a722, 0x00a723, 0x00a722, 0x00a722},
+ {0x00a723, 0x00a723, 0x00a722, 0x00a722},
+ {0x00a724, 0x00a725, 0x00a724, 0x00a724},
+ {0x00a725, 0x00a725, 0x00a724, 0x00a724},
+ {0x00a726, 0x00a727, 0x00a726, 0x00a726},
+ {0x00a727, 0x00a727, 0x00a726, 0x00a726},
+ {0x00a728, 0x00a729, 0x00a728, 0x00a728},
+ {0x00a729, 0x00a729, 0x00a728, 0x00a728},
+ {0x00a72a, 0x00a72b, 0x00a72a, 0x00a72a},
+ {0x00a72b, 0x00a72b, 0x00a72a, 0x00a72a},
+ {0x00a72c, 0x00a72d, 0x00a72c, 0x00a72c},
+ {0x00a72d, 0x00a72d, 0x00a72c, 0x00a72c},
+ {0x00a72e, 0x00a72f, 0x00a72e, 0x00a72e},
+ {0x00a72f, 0x00a72f, 0x00a72e, 0x00a72e},
+ {0x00a732, 0x00a733, 0x00a732, 0x00a732},
+ {0x00a733, 0x00a733, 0x00a732, 0x00a732},
+ {0x00a734, 0x00a735, 0x00a734, 0x00a734},
+ {0x00a735, 0x00a735, 0x00a734, 0x00a734},
+ {0x00a736, 0x00a737, 0x00a736, 0x00a736},
+ {0x00a737, 0x00a737, 0x00a736, 0x00a736},
+ {0x00a738, 0x00a739, 0x00a738, 0x00a738},
+ {0x00a739, 0x00a739, 0x00a738, 0x00a738},
+ {0x00a73a, 0x00a73b, 0x00a73a, 0x00a73a},
+ {0x00a73b, 0x00a73b, 0x00a73a, 0x00a73a},
+ {0x00a73c, 0x00a73d, 0x00a73c, 0x00a73c},
+ {0x00a73d, 0x00a73d, 0x00a73c, 0x00a73c},
+ {0x00a73e, 0x00a73f, 0x00a73e, 0x00a73e},
+ {0x00a73f, 0x00a73f, 0x00a73e, 0x00a73e},
+ {0x00a740, 0x00a741, 0x00a740, 0x00a740},
+ {0x00a741, 0x00a741, 0x00a740, 0x00a740},
+ {0x00a742, 0x00a743, 0x00a742, 0x00a742},
+ {0x00a743, 0x00a743, 0x00a742, 0x00a742},
+ {0x00a744, 0x00a745, 0x00a744, 0x00a744},
+ {0x00a745, 0x00a745, 0x00a744, 0x00a744},
+ {0x00a746, 0x00a747, 0x00a746, 0x00a746},
+ {0x00a747, 0x00a747, 0x00a746, 0x00a746},
+ {0x00a748, 0x00a749, 0x00a748, 0x00a748},
+ {0x00a749, 0x00a749, 0x00a748, 0x00a748},
+ {0x00a74a, 0x00a74b, 0x00a74a, 0x00a74a},
+ {0x00a74b, 0x00a74b, 0x00a74a, 0x00a74a},
+ {0x00a74c, 0x00a74d, 0x00a74c, 0x00a74c},
+ {0x00a74d, 0x00a74d, 0x00a74c, 0x00a74c},
+ {0x00a74e, 0x00a74f, 0x00a74e, 0x00a74e},
+ {0x00a74f, 0x00a74f, 0x00a74e, 0x00a74e},
+ {0x00a750, 0x00a751, 0x00a750, 0x00a750},
+ {0x00a751, 0x00a751, 0x00a750, 0x00a750},
+ {0x00a752, 0x00a753, 0x00a752, 0x00a752},
+ {0x00a753, 0x00a753, 0x00a752, 0x00a752},
+ {0x00a754, 0x00a755, 0x00a754, 0x00a754},
+ {0x00a755, 0x00a755, 0x00a754, 0x00a754},
+ {0x00a756, 0x00a757, 0x00a756, 0x00a756},
+ {0x00a757, 0x00a757, 0x00a756, 0x00a756},
+ {0x00a758, 0x00a759, 0x00a758, 0x00a758},
+ {0x00a759, 0x00a759, 0x00a758, 0x00a758},
+ {0x00a75a, 0x00a75b, 0x00a75a, 0x00a75a},
+ {0x00a75b, 0x00a75b, 0x00a75a, 0x00a75a},
+ {0x00a75c, 0x00a75d, 0x00a75c, 0x00a75c},
+ {0x00a75d, 0x00a75d, 0x00a75c, 0x00a75c},
+ {0x00a75e, 0x00a75f, 0x00a75e, 0x00a75e},
+ {0x00a75f, 0x00a75f, 0x00a75e, 0x00a75e},
+ {0x00a760, 0x00a761, 0x00a760, 0x00a760},
+ {0x00a761, 0x00a761, 0x00a760, 0x00a760},
+ {0x00a762, 0x00a763, 0x00a762, 0x00a762},
+ {0x00a763, 0x00a763, 0x00a762, 0x00a762},
+ {0x00a764, 0x00a765, 0x00a764, 0x00a764},
+ {0x00a765, 0x00a765, 0x00a764, 0x00a764},
+ {0x00a766, 0x00a767, 0x00a766, 0x00a766},
+ {0x00a767, 0x00a767, 0x00a766, 0x00a766},
+ {0x00a768, 0x00a769, 0x00a768, 0x00a768},
+ {0x00a769, 0x00a769, 0x00a768, 0x00a768},
+ {0x00a76a, 0x00a76b, 0x00a76a, 0x00a76a},
+ {0x00a76b, 0x00a76b, 0x00a76a, 0x00a76a},
+ {0x00a76c, 0x00a76d, 0x00a76c, 0x00a76c},
+ {0x00a76d, 0x00a76d, 0x00a76c, 0x00a76c},
+ {0x00a76e, 0x00a76f, 0x00a76e, 0x00a76e},
+ {0x00a76f, 0x00a76f, 0x00a76e, 0x00a76e},
+ {0x00a779, 0x00a77a, 0x00a779, 0x00a779},
+ {0x00a77a, 0x00a77a, 0x00a779, 0x00a779},
+ {0x00a77b, 0x00a77c, 0x00a77b, 0x00a77b},
+ {0x00a77c, 0x00a77c, 0x00a77b, 0x00a77b},
+ {0x00a77d, 0x001d79, 0x00a77d, 0x00a77d},
+ {0x00a77e, 0x00a77f, 0x00a77e, 0x00a77e},
+ {0x00a77f, 0x00a77f, 0x00a77e, 0x00a77e},
+ {0x00a780, 0x00a781, 0x00a780, 0x00a780},
+ {0x00a781, 0x00a781, 0x00a780, 0x00a780},
+ {0x00a782, 0x00a783, 0x00a782, 0x00a782},
+ {0x00a783, 0x00a783, 0x00a782, 0x00a782},
+ {0x00a784, 0x00a785, 0x00a784, 0x00a784},
+ {0x00a785, 0x00a785, 0x00a784, 0x00a784},
+ {0x00a786, 0x00a787, 0x00a786, 0x00a786},
+ {0x00a787, 0x00a787, 0x00a786, 0x00a786},
+ {0x00a78b, 0x00a78c, 0x00a78b, 0x00a78b},
+ {0x00a78c, 0x00a78c, 0x00a78b, 0x00a78b},
+ {0x00a78d, 0x000265, 0x00a78d, 0x00a78d},
+ {0x00a790, 0x00a791, 0x00a790, 0x00a790},
+ {0x00a791, 0x00a791, 0x00a790, 0x00a790},
+ {0x00a792, 0x00a793, 0x00a792, 0x00a792},
+ {0x00a793, 0x00a793, 0x00a792, 0x00a792},
+ {0x00a794, 0x00a794, 0x00a7c4, 0x00a7c4},
+ {0x00a796, 0x00a797, 0x00a796, 0x00a796},
+ {0x00a797, 0x00a797, 0x00a796, 0x00a796},
+ {0x00a798, 0x00a799, 0x00a798, 0x00a798},
+ {0x00a799, 0x00a799, 0x00a798, 0x00a798},
+ {0x00a79a, 0x00a79b, 0x00a79a, 0x00a79a},
+ {0x00a79b, 0x00a79b, 0x00a79a, 0x00a79a},
+ {0x00a79c, 0x00a79d, 0x00a79c, 0x00a79c},
+ {0x00a79d, 0x00a79d, 0x00a79c, 0x00a79c},
+ {0x00a79e, 0x00a79f, 0x00a79e, 0x00a79e},
+ {0x00a79f, 0x00a79f, 0x00a79e, 0x00a79e},
+ {0x00a7a0, 0x00a7a1, 0x00a7a0, 0x00a7a0},
+ {0x00a7a1, 0x00a7a1, 0x00a7a0, 0x00a7a0},
+ {0x00a7a2, 0x00a7a3, 0x00a7a2, 0x00a7a2},
+ {0x00a7a3, 0x00a7a3, 0x00a7a2, 0x00a7a2},
+ {0x00a7a4, 0x00a7a5, 0x00a7a4, 0x00a7a4},
+ {0x00a7a5, 0x00a7a5, 0x00a7a4, 0x00a7a4},
+ {0x00a7a6, 0x00a7a7, 0x00a7a6, 0x00a7a6},
+ {0x00a7a7, 0x00a7a7, 0x00a7a6, 0x00a7a6},
+ {0x00a7a8, 0x00a7a9, 0x00a7a8, 0x00a7a8},
+ {0x00a7a9, 0x00a7a9, 0x00a7a8, 0x00a7a8},
+ {0x00a7aa, 0x000266, 0x00a7aa, 0x00a7aa},
+ {0x00a7ab, 0x00025c, 0x00a7ab, 0x00a7ab},
+ {0x00a7ac, 0x000261, 0x00a7ac, 0x00a7ac},
+ {0x00a7ad, 0x00026c, 0x00a7ad, 0x00a7ad},
+ {0x00a7ae, 0x00026a, 0x00a7ae, 0x00a7ae},
+ {0x00a7b0, 0x00029e, 0x00a7b0, 0x00a7b0},
+ {0x00a7b1, 0x000287, 0x00a7b1, 0x00a7b1},
+ {0x00a7b2, 0x00029d, 0x00a7b2, 0x00a7b2},
+ {0x00a7b3, 0x00ab53, 0x00a7b3, 0x00a7b3},
+ {0x00a7b4, 0x00a7b5, 0x00a7b4, 0x00a7b4},
+ {0x00a7b5, 0x00a7b5, 0x00a7b4, 0x00a7b4},
+ {0x00a7b6, 0x00a7b7, 0x00a7b6, 0x00a7b6},
+ {0x00a7b7, 0x00a7b7, 0x00a7b6, 0x00a7b6},
+ {0x00a7b8, 0x00a7b9, 0x00a7b8, 0x00a7b8},
+ {0x00a7b9, 0x00a7b9, 0x00a7b8, 0x00a7b8},
+ {0x00a7ba, 0x00a7bb, 0x00a7ba, 0x00a7ba},
+ {0x00a7bb, 0x00a7bb, 0x00a7ba, 0x00a7ba},
+ {0x00a7bc, 0x00a7bd, 0x00a7bc, 0x00a7bc},
+ {0x00a7bd, 0x00a7bd, 0x00a7bc, 0x00a7bc},
+ {0x00a7be, 0x00a7bf, 0x00a7be, 0x00a7be},
+ {0x00a7bf, 0x00a7bf, 0x00a7be, 0x00a7be},
+ {0x00a7c0, 0x00a7c1, 0x00a7c0, 0x00a7c0},
+ {0x00a7c1, 0x00a7c1, 0x00a7c0, 0x00a7c0},
+ {0x00a7c2, 0x00a7c3, 0x00a7c2, 0x00a7c2},
+ {0x00a7c3, 0x00a7c3, 0x00a7c2, 0x00a7c2},
+ {0x00a7c4, 0x00a794, 0x00a7c4, 0x00a7c4},
+ {0x00a7c5, 0x000282, 0x00a7c5, 0x00a7c5},
+ {0x00a7c6, 0x001d8e, 0x00a7c6, 0x00a7c6},
+ {0x00a7c7, 0x00a7c8, 0x00a7c7, 0x00a7c7},
+ {0x00a7c8, 0x00a7c8, 0x00a7c7, 0x00a7c7},
+ {0x00a7c9, 0x00a7ca, 0x00a7c9, 0x00a7c9},
+ {0x00a7ca, 0x00a7ca, 0x00a7c9, 0x00a7c9},
+ {0x00a7d0, 0x00a7d1, 0x00a7d0, 0x00a7d0},
+ {0x00a7d1, 0x00a7d1, 0x00a7d0, 0x00a7d0},
+ {0x00a7d6, 0x00a7d7, 0x00a7d6, 0x00a7d6},
+ {0x00a7d7, 0x00a7d7, 0x00a7d6, 0x00a7d6},
+ {0x00a7d8, 0x00a7d9, 0x00a7d8, 0x00a7d8},
+ {0x00a7d9, 0x00a7d9, 0x00a7d8, 0x00a7d8},
+ {0x00a7f5, 0x00a7f6, 0x00a7f5, 0x00a7f5},
+ {0x00a7f6, 0x00a7f6, 0x00a7f5, 0x00a7f5},
+ {0x00ab53, 0x00ab53, 0x00a7b3, 0x00a7b3},
+ {0x00ab70, 0x00ab70, 0x0013a0, 0x0013a0},
+ {0x00ab71, 0x00ab71, 0x0013a1, 0x0013a1},
+ {0x00ab72, 0x00ab72, 0x0013a2, 0x0013a2},
+ {0x00ab73, 0x00ab73, 0x0013a3, 0x0013a3},
+ {0x00ab74, 0x00ab74, 0x0013a4, 0x0013a4},
+ {0x00ab75, 0x00ab75, 0x0013a5, 0x0013a5},
+ {0x00ab76, 0x00ab76, 0x0013a6, 0x0013a6},
+ {0x00ab77, 0x00ab77, 0x0013a7, 0x0013a7},
+ {0x00ab78, 0x00ab78, 0x0013a8, 0x0013a8},
+ {0x00ab79, 0x00ab79, 0x0013a9, 0x0013a9},
+ {0x00ab7a, 0x00ab7a, 0x0013aa, 0x0013aa},
+ {0x00ab7b, 0x00ab7b, 0x0013ab, 0x0013ab},
+ {0x00ab7c, 0x00ab7c, 0x0013ac, 0x0013ac},
+ {0x00ab7d, 0x00ab7d, 0x0013ad, 0x0013ad},
+ {0x00ab7e, 0x00ab7e, 0x0013ae, 0x0013ae},
+ {0x00ab7f, 0x00ab7f, 0x0013af, 0x0013af},
+ {0x00ab80, 0x00ab80, 0x0013b0, 0x0013b0},
+ {0x00ab81, 0x00ab81, 0x0013b1, 0x0013b1},
+ {0x00ab82, 0x00ab82, 0x0013b2, 0x0013b2},
+ {0x00ab83, 0x00ab83, 0x0013b3, 0x0013b3},
+ {0x00ab84, 0x00ab84, 0x0013b4, 0x0013b4},
+ {0x00ab85, 0x00ab85, 0x0013b5, 0x0013b5},
+ {0x00ab86, 0x00ab86, 0x0013b6, 0x0013b6},
+ {0x00ab87, 0x00ab87, 0x0013b7, 0x0013b7},
+ {0x00ab88, 0x00ab88, 0x0013b8, 0x0013b8},
+ {0x00ab89, 0x00ab89, 0x0013b9, 0x0013b9},
+ {0x00ab8a, 0x00ab8a, 0x0013ba, 0x0013ba},
+ {0x00ab8b, 0x00ab8b, 0x0013bb, 0x0013bb},
+ {0x00ab8c, 0x00ab8c, 0x0013bc, 0x0013bc},
+ {0x00ab8d, 0x00ab8d, 0x0013bd, 0x0013bd},
+ {0x00ab8e, 0x00ab8e, 0x0013be, 0x0013be},
+ {0x00ab8f, 0x00ab8f, 0x0013bf, 0x0013bf},
+ {0x00ab90, 0x00ab90, 0x0013c0, 0x0013c0},
+ {0x00ab91, 0x00ab91, 0x0013c1, 0x0013c1},
+ {0x00ab92, 0x00ab92, 0x0013c2, 0x0013c2},
+ {0x00ab93, 0x00ab93, 0x0013c3, 0x0013c3},
+ {0x00ab94, 0x00ab94, 0x0013c4, 0x0013c4},
+ {0x00ab95, 0x00ab95, 0x0013c5, 0x0013c5},
+ {0x00ab96, 0x00ab96, 0x0013c6, 0x0013c6},
+ {0x00ab97, 0x00ab97, 0x0013c7, 0x0013c7},
+ {0x00ab98, 0x00ab98, 0x0013c8, 0x0013c8},
+ {0x00ab99, 0x00ab99, 0x0013c9, 0x0013c9},
+ {0x00ab9a, 0x00ab9a, 0x0013ca, 0x0013ca},
+ {0x00ab9b, 0x00ab9b, 0x0013cb, 0x0013cb},
+ {0x00ab9c, 0x00ab9c, 0x0013cc, 0x0013cc},
+ {0x00ab9d, 0x00ab9d, 0x0013cd, 0x0013cd},
+ {0x00ab9e, 0x00ab9e, 0x0013ce, 0x0013ce},
+ {0x00ab9f, 0x00ab9f, 0x0013cf, 0x0013cf},
+ {0x00aba0, 0x00aba0, 0x0013d0, 0x0013d0},
+ {0x00aba1, 0x00aba1, 0x0013d1, 0x0013d1},
+ {0x00aba2, 0x00aba2, 0x0013d2, 0x0013d2},
+ {0x00aba3, 0x00aba3, 0x0013d3, 0x0013d3},
+ {0x00aba4, 0x00aba4, 0x0013d4, 0x0013d4},
+ {0x00aba5, 0x00aba5, 0x0013d5, 0x0013d5},
+ {0x00aba6, 0x00aba6, 0x0013d6, 0x0013d6},
+ {0x00aba7, 0x00aba7, 0x0013d7, 0x0013d7},
+ {0x00aba8, 0x00aba8, 0x0013d8, 0x0013d8},
+ {0x00aba9, 0x00aba9, 0x0013d9, 0x0013d9},
+ {0x00abaa, 0x00abaa, 0x0013da, 0x0013da},
+ {0x00abab, 0x00abab, 0x0013db, 0x0013db},
+ {0x00abac, 0x00abac, 0x0013dc, 0x0013dc},
+ {0x00abad, 0x00abad, 0x0013dd, 0x0013dd},
+ {0x00abae, 0x00abae, 0x0013de, 0x0013de},
+ {0x00abaf, 0x00abaf, 0x0013df, 0x0013df},
+ {0x00abb0, 0x00abb0, 0x0013e0, 0x0013e0},
+ {0x00abb1, 0x00abb1, 0x0013e1, 0x0013e1},
+ {0x00abb2, 0x00abb2, 0x0013e2, 0x0013e2},
+ {0x00abb3, 0x00abb3, 0x0013e3, 0x0013e3},
+ {0x00abb4, 0x00abb4, 0x0013e4, 0x0013e4},
+ {0x00abb5, 0x00abb5, 0x0013e5, 0x0013e5},
+ {0x00abb6, 0x00abb6, 0x0013e6, 0x0013e6},
+ {0x00abb7, 0x00abb7, 0x0013e7, 0x0013e7},
+ {0x00abb8, 0x00abb8, 0x0013e8, 0x0013e8},
+ {0x00abb9, 0x00abb9, 0x0013e9, 0x0013e9},
+ {0x00abba, 0x00abba, 0x0013ea, 0x0013ea},
+ {0x00abbb, 0x00abbb, 0x0013eb, 0x0013eb},
+ {0x00abbc, 0x00abbc, 0x0013ec, 0x0013ec},
+ {0x00abbd, 0x00abbd, 0x0013ed, 0x0013ed},
+ {0x00abbe, 0x00abbe, 0x0013ee, 0x0013ee},
+ {0x00abbf, 0x00abbf, 0x0013ef, 0x0013ef},
+ {0x00ff21, 0x00ff41, 0x00ff21, 0x00ff21},
+ {0x00ff22, 0x00ff42, 0x00ff22, 0x00ff22},
+ {0x00ff23, 0x00ff43, 0x00ff23, 0x00ff23},
+ {0x00ff24, 0x00ff44, 0x00ff24, 0x00ff24},
+ {0x00ff25, 0x00ff45, 0x00ff25, 0x00ff25},
+ {0x00ff26, 0x00ff46, 0x00ff26, 0x00ff26},
+ {0x00ff27, 0x00ff47, 0x00ff27, 0x00ff27},
+ {0x00ff28, 0x00ff48, 0x00ff28, 0x00ff28},
+ {0x00ff29, 0x00ff49, 0x00ff29, 0x00ff29},
+ {0x00ff2a, 0x00ff4a, 0x00ff2a, 0x00ff2a},
+ {0x00ff2b, 0x00ff4b, 0x00ff2b, 0x00ff2b},
+ {0x00ff2c, 0x00ff4c, 0x00ff2c, 0x00ff2c},
+ {0x00ff2d, 0x00ff4d, 0x00ff2d, 0x00ff2d},
+ {0x00ff2e, 0x00ff4e, 0x00ff2e, 0x00ff2e},
+ {0x00ff2f, 0x00ff4f, 0x00ff2f, 0x00ff2f},
+ {0x00ff30, 0x00ff50, 0x00ff30, 0x00ff30},
+ {0x00ff31, 0x00ff51, 0x00ff31, 0x00ff31},
+ {0x00ff32, 0x00ff52, 0x00ff32, 0x00ff32},
+ {0x00ff33, 0x00ff53, 0x00ff33, 0x00ff33},
+ {0x00ff34, 0x00ff54, 0x00ff34, 0x00ff34},
+ {0x00ff35, 0x00ff55, 0x00ff35, 0x00ff35},
+ {0x00ff36, 0x00ff56, 0x00ff36, 0x00ff36},
+ {0x00ff37, 0x00ff57, 0x00ff37, 0x00ff37},
+ {0x00ff38, 0x00ff58, 0x00ff38, 0x00ff38},
+ {0x00ff39, 0x00ff59, 0x00ff39, 0x00ff39},
+ {0x00ff3a, 0x00ff5a, 0x00ff3a, 0x00ff3a},
+ {0x00ff41, 0x00ff41, 0x00ff21, 0x00ff21},
+ {0x00ff42, 0x00ff42, 0x00ff22, 0x00ff22},
+ {0x00ff43, 0x00ff43, 0x00ff23, 0x00ff23},
+ {0x00ff44, 0x00ff44, 0x00ff24, 0x00ff24},
+ {0x00ff45, 0x00ff45, 0x00ff25, 0x00ff25},
+ {0x00ff46, 0x00ff46, 0x00ff26, 0x00ff26},
+ {0x00ff47, 0x00ff47, 0x00ff27, 0x00ff27},
+ {0x00ff48, 0x00ff48, 0x00ff28, 0x00ff28},
+ {0x00ff49, 0x00ff49, 0x00ff29, 0x00ff29},
+ {0x00ff4a, 0x00ff4a, 0x00ff2a, 0x00ff2a},
+ {0x00ff4b, 0x00ff4b, 0x00ff2b, 0x00ff2b},
+ {0x00ff4c, 0x00ff4c, 0x00ff2c, 0x00ff2c},
+ {0x00ff4d, 0x00ff4d, 0x00ff2d, 0x00ff2d},
+ {0x00ff4e, 0x00ff4e, 0x00ff2e, 0x00ff2e},
+ {0x00ff4f, 0x00ff4f, 0x00ff2f, 0x00ff2f},
+ {0x00ff50, 0x00ff50, 0x00ff30, 0x00ff30},
+ {0x00ff51, 0x00ff51, 0x00ff31, 0x00ff31},
+ {0x00ff52, 0x00ff52, 0x00ff32, 0x00ff32},
+ {0x00ff53, 0x00ff53, 0x00ff33, 0x00ff33},
+ {0x00ff54, 0x00ff54, 0x00ff34, 0x00ff34},
+ {0x00ff55, 0x00ff55, 0x00ff35, 0x00ff35},
+ {0x00ff56, 0x00ff56, 0x00ff36, 0x00ff36},
+ {0x00ff57, 0x00ff57, 0x00ff37, 0x00ff37},
+ {0x00ff58, 0x00ff58, 0x00ff38, 0x00ff38},
+ {0x00ff59, 0x00ff59, 0x00ff39, 0x00ff39},
+ {0x00ff5a, 0x00ff5a, 0x00ff3a, 0x00ff3a},
+ {0x010400, 0x010428, 0x010400, 0x010400},
+ {0x010401, 0x010429, 0x010401, 0x010401},
+ {0x010402, 0x01042a, 0x010402, 0x010402},
+ {0x010403, 0x01042b, 0x010403, 0x010403},
+ {0x010404, 0x01042c, 0x010404, 0x010404},
+ {0x010405, 0x01042d, 0x010405, 0x010405},
+ {0x010406, 0x01042e, 0x010406, 0x010406},
+ {0x010407, 0x01042f, 0x010407, 0x010407},
+ {0x010408, 0x010430, 0x010408, 0x010408},
+ {0x010409, 0x010431, 0x010409, 0x010409},
+ {0x01040a, 0x010432, 0x01040a, 0x01040a},
+ {0x01040b, 0x010433, 0x01040b, 0x01040b},
+ {0x01040c, 0x010434, 0x01040c, 0x01040c},
+ {0x01040d, 0x010435, 0x01040d, 0x01040d},
+ {0x01040e, 0x010436, 0x01040e, 0x01040e},
+ {0x01040f, 0x010437, 0x01040f, 0x01040f},
+ {0x010410, 0x010438, 0x010410, 0x010410},
+ {0x010411, 0x010439, 0x010411, 0x010411},
+ {0x010412, 0x01043a, 0x010412, 0x010412},
+ {0x010413, 0x01043b, 0x010413, 0x010413},
+ {0x010414, 0x01043c, 0x010414, 0x010414},
+ {0x010415, 0x01043d, 0x010415, 0x010415},
+ {0x010416, 0x01043e, 0x010416, 0x010416},
+ {0x010417, 0x01043f, 0x010417, 0x010417},
+ {0x010418, 0x010440, 0x010418, 0x010418},
+ {0x010419, 0x010441, 0x010419, 0x010419},
+ {0x01041a, 0x010442, 0x01041a, 0x01041a},
+ {0x01041b, 0x010443, 0x01041b, 0x01041b},
+ {0x01041c, 0x010444, 0x01041c, 0x01041c},
+ {0x01041d, 0x010445, 0x01041d, 0x01041d},
+ {0x01041e, 0x010446, 0x01041e, 0x01041e},
+ {0x01041f, 0x010447, 0x01041f, 0x01041f},
+ {0x010420, 0x010448, 0x010420, 0x010420},
+ {0x010421, 0x010449, 0x010421, 0x010421},
+ {0x010422, 0x01044a, 0x010422, 0x010422},
+ {0x010423, 0x01044b, 0x010423, 0x010423},
+ {0x010424, 0x01044c, 0x010424, 0x010424},
+ {0x010425, 0x01044d, 0x010425, 0x010425},
+ {0x010426, 0x01044e, 0x010426, 0x010426},
+ {0x010427, 0x01044f, 0x010427, 0x010427},
+ {0x010428, 0x010428, 0x010400, 0x010400},
+ {0x010429, 0x010429, 0x010401, 0x010401},
+ {0x01042a, 0x01042a, 0x010402, 0x010402},
+ {0x01042b, 0x01042b, 0x010403, 0x010403},
+ {0x01042c, 0x01042c, 0x010404, 0x010404},
+ {0x01042d, 0x01042d, 0x010405, 0x010405},
+ {0x01042e, 0x01042e, 0x010406, 0x010406},
+ {0x01042f, 0x01042f, 0x010407, 0x010407},
+ {0x010430, 0x010430, 0x010408, 0x010408},
+ {0x010431, 0x010431, 0x010409, 0x010409},
+ {0x010432, 0x010432, 0x01040a, 0x01040a},
+ {0x010433, 0x010433, 0x01040b, 0x01040b},
+ {0x010434, 0x010434, 0x01040c, 0x01040c},
+ {0x010435, 0x010435, 0x01040d, 0x01040d},
+ {0x010436, 0x010436, 0x01040e, 0x01040e},
+ {0x010437, 0x010437, 0x01040f, 0x01040f},
+ {0x010438, 0x010438, 0x010410, 0x010410},
+ {0x010439, 0x010439, 0x010411, 0x010411},
+ {0x01043a, 0x01043a, 0x010412, 0x010412},
+ {0x01043b, 0x01043b, 0x010413, 0x010413},
+ {0x01043c, 0x01043c, 0x010414, 0x010414},
+ {0x01043d, 0x01043d, 0x010415, 0x010415},
+ {0x01043e, 0x01043e, 0x010416, 0x010416},
+ {0x01043f, 0x01043f, 0x010417, 0x010417},
+ {0x010440, 0x010440, 0x010418, 0x010418},
+ {0x010441, 0x010441, 0x010419, 0x010419},
+ {0x010442, 0x010442, 0x01041a, 0x01041a},
+ {0x010443, 0x010443, 0x01041b, 0x01041b},
+ {0x010444, 0x010444, 0x01041c, 0x01041c},
+ {0x010445, 0x010445, 0x01041d, 0x01041d},
+ {0x010446, 0x010446, 0x01041e, 0x01041e},
+ {0x010447, 0x010447, 0x01041f, 0x01041f},
+ {0x010448, 0x010448, 0x010420, 0x010420},
+ {0x010449, 0x010449, 0x010421, 0x010421},
+ {0x01044a, 0x01044a, 0x010422, 0x010422},
+ {0x01044b, 0x01044b, 0x010423, 0x010423},
+ {0x01044c, 0x01044c, 0x010424, 0x010424},
+ {0x01044d, 0x01044d, 0x010425, 0x010425},
+ {0x01044e, 0x01044e, 0x010426, 0x010426},
+ {0x01044f, 0x01044f, 0x010427, 0x010427},
+ {0x0104b0, 0x0104d8, 0x0104b0, 0x0104b0},
+ {0x0104b1, 0x0104d9, 0x0104b1, 0x0104b1},
+ {0x0104b2, 0x0104da, 0x0104b2, 0x0104b2},
+ {0x0104b3, 0x0104db, 0x0104b3, 0x0104b3},
+ {0x0104b4, 0x0104dc, 0x0104b4, 0x0104b4},
+ {0x0104b5, 0x0104dd, 0x0104b5, 0x0104b5},
+ {0x0104b6, 0x0104de, 0x0104b6, 0x0104b6},
+ {0x0104b7, 0x0104df, 0x0104b7, 0x0104b7},
+ {0x0104b8, 0x0104e0, 0x0104b8, 0x0104b8},
+ {0x0104b9, 0x0104e1, 0x0104b9, 0x0104b9},
+ {0x0104ba, 0x0104e2, 0x0104ba, 0x0104ba},
+ {0x0104bb, 0x0104e3, 0x0104bb, 0x0104bb},
+ {0x0104bc, 0x0104e4, 0x0104bc, 0x0104bc},
+ {0x0104bd, 0x0104e5, 0x0104bd, 0x0104bd},
+ {0x0104be, 0x0104e6, 0x0104be, 0x0104be},
+ {0x0104bf, 0x0104e7, 0x0104bf, 0x0104bf},
+ {0x0104c0, 0x0104e8, 0x0104c0, 0x0104c0},
+ {0x0104c1, 0x0104e9, 0x0104c1, 0x0104c1},
+ {0x0104c2, 0x0104ea, 0x0104c2, 0x0104c2},
+ {0x0104c3, 0x0104eb, 0x0104c3, 0x0104c3},
+ {0x0104c4, 0x0104ec, 0x0104c4, 0x0104c4},
+ {0x0104c5, 0x0104ed, 0x0104c5, 0x0104c5},
+ {0x0104c6, 0x0104ee, 0x0104c6, 0x0104c6},
+ {0x0104c7, 0x0104ef, 0x0104c7, 0x0104c7},
+ {0x0104c8, 0x0104f0, 0x0104c8, 0x0104c8},
+ {0x0104c9, 0x0104f1, 0x0104c9, 0x0104c9},
+ {0x0104ca, 0x0104f2, 0x0104ca, 0x0104ca},
+ {0x0104cb, 0x0104f3, 0x0104cb, 0x0104cb},
+ {0x0104cc, 0x0104f4, 0x0104cc, 0x0104cc},
+ {0x0104cd, 0x0104f5, 0x0104cd, 0x0104cd},
+ {0x0104ce, 0x0104f6, 0x0104ce, 0x0104ce},
+ {0x0104cf, 0x0104f7, 0x0104cf, 0x0104cf},
+ {0x0104d0, 0x0104f8, 0x0104d0, 0x0104d0},
+ {0x0104d1, 0x0104f9, 0x0104d1, 0x0104d1},
+ {0x0104d2, 0x0104fa, 0x0104d2, 0x0104d2},
+ {0x0104d3, 0x0104fb, 0x0104d3, 0x0104d3},
+ {0x0104d8, 0x0104d8, 0x0104b0, 0x0104b0},
+ {0x0104d9, 0x0104d9, 0x0104b1, 0x0104b1},
+ {0x0104da, 0x0104da, 0x0104b2, 0x0104b2},
+ {0x0104db, 0x0104db, 0x0104b3, 0x0104b3},
+ {0x0104dc, 0x0104dc, 0x0104b4, 0x0104b4},
+ {0x0104dd, 0x0104dd, 0x0104b5, 0x0104b5},
+ {0x0104de, 0x0104de, 0x0104b6, 0x0104b6},
+ {0x0104df, 0x0104df, 0x0104b7, 0x0104b7},
+ {0x0104e0, 0x0104e0, 0x0104b8, 0x0104b8},
+ {0x0104e1, 0x0104e1, 0x0104b9, 0x0104b9},
+ {0x0104e2, 0x0104e2, 0x0104ba, 0x0104ba},
+ {0x0104e3, 0x0104e3, 0x0104bb, 0x0104bb},
+ {0x0104e4, 0x0104e4, 0x0104bc, 0x0104bc},
+ {0x0104e5, 0x0104e5, 0x0104bd, 0x0104bd},
+ {0x0104e6, 0x0104e6, 0x0104be, 0x0104be},
+ {0x0104e7, 0x0104e7, 0x0104bf, 0x0104bf},
+ {0x0104e8, 0x0104e8, 0x0104c0, 0x0104c0},
+ {0x0104e9, 0x0104e9, 0x0104c1, 0x0104c1},
+ {0x0104ea, 0x0104ea, 0x0104c2, 0x0104c2},
+ {0x0104eb, 0x0104eb, 0x0104c3, 0x0104c3},
+ {0x0104ec, 0x0104ec, 0x0104c4, 0x0104c4},
+ {0x0104ed, 0x0104ed, 0x0104c5, 0x0104c5},
+ {0x0104ee, 0x0104ee, 0x0104c6, 0x0104c6},
+ {0x0104ef, 0x0104ef, 0x0104c7, 0x0104c7},
+ {0x0104f0, 0x0104f0, 0x0104c8, 0x0104c8},
+ {0x0104f1, 0x0104f1, 0x0104c9, 0x0104c9},
+ {0x0104f2, 0x0104f2, 0x0104ca, 0x0104ca},
+ {0x0104f3, 0x0104f3, 0x0104cb, 0x0104cb},
+ {0x0104f4, 0x0104f4, 0x0104cc, 0x0104cc},
+ {0x0104f5, 0x0104f5, 0x0104cd, 0x0104cd},
+ {0x0104f6, 0x0104f6, 0x0104ce, 0x0104ce},
+ {0x0104f7, 0x0104f7, 0x0104cf, 0x0104cf},
+ {0x0104f8, 0x0104f8, 0x0104d0, 0x0104d0},
+ {0x0104f9, 0x0104f9, 0x0104d1, 0x0104d1},
+ {0x0104fa, 0x0104fa, 0x0104d2, 0x0104d2},
+ {0x0104fb, 0x0104fb, 0x0104d3, 0x0104d3},
+ {0x010570, 0x010597, 0x010570, 0x010570},
+ {0x010571, 0x010598, 0x010571, 0x010571},
+ {0x010572, 0x010599, 0x010572, 0x010572},
+ {0x010573, 0x01059a, 0x010573, 0x010573},
+ {0x010574, 0x01059b, 0x010574, 0x010574},
+ {0x010575, 0x01059c, 0x010575, 0x010575},
+ {0x010576, 0x01059d, 0x010576, 0x010576},
+ {0x010577, 0x01059e, 0x010577, 0x010577},
+ {0x010578, 0x01059f, 0x010578, 0x010578},
+ {0x010579, 0x0105a0, 0x010579, 0x010579},
+ {0x01057a, 0x0105a1, 0x01057a, 0x01057a},
+ {0x01057c, 0x0105a3, 0x01057c, 0x01057c},
+ {0x01057d, 0x0105a4, 0x01057d, 0x01057d},
+ {0x01057e, 0x0105a5, 0x01057e, 0x01057e},
+ {0x01057f, 0x0105a6, 0x01057f, 0x01057f},
+ {0x010580, 0x0105a7, 0x010580, 0x010580},
+ {0x010581, 0x0105a8, 0x010581, 0x010581},
+ {0x010582, 0x0105a9, 0x010582, 0x010582},
+ {0x010583, 0x0105aa, 0x010583, 0x010583},
+ {0x010584, 0x0105ab, 0x010584, 0x010584},
+ {0x010585, 0x0105ac, 0x010585, 0x010585},
+ {0x010586, 0x0105ad, 0x010586, 0x010586},
+ {0x010587, 0x0105ae, 0x010587, 0x010587},
+ {0x010588, 0x0105af, 0x010588, 0x010588},
+ {0x010589, 0x0105b0, 0x010589, 0x010589},
+ {0x01058a, 0x0105b1, 0x01058a, 0x01058a},
+ {0x01058c, 0x0105b3, 0x01058c, 0x01058c},
+ {0x01058d, 0x0105b4, 0x01058d, 0x01058d},
+ {0x01058e, 0x0105b5, 0x01058e, 0x01058e},
+ {0x01058f, 0x0105b6, 0x01058f, 0x01058f},
+ {0x010590, 0x0105b7, 0x010590, 0x010590},
+ {0x010591, 0x0105b8, 0x010591, 0x010591},
+ {0x010592, 0x0105b9, 0x010592, 0x010592},
+ {0x010594, 0x0105bb, 0x010594, 0x010594},
+ {0x010595, 0x0105bc, 0x010595, 0x010595},
+ {0x010597, 0x010597, 0x010570, 0x010570},
+ {0x010598, 0x010598, 0x010571, 0x010571},
+ {0x010599, 0x010599, 0x010572, 0x010572},
+ {0x01059a, 0x01059a, 0x010573, 0x010573},
+ {0x01059b, 0x01059b, 0x010574, 0x010574},
+ {0x01059c, 0x01059c, 0x010575, 0x010575},
+ {0x01059d, 0x01059d, 0x010576, 0x010576},
+ {0x01059e, 0x01059e, 0x010577, 0x010577},
+ {0x01059f, 0x01059f, 0x010578, 0x010578},
+ {0x0105a0, 0x0105a0, 0x010579, 0x010579},
+ {0x0105a1, 0x0105a1, 0x01057a, 0x01057a},
+ {0x0105a3, 0x0105a3, 0x01057c, 0x01057c},
+ {0x0105a4, 0x0105a4, 0x01057d, 0x01057d},
+ {0x0105a5, 0x0105a5, 0x01057e, 0x01057e},
+ {0x0105a6, 0x0105a6, 0x01057f, 0x01057f},
+ {0x0105a7, 0x0105a7, 0x010580, 0x010580},
+ {0x0105a8, 0x0105a8, 0x010581, 0x010581},
+ {0x0105a9, 0x0105a9, 0x010582, 0x010582},
+ {0x0105aa, 0x0105aa, 0x010583, 0x010583},
+ {0x0105ab, 0x0105ab, 0x010584, 0x010584},
+ {0x0105ac, 0x0105ac, 0x010585, 0x010585},
+ {0x0105ad, 0x0105ad, 0x010586, 0x010586},
+ {0x0105ae, 0x0105ae, 0x010587, 0x010587},
+ {0x0105af, 0x0105af, 0x010588, 0x010588},
+ {0x0105b0, 0x0105b0, 0x010589, 0x010589},
+ {0x0105b1, 0x0105b1, 0x01058a, 0x01058a},
+ {0x0105b3, 0x0105b3, 0x01058c, 0x01058c},
+ {0x0105b4, 0x0105b4, 0x01058d, 0x01058d},
+ {0x0105b5, 0x0105b5, 0x01058e, 0x01058e},
+ {0x0105b6, 0x0105b6, 0x01058f, 0x01058f},
+ {0x0105b7, 0x0105b7, 0x010590, 0x010590},
+ {0x0105b8, 0x0105b8, 0x010591, 0x010591},
+ {0x0105b9, 0x0105b9, 0x010592, 0x010592},
+ {0x0105bb, 0x0105bb, 0x010594, 0x010594},
+ {0x0105bc, 0x0105bc, 0x010595, 0x010595},
+ {0x010c80, 0x010cc0, 0x010c80, 0x010c80},
+ {0x010c81, 0x010cc1, 0x010c81, 0x010c81},
+ {0x010c82, 0x010cc2, 0x010c82, 0x010c82},
+ {0x010c83, 0x010cc3, 0x010c83, 0x010c83},
+ {0x010c84, 0x010cc4, 0x010c84, 0x010c84},
+ {0x010c85, 0x010cc5, 0x010c85, 0x010c85},
+ {0x010c86, 0x010cc6, 0x010c86, 0x010c86},
+ {0x010c87, 0x010cc7, 0x010c87, 0x010c87},
+ {0x010c88, 0x010cc8, 0x010c88, 0x010c88},
+ {0x010c89, 0x010cc9, 0x010c89, 0x010c89},
+ {0x010c8a, 0x010cca, 0x010c8a, 0x010c8a},
+ {0x010c8b, 0x010ccb, 0x010c8b, 0x010c8b},
+ {0x010c8c, 0x010ccc, 0x010c8c, 0x010c8c},
+ {0x010c8d, 0x010ccd, 0x010c8d, 0x010c8d},
+ {0x010c8e, 0x010cce, 0x010c8e, 0x010c8e},
+ {0x010c8f, 0x010ccf, 0x010c8f, 0x010c8f},
+ {0x010c90, 0x010cd0, 0x010c90, 0x010c90},
+ {0x010c91, 0x010cd1, 0x010c91, 0x010c91},
+ {0x010c92, 0x010cd2, 0x010c92, 0x010c92},
+ {0x010c93, 0x010cd3, 0x010c93, 0x010c93},
+ {0x010c94, 0x010cd4, 0x010c94, 0x010c94},
+ {0x010c95, 0x010cd5, 0x010c95, 0x010c95},
+ {0x010c96, 0x010cd6, 0x010c96, 0x010c96},
+ {0x010c97, 0x010cd7, 0x010c97, 0x010c97},
+ {0x010c98, 0x010cd8, 0x010c98, 0x010c98},
+ {0x010c99, 0x010cd9, 0x010c99, 0x010c99},
+ {0x010c9a, 0x010cda, 0x010c9a, 0x010c9a},
+ {0x010c9b, 0x010cdb, 0x010c9b, 0x010c9b},
+ {0x010c9c, 0x010cdc, 0x010c9c, 0x010c9c},
+ {0x010c9d, 0x010cdd, 0x010c9d, 0x010c9d},
+ {0x010c9e, 0x010cde, 0x010c9e, 0x010c9e},
+ {0x010c9f, 0x010cdf, 0x010c9f, 0x010c9f},
+ {0x010ca0, 0x010ce0, 0x010ca0, 0x010ca0},
+ {0x010ca1, 0x010ce1, 0x010ca1, 0x010ca1},
+ {0x010ca2, 0x010ce2, 0x010ca2, 0x010ca2},
+ {0x010ca3, 0x010ce3, 0x010ca3, 0x010ca3},
+ {0x010ca4, 0x010ce4, 0x010ca4, 0x010ca4},
+ {0x010ca5, 0x010ce5, 0x010ca5, 0x010ca5},
+ {0x010ca6, 0x010ce6, 0x010ca6, 0x010ca6},
+ {0x010ca7, 0x010ce7, 0x010ca7, 0x010ca7},
+ {0x010ca8, 0x010ce8, 0x010ca8, 0x010ca8},
+ {0x010ca9, 0x010ce9, 0x010ca9, 0x010ca9},
+ {0x010caa, 0x010cea, 0x010caa, 0x010caa},
+ {0x010cab, 0x010ceb, 0x010cab, 0x010cab},
+ {0x010cac, 0x010cec, 0x010cac, 0x010cac},
+ {0x010cad, 0x010ced, 0x010cad, 0x010cad},
+ {0x010cae, 0x010cee, 0x010cae, 0x010cae},
+ {0x010caf, 0x010cef, 0x010caf, 0x010caf},
+ {0x010cb0, 0x010cf0, 0x010cb0, 0x010cb0},
+ {0x010cb1, 0x010cf1, 0x010cb1, 0x010cb1},
+ {0x010cb2, 0x010cf2, 0x010cb2, 0x010cb2},
+ {0x010cc0, 0x010cc0, 0x010c80, 0x010c80},
+ {0x010cc1, 0x010cc1, 0x010c81, 0x010c81},
+ {0x010cc2, 0x010cc2, 0x010c82, 0x010c82},
+ {0x010cc3, 0x010cc3, 0x010c83, 0x010c83},
+ {0x010cc4, 0x010cc4, 0x010c84, 0x010c84},
+ {0x010cc5, 0x010cc5, 0x010c85, 0x010c85},
+ {0x010cc6, 0x010cc6, 0x010c86, 0x010c86},
+ {0x010cc7, 0x010cc7, 0x010c87, 0x010c87},
+ {0x010cc8, 0x010cc8, 0x010c88, 0x010c88},
+ {0x010cc9, 0x010cc9, 0x010c89, 0x010c89},
+ {0x010cca, 0x010cca, 0x010c8a, 0x010c8a},
+ {0x010ccb, 0x010ccb, 0x010c8b, 0x010c8b},
+ {0x010ccc, 0x010ccc, 0x010c8c, 0x010c8c},
+ {0x010ccd, 0x010ccd, 0x010c8d, 0x010c8d},
+ {0x010cce, 0x010cce, 0x010c8e, 0x010c8e},
+ {0x010ccf, 0x010ccf, 0x010c8f, 0x010c8f},
+ {0x010cd0, 0x010cd0, 0x010c90, 0x010c90},
+ {0x010cd1, 0x010cd1, 0x010c91, 0x010c91},
+ {0x010cd2, 0x010cd2, 0x010c92, 0x010c92},
+ {0x010cd3, 0x010cd3, 0x010c93, 0x010c93},
+ {0x010cd4, 0x010cd4, 0x010c94, 0x010c94},
+ {0x010cd5, 0x010cd5, 0x010c95, 0x010c95},
+ {0x010cd6, 0x010cd6, 0x010c96, 0x010c96},
+ {0x010cd7, 0x010cd7, 0x010c97, 0x010c97},
+ {0x010cd8, 0x010cd8, 0x010c98, 0x010c98},
+ {0x010cd9, 0x010cd9, 0x010c99, 0x010c99},
+ {0x010cda, 0x010cda, 0x010c9a, 0x010c9a},
+ {0x010cdb, 0x010cdb, 0x010c9b, 0x010c9b},
+ {0x010cdc, 0x010cdc, 0x010c9c, 0x010c9c},
+ {0x010cdd, 0x010cdd, 0x010c9d, 0x010c9d},
+ {0x010cde, 0x010cde, 0x010c9e, 0x010c9e},
+ {0x010cdf, 0x010cdf, 0x010c9f, 0x010c9f},
+ {0x010ce0, 0x010ce0, 0x010ca0, 0x010ca0},
+ {0x010ce1, 0x010ce1, 0x010ca1, 0x010ca1},
+ {0x010ce2, 0x010ce2, 0x010ca2, 0x010ca2},
+ {0x010ce3, 0x010ce3, 0x010ca3, 0x010ca3},
+ {0x010ce4, 0x010ce4, 0x010ca4, 0x010ca4},
+ {0x010ce5, 0x010ce5, 0x010ca5, 0x010ca5},
+ {0x010ce6, 0x010ce6, 0x010ca6, 0x010ca6},
+ {0x010ce7, 0x010ce7, 0x010ca7, 0x010ca7},
+ {0x010ce8, 0x010ce8, 0x010ca8, 0x010ca8},
+ {0x010ce9, 0x010ce9, 0x010ca9, 0x010ca9},
+ {0x010cea, 0x010cea, 0x010caa, 0x010caa},
+ {0x010ceb, 0x010ceb, 0x010cab, 0x010cab},
+ {0x010cec, 0x010cec, 0x010cac, 0x010cac},
+ {0x010ced, 0x010ced, 0x010cad, 0x010cad},
+ {0x010cee, 0x010cee, 0x010cae, 0x010cae},
+ {0x010cef, 0x010cef, 0x010caf, 0x010caf},
+ {0x010cf0, 0x010cf0, 0x010cb0, 0x010cb0},
+ {0x010cf1, 0x010cf1, 0x010cb1, 0x010cb1},
+ {0x010cf2, 0x010cf2, 0x010cb2, 0x010cb2},
+ {0x0118a0, 0x0118c0, 0x0118a0, 0x0118a0},
+ {0x0118a1, 0x0118c1, 0x0118a1, 0x0118a1},
+ {0x0118a2, 0x0118c2, 0x0118a2, 0x0118a2},
+ {0x0118a3, 0x0118c3, 0x0118a3, 0x0118a3},
+ {0x0118a4, 0x0118c4, 0x0118a4, 0x0118a4},
+ {0x0118a5, 0x0118c5, 0x0118a5, 0x0118a5},
+ {0x0118a6, 0x0118c6, 0x0118a6, 0x0118a6},
+ {0x0118a7, 0x0118c7, 0x0118a7, 0x0118a7},
+ {0x0118a8, 0x0118c8, 0x0118a8, 0x0118a8},
+ {0x0118a9, 0x0118c9, 0x0118a9, 0x0118a9},
+ {0x0118aa, 0x0118ca, 0x0118aa, 0x0118aa},
+ {0x0118ab, 0x0118cb, 0x0118ab, 0x0118ab},
+ {0x0118ac, 0x0118cc, 0x0118ac, 0x0118ac},
+ {0x0118ad, 0x0118cd, 0x0118ad, 0x0118ad},
+ {0x0118ae, 0x0118ce, 0x0118ae, 0x0118ae},
+ {0x0118af, 0x0118cf, 0x0118af, 0x0118af},
+ {0x0118b0, 0x0118d0, 0x0118b0, 0x0118b0},
+ {0x0118b1, 0x0118d1, 0x0118b1, 0x0118b1},
+ {0x0118b2, 0x0118d2, 0x0118b2, 0x0118b2},
+ {0x0118b3, 0x0118d3, 0x0118b3, 0x0118b3},
+ {0x0118b4, 0x0118d4, 0x0118b4, 0x0118b4},
+ {0x0118b5, 0x0118d5, 0x0118b5, 0x0118b5},
+ {0x0118b6, 0x0118d6, 0x0118b6, 0x0118b6},
+ {0x0118b7, 0x0118d7, 0x0118b7, 0x0118b7},
+ {0x0118b8, 0x0118d8, 0x0118b8, 0x0118b8},
+ {0x0118b9, 0x0118d9, 0x0118b9, 0x0118b9},
+ {0x0118ba, 0x0118da, 0x0118ba, 0x0118ba},
+ {0x0118bb, 0x0118db, 0x0118bb, 0x0118bb},
+ {0x0118bc, 0x0118dc, 0x0118bc, 0x0118bc},
+ {0x0118bd, 0x0118dd, 0x0118bd, 0x0118bd},
+ {0x0118be, 0x0118de, 0x0118be, 0x0118be},
+ {0x0118bf, 0x0118df, 0x0118bf, 0x0118bf},
+ {0x0118c0, 0x0118c0, 0x0118a0, 0x0118a0},
+ {0x0118c1, 0x0118c1, 0x0118a1, 0x0118a1},
+ {0x0118c2, 0x0118c2, 0x0118a2, 0x0118a2},
+ {0x0118c3, 0x0118c3, 0x0118a3, 0x0118a3},
+ {0x0118c4, 0x0118c4, 0x0118a4, 0x0118a4},
+ {0x0118c5, 0x0118c5, 0x0118a5, 0x0118a5},
+ {0x0118c6, 0x0118c6, 0x0118a6, 0x0118a6},
+ {0x0118c7, 0x0118c7, 0x0118a7, 0x0118a7},
+ {0x0118c8, 0x0118c8, 0x0118a8, 0x0118a8},
+ {0x0118c9, 0x0118c9, 0x0118a9, 0x0118a9},
+ {0x0118ca, 0x0118ca, 0x0118aa, 0x0118aa},
+ {0x0118cb, 0x0118cb, 0x0118ab, 0x0118ab},
+ {0x0118cc, 0x0118cc, 0x0118ac, 0x0118ac},
+ {0x0118cd, 0x0118cd, 0x0118ad, 0x0118ad},
+ {0x0118ce, 0x0118ce, 0x0118ae, 0x0118ae},
+ {0x0118cf, 0x0118cf, 0x0118af, 0x0118af},
+ {0x0118d0, 0x0118d0, 0x0118b0, 0x0118b0},
+ {0x0118d1, 0x0118d1, 0x0118b1, 0x0118b1},
+ {0x0118d2, 0x0118d2, 0x0118b2, 0x0118b2},
+ {0x0118d3, 0x0118d3, 0x0118b3, 0x0118b3},
+ {0x0118d4, 0x0118d4, 0x0118b4, 0x0118b4},
+ {0x0118d5, 0x0118d5, 0x0118b5, 0x0118b5},
+ {0x0118d6, 0x0118d6, 0x0118b6, 0x0118b6},
+ {0x0118d7, 0x0118d7, 0x0118b7, 0x0118b7},
+ {0x0118d8, 0x0118d8, 0x0118b8, 0x0118b8},
+ {0x0118d9, 0x0118d9, 0x0118b9, 0x0118b9},
+ {0x0118da, 0x0118da, 0x0118ba, 0x0118ba},
+ {0x0118db, 0x0118db, 0x0118bb, 0x0118bb},
+ {0x0118dc, 0x0118dc, 0x0118bc, 0x0118bc},
+ {0x0118dd, 0x0118dd, 0x0118bd, 0x0118bd},
+ {0x0118de, 0x0118de, 0x0118be, 0x0118be},
+ {0x0118df, 0x0118df, 0x0118bf, 0x0118bf},
+ {0x016e40, 0x016e60, 0x016e40, 0x016e40},
+ {0x016e41, 0x016e61, 0x016e41, 0x016e41},
+ {0x016e42, 0x016e62, 0x016e42, 0x016e42},
+ {0x016e43, 0x016e63, 0x016e43, 0x016e43},
+ {0x016e44, 0x016e64, 0x016e44, 0x016e44},
+ {0x016e45, 0x016e65, 0x016e45, 0x016e45},
+ {0x016e46, 0x016e66, 0x016e46, 0x016e46},
+ {0x016e47, 0x016e67, 0x016e47, 0x016e47},
+ {0x016e48, 0x016e68, 0x016e48, 0x016e48},
+ {0x016e49, 0x016e69, 0x016e49, 0x016e49},
+ {0x016e4a, 0x016e6a, 0x016e4a, 0x016e4a},
+ {0x016e4b, 0x016e6b, 0x016e4b, 0x016e4b},
+ {0x016e4c, 0x016e6c, 0x016e4c, 0x016e4c},
+ {0x016e4d, 0x016e6d, 0x016e4d, 0x016e4d},
+ {0x016e4e, 0x016e6e, 0x016e4e, 0x016e4e},
+ {0x016e4f, 0x016e6f, 0x016e4f, 0x016e4f},
+ {0x016e50, 0x016e70, 0x016e50, 0x016e50},
+ {0x016e51, 0x016e71, 0x016e51, 0x016e51},
+ {0x016e52, 0x016e72, 0x016e52, 0x016e52},
+ {0x016e53, 0x016e73, 0x016e53, 0x016e53},
+ {0x016e54, 0x016e74, 0x016e54, 0x016e54},
+ {0x016e55, 0x016e75, 0x016e55, 0x016e55},
+ {0x016e56, 0x016e76, 0x016e56, 0x016e56},
+ {0x016e57, 0x016e77, 0x016e57, 0x016e57},
+ {0x016e58, 0x016e78, 0x016e58, 0x016e58},
+ {0x016e59, 0x016e79, 0x016e59, 0x016e59},
+ {0x016e5a, 0x016e7a, 0x016e5a, 0x016e5a},
+ {0x016e5b, 0x016e7b, 0x016e5b, 0x016e5b},
+ {0x016e5c, 0x016e7c, 0x016e5c, 0x016e5c},
+ {0x016e5d, 0x016e7d, 0x016e5d, 0x016e5d},
+ {0x016e5e, 0x016e7e, 0x016e5e, 0x016e5e},
+ {0x016e5f, 0x016e7f, 0x016e5f, 0x016e5f},
+ {0x016e60, 0x016e60, 0x016e40, 0x016e40},
+ {0x016e61, 0x016e61, 0x016e41, 0x016e41},
+ {0x016e62, 0x016e62, 0x016e42, 0x016e42},
+ {0x016e63, 0x016e63, 0x016e43, 0x016e43},
+ {0x016e64, 0x016e64, 0x016e44, 0x016e44},
+ {0x016e65, 0x016e65, 0x016e45, 0x016e45},
+ {0x016e66, 0x016e66, 0x016e46, 0x016e46},
+ {0x016e67, 0x016e67, 0x016e47, 0x016e47},
+ {0x016e68, 0x016e68, 0x016e48, 0x016e48},
+ {0x016e69, 0x016e69, 0x016e49, 0x016e49},
+ {0x016e6a, 0x016e6a, 0x016e4a, 0x016e4a},
+ {0x016e6b, 0x016e6b, 0x016e4b, 0x016e4b},
+ {0x016e6c, 0x016e6c, 0x016e4c, 0x016e4c},
+ {0x016e6d, 0x016e6d, 0x016e4d, 0x016e4d},
+ {0x016e6e, 0x016e6e, 0x016e4e, 0x016e4e},
+ {0x016e6f, 0x016e6f, 0x016e4f, 0x016e4f},
+ {0x016e70, 0x016e70, 0x016e50, 0x016e50},
+ {0x016e71, 0x016e71, 0x016e51, 0x016e51},
+ {0x016e72, 0x016e72, 0x016e52, 0x016e52},
+ {0x016e73, 0x016e73, 0x016e53, 0x016e53},
+ {0x016e74, 0x016e74, 0x016e54, 0x016e54},
+ {0x016e75, 0x016e75, 0x016e55, 0x016e55},
+ {0x016e76, 0x016e76, 0x016e56, 0x016e56},
+ {0x016e77, 0x016e77, 0x016e57, 0x016e57},
+ {0x016e78, 0x016e78, 0x016e58, 0x016e58},
+ {0x016e79, 0x016e79, 0x016e59, 0x016e59},
+ {0x016e7a, 0x016e7a, 0x016e5a, 0x016e5a},
+ {0x016e7b, 0x016e7b, 0x016e5b, 0x016e5b},
+ {0x016e7c, 0x016e7c, 0x016e5c, 0x016e5c},
+ {0x016e7d, 0x016e7d, 0x016e5d, 0x016e5d},
+ {0x016e7e, 0x016e7e, 0x016e5e, 0x016e5e},
+ {0x016e7f, 0x016e7f, 0x016e5f, 0x016e5f},
+ {0x01e900, 0x01e922, 0x01e900, 0x01e900},
+ {0x01e901, 0x01e923, 0x01e901, 0x01e901},
+ {0x01e902, 0x01e924, 0x01e902, 0x01e902},
+ {0x01e903, 0x01e925, 0x01e903, 0x01e903},
+ {0x01e904, 0x01e926, 0x01e904, 0x01e904},
+ {0x01e905, 0x01e927, 0x01e905, 0x01e905},
+ {0x01e906, 0x01e928, 0x01e906, 0x01e906},
+ {0x01e907, 0x01e929, 0x01e907, 0x01e907},
+ {0x01e908, 0x01e92a, 0x01e908, 0x01e908},
+ {0x01e909, 0x01e92b, 0x01e909, 0x01e909},
+ {0x01e90a, 0x01e92c, 0x01e90a, 0x01e90a},
+ {0x01e90b, 0x01e92d, 0x01e90b, 0x01e90b},
+ {0x01e90c, 0x01e92e, 0x01e90c, 0x01e90c},
+ {0x01e90d, 0x01e92f, 0x01e90d, 0x01e90d},
+ {0x01e90e, 0x01e930, 0x01e90e, 0x01e90e},
+ {0x01e90f, 0x01e931, 0x01e90f, 0x01e90f},
+ {0x01e910, 0x01e932, 0x01e910, 0x01e910},
+ {0x01e911, 0x01e933, 0x01e911, 0x01e911},
+ {0x01e912, 0x01e934, 0x01e912, 0x01e912},
+ {0x01e913, 0x01e935, 0x01e913, 0x01e913},
+ {0x01e914, 0x01e936, 0x01e914, 0x01e914},
+ {0x01e915, 0x01e937, 0x01e915, 0x01e915},
+ {0x01e916, 0x01e938, 0x01e916, 0x01e916},
+ {0x01e917, 0x01e939, 0x01e917, 0x01e917},
+ {0x01e918, 0x01e93a, 0x01e918, 0x01e918},
+ {0x01e919, 0x01e93b, 0x01e919, 0x01e919},
+ {0x01e91a, 0x01e93c, 0x01e91a, 0x01e91a},
+ {0x01e91b, 0x01e93d, 0x01e91b, 0x01e91b},
+ {0x01e91c, 0x01e93e, 0x01e91c, 0x01e91c},
+ {0x01e91d, 0x01e93f, 0x01e91d, 0x01e91d},
+ {0x01e91e, 0x01e940, 0x01e91e, 0x01e91e},
+ {0x01e91f, 0x01e941, 0x01e91f, 0x01e91f},
+ {0x01e920, 0x01e942, 0x01e920, 0x01e920},
+ {0x01e921, 0x01e943, 0x01e921, 0x01e921},
+ {0x01e922, 0x01e922, 0x01e900, 0x01e900},
+ {0x01e923, 0x01e923, 0x01e901, 0x01e901},
+ {0x01e924, 0x01e924, 0x01e902, 0x01e902},
+ {0x01e925, 0x01e925, 0x01e903, 0x01e903},
+ {0x01e926, 0x01e926, 0x01e904, 0x01e904},
+ {0x01e927, 0x01e927, 0x01e905, 0x01e905},
+ {0x01e928, 0x01e928, 0x01e906, 0x01e906},
+ {0x01e929, 0x01e929, 0x01e907, 0x01e907},
+ {0x01e92a, 0x01e92a, 0x01e908, 0x01e908},
+ {0x01e92b, 0x01e92b, 0x01e909, 0x01e909},
+ {0x01e92c, 0x01e92c, 0x01e90a, 0x01e90a},
+ {0x01e92d, 0x01e92d, 0x01e90b, 0x01e90b},
+ {0x01e92e, 0x01e92e, 0x01e90c, 0x01e90c},
+ {0x01e92f, 0x01e92f, 0x01e90d, 0x01e90d},
+ {0x01e930, 0x01e930, 0x01e90e, 0x01e90e},
+ {0x01e931, 0x01e931, 0x01e90f, 0x01e90f},
+ {0x01e932, 0x01e932, 0x01e910, 0x01e910},
+ {0x01e933, 0x01e933, 0x01e911, 0x01e911},
+ {0x01e934, 0x01e934, 0x01e912, 0x01e912},
+ {0x01e935, 0x01e935, 0x01e913, 0x01e913},
+ {0x01e936, 0x01e936, 0x01e914, 0x01e914},
+ {0x01e937, 0x01e937, 0x01e915, 0x01e915},
+ {0x01e938, 0x01e938, 0x01e916, 0x01e916},
+ {0x01e939, 0x01e939, 0x01e917, 0x01e917},
+ {0x01e93a, 0x01e93a, 0x01e918, 0x01e918},
+ {0x01e93b, 0x01e93b, 0x01e919, 0x01e919},
+ {0x01e93c, 0x01e93c, 0x01e91a, 0x01e91a},
+ {0x01e93d, 0x01e93d, 0x01e91b, 0x01e91b},
+ {0x01e93e, 0x01e93e, 0x01e91c, 0x01e91c},
+ {0x01e93f, 0x01e93f, 0x01e91d, 0x01e91d},
+ {0x01e940, 0x01e940, 0x01e91e, 0x01e91e},
+ {0x01e941, 0x01e941, 0x01e91f, 0x01e91f},
+ {0x01e942, 0x01e942, 0x01e920, 0x01e920},
+ {0x01e943, 0x01e943, 0x01e921, 0x01e921},
+};
--
2.34.1
[text/x-patch] v17-0003-Catalog-changes-preparing-for-builtin-collation-.patch (48.5K, ../../[email protected]/4-v17-0003-Catalog-changes-preparing-for-builtin-collation-.patch)
download | inline diff:
From dba9413ead941c64fe8498290c9bafeeb66bd457 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Tue, 26 Dec 2023 13:32:48 -0800
Subject: [PATCH v17 3/4] Catalog changes preparing for builtin collation
provider.
daticulocale -> datlocale, colliculocale -> colllocale.
---
doc/src/sgml/bki.sgml | 2 +-
doc/src/sgml/catalogs.sgml | 8 +--
src/backend/catalog/pg_collation.c | 10 +--
src/backend/commands/collationcmds.c | 34 +++++-----
src/backend/commands/dbcommands.c | 68 +++++++++----------
src/backend/utils/adt/pg_locale.c | 4 +-
src/backend/utils/init/postinit.c | 12 ++--
src/bin/initdb/initdb.c | 32 ++++-----
src/bin/pg_dump/pg_dump.c | 56 ++++++++-------
src/bin/pg_upgrade/info.c | 31 ++++++---
src/bin/pg_upgrade/pg_upgrade.c | 34 +++++++---
src/bin/pg_upgrade/pg_upgrade.h | 2 +-
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 27 +++++---
src/bin/psql/describe.c | 20 ++++--
src/include/catalog/pg_collation.dat | 2 +-
src/include/catalog/pg_collation.h | 4 +-
src/include/catalog/pg_database.dat | 2 +-
src/include/catalog/pg_database.h | 2 +-
.../regress/expected/collate.icu.utf8.out | 4 +-
src/test/regress/expected/psql.out | 18 ++---
src/test/regress/sql/collate.icu.utf8.sql | 4 +-
21 files changed, 211 insertions(+), 165 deletions(-)
diff --git a/doc/src/sgml/bki.sgml b/doc/src/sgml/bki.sgml
index 315ba81951..3cd5bee7ff 100644
--- a/doc/src/sgml/bki.sgml
+++ b/doc/src/sgml/bki.sgml
@@ -186,7 +186,7 @@
datlocprovider => 'LOCALE_PROVIDER', datistemplate => 't',
datallowconn => 't', dathasloginevt => 'f', datconnlimit => '-1', datfrozenxid => '0',
datminmxid => '1', dattablespace => 'pg_default', datcollate => 'LC_COLLATE',
- datctype => 'LC_CTYPE', daticulocale => 'ICU_LOCALE', datacl => '_null_' },
+ datctype => 'LC_CTYPE', datlocale => 'DATLOCALE', datacl => '_null_' },
]
]]></programlisting>
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 880f717b10..734b1cf74f 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2422,10 +2422,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>colliculocale</structfield> <type>text</type>
+ <structfield>colllocale</structfield> <type>text</type>
</para>
<para>
- ICU locale ID for this collation object
+ Locale name for builtin or ICU provider
</para></entry>
</row>
@@ -3131,10 +3131,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>daticulocale</structfield> <type>text</type>
+ <structfield>datlocale</structfield> <type>text</type>
</para>
<para>
- ICU locale ID for this database
+ Locale name for builtin or ICU provider
</para></entry>
</row>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5c8ccb8b3b..7bad94f908 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,7 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *colliculocale,
+ const char *colllocale,
const char *collicurules,
const char *collversion,
bool if_not_exists,
@@ -68,7 +68,7 @@ CollationCreate(const char *collname, Oid collnamespace,
Assert(collname);
Assert(collnamespace);
Assert(collowner);
- Assert((collcollate && collctype) || colliculocale);
+ Assert((collcollate && collctype) || colllocale);
/*
* Make sure there is no existing collation of same name & encoding.
@@ -191,10 +191,10 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collctype - 1] = CStringGetTextDatum(collctype);
else
nulls[Anum_pg_collation_collctype - 1] = true;
- if (colliculocale)
- values[Anum_pg_collation_colliculocale - 1] = CStringGetTextDatum(colliculocale);
+ if (colllocale)
+ values[Anum_pg_collation_colllocale - 1] = CStringGetTextDatum(colllocale);
else
- nulls[Anum_pg_collation_colliculocale - 1] = true;
+ nulls[Anum_pg_collation_colllocale - 1] = true;
if (collicurules)
values[Anum_pg_collation_collicurules - 1] = CStringGetTextDatum(collicurules);
else
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 58c059fdb7..27564e569a 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *versionEl = NULL;
char *collcollate;
char *collctype;
- char *colliculocale;
+ char *colllocale;
char *collicurules;
bool collisdeterministic;
int collencoding;
@@ -159,11 +159,11 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
else
collctype = NULL;
- datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_colliculocale, &isnull);
+ datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_colllocale, &isnull);
if (!isnull)
- colliculocale = TextDatumGetCString(datum);
+ colllocale = TextDatumGetCString(datum);
else
- colliculocale = NULL;
+ colllocale = NULL;
/*
* When the ICU locale comes from an existing collation, do not
@@ -196,7 +196,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collcollate = NULL;
collctype = NULL;
- colliculocale = NULL;
+ colllocale = NULL;
collicurules = NULL;
if (providerEl)
@@ -236,7 +236,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collctype = defGetString(localeEl);
}
else
- colliculocale = defGetString(localeEl);
+ colllocale = defGetString(localeEl);
}
if (lccollateEl)
@@ -261,7 +261,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
else if (collprovider == COLLPROVIDER_ICU)
{
- if (!colliculocale)
+ if (!colllocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("parameter \"%s\" must be specified",
@@ -273,20 +273,20 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
*/
if (!IsBinaryUpgrade)
{
- char *langtag = icu_language_tag(colliculocale,
+ char *langtag = icu_language_tag(colllocale,
icu_validation_level);
- if (langtag && strcmp(colliculocale, langtag) != 0)
+ if (langtag && strcmp(colllocale, langtag) != 0)
{
ereport(NOTICE,
(errmsg("using standard form \"%s\" for ICU locale \"%s\"",
- langtag, colliculocale)));
+ langtag, colllocale)));
- colliculocale = langtag;
+ colllocale = langtag;
}
}
- icu_validate_locale(colliculocale);
+ icu_validate_locale(colllocale);
}
/*
@@ -334,7 +334,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
if (!collversion)
- collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colliculocale : collcollate);
+ collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colllocale : collcollate);
newoid = CollationCreate(collName,
collNamespace,
@@ -344,7 +344,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- colliculocale,
+ colllocale,
collicurules,
collversion,
if_not_exists,
@@ -435,7 +435,7 @@ AlterCollation(AlterCollationStmt *stmt)
datum = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion, &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
newversion = get_collation_actual_version(collForm->collprovider, TextDatumGetCString(datum));
/* cannot change from NULL to non-NULL or vice versa */
@@ -502,7 +502,7 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup,
provider == COLLPROVIDER_ICU ?
- Anum_pg_database_daticulocale : Anum_pg_database_datcollate);
+ Anum_pg_database_datlocale : Anum_pg_database_datcollate);
locale = TextDatumGetCString(datum);
@@ -523,7 +523,7 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
Assert(provider != COLLPROVIDER_DEFAULT);
datum = SysCacheGetAttrNotNull(COLLOID, colltp,
provider == COLLPROVIDER_ICU ?
- Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
locale = TextDatumGetCString(datum);
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index b1327de71e..d1de46e759 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -118,7 +118,7 @@ static bool get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, bool *dbHasLoginEvtP,
TransactionId *dbFrozenXidP, MultiXactId *dbMinMultiP,
- Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbIculocale,
+ Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbLocale,
char **dbIcurules,
char *dbLocProvider,
char **dbCollversion);
@@ -675,7 +675,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
int src_encoding = -1;
char *src_collate = NULL;
char *src_ctype = NULL;
- char *src_iculocale = NULL;
+ char *src_locale = NULL;
char *src_icurules = NULL;
char src_locprovider = '\0';
char *src_collversion = NULL;
@@ -713,7 +713,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
const char *dbtemplate = NULL;
char *dbcollate = NULL;
char *dbctype = NULL;
- char *dbiculocale = NULL;
+ char *dblocale = NULL;
char *dbicurules = NULL;
char dblocprovider = '\0';
char *canonname;
@@ -903,7 +903,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
if (dctype && dctype->arg)
dbctype = defGetString(dctype);
if (diculocale && diculocale->arg)
- dbiculocale = defGetString(diculocale);
+ dblocale = defGetString(diculocale);
if (dicurules && dicurules->arg)
dbicurules = defGetString(dicurules);
if (dlocprovider && dlocprovider->arg)
@@ -971,7 +971,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
&src_dboid, &src_owner, &src_encoding,
&src_istemplate, &src_allowconn, &src_hasloginevt,
&src_frozenxid, &src_minmxid, &src_deftablespace,
- &src_collate, &src_ctype, &src_iculocale, &src_icurules, &src_locprovider,
+ &src_collate, &src_ctype, &src_locale, &src_icurules, &src_locprovider,
&src_collversion))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_DATABASE),
@@ -1027,12 +1027,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
dbctype = src_ctype;
if (dblocprovider == '\0')
dblocprovider = src_locprovider;
- if (dbiculocale == NULL && dblocprovider == COLLPROVIDER_ICU)
+ if (dblocale == NULL && dblocprovider == COLLPROVIDER_ICU)
{
if (dlocale && dlocale->arg)
- dbiculocale = defGetString(dlocale);
+ dblocale = defGetString(dlocale);
else
- dbiculocale = src_iculocale;
+ dblocale = src_locale;
}
if (dbicurules == NULL && dblocprovider == COLLPROVIDER_ICU)
dbicurules = src_icurules;
@@ -1071,7 +1071,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* This would happen if template0 uses the libc provider but the new
* database uses icu.
*/
- if (!dbiculocale)
+ if (!dblocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("LOCALE or ICU_LOCALE must be specified")));
@@ -1081,26 +1081,26 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* database, preserve locale string. Otherwise, canonicalize to a
* language tag.
*/
- if (!IsBinaryUpgrade && dbiculocale != src_iculocale)
+ if (!IsBinaryUpgrade && dblocale != src_locale)
{
- char *langtag = icu_language_tag(dbiculocale,
+ char *langtag = icu_language_tag(dblocale,
icu_validation_level);
- if (langtag && strcmp(dbiculocale, langtag) != 0)
+ if (langtag && strcmp(dblocale, langtag) != 0)
{
ereport(NOTICE,
(errmsg("using standard form \"%s\" for ICU locale \"%s\"",
- langtag, dbiculocale)));
+ langtag, dblocale)));
- dbiculocale = langtag;
+ dblocale = langtag;
}
}
- icu_validate_locale(dbiculocale);
+ icu_validate_locale(dblocale);
}
else
{
- if (dbiculocale)
+ if (dblocale)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU locale cannot be specified unless locale provider is ICU")));
@@ -1157,13 +1157,13 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
char *val1;
char *val2;
- Assert(dbiculocale);
- Assert(src_iculocale);
- if (strcmp(dbiculocale, src_iculocale) != 0)
+ Assert(dblocale);
+ Assert(src_locale);
+ if (strcmp(dblocale, src_locale) != 0)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("new ICU locale (%s) is incompatible with the ICU locale of the template database (%s)",
- dbiculocale, src_iculocale),
+ dblocale, src_locale),
errhint("Use the same ICU locale as in the template database, or use template0 as template.")));
val1 = dbicurules;
@@ -1197,7 +1197,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
char *actual_versionstr;
- actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dbiculocale : dbcollate);
+ actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
if (!actual_versionstr)
ereport(ERROR,
(errmsg("template database \"%s\" has a collation version, but no actual collation version could be determined",
@@ -1225,7 +1225,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* collation version, which is normally only the case for template0.
*/
if (dbcollversion == NULL)
- dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dbiculocale : dbcollate);
+ dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
/* Resolve default tablespace for new database */
if (dtablespacename && dtablespacename->arg)
@@ -1364,8 +1364,8 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* block on the unique index, and fail after we commit).
*/
- Assert((dblocprovider == COLLPROVIDER_ICU && dbiculocale) ||
- (dblocprovider != COLLPROVIDER_ICU && !dbiculocale));
+ Assert((dblocprovider == COLLPROVIDER_ICU && dblocale) ||
+ (dblocprovider != COLLPROVIDER_ICU && !dblocale));
/* Form tuple */
new_record[Anum_pg_database_oid - 1] = ObjectIdGetDatum(dboid);
@@ -1383,10 +1383,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
new_record[Anum_pg_database_dattablespace - 1] = ObjectIdGetDatum(dst_deftablespace);
new_record[Anum_pg_database_datcollate - 1] = CStringGetTextDatum(dbcollate);
new_record[Anum_pg_database_datctype - 1] = CStringGetTextDatum(dbctype);
- if (dbiculocale)
- new_record[Anum_pg_database_daticulocale - 1] = CStringGetTextDatum(dbiculocale);
+ if (dblocale)
+ new_record[Anum_pg_database_datlocale - 1] = CStringGetTextDatum(dblocale);
else
- new_record_nulls[Anum_pg_database_daticulocale - 1] = true;
+ new_record_nulls[Anum_pg_database_datlocale - 1] = true;
if (dbicurules)
new_record[Anum_pg_database_daticurules - 1] = CStringGetTextDatum(dbicurules);
else
@@ -2472,7 +2472,7 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
datum = heap_getattr(tuple, Anum_pg_database_datcollversion, RelationGetDescr(rel), &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_daticulocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
+ datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
if (isnull)
elog(ERROR, "unexpected null in pg_database");
newversion = get_collation_actual_version(datForm->datlocprovider, TextDatumGetCString(datum));
@@ -2670,7 +2670,7 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
datlocprovider = ((Form_pg_database) GETSTRUCT(tp))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_daticulocale : Anum_pg_database_datcollate);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate);
version = get_collation_actual_version(datlocprovider, TextDatumGetCString(datum));
ReleaseSysCache(tp);
@@ -2697,7 +2697,7 @@ get_db_info(const char *name, LOCKMODE lockmode,
Oid *dbIdP, Oid *ownerIdP,
int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, bool *dbHasLoginEvtP,
TransactionId *dbFrozenXidP, MultiXactId *dbMinMultiP,
- Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbIculocale,
+ Oid *dbTablespace, char **dbCollate, char **dbCtype, char **dbLocale,
char **dbIcurules,
char *dbLocProvider,
char **dbCollversion)
@@ -2808,13 +2808,13 @@ get_db_info(const char *name, LOCKMODE lockmode,
datum = SysCacheGetAttrNotNull(DATABASEOID, tuple, Anum_pg_database_datctype);
*dbCtype = TextDatumGetCString(datum);
}
- if (dbIculocale)
+ if (dbLocale)
{
- datum = SysCacheGetAttr(DATABASEOID, tuple, Anum_pg_database_daticulocale, &isnull);
+ datum = SysCacheGetAttr(DATABASEOID, tuple, Anum_pg_database_datlocale, &isnull);
if (isnull)
- *dbIculocale = NULL;
+ *dbLocale = NULL;
else
- *dbIculocale = TextDatumGetCString(datum);
+ *dbLocale = TextDatumGetCString(datum);
}
if (dbIcurules)
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 79b59b0af7..45fe847320 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1606,7 +1606,7 @@ pg_newlocale_from_collation(Oid collid)
const char *iculocstr;
const char *icurules;
- datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colliculocale);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
iculocstr = TextDatumGetCString(datum);
datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collicurules, &isnull);
@@ -1627,7 +1627,7 @@ pg_newlocale_from_collation(Oid collid)
collversionstr = TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate);
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
actual_versionstr = get_collation_actual_version(collform->collprovider,
TextDatumGetCString(datum));
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1ad3367159..9818077d51 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -318,7 +318,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
bool isnull;
char *collate;
char *ctype;
- char *iculocale;
+ char *datlocale;
/* Fetch our pg_database row normally, via syscache */
tup = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
@@ -427,8 +427,8 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
{
char *icurules;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_daticulocale);
- iculocale = TextDatumGetCString(datum);
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
+ datlocale = TextDatumGetCString(datum);
datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_daticurules, &isnull);
if (!isnull)
@@ -436,10 +436,10 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
else
icurules = NULL;
- make_icu_collator(iculocale, icurules, &default_locale);
+ make_icu_collator(datlocale, icurules, &default_locale);
}
else
- iculocale = NULL;
+ datlocale = NULL;
default_locale.provider = dbform->datlocprovider;
@@ -464,7 +464,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
collversionstr = TextDatumGetCString(datum);
- actual_versionstr = get_collation_actual_version(dbform->datlocprovider, dbform->datlocprovider == COLLPROVIDER_ICU ? iculocale : collate);
+ actual_versionstr = get_collation_actual_version(dbform->datlocprovider, dbform->datlocprovider == COLLPROVIDER_ICU ? datlocale : collate);
if (!actual_versionstr)
/* should not happen */
elog(WARNING,
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index ac409b0006..90f793632a 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -145,7 +145,7 @@ static char *lc_numeric = NULL;
static char *lc_time = NULL;
static char *lc_messages = NULL;
static char locale_provider = COLLPROVIDER_LIBC;
-static char *icu_locale = NULL;
+static char *datlocale = NULL;
static char *icu_rules = NULL;
static const char *default_text_search_config = NULL;
static char *username = NULL;
@@ -1515,8 +1515,8 @@ bootstrap_template1(void)
bki_lines = replace_token(bki_lines, "LC_CTYPE",
escape_quotes_bki(lc_ctype));
- bki_lines = replace_token(bki_lines, "ICU_LOCALE",
- icu_locale ? escape_quotes_bki(icu_locale) : "_null_");
+ bki_lines = replace_token(bki_lines, "DATLOCALE",
+ datlocale ? escape_quotes_bki(datlocale) : "_null_");
bki_lines = replace_token(bki_lines, "ICU_RULES",
icu_rules ? escape_quotes_bki(icu_rules) : "_null_");
@@ -2363,8 +2363,8 @@ setlocales(void)
lc_monetary = locale;
if (!lc_messages)
lc_messages = locale;
- if (!icu_locale && locale_provider == COLLPROVIDER_ICU)
- icu_locale = locale;
+ if (!datlocale && locale_provider != COLLPROVIDER_LIBC)
+ datlocale = locale;
}
/*
@@ -2395,17 +2395,17 @@ setlocales(void)
char *langtag;
/* acquire default locale from the environment, if not specified */
- if (icu_locale == NULL)
+ if (datlocale == NULL)
pg_fatal("ICU locale must be specified");
/* canonicalize to a language tag */
- langtag = icu_language_tag(icu_locale);
+ langtag = icu_language_tag(datlocale);
printf(_("Using language tag \"%s\" for ICU locale \"%s\".\n"),
- langtag, icu_locale);
- pg_free(icu_locale);
- icu_locale = langtag;
+ langtag, datlocale);
+ pg_free(datlocale);
+ datlocale = langtag;
- icu_validate_locale(icu_locale);
+ icu_validate_locale(datlocale);
/*
* In supported builds, the ICU locale ID will be opened during
@@ -2599,14 +2599,14 @@ setup_locale_encoding(void)
strcmp(lc_ctype, lc_numeric) == 0 &&
strcmp(lc_ctype, lc_monetary) == 0 &&
strcmp(lc_ctype, lc_messages) == 0 &&
- (!icu_locale || strcmp(lc_ctype, icu_locale) == 0))
+ (!datlocale || strcmp(lc_ctype, datlocale) == 0))
printf(_("The database cluster will be initialized with locale \"%s\".\n"), lc_ctype);
else
{
printf(_("The database cluster will be initialized with this locale configuration:\n"));
printf(_(" provider: %s\n"), collprovider_name(locale_provider));
- if (icu_locale)
- printf(_(" ICU locale: %s\n"), icu_locale);
+ if (datlocale)
+ printf(_(" ICU locale: %s\n"), datlocale);
printf(_(" LC_COLLATE: %s\n"
" LC_CTYPE: %s\n"
" LC_MESSAGES: %s\n"
@@ -3277,7 +3277,7 @@ main(int argc, char *argv[])
pg_fatal("unrecognized locale provider: %s", optarg);
break;
case 16:
- icu_locale = pg_strdup(optarg);
+ datlocale = pg_strdup(optarg);
break;
case 17:
icu_rules = pg_strdup(optarg);
@@ -3312,7 +3312,7 @@ main(int argc, char *argv[])
exit(1);
}
- if (icu_locale && locale_provider != COLLPROVIDER_ICU)
+ if (datlocale && locale_provider != COLLPROVIDER_ICU)
pg_fatal("%s cannot be specified unless locale provider \"%s\" is chosen",
"--icu-locale", "icu");
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f40bc759c5..303d1ff4a8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -2982,7 +2982,7 @@ dumpDatabase(Archive *fout)
i_datlocprovider,
i_collate,
i_ctype,
- i_daticulocale,
+ i_datlocale,
i_daticurules,
i_frozenxid,
i_minmxid,
@@ -3001,7 +3001,7 @@ dumpDatabase(Archive *fout)
*datlocprovider,
*collate,
*ctype,
- *iculocale,
+ *locale,
*icurules,
*datistemplate,
*datconnlimit,
@@ -3025,10 +3025,12 @@ dumpDatabase(Archive *fout)
appendPQExpBufferStr(dbQry, "datminmxid, ");
else
appendPQExpBufferStr(dbQry, "0 AS datminmxid, ");
- if (fout->remoteVersion >= 150000)
- appendPQExpBufferStr(dbQry, "datlocprovider, daticulocale, datcollversion, ");
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(dbQry, "datlocprovider, datlocale, datcollversion, ");
+ else if (fout->remoteVersion >= 150000)
+ appendPQExpBufferStr(dbQry, "datlocprovider, daticulocale AS datlocale, datcollversion, ");
else
- appendPQExpBufferStr(dbQry, "'c' AS datlocprovider, NULL AS daticulocale, NULL AS datcollversion, ");
+ appendPQExpBufferStr(dbQry, "'c' AS datlocprovider, NULL AS datlocale, NULL AS datcollversion, ");
if (fout->remoteVersion >= 160000)
appendPQExpBufferStr(dbQry, "daticurules, ");
else
@@ -3049,7 +3051,7 @@ dumpDatabase(Archive *fout)
i_datlocprovider = PQfnumber(res, "datlocprovider");
i_collate = PQfnumber(res, "datcollate");
i_ctype = PQfnumber(res, "datctype");
- i_daticulocale = PQfnumber(res, "daticulocale");
+ i_datlocale = PQfnumber(res, "datlocale");
i_daticurules = PQfnumber(res, "daticurules");
i_frozenxid = PQfnumber(res, "datfrozenxid");
i_minmxid = PQfnumber(res, "datminmxid");
@@ -3068,10 +3070,10 @@ dumpDatabase(Archive *fout)
datlocprovider = PQgetvalue(res, 0, i_datlocprovider);
collate = PQgetvalue(res, 0, i_collate);
ctype = PQgetvalue(res, 0, i_ctype);
- if (!PQgetisnull(res, 0, i_daticulocale))
- iculocale = PQgetvalue(res, 0, i_daticulocale);
+ if (!PQgetisnull(res, 0, i_datlocale))
+ locale = PQgetvalue(res, 0, i_datlocale);
else
- iculocale = NULL;
+ locale = NULL;
if (!PQgetisnull(res, 0, i_daticurules))
icurules = PQgetvalue(res, 0, i_daticurules);
else
@@ -3136,11 +3138,12 @@ dumpDatabase(Archive *fout)
appendStringLiteralAH(creaQry, ctype, fout);
}
}
- if (iculocale)
+ if (locale)
{
appendPQExpBufferStr(creaQry, " ICU_LOCALE = ");
- appendStringLiteralAH(creaQry, iculocale, fout);
+ appendStringLiteralAH(creaQry, locale, fout);
}
+
if (icurules)
{
appendPQExpBufferStr(creaQry, " ICU_RULES = ");
@@ -13754,12 +13757,12 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
int i_collisdeterministic;
int i_collcollate;
int i_collctype;
- int i_colliculocale;
+ int i_colllocale;
int i_collicurules;
const char *collprovider;
const char *collcollate;
const char *collctype;
- const char *colliculocale;
+ const char *colllocale;
const char *collicurules;
/* Do nothing in data-only dump */
@@ -13791,12 +13794,15 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
appendPQExpBufferStr(query,
"true AS collisdeterministic, ");
- if (fout->remoteVersion >= 150000)
+ if (fout->remoteVersion >= 170000)
+ appendPQExpBufferStr(query,
+ "colllocale, ");
+ else if (fout->remoteVersion >= 150000)
appendPQExpBufferStr(query,
- "colliculocale, ");
+ "colliculocale AS colllocale, ");
else
appendPQExpBufferStr(query,
- "NULL AS colliculocale, ");
+ "NULL AS colllocale, ");
if (fout->remoteVersion >= 160000)
appendPQExpBufferStr(query,
@@ -13818,7 +13824,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
i_collisdeterministic = PQfnumber(res, "collisdeterministic");
i_collcollate = PQfnumber(res, "collcollate");
i_collctype = PQfnumber(res, "collctype");
- i_colliculocale = PQfnumber(res, "colliculocale");
+ i_colllocale = PQfnumber(res, "colllocale");
i_collicurules = PQfnumber(res, "collicurules");
collprovider = PQgetvalue(res, 0, i_collprovider);
@@ -13845,10 +13851,10 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
collctype = NULL;
}
- if (!PQgetisnull(res, 0, i_colliculocale))
- colliculocale = PQgetvalue(res, 0, i_colliculocale);
+ if (!PQgetisnull(res, 0, i_colllocale))
+ colllocale = PQgetvalue(res, 0, i_colllocale);
else
- colliculocale = NULL;
+ colllocale = NULL;
if (!PQgetisnull(res, 0, i_collicurules))
collicurules = PQgetvalue(res, 0, i_collicurules);
@@ -13878,7 +13884,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
if (collprovider[0] == 'd')
{
- if (collcollate || collctype || colliculocale || collicurules)
+ if (collcollate || collctype || colllocale || collicurules)
pg_log_warning("invalid collation \"%s\"", qcollname);
/* no locale -- the default collation cannot be reloaded anyway */
@@ -13887,16 +13893,16 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
{
if (fout->remoteVersion >= 150000)
{
- if (collcollate || collctype || !colliculocale)
+ if (collcollate || collctype || !colllocale)
pg_log_warning("invalid collation \"%s\"", qcollname);
appendPQExpBufferStr(q, ", locale = ");
- appendStringLiteralAH(q, colliculocale ? colliculocale : "",
+ appendStringLiteralAH(q, colllocale ? colllocale : "",
fout);
}
else
{
- if (!collcollate || !collctype || colliculocale ||
+ if (!collcollate || !collctype || colllocale ||
strcmp(collcollate, collctype) != 0)
pg_log_warning("invalid collation \"%s\"", qcollname);
@@ -13912,7 +13918,7 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
}
else if (collprovider[0] == 'c')
{
- if (colliculocale || collicurules || !collcollate || !collctype)
+ if (colllocale || collicurules || !collcollate || !collctype)
pg_log_warning("invalid collation \"%s\"", qcollname);
if (collcollate && collctype && strcmp(collcollate, collctype) == 0)
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 183c2f84eb..101fe855fc 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -328,18 +328,24 @@ get_template0_info(ClusterInfo *cluster)
int i_datlocprovider;
int i_datcollate;
int i_datctype;
- int i_daticulocale;
+ int i_datlocale;
- if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
+ if (GET_MAJOR_VERSION(cluster->major_version) >= 1700)
dbres = executeQueryOrDie(conn,
"SELECT encoding, datlocprovider, "
- " datcollate, datctype, daticulocale "
+ " datcollate, datctype, datlocale "
+ "FROM pg_catalog.pg_database "
+ "WHERE datname='template0'");
+ else if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
+ dbres = executeQueryOrDie(conn,
+ "SELECT encoding, datlocprovider, "
+ " datcollate, datctype, daticulocale AS datlocale"
"FROM pg_catalog.pg_database "
"WHERE datname='template0'");
else
dbres = executeQueryOrDie(conn,
"SELECT encoding, 'c' AS datlocprovider, "
- " datcollate, datctype, NULL AS daticulocale "
+ " datcollate, datctype, NULL AS datlocale "
"FROM pg_catalog.pg_database "
"WHERE datname='template0'");
@@ -353,16 +359,16 @@ get_template0_info(ClusterInfo *cluster)
i_datlocprovider = PQfnumber(dbres, "datlocprovider");
i_datcollate = PQfnumber(dbres, "datcollate");
i_datctype = PQfnumber(dbres, "datctype");
- i_daticulocale = PQfnumber(dbres, "daticulocale");
+ i_datlocale = PQfnumber(dbres, "datlocale");
locale->db_encoding = atoi(PQgetvalue(dbres, 0, i_datencoding));
locale->db_collprovider = PQgetvalue(dbres, 0, i_datlocprovider)[0];
locale->db_collate = pg_strdup(PQgetvalue(dbres, 0, i_datcollate));
locale->db_ctype = pg_strdup(PQgetvalue(dbres, 0, i_datctype));
- if (PQgetisnull(dbres, 0, i_daticulocale))
- locale->db_iculocale = NULL;
+ if (PQgetisnull(dbres, 0, i_datlocale))
+ locale->db_locale = NULL;
else
- locale->db_iculocale = pg_strdup(PQgetvalue(dbres, 0, i_daticulocale));
+ locale->db_locale = pg_strdup(PQgetvalue(dbres, 0, i_datlocale));
cluster->template0 = locale;
@@ -392,12 +398,15 @@ get_db_infos(ClusterInfo *cluster)
snprintf(query, sizeof(query),
"SELECT d.oid, d.datname, d.encoding, d.datcollate, d.datctype, ");
- if (GET_MAJOR_VERSION(cluster->major_version) < 1500)
+ if (GET_MAJOR_VERSION(cluster->major_version) >= 1700)
+ snprintf(query + strlen(query), sizeof(query) - strlen(query),
+ "datlocprovider, datlocale, ");
+ else if (GET_MAJOR_VERSION(cluster->major_version) >= 1500)
snprintf(query + strlen(query), sizeof(query) - strlen(query),
- "'c' AS datlocprovider, NULL AS daticulocale, ");
+ "datlocprovider, daticulocale AS datlocale, ");
else
snprintf(query + strlen(query), sizeof(query) - strlen(query),
- "datlocprovider, daticulocale, ");
+ "'c' AS datlocprovider, NULL AS datlocale, ");
snprintf(query + strlen(query), sizeof(query) - strlen(query),
"pg_catalog.pg_tablespace_location(t.oid) AS spclocation "
"FROM pg_catalog.pg_database d "
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 10c94a6c1f..bb261353bd 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -391,7 +391,7 @@ setup(char *argv0, bool *live_check)
* Copy locale and encoding information into the new cluster's template0.
*
* We need to copy the encoding, datlocprovider, datcollate, datctype, and
- * daticulocale. We don't need datcollversion because that's never set for
+ * datlocale. We don't need datcollversion because that's never set for
* template0.
*/
static void
@@ -400,7 +400,7 @@ set_locale_and_encoding(void)
PGconn *conn_new_template1;
char *datcollate_literal;
char *datctype_literal;
- char *daticulocale_literal = NULL;
+ char *datlocale_literal = NULL;
DbLocaleInfo *locale = old_cluster.template0;
prep_status("Setting locale and encoding for new cluster");
@@ -414,15 +414,29 @@ set_locale_and_encoding(void)
datctype_literal = PQescapeLiteral(conn_new_template1,
locale->db_ctype,
strlen(locale->db_ctype));
- if (locale->db_iculocale)
- daticulocale_literal = PQescapeLiteral(conn_new_template1,
- locale->db_iculocale,
- strlen(locale->db_iculocale));
+ if (locale->db_locale)
+ datlocale_literal = PQescapeLiteral(conn_new_template1,
+ locale->db_locale,
+ strlen(locale->db_locale));
else
- daticulocale_literal = pg_strdup("NULL");
+ datlocale_literal = pg_strdup("NULL");
/* update template0 in new cluster */
- if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1500)
+ if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1700)
+ PQclear(executeQueryOrDie(conn_new_template1,
+ "UPDATE pg_catalog.pg_database "
+ " SET encoding = %d, "
+ " datlocprovider = '%c', "
+ " datcollate = %s, "
+ " datctype = %s, "
+ " datlocale = %s "
+ " WHERE datname = 'template0' ",
+ locale->db_encoding,
+ locale->db_collprovider,
+ datcollate_literal,
+ datctype_literal,
+ datlocale_literal));
+ else if (GET_MAJOR_VERSION(new_cluster.major_version) >= 1500)
PQclear(executeQueryOrDie(conn_new_template1,
"UPDATE pg_catalog.pg_database "
" SET encoding = %d, "
@@ -435,7 +449,7 @@ set_locale_and_encoding(void)
locale->db_collprovider,
datcollate_literal,
datctype_literal,
- daticulocale_literal));
+ datlocale_literal));
else
PQclear(executeQueryOrDie(conn_new_template1,
"UPDATE pg_catalog.pg_database "
@@ -449,7 +463,7 @@ set_locale_and_encoding(void)
PQfreemem(datcollate_literal);
PQfreemem(datctype_literal);
- PQfreemem(daticulocale_literal);
+ PQfreemem(datlocale_literal);
PQfinish(conn_new_template1);
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index d9a848cbfd..70ac424bc9 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -208,7 +208,7 @@ typedef struct
char *db_collate;
char *db_ctype;
char db_collprovider;
- char *db_iculocale;
+ char *db_locale;
int db_encoding;
} DbLocaleInfo;
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index d951ed3af0..41d06d272b 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -104,6 +104,8 @@ if ($oldnode->pg_version >= 11)
push @custom_opts, '--allow-group-access';
}
+my $oldversion = int($oldnode->pg_version =~ s/([0-9]*).*/$1/rg);
+
# Set up the locale settings for the original cluster, so that we
# can test that pg_upgrade copies the locale settings of template0
# from the old to the new cluster.
@@ -111,15 +113,22 @@ if ($oldnode->pg_version >= 11)
my $original_encoding = "6"; # UTF-8
my $original_provider = "c";
my $original_locale = "C";
-my $original_iculocale = "";
+my $original_datlocale = "";
my $provider_field = "'c' AS datlocprovider";
-my $iculocale_field = "NULL AS daticulocale";
-if ($oldnode->pg_version >= 15 && $ENV{with_icu} eq 'yes')
+my $datlocale_field = "NULL AS datlocale";
+if ($oldversion >= 15 && $ENV{with_icu} eq 'yes')
{
$provider_field = "datlocprovider";
- $iculocale_field = "daticulocale";
+ if ($oldversion >= 17)
+ {
+ $datlocale_field = "datlocale";
+ }
+ else
+ {
+ $datlocale_field = "daticulocale AS datlocale";
+ }
$original_provider = "i";
- $original_iculocale = "fr-CA";
+ $original_datlocale = "fr-CA";
}
my @initdb_params = @custom_opts;
@@ -139,10 +148,10 @@ $oldnode->start;
my $result;
$result = $oldnode->safe_psql(
'postgres',
- "SELECT encoding, $provider_field, datcollate, datctype, $iculocale_field
+ "SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_iculocale",
+ "$original_encoding|$original_provider|$original_locale|$original_locale|$original_datlocale",
"check locales in original cluster");
# The default location of the source code is the root of this directory.
@@ -422,10 +431,10 @@ if (-d $log_path)
# Test that upgraded cluster has original locale settings.
$result = $newnode->safe_psql(
'postgres',
- "SELECT encoding, $provider_field, datcollate, datctype, $iculocale_field
+ "SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_iculocale",
+ "$original_encoding|$original_provider|$original_locale|$original_locale|$original_datlocale",
"check that locales in new cluster match original cluster");
# Second dump from the upgraded instance.
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index b6a4eb1d56..b943569050 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -937,14 +937,18 @@ listAllDbs(const char *pattern, bool verbose)
" d.datctype as \"%s\",\n",
gettext_noop("Collate"),
gettext_noop("Ctype"));
- if (pset.sversion >= 150000)
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ " d.datlocale as \"%s\",\n",
+ gettext_noop("Locale"));
+ else if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
" d.daticulocale as \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
else
appendPQExpBuffer(&buf,
" NULL as \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
" d.daticurules as \"%s\",\n",
@@ -4983,14 +4987,18 @@ listCollations(const char *pattern, bool verbose, bool showSystem)
gettext_noop("Collate"),
gettext_noop("Ctype"));
- if (pset.sversion >= 150000)
+ if (pset.sversion >= 170000)
+ appendPQExpBuffer(&buf,
+ " c.colllocale AS \"%s\",\n",
+ gettext_noop("Locale"));
+ else if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
" c.colliculocale AS \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
else
appendPQExpBuffer(&buf,
" c.collcollate AS \"%s\",\n",
- gettext_noop("ICU Locale"));
+ gettext_noop("Locale"));
if (pset.sversion >= 160000)
appendPQExpBuffer(&buf,
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 10c363d2ee..7396ff10c4 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -29,6 +29,6 @@
{ oid => '963',
descr => 'sorts using the Unicode Collation Algorithm with default settings',
collname => 'unicode', collprovider => 'i', collencoding => '-1',
- colliculocale => 'und' },
+ colllocale => 'und' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 5f08eb0a4a..85cb09c4f8 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -42,7 +42,7 @@ CATALOG(pg_collation,3456,CollationRelationId)
#ifdef CATALOG_VARLEN /* variable-length fields start here */
text collcollate BKI_DEFAULT(_null_); /* LC_COLLATE setting */
text collctype BKI_DEFAULT(_null_); /* LC_CTYPE setting */
- text colliculocale BKI_DEFAULT(_null_); /* ICU locale ID */
+ text colllocale BKI_DEFAULT(_null_); /* locale ID */
text collicurules BKI_DEFAULT(_null_); /* ICU collation rules */
text collversion BKI_DEFAULT(_null_); /* provider-dependent
* version of collation
@@ -94,7 +94,7 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *colliculocale,
+ const char *colllocale,
const char *collicurules,
const char *collversion,
bool if_not_exists,
diff --git a/src/include/catalog/pg_database.dat b/src/include/catalog/pg_database.dat
index 4306e8a3e8..c2ba636f8d 100644
--- a/src/include/catalog/pg_database.dat
+++ b/src/include/catalog/pg_database.dat
@@ -18,7 +18,7 @@
datlocprovider => 'LOCALE_PROVIDER', datistemplate => 't',
datallowconn => 't', dathasloginevt => 'f', datconnlimit => '-1', datfrozenxid => '0',
datminmxid => '1', dattablespace => 'pg_default', datcollate => 'LC_COLLATE',
- datctype => 'LC_CTYPE', daticulocale => 'ICU_LOCALE',
+ datctype => 'LC_CTYPE', datlocale => 'DATLOCALE',
daticurules => 'ICU_RULES', datacl => '_null_' },
]
diff --git a/src/include/catalog/pg_database.h b/src/include/catalog/pg_database.h
index 014baa7bab..dbd4379ffa 100644
--- a/src/include/catalog/pg_database.h
+++ b/src/include/catalog/pg_database.h
@@ -75,7 +75,7 @@ CATALOG(pg_database,1262,DatabaseRelationId) BKI_SHARED_RELATION BKI_ROWTYPE_OID
text datctype BKI_FORCE_NOT_NULL;
/* ICU locale ID */
- text daticulocale;
+ text datlocale;
/* ICU collation rules */
text daticurules;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 7a05c75967..8ca93f4dea 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1024,7 +1024,7 @@ SET icu_validation_level = disabled;
do $$
BEGIN
EXECUTE 'CREATE COLLATION test0 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
CREATE COLLATION test0 FROM "C"; -- fail, duplicate name
@@ -1032,7 +1032,7 @@ ERROR: collation "test0" already exists
do $$
BEGIN
EXECUTE 'CREATE COLLATION test1 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
RESET icu_validation_level;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index ad02772562..69060fe3c0 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -6221,9 +6221,9 @@ List of schemas
(0 rows)
\dO "no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp "no.such.access.privilege"
@@ -6410,9 +6410,9 @@ cross-database references are not implemented: "no.such.schema"."no.such.languag
(0 rows)
\dO "no.such.schema"."no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp "no.such.schema"."no.such.access.privilege"
@@ -6553,9 +6553,9 @@ List of text search templates
(0 rows)
\dO regression."no.such.schema"."no.such.collation"
- List of collations
- Schema | Name | Provider | Collate | Ctype | ICU Locale | ICU Rules | Deterministic?
---------+------+----------+---------+-------+------------+-----------+----------------
+ List of collations
+ Schema | Name | Provider | Collate | Ctype | Locale | ICU Rules | Deterministic?
+--------+------+----------+---------+-------+--------+-----------+----------------
(0 rows)
\dp regression."no.such.schema"."no.such.access.privilege"
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 3db9e25913..03837de846 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -363,14 +363,14 @@ SET icu_validation_level = disabled;
do $$
BEGIN
EXECUTE 'CREATE COLLATION test0 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
CREATE COLLATION test0 FROM "C"; -- fail, duplicate name
do $$
BEGIN
EXECUTE 'CREATE COLLATION test1 (provider = icu, locale = ' ||
- quote_literal((SELECT CASE WHEN datlocprovider='i' THEN daticulocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
+ quote_literal((SELECT CASE WHEN datlocprovider='i' THEN datlocale ELSE datcollate END FROM pg_database WHERE datname = current_database())) || ');';
END
$$;
--
2.34.1
[text/x-patch] v17-0004-Introduce-collation-provider-builtin-for-C-and-C.patch (80.3K, ../../[email protected]/5-v17-0004-Introduce-collation-provider-builtin-for-C-and-C.patch)
download | inline diff:
From befdcd94fe45bf9ef6fbecdc6859b8f837502360 Mon Sep 17 00:00:00 2001
From: Jeff Davis <[email protected]>
Date: Mon, 1 May 2023 15:38:29 -0700
Subject: [PATCH v17 4/4] Introduce collation provider "builtin" for "C" and
"C.UTF-8".
The builtin "C" locale is equal (in semantics and implementation) to
the libc "C" locale.
The builtin "C.UTF-8" locale is especially useful. It provides a fast
memcmp-based collation (like "C") that supports abbrevated keys, while
also providing richer ctype semantics (upper/lower and regexes). The
semantics are derived from Unicode by building in lookup tables in the
same way as for text normalization. By using built-in semantics, the
behavior is stabilized within a Postgres major version, and also
matches the behavior of other built-in Unicode functionality, such as
normalization.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
---
doc/src/sgml/charset.sgml | 88 +++++++--
doc/src/sgml/ref/create_collation.sgml | 11 +-
doc/src/sgml/ref/create_database.sgml | 8 +-
doc/src/sgml/ref/createdb.sgml | 2 +-
doc/src/sgml/ref/initdb.sgml | 17 +-
src/backend/catalog/pg_collation.c | 5 +-
src/backend/commands/collationcmds.c | 93 ++++++++--
src/backend/commands/dbcommands.c | 121 +++++++++---
src/backend/regex/regc_pg_locale.c | 41 +++-
src/backend/utils/adt/formatting.c | 185 +++++++++++++++++++
src/backend/utils/adt/pg_locale.c | 130 +++++++++++--
src/backend/utils/init/postinit.c | 19 +-
src/bin/initdb/initdb.c | 58 +++---
src/bin/initdb/t/001_initdb.pl | 57 +++++-
src/bin/pg_dump/pg_dump.c | 49 +++--
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 70 +++++--
src/bin/psql/describe.c | 4 +-
src/bin/scripts/createdb.c | 18 +-
src/bin/scripts/t/020_createdb.pl | 78 ++++++++
src/common/wchar.c | 4 +-
src/include/catalog/pg_collation.dat | 9 +-
src/include/catalog/pg_collation.h | 5 +-
src/include/mb/pg_wchar.h | 15 ++
src/include/utils/pg_locale.h | 7 +-
src/test/icu/t/010_database.pl | 22 +--
src/test/regress/expected/collate.out | 24 ++-
src/test/regress/expected/collate.utf8.out | 109 +++++++++++
src/test/regress/expected/collate.utf8_1.out | 8 +
src/test/regress/parallel_schedule | 4 +-
src/test/regress/sql/collate.sql | 10 +
src/test/regress/sql/collate.utf8.sql | 54 ++++++
31 files changed, 1159 insertions(+), 166 deletions(-)
create mode 100644 src/test/regress/expected/collate.utf8.out
create mode 100644 src/test/regress/expected/collate.utf8_1.out
create mode 100644 src/test/regress/sql/collate.utf8.sql
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 74783d148f..1553deea20 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -342,22 +342,14 @@ initdb --locale=sv_SE
<title>Locale Providers</title>
<para>
- <productname>PostgreSQL</productname> supports multiple <firstterm>locale
- providers</firstterm>. This specifies which library supplies the locale
- data. One standard provider name is <literal>libc</literal>, which uses
- the locales provided by the operating system C library. These are the
- locales used by most tools provided by the operating system. Another
- provider is <literal>icu</literal>, which uses the external
- ICU<indexterm><primary>ICU</primary></indexterm> library. ICU locales can
- only be used if support for ICU was configured when PostgreSQL was built.
+ A locale provider specifies which library defines the locale behavior for
+ collations and character classifications.
</para>
<para>
The commands and tools that select the locale settings, as described
- above, each have an option to select the locale provider. The examples
- shown earlier all use the <literal>libc</literal> provider, which is the
- default. Here is an example to initialize a database cluster using the
- ICU provider:
+ above, each have an option to select the locale provider. Here is an
+ example to initialize a database cluster using the ICU provider:
<programlisting>
initdb --locale-provider=icu --icu-locale=en
</programlisting>
@@ -370,12 +362,74 @@ initdb --locale-provider=icu --icu-locale=en
</para>
<para>
- Which locale provider to use depends on individual requirements. For most
- basic uses, either provider will give adequate results. For the libc
- provider, it depends on what the operating system offers; some operating
- systems are better than others. For advanced uses, ICU offers more locale
- variants and customization options.
+ Regardless of the locale provider, the operating system is still used to
+ provide some locale-aware behavior, such as messages (see <xref
+ linkend="guc-lc-messages"/>).
</para>
+
+ <para>
+ The available locale providers are listed below.
+ </para>
+
+ <sect3 id="locale-provider-builtin">
+ <title>Builtin</title>
+ <para>
+ The <literal>builtin</literal> provider uses built-in operations. Only
+ the <literal>C</literal> and <literal>C.UTF-8</literal> locales are
+ supported for this provider.
+ </para>
+ <para>
+ The <literal>C</literal> locale behavior is identical to the
+ <literal>C</literal> locale in the libc provider. When using this locale,
+ the behavior may depend on the database encoding.
+ </para>
+ <para>
+ The <literal>C.UTF-8</literal> locale is available only for when the
+ database encoding is <literal>UTF-8</literal>, and the behavior is based
+ on Unicode. The collation uses the code point values only. The regular
+ expression character classes are based on the "POSIX Compatible"
+ semantics, and the case mapping is the "simple" variant.
+ </para>
+ </sect3>
+ <sect3 id="locale-provider-icu">
+ <title>ICU</title>
+ <para>
+ The <literal>icu</literal> provider uses the external
+ ICU<indexterm><primary>ICU</primary></indexterm>
+ library. <productname>PostgreSQL</productname> must have been configured
+ with support.
+ </para>
+ <para>
+ ICU provides collation and character classification behavior that is
+ independent of the operating system and database encoding, which is
+ preferable if you expect to transition to other platforms without any
+ change in results. <literal>LC_COLLATE</literal> and
+ <literal>LC_CTYPE</literal> can be set independently of the ICU locale.
+ </para>
+ <note>
+ <para>
+ For the ICU provider, results may depend on the version of the ICU
+ library used, as it is updated to reflect changes in natural language
+ over time.
+ </para>
+ </note>
+ </sect3>
+ <sect3 id="locale-provider-libc">
+ <title>libc</title>
+ <para>
+ The <literal>libc</literal> provider uses the operating system's C
+ library. The collation and character classification behavior is
+ controlled by the settings <literal>LC_COLLATE</literal> and
+ <literal>LC_CTYPE</literal>, so they cannot be set independently.
+ </para>
+ <note>
+ <para>
+ The same locale name may have different behavior on different platforms
+ when using the libc provider.
+ </para>
+ </note>
+ </sect3>
+
</sect2>
<sect2 id="icu-locales">
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 5cf9777764..85f18cbbe5 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -96,6 +96,11 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
<replaceable>locale</replaceable>, you cannot specify either of those
parameters.
</para>
+ <para>
+ If <replaceable>provider</replaceable> is <literal>builtin</literal>,
+ then <replaceable>locale</replaceable> must be specified and set to
+ either <literal>C</literal> or <literal>C.UTF-8</literal>.
+ </para>
</listitem>
</varlistentry>
@@ -129,9 +134,9 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
<listitem>
<para>
Specifies the provider to use for locale services associated with this
- collation. Possible values are
- <literal>icu</literal><indexterm><primary>ICU</primary></indexterm>
- (if the server was built with ICU support) or <literal>libc</literal>.
+ collation. Possible values are <literal>builtin</literal>,
+ <literal>icu</literal><indexterm><primary>ICU</primary></indexterm> (if
+ the server was built with ICU support) or <literal>libc</literal>.
<literal>libc</literal> is the default. See <xref
linkend="locale-providers"/> for details.
</para>
diff --git a/doc/src/sgml/ref/create_database.sgml b/doc/src/sgml/ref/create_database.sgml
index 72927960eb..1f5cdf1271 100644
--- a/doc/src/sgml/ref/create_database.sgml
+++ b/doc/src/sgml/ref/create_database.sgml
@@ -162,6 +162,12 @@ CREATE DATABASE <replaceable class="parameter">name</replaceable>
linkend="create-database-lc-ctype"/>, or <xref
linkend="create-database-icu-locale"/> individually.
</para>
+ <para>
+ If <xref linkend="create-database-locale-provider"/> is
+ <literal>builtin</literal>, then <replaceable>locale</replaceable>
+ must be specified and set to either <literal>C</literal> or
+ <literal>C.UTF-8</literal>.
+ </para>
<tip>
<para>
The other locale settings <xref linkend="guc-lc-messages"/>, <xref
@@ -243,7 +249,7 @@ CREATE DATABASE <replaceable class="parameter">name</replaceable>
<listitem>
<para>
Specifies the provider to use for the default collation in this
- database. Possible values are
+ database. Possible values are <literal>builtin</literal>,
<literal>icu</literal><indexterm><primary>ICU</primary></indexterm>
(if the server was built with ICU support) or <literal>libc</literal>.
By default, the provider is the same as that of the <xref
diff --git a/doc/src/sgml/ref/createdb.sgml b/doc/src/sgml/ref/createdb.sgml
index e4647d5ce7..d3e815f659 100644
--- a/doc/src/sgml/ref/createdb.sgml
+++ b/doc/src/sgml/ref/createdb.sgml
@@ -171,7 +171,7 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
- <term><option>--locale-provider={<literal>libc</literal>|<literal>icu</literal>}</option></term>
+ <term><option>--locale-provider={<literal>builtin</literal>|<literal>libc</literal>|<literal>icu</literal>}</option></term>
<listitem>
<para>
Specifies the locale provider for the database's default collation.
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
index cd75cae10e..08a1c2538f 100644
--- a/doc/src/sgml/ref/initdb.sgml
+++ b/doc/src/sgml/ref/initdb.sgml
@@ -286,6 +286,11 @@ PostgreSQL documentation
environment that <command>initdb</command> runs in. Locale
support is described in <xref linkend="locale"/>.
</para>
+ <para>
+ If <option>--locale-provider</option> is <literal>builtin</literal>,
+ <option>--locale</option> must be specified and set to
+ <literal>C</literal> or <literal>C.UTF-8</literal>.
+ </para>
</listitem>
</varlistentry>
@@ -314,8 +319,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-builtin-locale">
+ <term><option>--builtin-locale=<replaceable>locale</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the locale name when the builtin provider is used. Locale support
+ is described in <xref linkend="locale"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="app-initdb-option-locale-provider">
- <term><option>--locale-provider={<literal>libc</literal>|<literal>icu</literal>}</option></term>
+ <term><option>--locale-provider={<literal>builtin</literal>|<literal>libc</literal>|<literal>icu</literal>}</option></term>
<listitem>
<para>
This option sets the locale provider for databases created in the new
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 7bad94f908..01e91000af 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -68,7 +68,10 @@ CollationCreate(const char *collname, Oid collnamespace,
Assert(collname);
Assert(collnamespace);
Assert(collowner);
- Assert((collcollate && collctype) || colllocale);
+ Assert((collprovider == COLLPROVIDER_LIBC &&
+ collcollate && collctype && !colllocale) ||
+ (collprovider != COLLPROVIDER_LIBC &&
+ !collcollate && !collctype && colllocale));
/*
* Make sure there is no existing collation of same name & encoding.
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 27564e569a..0fa073496e 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,7 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *versionEl = NULL;
char *collcollate;
char *collctype;
- char *colllocale;
+ const char *colllocale;
char *collicurules;
bool collisdeterministic;
int collencoding;
@@ -215,7 +215,9 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (collproviderstr)
{
- if (pg_strcasecmp(collproviderstr, "icu") == 0)
+ if (pg_strcasecmp(collproviderstr, "builtin") == 0)
+ collprovider = COLLPROVIDER_BUILTIN;
+ else if (pg_strcasecmp(collproviderstr, "icu") == 0)
collprovider = COLLPROVIDER_ICU;
else if (pg_strcasecmp(collproviderstr, "libc") == 0)
collprovider = COLLPROVIDER_LIBC;
@@ -245,7 +247,17 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (lcctypeEl)
collctype = defGetString(lcctypeEl);
- if (collprovider == COLLPROVIDER_LIBC)
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ {
+ if (!colllocale)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("parameter \"locale\" must be specified")));
+
+ colllocale = builtin_validate_locale(GetDatabaseEncoding(),
+ colllocale);
+ }
+ else if (collprovider == COLLPROVIDER_LIBC)
{
if (!collcollate)
ereport(ERROR,
@@ -305,7 +317,17 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU rules cannot be specified unless locale provider is ICU")));
- if (collprovider == COLLPROVIDER_ICU)
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ {
+ /*
+ * Behavior may be different in different encodings, so set
+ * collencoding to the current database encoding. No validation is
+ * required, because the "builtin" provider is compatible with any
+ * encoding.
+ */
+ collencoding = GetDatabaseEncoding();
+ }
+ else if (collprovider == COLLPROVIDER_ICU)
{
#ifdef USE_ICU
/*
@@ -334,7 +356,18 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
if (!collversion)
- collversion = get_collation_actual_version(collprovider, collprovider == COLLPROVIDER_ICU ? colllocale : collcollate);
+ {
+ const char *locale;
+
+ if (collprovider == COLLPROVIDER_ICU)
+ locale = colllocale;
+ else if (collprovider == COLLPROVIDER_LIBC)
+ locale = collcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ collversion = get_collation_actual_version(collprovider, locale);
+ }
newoid = CollationCreate(collName,
collNamespace,
@@ -409,6 +442,7 @@ AlterCollation(AlterCollationStmt *stmt)
Form_pg_collation collForm;
Datum datum;
bool isnull;
+ char *locale;
char *oldversion;
char *newversion;
ObjectAddress address;
@@ -435,8 +469,20 @@ AlterCollation(AlterCollationStmt *stmt)
datum = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion, &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = SysCacheGetAttrNotNull(COLLOID, tup, collForm->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
- newversion = get_collation_actual_version(collForm->collprovider, TextDatumGetCString(datum));
+ if (collForm->collprovider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, Anum_pg_collation_colllocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (collForm->collprovider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, tup, Anum_pg_collation_collcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ newversion = get_collation_actual_version(collForm->collprovider, locale);
/* cannot change from NULL to non-NULL or vice versa */
if ((!oldversion && newversion) || (oldversion && !newversion))
@@ -500,11 +546,18 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
provider = ((Form_pg_database) GETSTRUCT(dbtup))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup,
- provider == COLLPROVIDER_ICU ?
- Anum_pg_database_datlocale : Anum_pg_database_datcollate);
-
- locale = TextDatumGetCString(datum);
+ if (provider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup, Anum_pg_database_datlocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (provider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, dbtup, Anum_pg_database_datcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
ReleaseSysCache(dbtup);
}
@@ -521,11 +574,19 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
provider = ((Form_pg_collation) GETSTRUCT(colltp))->collprovider;
Assert(provider != COLLPROVIDER_DEFAULT);
- datum = SysCacheGetAttrNotNull(COLLOID, colltp,
- provider == COLLPROVIDER_ICU ?
- Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
- locale = TextDatumGetCString(datum);
+ if (provider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, colltp, Anum_pg_collation_colllocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (provider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(COLLOID, colltp, Anum_pg_collation_collcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
ReleaseSysCache(colltp);
}
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index d1de46e759..d7a21adc5c 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -698,6 +698,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
DefElem *dtemplate = NULL;
DefElem *dencoding = NULL;
DefElem *dlocale = NULL;
+ DefElem *dbuiltinlocale = NULL;
DefElem *dcollate = NULL;
DefElem *dctype = NULL;
DefElem *diculocale = NULL;
@@ -713,7 +714,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
const char *dbtemplate = NULL;
char *dbcollate = NULL;
char *dbctype = NULL;
- char *dblocale = NULL;
+ const char *dblocale = NULL;
char *dbicurules = NULL;
char dblocprovider = '\0';
char *canonname;
@@ -762,6 +763,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
errorConflictingDefElem(defel, pstate);
dlocale = defel;
}
+ else if (strcmp(defel->defname, "builtin_locale") == 0)
+ {
+ if (dbuiltinlocale)
+ errorConflictingDefElem(defel, pstate);
+ dbuiltinlocale = defel;
+ }
else if (strcmp(defel->defname, "lc_collate") == 0)
{
if (dcollate)
@@ -897,7 +904,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
dbcollate = defGetString(dlocale);
dbctype = defGetString(dlocale);
+ dblocale = defGetString(dlocale);
}
+ if (dbuiltinlocale && dbuiltinlocale->arg)
+ dblocale = defGetString(dbuiltinlocale);
if (dcollate && dcollate->arg)
dbcollate = defGetString(dcollate);
if (dctype && dctype->arg)
@@ -910,7 +920,9 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
{
char *locproviderstr = defGetString(dlocprovider);
- if (pg_strcasecmp(locproviderstr, "icu") == 0)
+ if (pg_strcasecmp(locproviderstr, "builtin") == 0)
+ dblocprovider = COLLPROVIDER_BUILTIN;
+ else if (pg_strcasecmp(locproviderstr, "icu") == 0)
dblocprovider = COLLPROVIDER_ICU;
else if (pg_strcasecmp(locproviderstr, "libc") == 0)
dblocprovider = COLLPROVIDER_LIBC;
@@ -1027,14 +1039,9 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
dbctype = src_ctype;
if (dblocprovider == '\0')
dblocprovider = src_locprovider;
- if (dblocale == NULL && dblocprovider == COLLPROVIDER_ICU)
- {
- if (dlocale && dlocale->arg)
- dblocale = defGetString(dlocale);
- else
- dblocale = src_locale;
- }
- if (dbicurules == NULL && dblocprovider == COLLPROVIDER_ICU)
+ if (dblocale == NULL)
+ dblocale = src_locale;
+ if (dbicurules == NULL)
dbicurules = src_icurules;
/* Some encodings are client only */
@@ -1059,6 +1066,27 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
check_encoding_locale_matches(encoding, dbcollate, dbctype);
+ if (dblocprovider == COLLPROVIDER_BUILTIN)
+ {
+ /*
+ * This would happen if template0 uses the libc provider but the new
+ * database uses builtin.
+ */
+ if (!dblocale)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("LOCALE must be specified for the builtin provider")));
+
+ dblocale = builtin_validate_locale(encoding, dblocale);
+ }
+ else
+ {
+ if (dbuiltinlocale && dbuiltinlocale->arg)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("BUILTIN_LOCALE cannot be specified unless locale provider is builtin")));
+ }
+
if (dblocprovider == COLLPROVIDER_ICU)
{
if (!(is_encoding_supported_by_icu(encoding)))
@@ -1100,7 +1128,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
}
else
{
- if (dblocale)
+ if (diculocale && diculocale->arg)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("ICU locale cannot be specified unless locale provider is ICU")));
@@ -1111,6 +1139,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
errmsg("ICU rules cannot be specified unless locale provider is ICU")));
}
+ /* for libc, locale comes from datcollate and datctype */
+ if (dblocprovider == COLLPROVIDER_LIBC)
+ dblocale = NULL;
+
/*
* Check that the new encoding and locale settings match the source
* database. We insist on this because we simply copy the source data ---
@@ -1196,8 +1228,16 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
if (src_collversion && !dcollversion)
{
char *actual_versionstr;
+ const char *locale;
- actual_versionstr = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
+ if (dblocprovider == COLLPROVIDER_ICU)
+ locale = dblocale;
+ else if (dblocprovider == COLLPROVIDER_LIBC)
+ locale = dbcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ actual_versionstr = get_collation_actual_version(dblocprovider, locale);
if (!actual_versionstr)
ereport(ERROR,
(errmsg("template database \"%s\" has a collation version, but no actual collation version could be determined",
@@ -1225,7 +1265,18 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* collation version, which is normally only the case for template0.
*/
if (dbcollversion == NULL)
- dbcollversion = get_collation_actual_version(dblocprovider, dblocprovider == COLLPROVIDER_ICU ? dblocale : dbcollate);
+ {
+ const char *locale;
+
+ if (dblocprovider == COLLPROVIDER_ICU)
+ locale = dblocale;
+ else if (dblocprovider == COLLPROVIDER_LIBC)
+ locale = dbcollate;
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ dbcollversion = get_collation_actual_version(dblocprovider, locale);
+ }
/* Resolve default tablespace for new database */
if (dtablespacename && dtablespacename->arg)
@@ -1364,8 +1415,8 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
* block on the unique index, and fail after we commit).
*/
- Assert((dblocprovider == COLLPROVIDER_ICU && dblocale) ||
- (dblocprovider != COLLPROVIDER_ICU && !dblocale));
+ Assert((dblocprovider != COLLPROVIDER_LIBC && dblocale) ||
+ (dblocprovider == COLLPROVIDER_LIBC && !dblocale));
/* Form tuple */
new_record[Anum_pg_database_oid - 1] = ObjectIdGetDatum(dboid);
@@ -2446,6 +2497,7 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
ObjectAddress address;
Datum datum;
bool isnull;
+ char *locale;
char *oldversion;
char *newversion;
@@ -2472,10 +2524,24 @@ AlterDatabaseRefreshColl(AlterDatabaseRefreshCollStmt *stmt)
datum = heap_getattr(tuple, Anum_pg_database_datcollversion, RelationGetDescr(rel), &isnull);
oldversion = isnull ? NULL : TextDatumGetCString(datum);
- datum = heap_getattr(tuple, datForm->datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
- if (isnull)
- elog(ERROR, "unexpected null in pg_database");
- newversion = get_collation_actual_version(datForm->datlocprovider, TextDatumGetCString(datum));
+ if (datForm->datlocprovider == COLLPROVIDER_ICU)
+ {
+ datum = heap_getattr(tuple, Anum_pg_database_datlocale, RelationGetDescr(rel), &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected null in pg_database");
+ locale = TextDatumGetCString(datum);
+ }
+ else if (datForm->datlocprovider == COLLPROVIDER_LIBC)
+ {
+ datum = heap_getattr(tuple, Anum_pg_database_datcollate, RelationGetDescr(rel), &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected null in pg_database");
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ newversion = get_collation_actual_version(datForm->datlocprovider, locale);
/* cannot change from NULL to non-NULL or vice versa */
if ((!oldversion && newversion) || (oldversion && !newversion))
@@ -2660,6 +2726,7 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
HeapTuple tp;
char datlocprovider;
Datum datum;
+ char *locale;
char *version;
tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(dbid));
@@ -2670,8 +2737,20 @@ pg_database_collation_actual_version(PG_FUNCTION_ARGS)
datlocprovider = ((Form_pg_database) GETSTRUCT(tp))->datlocprovider;
- datum = SysCacheGetAttrNotNull(DATABASEOID, tp, datlocprovider == COLLPROVIDER_ICU ? Anum_pg_database_datlocale : Anum_pg_database_datcollate);
- version = get_collation_actual_version(datlocprovider, TextDatumGetCString(datum));
+ if (datlocprovider == COLLPROVIDER_ICU)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, Anum_pg_database_datlocale);
+ locale = TextDatumGetCString(datum);
+ }
+ else if (datlocprovider == COLLPROVIDER_LIBC)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tp, Anum_pg_database_datcollate);
+ locale = TextDatumGetCString(datum);
+ }
+ else
+ locale = NULL; /* COLLPROVIDER_BUILTIN */
+
+ version = get_collation_actual_version(datlocprovider, locale);
ReleaseSysCache(tp);
diff --git a/src/backend/regex/regc_pg_locale.c b/src/backend/regex/regc_pg_locale.c
index 6a26388bfa..3cb8678298 100644
--- a/src/backend/regex/regc_pg_locale.c
+++ b/src/backend/regex/regc_pg_locale.c
@@ -16,6 +16,8 @@
*/
#include "catalog/pg_collation.h"
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
#include "utils/pg_locale.h"
/*
@@ -64,6 +66,7 @@
typedef enum
{
PG_REGEX_LOCALE_C, /* C locale (encoding independent) */
+ PG_REGEX_BUILTIN, /* built-in Unicode semantics */
PG_REGEX_LOCALE_WIDE, /* Use <wctype.h> functions */
PG_REGEX_LOCALE_1BYTE, /* Use <ctype.h> functions */
PG_REGEX_LOCALE_WIDE_L, /* Use locale_t <wctype.h> functions */
@@ -75,6 +78,8 @@ static PG_Locale_Strategy pg_regex_strategy;
static pg_locale_t pg_regex_locale;
static Oid pg_regex_collation;
+static bool regex_builtin_cclass_posix = false;
+
/*
* Hard-wired character properties for C locale
*/
@@ -266,7 +271,15 @@ pg_set_regex_collation(Oid collation)
if (GetDatabaseEncoding() == PG_UTF8)
{
if (pg_regex_locale)
- pg_regex_strategy = PG_REGEX_LOCALE_WIDE_L;
+ {
+ if (pg_regex_locale->provider == COLLPROVIDER_BUILTIN)
+ {
+ pg_regex_strategy = PG_REGEX_BUILTIN;
+ regex_builtin_cclass_posix = pg_regex_locale->info.builtin.cclass_posix;
+ }
+ else
+ pg_regex_strategy = PG_REGEX_LOCALE_WIDE_L;
+ }
else
pg_regex_strategy = PG_REGEX_LOCALE_WIDE;
}
@@ -290,6 +303,8 @@ pg_wc_isdigit(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISDIGIT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isdigit(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswdigit((wint_t) c);
@@ -322,6 +337,8 @@ pg_wc_isalpha(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISALPHA));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isalpha(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswalpha((wint_t) c);
@@ -354,6 +371,8 @@ pg_wc_isalnum(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISALNUM));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isalnum(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswalnum((wint_t) c);
@@ -395,6 +414,8 @@ pg_wc_isupper(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISUPPER));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isupper(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswupper((wint_t) c);
@@ -427,6 +448,8 @@ pg_wc_islower(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISLOWER));
+ case PG_REGEX_BUILTIN:
+ return pg_u_islower(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswlower((wint_t) c);
@@ -459,6 +482,8 @@ pg_wc_isgraph(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISGRAPH));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isgraph(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswgraph((wint_t) c);
@@ -491,6 +516,8 @@ pg_wc_isprint(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISPRINT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isprint(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswprint((wint_t) c);
@@ -523,6 +550,8 @@ pg_wc_ispunct(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISPUNCT));
+ case PG_REGEX_BUILTIN:
+ return pg_u_ispunct(c, regex_builtin_cclass_posix);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswpunct((wint_t) c);
@@ -555,6 +584,8 @@ pg_wc_isspace(pg_wchar c)
case PG_REGEX_LOCALE_C:
return (c <= (pg_wchar) 127 &&
(pg_char_properties[c] & PG_ISSPACE));
+ case PG_REGEX_BUILTIN:
+ return pg_u_isspace(c);
case PG_REGEX_LOCALE_WIDE:
if (sizeof(wchar_t) >= 4 || c <= (pg_wchar) 0xFFFF)
return iswspace((wint_t) c);
@@ -588,6 +619,8 @@ pg_wc_toupper(pg_wchar c)
if (c <= (pg_wchar) 127)
return pg_ascii_toupper((unsigned char) c);
return c;
+ case PG_REGEX_BUILTIN:
+ return unicode_uppercase_simple(c);
case PG_REGEX_LOCALE_WIDE:
/* force C behavior for ASCII characters, per comments above */
if (c <= (pg_wchar) 127)
@@ -628,6 +661,8 @@ pg_wc_tolower(pg_wchar c)
if (c <= (pg_wchar) 127)
return pg_ascii_tolower((unsigned char) c);
return c;
+ case PG_REGEX_BUILTIN:
+ return unicode_lowercase_simple(c);
case PG_REGEX_LOCALE_WIDE:
/* force C behavior for ASCII characters, per comments above */
if (c <= (pg_wchar) 127)
@@ -792,6 +827,9 @@ pg_ctype_get_cache(pg_wc_probefunc probefunc, int cclasscode)
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
#endif
break;
+ case PG_REGEX_BUILTIN:
+ max_chr = (pg_wchar) MAX_SIMPLE_CHR;
+ break;
case PG_REGEX_LOCALE_WIDE:
case PG_REGEX_LOCALE_WIDE_L:
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
@@ -809,6 +847,7 @@ pg_ctype_get_cache(pg_wc_probefunc probefunc, int cclasscode)
max_chr = (pg_wchar) MAX_SIMPLE_CHR;
break;
default:
+ Assert(false);
max_chr = 0; /* can't get here, but keep compiler quiet */
break;
}
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index 829aaa8d0e..9895368ff8 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -77,6 +77,8 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "common/unicode_case.h"
+#include "common/unicode_category.h"
#include "mb/pg_wchar.h"
#include "nodes/miscnodes.h"
#include "parser/scansup.h"
@@ -1680,6 +1682,64 @@ str_tolower(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *src = (unsigned char *) buff;
+ unsigned char *dst;
+ size_t dstsize = nbytes + 1;
+ int srcoff = 0;
+ int dstoff = 0;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ dst = (unsigned char *) palloc(dstsize);
+
+ while (srcoff < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(src + srcoff);
+ pg_wchar u2 = unicode_lowercase_simple(u1);
+ int u1len = unicode_utf8len(u1);
+ int u2len = unicode_utf8len(u2);
+
+ /*
+ * If we can't fit the necessary bytes and a terminating NUL,
+ * reallocate buffer to the maximum size we might need, and
+ * shrink it later.
+ */
+ if (dstoff + u2len + 1 > dstsize)
+ {
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ dstsize = (nbytes + 1) * sizeof(pg_wchar);
+ dst = repalloc(dst, dstsize);
+ }
+
+ unicode_to_utf8(u2, dst + dstoff);
+ srcoff += u1len;
+ dstoff += u2len;
+ }
+
+ *(dst + dstoff) = '\0';
+ dstoff++;
+
+ if (dstsize == dstoff)
+ {
+ result = (char *) dst;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(dstoff);
+ memcpy(result, dst, dstoff);
+ pfree(dst);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
@@ -1798,6 +1858,64 @@ str_toupper(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *src = (unsigned char *) buff;
+ unsigned char *dst;
+ size_t dstsize = nbytes + 1;
+ int srcoff = 0;
+ int dstoff = 0;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ dst = (unsigned char *) palloc(dstsize);
+
+ while (srcoff < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(src + srcoff);
+ pg_wchar u2 = unicode_uppercase_simple(u1);
+ int u1len = unicode_utf8len(u1);
+ int u2len = unicode_utf8len(u2);
+
+ /*
+ * If we can't fit the necessary bytes and a terminating NUL,
+ * reallocate buffer to the maximum size we might need, and
+ * shrink it later.
+ */
+ if (dstoff + u2len + 1 > dstsize)
+ {
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ dstsize = (nbytes + 1) * sizeof(pg_wchar);
+ dst = repalloc(dst, dstsize);
+ }
+
+ unicode_to_utf8(u2, dst + dstoff);
+ srcoff += u1len;
+ dstoff += u2len;
+ }
+
+ *(dst + dstoff) = '\0';
+ dstoff++;
+
+ if (dstsize == dstoff)
+ {
+ result = (char *) dst;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(dstoff);
+ memcpy(result, dst, dstoff);
+ pfree(dst);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
@@ -1917,6 +2035,73 @@ str_initcap(const char *buff, size_t nbytes, Oid collid)
}
else
#endif
+ if (mylocale && mylocale->provider == COLLPROVIDER_BUILTIN)
+ {
+ const unsigned char *src = (unsigned char *) buff;
+ unsigned char *dst;
+ size_t dstsize = nbytes + 1;
+ int srcoff = 0;
+ int dstoff = 0;
+
+ Assert(GetDatabaseEncoding() == PG_UTF8);
+
+ /* Output workspace cannot have more codes than input bytes */
+ dst = (unsigned char *) palloc(dstsize);
+
+ while (srcoff < nbytes)
+ {
+ pg_wchar u1 = utf8_to_unicode(src + srcoff);
+ pg_wchar u2;
+ int u1len = unicode_utf8len(u1);
+ int u2len;
+
+ if (wasalnum)
+ u2 = unicode_lowercase_simple(u1);
+ else
+ u2 = unicode_uppercase_simple(u1);
+
+ u2len = unicode_utf8len(u2);
+
+ wasalnum = pg_u_isalnum(u2, mylocale->info.builtin.cclass_posix);
+
+ /*
+ * If we can't fit the necessary bytes and a terminating NUL,
+ * reallocate buffer to the maximum size we might need, and
+ * shrink it later.
+ */
+ if (dstoff + u2len + 1 > dstsize)
+ {
+ /* Overflow paranoia */
+ if ((nbytes + 1) > (INT_MAX / sizeof(pg_wchar)))
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+
+ dstsize = (nbytes + 1) * sizeof(pg_wchar);
+ dst = repalloc(dst, dstsize);
+ }
+
+ unicode_to_utf8(u2, dst + dstoff);
+ srcoff += u1len;
+ dstoff += u2len;
+ }
+
+ *(dst + dstoff) = '\0';
+ dstoff++;
+
+ if (dstsize == dstoff)
+ {
+ result = (char *) dst;
+ }
+ else
+ {
+ /* shrink buffer and store result */
+ result = palloc(dstoff);
+ memcpy(result, dst, dstoff);
+ pfree(dst);
+ }
+ }
+ else
{
if (pg_database_encoding_max_length() > 1)
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 45fe847320..850af2daf4 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1269,7 +1269,19 @@ lookup_collation_cache(Oid collation, bool set_flags)
elog(ERROR, "cache lookup failed for collation %u", collation);
collform = (Form_pg_collation) GETSTRUCT(tp);
- if (collform->collprovider == COLLPROVIDER_LIBC)
+ if (collform->collprovider == COLLPROVIDER_BUILTIN)
+ {
+ Datum datum;
+ const char *colllocale;
+
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
+ colllocale = TextDatumGetCString(datum);
+
+ cache_entry->collate_is_c = true;
+ cache_entry->ctype_is_c = ((strcmp(colllocale, "C") == 0) ||
+ (strcmp(colllocale, "POSIX") == 0));
+ }
+ else if (collform->collprovider == COLLPROVIDER_LIBC)
{
Datum datum;
const char *collcollate;
@@ -1320,16 +1332,30 @@ lc_collate_is_c(Oid collation)
if (collation == DEFAULT_COLLATION_OID)
{
static int result = -1;
- char *localeptr;
-
- if (default_locale.provider == COLLPROVIDER_ICU)
- return false;
+ const char *localeptr;
if (result >= 0)
return (bool) result;
- localeptr = setlocale(LC_COLLATE, NULL);
- if (!localeptr)
- elog(ERROR, "invalid LC_COLLATE setting");
+
+ if (default_locale.provider == COLLPROVIDER_BUILTIN)
+ {
+ result = true;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_ICU)
+ {
+ result = false;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_LIBC)
+ {
+ localeptr = setlocale(LC_CTYPE, NULL);
+ if (!localeptr)
+ elog(ERROR, "invalid LC_CTYPE setting");
+ }
+ else
+ elog(ERROR, "unexpected collation provider '%c'",
+ default_locale.provider);
if (strcmp(localeptr, "C") == 0)
result = true;
@@ -1373,16 +1399,29 @@ lc_ctype_is_c(Oid collation)
if (collation == DEFAULT_COLLATION_OID)
{
static int result = -1;
- char *localeptr;
-
- if (default_locale.provider == COLLPROVIDER_ICU)
- return false;
+ const char *localeptr;
if (result >= 0)
return (bool) result;
- localeptr = setlocale(LC_CTYPE, NULL);
- if (!localeptr)
- elog(ERROR, "invalid LC_CTYPE setting");
+
+ if (default_locale.provider == COLLPROVIDER_BUILTIN)
+ {
+ localeptr = default_locale.info.builtin.locale;
+ }
+ else if (default_locale.provider == COLLPROVIDER_ICU)
+ {
+ result = false;
+ return (bool) result;
+ }
+ else if (default_locale.provider == COLLPROVIDER_LIBC)
+ {
+ localeptr = setlocale(LC_CTYPE, NULL);
+ if (!localeptr)
+ elog(ERROR, "invalid LC_CTYPE setting");
+ }
+ else
+ elog(ERROR, "unexpected collation provider '%c'",
+ default_locale.provider);
if (strcmp(localeptr, "C") == 0)
result = true;
@@ -1390,6 +1429,7 @@ lc_ctype_is_c(Oid collation)
result = true;
else
result = false;
+
return (bool) result;
}
@@ -1520,10 +1560,10 @@ pg_newlocale_from_collation(Oid collid)
if (collid == DEFAULT_COLLATION_OID)
{
- if (default_locale.provider == COLLPROVIDER_ICU)
- return &default_locale;
- else
+ if (default_locale.provider == COLLPROVIDER_LIBC)
return (pg_locale_t) 0;
+ else
+ return &default_locale;
}
cache_entry = lookup_collation_cache(collid, false);
@@ -1548,7 +1588,18 @@ pg_newlocale_from_collation(Oid collid)
result.provider = collform->collprovider;
result.deterministic = collform->collisdeterministic;
- if (collform->collprovider == COLLPROVIDER_LIBC)
+ if (collform->collprovider == COLLPROVIDER_BUILTIN)
+ {
+ const char *locstr;
+
+ datum = SysCacheGetAttrNotNull(COLLOID, tp, Anum_pg_collation_colllocale);
+ locstr = TextDatumGetCString(datum);
+
+ result.info.builtin.locale = MemoryContextStrdup(TopMemoryContext,
+ locstr);
+ result.info.builtin.cclass_posix = true;
+ }
+ else if (collform->collprovider == COLLPROVIDER_LIBC)
{
const char *collcollate;
const char *collctype pg_attribute_unused();
@@ -1627,6 +1678,7 @@ pg_newlocale_from_collation(Oid collid)
collversionstr = TextDatumGetCString(datum);
+ Assert(collform->collprovider != COLLPROVIDER_BUILTIN);
datum = SysCacheGetAttrNotNull(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colllocale : Anum_pg_collation_collcollate);
actual_versionstr = get_collation_actual_version(collform->collprovider,
@@ -1678,6 +1730,14 @@ get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
+ /*
+ * The only two supported locales (C and C.UTF-8) are both based on memcmp
+ * and do not change. (The ctype behavior can change, but the versioning
+ * does not track that.)
+ */
+ if (collprovider == COLLPROVIDER_BUILTIN)
+ return NULL;
+
#ifdef USE_ICU
if (collprovider == COLLPROVIDER_ICU)
{
@@ -2444,6 +2504,38 @@ pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
return result;
}
+const char *
+builtin_validate_locale(int encoding, const char *locale)
+{
+ const char *canonical_name = NULL;
+ int required_encoding = -1;
+
+ if (strcmp(locale, "C") == 0 || strcmp(locale, "POSIX") == 0)
+ {
+ canonical_name = "C";
+ }
+ else if (strcmp(locale, "C.UTF-8") == 0 || strcmp(locale, "C.UTF8") == 0)
+ {
+ required_encoding = PG_UTF8;
+ canonical_name = "C.UTF-8";
+ }
+
+ if (!canonical_name)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("invalid locale name \"%s\" for builtin provider",
+ locale)));
+
+ if (required_encoding >= 0 && encoding != required_encoding)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("encoding \"%s\" does not match locale \"%s\"",
+ pg_encoding_to_char(encoding), locale)));
+
+ return canonical_name;
+}
+
+
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 9818077d51..575ba0281d 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -423,7 +423,16 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
strcmp(ctype, "POSIX") == 0)
database_ctype_is_c = true;
- if (dbform->datlocprovider == COLLPROVIDER_ICU)
+ if (dbform->datlocprovider == COLLPROVIDER_BUILTIN)
+ {
+ datum = SysCacheGetAttrNotNull(DATABASEOID, tup, Anum_pg_database_datlocale);
+ datlocale = TextDatumGetCString(datum);
+
+ default_locale.info.builtin.locale = MemoryContextStrdup(
+ TopMemoryContext, datlocale);
+ default_locale.info.builtin.cclass_posix = true;
+ }
+ else if (dbform->datlocprovider == COLLPROVIDER_ICU)
{
char *icurules;
@@ -461,10 +470,16 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
{
char *actual_versionstr;
char *collversionstr;
+ char *locale;
collversionstr = TextDatumGetCString(datum);
- actual_versionstr = get_collation_actual_version(dbform->datlocprovider, dbform->datlocprovider == COLLPROVIDER_ICU ? datlocale : collate);
+ if (dbform->datlocprovider == COLLPROVIDER_LIBC)
+ locale = collate;
+ else
+ locale = datlocale;
+
+ actual_versionstr = get_collation_actual_version(dbform->datlocprovider, locale);
if (!actual_versionstr)
/* should not happen */
elog(WARNING,
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 90f793632a..7419c38722 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -146,6 +146,7 @@ static char *lc_time = NULL;
static char *lc_messages = NULL;
static char locale_provider = COLLPROVIDER_LIBC;
static char *datlocale = NULL;
+static bool icu_locale_specified = false;
static char *icu_rules = NULL;
static const char *default_text_search_config = NULL;
static char *username = NULL;
@@ -2390,14 +2391,13 @@ setlocales(void)
lc_messages = canonname;
#endif
+ if (locale_provider != COLLPROVIDER_LIBC && datlocale == NULL)
+ pg_fatal("locale must be specified unless provider is libc");
+
if (locale_provider == COLLPROVIDER_ICU)
{
char *langtag;
- /* acquire default locale from the environment, if not specified */
- if (datlocale == NULL)
- pg_fatal("ICU locale must be specified");
-
/* canonicalize to a language tag */
langtag = icu_language_tag(datlocale);
printf(_("Using language tag \"%s\" for ICU locale \"%s\".\n"),
@@ -2442,7 +2442,8 @@ usage(const char *progname)
" set default locale in the respective category for\n"
" new databases (default taken from environment)\n"));
printf(_(" --no-locale equivalent to --locale=C\n"));
- printf(_(" --locale-provider={libc|icu}\n"
+ printf(_(" --builtin-locale=LOCALE set builtin locale name for new databases\n"));
+ printf(_(" --locale-provider={builtin|libc|icu}\n"
" set default locale provider for new databases\n"));
printf(_(" --pwfile=FILE read password for the new superuser from file\n"));
printf(_(" -T, --text-search-config=CFG\n"
@@ -2593,20 +2594,28 @@ setup_locale_encoding(void)
{
setlocales();
- if (locale_provider == COLLPROVIDER_LIBC &&
- strcmp(lc_ctype, lc_collate) == 0 &&
- strcmp(lc_ctype, lc_time) == 0 &&
- strcmp(lc_ctype, lc_numeric) == 0 &&
- strcmp(lc_ctype, lc_monetary) == 0 &&
- strcmp(lc_ctype, lc_messages) == 0 &&
- (!datlocale || strcmp(lc_ctype, datlocale) == 0))
+ if (locale_provider == COLLPROVIDER_BUILTIN &&
+ strcmp(lc_ctype, "C") == 0 &&
+ strcmp(lc_collate, "C") == 0 &&
+ strcmp(lc_time, "C") == 0 &&
+ strcmp(lc_numeric, "C") == 0 &&
+ strcmp(lc_monetary, "C") == 0 &&
+ strcmp(lc_messages, "C") == 0)
+ printf(_("The database cluster will be initialized with no locale.\n"));
+ else if (locale_provider == COLLPROVIDER_LIBC &&
+ strcmp(lc_ctype, lc_collate) == 0 &&
+ strcmp(lc_ctype, lc_time) == 0 &&
+ strcmp(lc_ctype, lc_numeric) == 0 &&
+ strcmp(lc_ctype, lc_monetary) == 0 &&
+ strcmp(lc_ctype, lc_messages) == 0 &&
+ (!datlocale || strcmp(lc_ctype, datlocale) == 0))
printf(_("The database cluster will be initialized with locale \"%s\".\n"), lc_ctype);
else
{
printf(_("The database cluster will be initialized with this locale configuration:\n"));
- printf(_(" provider: %s\n"), collprovider_name(locale_provider));
- if (datlocale)
- printf(_(" ICU locale: %s\n"), datlocale);
+ printf(_(" default collation provider: %s\n"), collprovider_name(locale_provider));
+ if (locale_provider != COLLPROVIDER_LIBC)
+ printf(_(" default collation locale: %s\n"), datlocale);
printf(_(" LC_COLLATE: %s\n"
" LC_CTYPE: %s\n"
" LC_MESSAGES: %s\n"
@@ -3099,9 +3108,10 @@ main(int argc, char *argv[])
{"allow-group-access", no_argument, NULL, 'g'},
{"discard-caches", no_argument, NULL, 14},
{"locale-provider", required_argument, NULL, 15},
- {"icu-locale", required_argument, NULL, 16},
- {"icu-rules", required_argument, NULL, 17},
- {"sync-method", required_argument, NULL, 18},
+ {"builtin-locale", required_argument, NULL, 16},
+ {"icu-locale", required_argument, NULL, 17},
+ {"icu-rules", required_argument, NULL, 18},
+ {"sync-method", required_argument, NULL, 19},
{NULL, 0, NULL, 0}
};
@@ -3269,7 +3279,9 @@ main(int argc, char *argv[])
"-c debug_discard_caches=1");
break;
case 15:
- if (strcmp(optarg, "icu") == 0)
+ if (strcmp(optarg, "builtin") == 0)
+ locale_provider = COLLPROVIDER_BUILTIN;
+ else if (strcmp(optarg, "icu") == 0)
locale_provider = COLLPROVIDER_ICU;
else if (strcmp(optarg, "libc") == 0)
locale_provider = COLLPROVIDER_LIBC;
@@ -3280,9 +3292,13 @@ main(int argc, char *argv[])
datlocale = pg_strdup(optarg);
break;
case 17:
- icu_rules = pg_strdup(optarg);
+ datlocale = pg_strdup(optarg);
+ icu_locale_specified = true;
break;
case 18:
+ icu_rules = pg_strdup(optarg);
+ break;
+ case 19:
if (!parse_sync_method(optarg, &sync_method))
exit(1);
break;
@@ -3312,7 +3328,7 @@ main(int argc, char *argv[])
exit(1);
}
- if (datlocale && locale_provider != COLLPROVIDER_ICU)
+ if (icu_locale_specified && locale_provider != COLLPROVIDER_ICU)
pg_fatal("%s cannot be specified unless locale provider \"%s\" is chosen",
"--icu-locale", "icu");
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index 03376cc0f7..242f4581a5 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -117,7 +117,7 @@ if ($ENV{with_icu} eq 'yes')
{
command_fails_like(
[ 'initdb', '--no-sync', '--locale-provider=icu', "$tempdir/data2" ],
- qr/initdb: error: ICU locale must be specified/,
+ qr/initdb: error: locale must be specified unless provider is libc/,
'locale provider ICU requires --icu-locale');
command_ok(
@@ -138,7 +138,7 @@ if ($ENV{with_icu} eq 'yes')
'--lc-monetary=C', '--lc-time=C',
"$tempdir/data4"
],
- qr/^\s+ICU locale:\s+und\n/ms,
+ qr/^\s+default collation locale:\s+und\n/ms,
'options --locale-provider=icu --locale=und --lc-*=C');
command_fails_like(
@@ -184,6 +184,59 @@ else
'locale provider ICU fails since no ICU support');
}
+command_fails(
+ [ 'initdb', '--no-sync', '--locale-provider=builtin', "$tempdir/data6" ],
+ 'locale provider builtin fails without --locale');
+
+command_ok(
+ [
+ 'initdb', '--no-sync',
+ '--locale-provider=builtin', '--locale=C',
+ "$tempdir/data7"
+ ],
+ 'locale provider builtin with --locale');
+
+command_ok(
+ [
+ 'initdb', '--no-sync',
+ '--locale-provider=builtin', '-E UTF-8',
+ '--builtin-locale=C.UTF-8', "$tempdir/data8"
+ ],
+ 'locale provider builtin with -E UTF-8 --builtin-locale=C.UTF-8');
+
+command_fails(
+ [
+ 'initdb', '--no-sync',
+ '--locale-provider=builtin', '-E SQL_ASCII',
+ '--builtin-locale=C.UTF-8', "$tempdir/data9"
+ ],
+ 'locale provider builtin with --builtin-locale=C.UTF-8 fails for SQL_ASCII'
+);
+
+command_ok(
+ [
+ 'initdb', '--no-sync',
+ '--locale-provider=builtin', '--lc-ctype=C',
+ '--locale=C', "$tempdir/data10"
+ ],
+ 'locale provider builtin with --lc-ctype');
+
+command_fails(
+ [
+ 'initdb', '--no-sync',
+ '--locale-provider=builtin', '--icu-locale=en',
+ "$tempdir/dataX"
+ ],
+ 'fails for locale provider builtin with ICU locale');
+
+command_fails(
+ [
+ 'initdb', '--no-sync',
+ '--locale-provider=builtin', '--icu-rules=""',
+ "$tempdir/dataX"
+ ],
+ 'fails for locale provider builtin with ICU rules');
+
command_fails(
[ 'initdb', '--no-sync', '--locale-provider=xyz', "$tempdir/dataX" ],
'fails for invalid locale provider');
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 303d1ff4a8..f16436d179 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3112,7 +3112,9 @@ dumpDatabase(Archive *fout)
}
appendPQExpBufferStr(creaQry, " LOCALE_PROVIDER = ");
- if (datlocprovider[0] == 'c')
+ if (datlocprovider[0] == 'b')
+ appendPQExpBufferStr(creaQry, "builtin");
+ else if (datlocprovider[0] == 'c')
appendPQExpBufferStr(creaQry, "libc");
else if (datlocprovider[0] == 'i')
appendPQExpBufferStr(creaQry, "icu");
@@ -3120,27 +3122,33 @@ dumpDatabase(Archive *fout)
pg_fatal("unrecognized locale provider: %s",
datlocprovider);
- if (strlen(collate) > 0 && strcmp(collate, ctype) == 0)
+ if (!locale && datlocprovider[0] != 'c')
+ pg_log_warning("database '%s' with provider '%s' missing datlocale",
+ datname, datlocprovider);
+
+ if (locale && datlocprovider[0] == 'c')
+ pg_log_warning("database '%s' with provider 'c' has non-NULL locale '%s'",
+ datname, locale);
+
+ /* if collate and ctype are equal, and locale is NULL, use LOCALE */
+ if (!locale && strlen(collate) > 0 && strcmp(collate, ctype) == 0)
+ locale = collate;
+
+ /* output LC_COLLATE and LC_CTYPE if different from LOCALE */
+ if (strlen(collate) > 0 && (!locale || strcmp(collate, locale) != 0))
{
- appendPQExpBufferStr(creaQry, " LOCALE = ");
+ appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
appendStringLiteralAH(creaQry, collate, fout);
}
- else
+ if (strlen(ctype) > 0 && (!locale || strcmp(ctype, locale) != 0))
{
- if (strlen(collate) > 0)
- {
- appendPQExpBufferStr(creaQry, " LC_COLLATE = ");
- appendStringLiteralAH(creaQry, collate, fout);
- }
- if (strlen(ctype) > 0)
- {
- appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
- appendStringLiteralAH(creaQry, ctype, fout);
- }
+ appendPQExpBufferStr(creaQry, " LC_CTYPE = ");
+ appendStringLiteralAH(creaQry, ctype, fout);
}
+
if (locale)
{
- appendPQExpBufferStr(creaQry, " ICU_LOCALE = ");
+ appendPQExpBufferStr(creaQry, " LOCALE = ");
appendStringLiteralAH(creaQry, locale, fout);
}
@@ -13868,7 +13876,9 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
fmtQualifiedDumpable(collinfo));
appendPQExpBufferStr(q, "provider = ");
- if (collprovider[0] == 'c')
+ if (collprovider[0] == 'b')
+ appendPQExpBufferStr(q, "builtin");
+ else if (collprovider[0] == 'c')
appendPQExpBufferStr(q, "libc");
else if (collprovider[0] == 'i')
appendPQExpBufferStr(q, "icu");
@@ -13889,6 +13899,13 @@ dumpCollation(Archive *fout, const CollInfo *collinfo)
/* no locale -- the default collation cannot be reloaded anyway */
}
+ else if (collprovider[0] == 'b')
+ {
+ if (collcollate || collctype || colllocale || collicurules)
+ pg_log_warning("invalid collation \"%s\"", qcollname);
+
+ appendPQExpBufferStr(q, ", locale = 'C'");
+ }
else if (collprovider[0] == 'i')
{
if (fout->remoteVersion >= 150000)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 41d06d272b..94bf086ba8 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -110,13 +110,16 @@ my $oldversion = int($oldnode->pg_version =~ s/([0-9]*).*/$1/rg);
# can test that pg_upgrade copies the locale settings of template0
# from the old to the new cluster.
-my $original_encoding = "6"; # UTF-8
-my $original_provider = "c";
-my $original_locale = "C";
-my $original_datlocale = "";
-my $provider_field = "'c' AS datlocprovider";
-my $datlocale_field = "NULL AS datlocale";
-if ($oldversion >= 15 && $ENV{with_icu} eq 'yes')
+my %encoding_number = ('UTF-8' => 6, 'SQL_ASCII' => 0);
+my $provider_field;
+my $datlocale_field;
+my $original_encoding;
+my $original_provider;
+my $original_datcollate = "C";
+my $original_datctype = "C";
+my $original_datlocale;
+
+if ($oldversion >= 15)
{
$provider_field = "datlocprovider";
if ($oldversion >= 17)
@@ -127,18 +130,52 @@ if ($oldversion >= 15 && $ENV{with_icu} eq 'yes')
{
$datlocale_field = "daticulocale AS datlocale";
}
+}
+else
+{
+ $provider_field = "'c' AS datlocprovider";
+ $datlocale_field = "NULL AS datlocale";
+}
+
+if ($oldversion >= 17)
+{
+ $original_encoding = "UTF-8";
+ $original_provider = "b";
+ $original_datlocale = "C.UTF-8";
+}
+elsif ($oldversion >= 15 && $ENV{with_icu} eq 'yes')
+{
+ $original_encoding = "UTF-8";
$original_provider = "i";
$original_datlocale = "fr-CA";
}
+else
+{
+ my $original_encoding = "SQL_ASCII";
+ my $original_provider = "c";
+ my $original_datlocale = "";
+}
my @initdb_params = @custom_opts;
-push @initdb_params, ('--encoding', 'UTF-8');
-push @initdb_params, ('--locale', $original_locale);
-if ($original_provider eq "i")
+push @initdb_params, ('--encoding', $original_encoding);
+push @initdb_params, ('--lc-collate', $original_datcollate);
+push @initdb_params, ('--lc-ctype', $original_datctype);
+
+# add --locale-provider, if supported
+my %provider_name = ('b' => 'builtin', 'i' => 'icu', 'c' => 'libc');
+if ($oldnode->pg_version >= 15)
{
- push @initdb_params, ('--locale-provider', 'icu');
- push @initdb_params, ('--icu-locale', 'fr-CA');
+ push @initdb_params,
+ ('--locale-provider', $provider_name{$original_provider});
+ if ($original_provider eq 'b')
+ {
+ push @initdb_params, ('--builtin-locale', $original_datlocale);
+ }
+ elsif ($original_provider eq 'i')
+ {
+ push @initdb_params, ('--icu-locale', $original_datlocale);
+ }
}
$node_params{extra} = \@initdb_params;
@@ -151,7 +188,7 @@ $result = $oldnode->safe_psql(
"SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_datlocale",
+ "$encoding_number{$original_encoding}|$original_provider|$original_datcollate|$original_datctype|$original_datlocale",
"check locales in original cluster");
# The default location of the source code is the root of this directory.
@@ -327,7 +364,8 @@ if (defined($ENV{oldinstall}))
}
# Create an invalid database, will be deleted below
-$oldnode->safe_psql('postgres', qq(
+$oldnode->safe_psql(
+ 'postgres', qq(
CREATE DATABASE regression_invalid;
UPDATE pg_database SET datconnlimit = -2 WHERE datname = 'regression_invalid';
));
@@ -370,7 +408,7 @@ command_checks_all(
$mode, '--check',
],
1,
- [qr/invalid/], # pg_upgrade prints errors on stdout :(
+ [qr/invalid/], # pg_upgrade prints errors on stdout :(
[qr//],
'invalid database causes failure');
rmtree($newnode->data_dir . "/pg_upgrade_output.d");
@@ -434,7 +472,7 @@ $result = $newnode->safe_psql(
"SELECT encoding, $provider_field, datcollate, datctype, $datlocale_field
FROM pg_database WHERE datname='template0'");
is( $result,
- "$original_encoding|$original_provider|$original_locale|$original_locale|$original_datlocale",
+ "$encoding_number{$original_encoding}|$original_provider|$original_datcollate|$original_datctype|$original_datlocale",
"check that locales in new cluster match original cluster");
# Second dump from the upgraded instance.
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index b943569050..c649477505 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -926,7 +926,7 @@ listAllDbs(const char *pattern, bool verbose)
gettext_noop("Encoding"));
if (pset.sversion >= 150000)
appendPQExpBuffer(&buf,
- " CASE d.datlocprovider WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
+ " CASE d.datlocprovider WHEN 'b' THEN 'builtin' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
gettext_noop("Locale Provider"));
else
appendPQExpBuffer(&buf,
@@ -4974,7 +4974,7 @@ listCollations(const char *pattern, bool verbose, bool showSystem)
if (pset.sversion >= 100000)
appendPQExpBuffer(&buf,
- " CASE c.collprovider WHEN 'd' THEN 'default' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
+ " CASE c.collprovider WHEN 'd' THEN 'default' WHEN 'b' THEN 'builtin' WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' END AS \"%s\",\n",
gettext_noop("Provider"));
else
appendPQExpBuffer(&buf,
diff --git a/src/bin/scripts/createdb.c b/src/bin/scripts/createdb.c
index 14970a6a5f..4af4b98181 100644
--- a/src/bin/scripts/createdb.c
+++ b/src/bin/scripts/createdb.c
@@ -40,8 +40,9 @@ main(int argc, char *argv[])
{"locale", required_argument, NULL, 'l'},
{"maintenance-db", required_argument, NULL, 3},
{"locale-provider", required_argument, NULL, 4},
- {"icu-locale", required_argument, NULL, 5},
- {"icu-rules", required_argument, NULL, 6},
+ {"builtin-locale", required_argument, NULL, 5},
+ {"icu-locale", required_argument, NULL, 6},
+ {"icu-rules", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
@@ -67,6 +68,7 @@ main(int argc, char *argv[])
char *lc_ctype = NULL;
char *locale = NULL;
char *locale_provider = NULL;
+ char *builtin_locale = NULL;
char *icu_locale = NULL;
char *icu_rules = NULL;
@@ -134,9 +136,12 @@ main(int argc, char *argv[])
locale_provider = pg_strdup(optarg);
break;
case 5:
- icu_locale = pg_strdup(optarg);
+ builtin_locale = pg_strdup(optarg);
break;
case 6:
+ icu_locale = pg_strdup(optarg);
+ break;
+ case 7:
icu_rules = pg_strdup(optarg);
break;
default:
@@ -216,6 +221,11 @@ main(int argc, char *argv[])
appendPQExpBufferStr(&sql, " LOCALE ");
appendStringLiteralConn(&sql, locale, conn);
}
+ if (builtin_locale)
+ {
+ appendPQExpBufferStr(&sql, " BUILTIN_LOCALE ");
+ appendStringLiteralConn(&sql, builtin_locale, conn);
+ }
if (lc_collate)
{
appendPQExpBufferStr(&sql, " LC_COLLATE ");
@@ -296,7 +306,7 @@ help(const char *progname)
printf(_(" --lc-ctype=LOCALE LC_CTYPE setting for the database\n"));
printf(_(" --icu-locale=LOCALE ICU locale setting for the database\n"));
printf(_(" --icu-rules=RULES ICU rules setting for the database\n"));
- printf(_(" --locale-provider={libc|icu}\n"
+ printf(_(" --locale-provider={builtin|libc|icu}\n"
" locale provider for the database's default collation\n"));
printf(_(" -O, --owner=OWNER database user to own the new database\n"));
printf(_(" -S, --strategy=STRATEGY database creation strategy wal_log or file_copy\n"));
diff --git a/src/bin/scripts/t/020_createdb.pl b/src/bin/scripts/t/020_createdb.pl
index 37e47b0078..3ba623f9d1 100644
--- a/src/bin/scripts/t/020_createdb.pl
+++ b/src/bin/scripts/t/020_createdb.pl
@@ -105,6 +105,84 @@ else
'create database with ICU fails since no ICU support');
}
+$node->command_fails(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ 'tbuiltin1'
+ ],
+ 'create database with provider "builtin" fails without --locale');
+
+$node->command_ok(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ '--locale=C', 'tbuiltin2'
+ ],
+ 'create database with provider "builtin" and locale "C"');
+
+$node->command_ok(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ '--locale=C', '--lc-collate=C',
+ 'tbuiltin3'
+ ],
+ 'create database with provider "builtin" and LC_COLLATE=C');
+
+$node->command_ok(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ '--locale=C', '--lc-ctype=C',
+ 'tbuiltin4'
+ ],
+ 'create database with provider "builtin" and LC_CTYPE=C');
+
+$node->command_ok(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ '-E UTF-8', '--builtin-locale=C.UTF8',
+ 'tbuiltin5'
+ ],
+ 'create database with provider "builtin" and --builtin-locale C.UTF-8');
+
+$node->command_fails(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ '-E LATIN1', '--builtin-locale=C.UTF-8',
+ 'tbuiltin6'
+ ],
+ 'create database with provider "builtin" and --builtin-locale C.UTF-8');
+
+$node->command_fails(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ '--locale=C', '--icu-locale=en',
+ 'tbuiltin7'
+ ],
+ 'create database with provider "builtin" and ICU_LOCALE="en"');
+
+$node->command_fails(
+ [
+ 'createdb', '-T',
+ 'template0', '--locale-provider=builtin',
+ '--locale=C', '--icu-rules=""',
+ 'tbuiltin8'
+ ],
+ 'create database with provider "builtin" and ICU_RULES=""');
+
+$node->command_fails(
+ [
+ 'createdb', '-T',
+ 'template1', '--locale-provider=builtin',
+ '--locale=C', 'tbuiltin9'
+ ],
+ 'create database with provider "builtin" not matching template');
+
$node->command_fails([ 'createdb', 'foobar1' ],
'fails if database already exists');
diff --git a/src/common/wchar.c b/src/common/wchar.c
index 7c5ce5ca08..95e6b499e3 100644
--- a/src/common/wchar.c
+++ b/src/common/wchar.c
@@ -477,8 +477,8 @@ pg_utf2wchar_with_len(const unsigned char *from, pg_wchar *to, int len)
/*
- * Map a Unicode code point to UTF-8. utf8string must have 4 bytes of
- * space allocated.
+ * Map a Unicode code point to UTF-8. utf8string must have at least
+ * unicode_utf8len(c) bytes available.
*/
unsigned char *
unicode_to_utf8(pg_wchar c, unsigned char *utf8string)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 7396ff10c4..1e439e6975 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -23,12 +23,15 @@
descr => 'standard POSIX collation',
collname => 'POSIX', collprovider => 'c', collencoding => '-1',
collcollate => 'POSIX', collctype => 'POSIX' },
-{ oid => '962', descr => 'sorts by Unicode code point',
- collname => 'ucs_basic', collprovider => 'c', collencoding => '6',
- collcollate => 'C', collctype => 'C' },
+{ oid => '962', descr => 'sorts by Unicode code point, C character semantics',
+ collname => 'ucs_basic', collprovider => 'b', collencoding => '6',
+ colllocale => 'C' },
{ oid => '963',
descr => 'sorts using the Unicode Collation Algorithm with default settings',
collname => 'unicode', collprovider => 'i', collencoding => '-1',
colllocale => 'und' },
+{ oid => '970', descr => 'sorts by Unicode code point; Unicode & POSIX character semantics',
+ collname => 'pg_c_utf8', collprovider => 'b', collencoding => '6',
+ colllocale => 'C.UTF8' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 85cb09c4f8..5ce289d74b 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -42,7 +42,7 @@ CATALOG(pg_collation,3456,CollationRelationId)
#ifdef CATALOG_VARLEN /* variable-length fields start here */
text collcollate BKI_DEFAULT(_null_); /* LC_COLLATE setting */
text collctype BKI_DEFAULT(_null_); /* LC_CTYPE setting */
- text colllocale BKI_DEFAULT(_null_); /* locale ID */
+ text colllocale BKI_DEFAULT(_null_); /* locale ID */
text collicurules BKI_DEFAULT(_null_); /* ICU collation rules */
text collversion BKI_DEFAULT(_null_); /* provider-dependent
* version of collation
@@ -68,6 +68,7 @@ MAKE_SYSCACHE(COLLOID, pg_collation_oid_index, 8);
#ifdef EXPOSE_TO_CLIENT_CODE
#define COLLPROVIDER_DEFAULT 'd'
+#define COLLPROVIDER_BUILTIN 'b'
#define COLLPROVIDER_ICU 'i'
#define COLLPROVIDER_LIBC 'c'
@@ -76,6 +77,8 @@ collprovider_name(char c)
{
switch (c)
{
+ case COLLPROVIDER_BUILTIN:
+ return "builtin";
case COLLPROVIDER_ICU:
return "icu";
case COLLPROVIDER_LIBC:
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 1d521bea24..fbd9e58ed3 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -562,6 +562,21 @@ surrogate_pair_to_codepoint(pg_wchar first, pg_wchar second)
return ((first & 0x3FF) << 10) + 0x10000 + (second & 0x3FF);
}
+/*
+ * Number of bytes needed to represent the given char in UTF8.
+ */
+static inline int
+unicode_utf8len(pg_wchar c)
+{
+ if (c <= 0x7F)
+ return 1;
+ else if (c <= 0x7FF)
+ return 2;
+ else if (c <= 0xFFFF)
+ return 3;
+ else
+ return 4;
+}
/*
* The functions in this list are exported by libpq, and we need to be sure
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 28c925b5af..ac5948dadd 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -76,6 +76,11 @@ struct pg_locale_struct
bool deterministic;
union
{
+ struct
+ {
+ const char *locale;
+ bool cclass_posix;
+ } builtin;
locale_t lt;
#ifdef USE_ICU
struct
@@ -112,7 +117,7 @@ extern size_t pg_strxfrm_prefix(char *dest, const char *src, size_t destsize,
pg_locale_t locale);
extern size_t pg_strnxfrm_prefix(char *dest, size_t destsize, const char *src,
size_t srclen, pg_locale_t locale);
-
+extern const char *builtin_validate_locale(int encoding, const char *loc_str);
extern void icu_validate_locale(const char *loc_str);
extern char *icu_language_tag(const char *loc_str, int elevel);
diff --git a/src/test/icu/t/010_database.pl b/src/test/icu/t/010_database.pl
index 8a1fc12ec6..5f8ef16803 100644
--- a/src/test/icu/t/010_database.pl
+++ b/src/test/icu/t/010_database.pl
@@ -27,9 +27,8 @@ CREATE TABLE icu (def text, en text COLLATE "en-x-icu", upfirst text COLLATE upp
INSERT INTO icu VALUES ('a', 'a', 'a'), ('b', 'b', 'b'), ('A', 'A', 'A'), ('B', 'B', 'B');
});
-is( $node1->safe_psql('dbicu', q{SELECT icu_unicode_version() IS NOT NULL}),
- qq(t),
- 'ICU unicode version defined');
+is($node1->safe_psql('dbicu', q{SELECT icu_unicode_version() IS NOT NULL}),
+ qq(t), 'ICU unicode version defined');
is( $node1->safe_psql('dbicu', q{SELECT def FROM icu ORDER BY def}),
qq(A
@@ -63,14 +62,13 @@ is( $node1->psql(
0,
"C locale works for ICU");
-# Test that LOCALE works for ICU locales if LC_COLLATE and LC_CTYPE
-# are specified
-is( $node1->psql(
- 'postgres',
- q{CREATE DATABASE dbicu2 LOCALE_PROVIDER icu LOCALE '@colStrength=primary'
- LC_COLLATE='C' LC_CTYPE='C' TEMPLATE template0 ENCODING UTF8}
- ),
- 0,
- "LOCALE works for ICU locales if LC_COLLATE and LC_CTYPE are specified");
+my ($ret, $stdout, $stderr) = $node1->psql('postgres',
+ q{CREATE DATABASE dbicu LOCALE_PROVIDER builtin LOCALE 'C' TEMPLATE dbicu}
+);
+isnt($ret, 0, "locale provider must match template: exit code not 0");
+like(
+ $stderr,
+ qr/ERROR: new locale provider \(builtin\) does not match locale provider of the template database \(icu\)/,
+ "locale provider must match template: error message");
done_testing();
diff --git a/src/test/regress/expected/collate.out b/src/test/regress/expected/collate.out
index 0649564485..ece4a8e99d 100644
--- a/src/test/regress/expected/collate.out
+++ b/src/test/regress/expected/collate.out
@@ -650,6 +650,26 @@ EXPLAIN (COSTS OFF)
(3 rows)
-- CREATE/DROP COLLATION
+CREATE COLLATION builtin_c ( PROVIDER = builtin, LOCALE = "C" );
+CREATE COLLATION builtin_posix ( PROVIDER = builtin, LOCALE = "POSIX" );
+SELECT b FROM collate_test1 ORDER BY b COLLATE builtin_c;
+ b
+-----
+ ABD
+ Abc
+ abc
+ bbc
+(4 rows)
+
+CREATE COLLATION builtin2 ( PROVIDER = builtin ); -- fails
+ERROR: parameter "locale" must be specified
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "en_US" ); -- fails
+ERROR: invalid locale name "en_US" for builtin provider
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LC_CTYPE = "C", LC_COLLATE = "C" ); -- fails
+ERROR: parameter "locale" must be specified
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "POSIX", LC_CTYPE = "POSIX" ); -- fails
+ERROR: conflicting or redundant options
+DETAIL: LOCALE cannot be specified together with LC_COLLATE or LC_CTYPE.
CREATE COLLATION mycoll1 FROM "C";
CREATE COLLATION mycoll2 ( LC_COLLATE = "POSIX", LC_CTYPE = "POSIX" );
CREATE COLLATION mycoll3 FROM "default"; -- intentionally unsupported
@@ -754,7 +774,7 @@ DETAIL: FROM cannot be specified together with any other options.
-- must get rid of them.
--
DROP SCHEMA collate_tests CASCADE;
-NOTICE: drop cascades to 19 other objects
+NOTICE: drop cascades to 21 other objects
DETAIL: drop cascades to table collate_test1
drop cascades to table collate_test_like
drop cascades to table collate_test2
@@ -771,6 +791,8 @@ drop cascades to function dup(anyelement)
drop cascades to table collate_test20
drop cascades to table collate_test21
drop cascades to table collate_test22
+drop cascades to collation builtin_c
+drop cascades to collation builtin_posix
drop cascades to collation mycoll2
drop cascades to table collate_test23
drop cascades to view collate_on_int
diff --git a/src/test/regress/expected/collate.utf8.out b/src/test/regress/expected/collate.utf8.out
new file mode 100644
index 0000000000..2ef1826d91
--- /dev/null
+++ b/src/test/regress/expected/collate.utf8.out
@@ -0,0 +1,109 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+SET client_encoding TO UTF8;
+--
+-- Test preinstalled PG_C_UTF8 collation.
+--
+CREATE TABLE builtin_test (
+ t TEXT COLLATE PG_C_UTF8
+);
+INSERT INTO builtin_test VALUES
+ ('abc DEF'),
+ ('ábc DÉF'),
+ ('DŽxxDŽ džxxDž Džxxdž'),
+ ('ȺȺȺ'),
+ ('ⱥⱥⱥ'),
+ ('ⱥȺ');
+SELECT
+ t, lower(t), initcap(t), upper(t),
+ length(convert_to(t, 'UTF8')) AS t_bytes,
+ length(convert_to(lower(t), 'UTF8')) AS lower_t_bytes,
+ length(convert_to(initcap(t), 'UTF8')) AS initcap_t_bytes,
+ length(convert_to(upper(t), 'UTF8')) AS upper_t_bytes
+ FROM builtin_test;
+ t | lower | initcap | upper | t_bytes | lower_t_bytes | initcap_t_bytes | upper_t_bytes
+----------------+----------------+----------------+----------------+---------+---------------+-----------------+---------------
+ abc DEF | abc def | Abc Def | ABC DEF | 7 | 7 | 7 | 7
+ ábc DÉF | ábc déf | Ábc Déf | ÁBC DÉF | 9 | 9 | 9 | 9
+ DŽxxDŽ džxxDž Džxxdž | džxxdž džxxdž džxxdž | DŽxxdž DŽxxdž DŽxxdž | DŽXXDŽ DŽXXDŽ DŽXXDŽ | 20 | 20 | 20 | 20
+ ȺȺȺ | ⱥⱥⱥ | Ⱥⱥⱥ | ȺȺȺ | 6 | 9 | 8 | 6
+ ⱥⱥⱥ | ⱥⱥⱥ | Ⱥⱥⱥ | ȺȺȺ | 9 | 9 | 8 | 6
+ ⱥȺ | ⱥⱥ | Ⱥⱥ | ȺȺ | 5 | 6 | 5 | 4
+(6 rows)
+
+DROP TABLE builtin_test;
+-- character classes
+SELECT 'xyz' ~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xyz' !~ '[[:upper:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '@' !~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '@' ~ '[[:punct:]]' COLLATE PG_C_UTF8; -- symbols are punctuation in posix
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'a8a' ~ '[[:digit:]]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT '൧' !~ '\d' COLLATE PG_C_UTF8; -- only 0-9 considered digits in posix
+ ?column?
+----------
+ t
+(1 row)
+
+-- case mapping
+SELECT 'xYz' ~* 'XyZ' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xAb' ~* '[W-Y]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'xAb' !~* '[c-d]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'Δ' ~* '[α-λ]' COLLATE PG_C_UTF8;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT 'δ' ~* '[Γ-Λ]' COLLATE PG_C_UTF8; -- same as above with cases reversed
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/collate.utf8_1.out b/src/test/regress/expected/collate.utf8_1.out
new file mode 100644
index 0000000000..e73fdf50c3
--- /dev/null
+++ b/src/test/regress/expected/collate.utf8_1.out
@@ -0,0 +1,8 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 1d8a414eea..e48cb4b7a3 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -78,9 +78,9 @@ test: brin_bloom brin_multi
# psql depends on create_am
# amutils depends on geometry, create_index_spgist, hash_index, brin
# ----------
-test: create_table_like alter_generic alter_operator misc async dbsize merge misc_functions sysviews tsrf tid tidscan tidrangescan collate.icu.utf8 incremental_sort create_role without_overlaps
+test: create_table_like alter_generic alter_operator misc async dbsize merge misc_functions sysviews tsrf tid tidscan tidrangescan collate.utf8 collate.icu.utf8 incremental_sort create_role without_overlaps
-# collate.*.utf8 tests cannot be run in parallel with each other
+# collate.linux.utf8 and collate.icu.utf8 tests cannot be run in parallel with each other
test: rules psql psql_crosstab amutils stats_ext collate.linux.utf8 collate.windows.win1252
# ----------
diff --git a/src/test/regress/sql/collate.sql b/src/test/regress/sql/collate.sql
index c3d40fc195..01d5c69fe4 100644
--- a/src/test/regress/sql/collate.sql
+++ b/src/test/regress/sql/collate.sql
@@ -244,6 +244,16 @@ EXPLAIN (COSTS OFF)
-- CREATE/DROP COLLATION
+CREATE COLLATION builtin_c ( PROVIDER = builtin, LOCALE = "C" );
+CREATE COLLATION builtin_posix ( PROVIDER = builtin, LOCALE = "POSIX" );
+
+SELECT b FROM collate_test1 ORDER BY b COLLATE builtin_c;
+
+CREATE COLLATION builtin2 ( PROVIDER = builtin ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "en_US" ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LC_CTYPE = "C", LC_COLLATE = "C" ); -- fails
+CREATE COLLATION builtin2 ( PROVIDER = builtin, LOCALE = "POSIX", LC_CTYPE = "POSIX" ); -- fails
+
CREATE COLLATION mycoll1 FROM "C";
CREATE COLLATION mycoll2 ( LC_COLLATE = "POSIX", LC_CTYPE = "POSIX" );
CREATE COLLATION mycoll3 FROM "default"; -- intentionally unsupported
diff --git a/src/test/regress/sql/collate.utf8.sql b/src/test/regress/sql/collate.utf8.sql
new file mode 100644
index 0000000000..584c50f915
--- /dev/null
+++ b/src/test/regress/sql/collate.utf8.sql
@@ -0,0 +1,54 @@
+/*
+ * This test is for collations and character operations when using the
+ * builtin provider with the C.UTF-8 locale.
+ */
+
+/* skip test if not UTF8 server encoding */
+SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+SET client_encoding TO UTF8;
+
+--
+-- Test preinstalled PG_C_UTF8 collation.
+--
+
+CREATE TABLE builtin_test (
+ t TEXT COLLATE PG_C_UTF8
+);
+INSERT INTO builtin_test VALUES
+ ('abc DEF'),
+ ('ábc DÉF'),
+ ('DŽxxDŽ džxxDž Džxxdž'),
+ ('ȺȺȺ'),
+ ('ⱥⱥⱥ'),
+ ('ⱥȺ');
+
+SELECT
+ t, lower(t), initcap(t), upper(t),
+ length(convert_to(t, 'UTF8')) AS t_bytes,
+ length(convert_to(lower(t), 'UTF8')) AS lower_t_bytes,
+ length(convert_to(initcap(t), 'UTF8')) AS initcap_t_bytes,
+ length(convert_to(upper(t), 'UTF8')) AS upper_t_bytes
+ FROM builtin_test;
+
+DROP TABLE builtin_test;
+
+-- character classes
+
+SELECT 'xyz' ~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+SELECT 'xyz' !~ '[[:upper:]]' COLLATE PG_C_UTF8;
+SELECT '@' !~ '[[:alnum:]]' COLLATE PG_C_UTF8;
+SELECT '@' ~ '[[:punct:]]' COLLATE PG_C_UTF8; -- symbols are punctuation in posix
+SELECT 'a8a' ~ '[[:digit:]]' COLLATE PG_C_UTF8;
+SELECT '൧' !~ '\d' COLLATE PG_C_UTF8; -- only 0-9 considered digits in posix
+
+-- case mapping
+
+SELECT 'xYz' ~* 'XyZ' COLLATE PG_C_UTF8;
+SELECT 'xAb' ~* '[W-Y]' COLLATE PG_C_UTF8;
+SELECT 'xAb' !~* '[c-d]' COLLATE PG_C_UTF8;
+SELECT 'Δ' ~* '[α-λ]' COLLATE PG_C_UTF8;
+SELECT 'δ' ~* '[Γ-Λ]' COLLATE PG_C_UTF8; -- same as above with cases reversed
--
2.34.1
^ permalink raw reply [nested|flat] 12+ messages in thread
end of thread, other threads:[~2024-02-16 00:13 UTC | newest]
Thread overview: 12+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-09-12 05:22 [PATCH v6 6/7] Row pattern recognition patch (tests). Tatsuo Ishii <[email protected]>
2024-01-12 02:02 Re: Built-in CTYPE provider Jeff Davis <[email protected]>
2024-01-12 16:58 ` Re: Built-in CTYPE provider Jeff Davis <[email protected]>
2024-01-15 14:30 ` Re: Built-in CTYPE provider Daniel Verite <[email protected]>
2024-01-18 12:53 ` Re: Built-in CTYPE provider Peter Eisentraut <[email protected]>
2024-01-18 19:42 ` Re: Built-in CTYPE provider Daniel Verite <[email protected]>
2024-01-18 22:03 ` Re: Built-in CTYPE provider Jeff Davis <[email protected]>
2024-01-22 18:49 ` Re: Built-in CTYPE provider Peter Eisentraut <[email protected]>
2024-01-22 23:33 ` Re: Built-in CTYPE provider Jeff Davis <[email protected]>
2024-02-07 09:53 ` Re: Built-in CTYPE provider Peter Eisentraut <[email protected]>
2024-02-13 02:01 ` Re: Built-in CTYPE provider Jeff Davis <[email protected]>
2024-02-16 00:13 ` Re: Built-in CTYPE provider Jeff Davis <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox