pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Marina Polyakova <m.polyakova@postgrespro.ru>
To: pgsql-hackers@postgresql.org
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Cc: Julien Rouhaud <rjuju123@gmail.com>
Cc: Daniel Verite <daniel@manitou-mail.org>
Cc: pryzby@telsasoft.com
Cc: AndrewBille@gmail.com
Cc: michael@paquier.xyz
Subject: Re: ICU for global collation
Date: Tue, 23 Aug 2022 20:59:02 +0300
Message-ID: <7ad26354e75259f59c4a6c6997b8ee32@postgrespro.ru> (raw)
In-Reply-To: <YwRUn4J8+uqlyUYQ@paquier.xyz>
References: <87e93704-b7f7-71f6-161d-8713e6334ed3@enterprisedb.com>
	<704405ed-f1fa-d2a7-b9f9-48a07fb45648@enterprisedb.com>
	<20220625151930.GH22452@telsasoft.com>
	<20220626035124.t36putp232nfl5im@jrouhaud>
	<54b7bd53-ae10-6258-21e6-b799f7480803@enterprisedb.com>
	<YrlRS847pNnxwu+v@paquier.xyz>
	<97f2c29e-24d9-3660-60cf-9138c0c47d90@enterprisedb.com>
	<20220628013205.GH19662@telsasoft.com>
	<f385ba25e7f8be427b8c582e5cca7d79@postgrespro.ru>
	<7d74b2d7-7fc6-f55b-3f77-c0e05569b64e@enterprisedb.com>
	<YwRUn4J8+uqlyUYQ@paquier.xyz>

My colleague Andrew Bille found another bug in master 
(b4e936859dc441102eb0b6fb7a104f3948c90490) and REL_15_STABLE 
(2c63b0930aee1bb5c265fad4a65c9d0b62b1f9da): pg_collation.colliculocale 
is not dumped. See check_icu_locale.sh:

In the old cluster:
SELECT collname, colliculocale FROM pg_collation WHERE collname = 
'testcoll_backwards'
       collname      |   colliculocale
--------------------+-------------------
  testcoll_backwards | @colBackwards=yes
(1 row)

In the new cluster:
SELECT collname, colliculocale FROM pg_collation WHERE collname = 
'testcoll_backwards'
       collname      | colliculocale
--------------------+---------------
  testcoll_backwards |
(1 row)

diff_dump_colliculocale.patch works for me.

-- 
Marina Polyakova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

check_icu_locale.sh (text/plain, 664B) download initdb -D data_old && pg_ctl -D data_old -l logfile_old start && psql -ac "CREATE COLLATION testcoll_backwards (provider = icu, locale = '@colBackwards=yes')" postgres && echo "In the old cluster:" && psql -ac "SELECT collname, colliculocale FROM pg_collation WHERE collname = 'testcoll_backwards'" postgres && pg_dump postgres > dump_postgres.sql && pg_ctl -D data_old stop && initdb -D data_new && pg_ctl -D data_new -l logfile_new start && psql -v ON_ERROR_STOP=1 -f dump_postgres.sql postgres && echo "In the new cluster:" && psql -ac "SELECT collname, colliculocale FROM pg_collation WHERE collname = 'testcoll_backwards'" postgres && pg_ctl -D data_new stop
diff_dump_colliculocale.patch (text/x-diff, 5.1K) download diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile index 2f524b09bf53a55037013d78148f8cbca4fa7eee..9dc5a784dd2d1ce58a6284f19c54730364779c4d 100644 --- a/src/bin/pg_dump/Makefile +++ b/src/bin/pg_dump/Makefile @@ -17,6 +17,7 @@ top_builddir = ../../.. include $(top_builddir)/src/Makefile.global export GZIP_PROGRAM=$(GZIP) +export with_icu override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS) LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index 2c6891573296b395c5bd6b627d061de657355e9c..e8e8f69e30c8a787e5dff157f5d1619917638d7d 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -13084,9 +13084,11 @@ dumpCollation(Archive *fout, const CollInfo *collinfo) int i_collisdeterministic; int i_collcollate; int i_collctype; + int i_colliculocale; const char *collprovider; const char *collcollate; const char *collctype; + const char *colliculocale; /* Do nothing in data-only dump */ if (dopt->dataOnly) @@ -13117,6 +13119,13 @@ dumpCollation(Archive *fout, const CollInfo *collinfo) appendPQExpBufferStr(query, "true AS collisdeterministic, "); + if (fout->remoteVersion >= 150000) + appendPQExpBufferStr(query, + "colliculocale, "); + else + appendPQExpBufferStr(query, + "NULL AS colliculocale, "); + appendPQExpBuffer(query, "collcollate, " "collctype " @@ -13130,10 +13139,24 @@ dumpCollation(Archive *fout, const CollInfo *collinfo) i_collisdeterministic = PQfnumber(res, "collisdeterministic"); i_collcollate = PQfnumber(res, "collcollate"); i_collctype = PQfnumber(res, "collctype"); + i_colliculocale = PQfnumber(res, "colliculocale"); collprovider = PQgetvalue(res, 0, i_collprovider); - collcollate = PQgetvalue(res, 0, i_collcollate); - collctype = PQgetvalue(res, 0, i_collctype); + + if (!PQgetisnull(res, 0, i_collcollate)) + collcollate = PQgetvalue(res, 0, i_collcollate); + else + collcollate = NULL; + + if (!PQgetisnull(res, 0, i_collctype)) + collctype = PQgetvalue(res, 0, i_collctype); + else + collctype = NULL; + + if (!PQgetisnull(res, 0, i_colliculocale)) + colliculocale = PQgetvalue(res, 0, i_colliculocale); + else + colliculocale = NULL; appendPQExpBuffer(delq, "DROP COLLATION %s;\n", fmtQualifiedDumpable(collinfo)); @@ -13156,17 +13179,28 @@ dumpCollation(Archive *fout, const CollInfo *collinfo) if (strcmp(PQgetvalue(res, 0, i_collisdeterministic), "f") == 0) appendPQExpBufferStr(q, ", deterministic = false"); - if (strcmp(collcollate, collctype) == 0) + if (colliculocale != NULL) { appendPQExpBufferStr(q, ", locale = "); - appendStringLiteralAH(q, collcollate, fout); + appendStringLiteralAH(q, colliculocale, fout); } else { - appendPQExpBufferStr(q, ", lc_collate = "); - appendStringLiteralAH(q, collcollate, fout); - appendPQExpBufferStr(q, ", lc_ctype = "); - appendStringLiteralAH(q, collctype, fout); + Assert(collcollate != NULL); + Assert(collctype != NULL); + + if (strcmp(collcollate, collctype) == 0) + { + appendPQExpBufferStr(q, ", locale = "); + appendStringLiteralAH(q, collcollate, fout); + } + else + { + appendPQExpBufferStr(q, ", lc_collate = "); + appendStringLiteralAH(q, collcollate, fout); + appendPQExpBufferStr(q, ", lc_ctype = "); + appendStringLiteralAH(q, collctype, fout); + } } /* diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl index b10e1c4c0d4d27bf1a427c8cfe36f8dea4dad78a..7f4be26bce6fb37b83e4735c541a2215841f5e95 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -1593,6 +1593,15 @@ my %tests = ( like => { %full_runs, section_pre_data => 1, }, }, + 'CREATE COLLATION icu_collation' => { + create_order => 76, + create_sql => "CREATE COLLATION icu_collation (PROVIDER = icu, LOCALE = 'C');", + regexp => + qr/CREATE COLLATION public.icu_collation \(provider = icu, locale = 'C'(, version = '[^']*')?\);/m, + icu_collation => 1, + like => { %full_runs, section_pre_data => 1, }, + }, + 'CREATE CAST FOR timestamptz' => { create_order => 51, create_sql => @@ -3931,6 +3940,11 @@ foreach my $test ( $test_db = $tests{$test}->{database}; } + if (defined($tests{$test}->{icu_collation})) + { + $tests{$test}->{collation} = 1; + } + if ($tests{$test}->{create_sql}) { @@ -3940,6 +3954,12 @@ foreach my $test ( next; } + # Skip any icu-related collation commands if build was without icu + if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu_collation})) + { + next; + } + # Skip tests specific to LZ4 if this build does not support # this option. if (!$supports_lz4 && defined($tests{$test}->{lz4})) @@ -4141,6 +4161,12 @@ foreach my $run (sort keys %pgdump_runs) next; } + # Skip any icu-related collation commands if build was without icu + if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu_collation})) + { + next; + } + # Skip tests specific to LZ4 if this build does not support # this option. if (!$supports_lz4 && defined($tests{$test}->{lz4}))

view thread (111+ messages)  latest in thread

Message-ID: <7ad26354e75259f59c4a6c6997b8ee32@postgrespro.ru>
Permalink:  ../7ad26354e75259f59c4a6c6997b8ee32@postgrespro.ru/
Also on:    postgresql.org/message-id/7ad26354e75259f59c4a6c6997b8ee32@postgrespro.ru

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: m.polyakova@postgrespro.ru, peter.eisentraut@enterprisedb.com, rjuju123@gmail.com, daniel@manitou-mail.org, pryzby@telsasoft.com, AndrewBille@gmail.com, michael@paquier.xyz
  Subject: Re: ICU for global collation
  In-Reply-To: <7ad26354e75259f59c4a6c6997b8ee32@postgrespro.ru>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox