pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedforeign_data test fails with non-C locale
24+ messages / 7 participants
[nested] [flat]
* foreign_data test fails with non-C locale
@ 2009-01-09 14:12 Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
0 siblings, 2 replies; 24+ messages in thread
From: Heikki Linnakangas @ 2009-01-09 14:12 UTC (permalink / raw)
To: pgsql-hackers
The foreign_data test case is failing when I run "make installcheck"
against a server that's been initialized with a locale other than C
(en_GB.UTF-8).
The reason is the different ordering of upper and lower case characters,
per attached diff file. We can simply add an alternative expected output
file, but I'd prefer not to if we can modify the test case instead. We
could rename some of the object so that they sort the same in all
locales, but that seems a bit awkward in this case.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
Attachments:
regression.diffs (text/plain, 8.5K) download
*** /home/hlinnaka/git-sandbox/pgsql/src/test/regress/expected/foreign_data.out Fri Jan 9 13:11:06 2009
--- /home/hlinnaka/git-sandbox/pgsql/src/test/regress/results/foreign_data.out Fri Jan 9 15:47:27 2009
***************
*** 658,667 ****
SELECT * FROM information_schema.foreign_servers ORDER BY 1, 2;
foreign_server_catalog | foreign_server_name | foreign_data_wrapper_catalog | foreign_data_wrapper_name | foreign_server_type | foreign_server_version | authorization_identifier
------------------------+---------------------+------------------------------+---------------------------+---------------------+------------------------+--------------------------
- regression | S6 | regression | foo | | | foreign_data_user
regression | s4 | regression | foo | oracle | | foreign_data_user
regression | s5 | regression | foo | | 15.0 | regress_test_role
regression | s6 | regression | foo | | 16.0 | regress_test_indirect
regression | s8 | regression | postgresql | | | foreign_data_user
regression | st1 | regression | foo | | | regress_test_indirect
regression | st2 | regression | foo | | | regress_test_role
--- 658,667 ----
SELECT * FROM information_schema.foreign_servers ORDER BY 1, 2;
foreign_server_catalog | foreign_server_name | foreign_data_wrapper_catalog | foreign_data_wrapper_name | foreign_server_type | foreign_server_version | authorization_identifier
------------------------+---------------------+------------------------------+---------------------------+---------------------+------------------------+--------------------------
regression | s4 | regression | foo | oracle | | foreign_data_user
regression | s5 | regression | foo | | 15.0 | regress_test_role
regression | s6 | regression | foo | | 16.0 | regress_test_indirect
+ regression | S6 | regression | foo | | | foreign_data_user
regression | s8 | regression | postgresql | | | foreign_data_user
regression | st1 | regression | foo | | | regress_test_indirect
regression | st2 | regression | foo | | | regress_test_role
***************
*** 670,680 ****
SELECT * FROM information_schema.foreign_server_options ORDER BY 1, 2, 3;
foreign_server_catalog | foreign_server_name | option_name | option_value
------------------------+---------------------+------------------+--------------
- regression | S6 | mixed_case_names | true
regression | s4 | dbname | b
regression | s4 | host | a
regression | s6 | dbname | b
regression | s6 | host | a
regression | s8 | connect_timeout | 30
regression | s8 | dbname | db1
(7 rows)
--- 670,680 ----
SELECT * FROM information_schema.foreign_server_options ORDER BY 1, 2, 3;
foreign_server_catalog | foreign_server_name | option_name | option_value
------------------------+---------------------+------------------+--------------
regression | s4 | dbname | b
regression | s4 | host | a
regression | s6 | dbname | b
regression | s6 | host | a
+ regression | S6 | mixed_case_names | true
regression | s8 | connect_timeout | 30
regression | s8 | dbname | db1
(7 rows)
***************
*** 682,693 ****
SELECT * FROM information_schema.user_mappings ORDER BY 1, 2, 3;
authorization_identifier | foreign_server_catalog | foreign_server_name
--------------------------+------------------------+---------------------
PUBLIC | regression | s4
PUBLIC | regression | s8
PUBLIC | regression | st1
- foreign_data_user | regression | S6
- foreign_data_user | regression | s4
- foreign_data_user | regression | s8
regress_test_role | regression | s5
regress_test_role | regression | s6
regress_test_role | regression | st1
--- 682,693 ----
SELECT * FROM information_schema.user_mappings ORDER BY 1, 2, 3;
authorization_identifier | foreign_server_catalog | foreign_server_name
--------------------------+------------------------+---------------------
+ foreign_data_user | regression | s4
+ foreign_data_user | regression | S6
+ foreign_data_user | regression | s8
PUBLIC | regression | s4
PUBLIC | regression | s8
PUBLIC | regression | st1
regress_test_role | regression | s5
regress_test_role | regression | s6
regress_test_role | regression | st1
***************
*** 696,705 ****
SELECT * FROM information_schema.user_mapping_options ORDER BY 1, 2, 3, 4;
authorization_identifier | foreign_server_catalog | foreign_server_name | option_name | option_value
--------------------------+------------------------+---------------------+-------------+-----------------
- PUBLIC | regression | s4 | mapping | is public
- PUBLIC | regression | st1 | modified | 1
foreign_data_user | regression | S6 | username | test_mixed_case
foreign_data_user | regression | s8 | password | public
regress_test_role | regression | s5 | modified | 1
regress_test_role | regression | s6 | username | test
regress_test_role | regression | st1 | password | boo
--- 696,705 ----
SELECT * FROM information_schema.user_mapping_options ORDER BY 1, 2, 3, 4;
authorization_identifier | foreign_server_catalog | foreign_server_name | option_name | option_value
--------------------------+------------------------+---------------------+-------------+-----------------
foreign_data_user | regression | S6 | username | test_mixed_case
foreign_data_user | regression | s8 | password | public
+ PUBLIC | regression | s4 | mapping | is public
+ PUBLIC | regression | st1 | modified | 1
regress_test_role | regression | s5 | modified | 1
regress_test_role | regression | s6 | username | test
regress_test_role | regression | st1 | password | boo
======================================================================
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 14:17 Andrew Dunstan <andrew@dunslane.net>
parent: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
1 sibling, 2 replies; 24+ messages in thread
From: Andrew Dunstan @ 2009-01-09 14:17 UTC (permalink / raw)
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; +Cc: pgsql-hackers
Heikki Linnakangas wrote:
> The foreign_data test case is failing when I run "make installcheck"
> against a server that's been initialized with a locale other than C
> (en_GB.UTF-8).
>
> The reason is the different ordering of upper and lower case
> characters, per attached diff file. We can simply add an alternative
> expected output file, but I'd prefer not to if we can modify the test
> case instead. We could rename some of the object so that they sort the
> same in all locales, but that seems a bit awkward in this case.
Regression tests have always failed on non-C locales AFAIK. The
buildfarm goes out of its way to avoid that.
cheers
andrew
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 14:23 Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
parent: Andrew Dunstan <andrew@dunslane.net>
1 sibling, 0 replies; 24+ messages in thread
From: Heikki Linnakangas @ 2009-01-09 14:23 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: pgsql-hackers
Andrew Dunstan wrote:
> Heikki Linnakangas wrote:
>> The foreign_data test case is failing when I run "make installcheck"
>> against a server that's been initialized with a locale other than C
>> (en_GB.UTF-8).
>>
>> The reason is the different ordering of upper and lower case
>> characters, per attached diff file. We can simply add an alternative
>> expected output file, but I'd prefer not to if we can modify the test
>> case instead. We could rename some of the object so that they sort the
>> same in all locales, but that seems a bit awkward in this case.
>
> Regression tests have always failed on non-C locales AFAIK. The
> buildfarm goes out of its way to avoid that.
No, that's the only test case that's failing.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 14:25 Peter Eisentraut <peter_e@gmx.net>
parent: Andrew Dunstan <andrew@dunslane.net>
1 sibling, 1 reply; 24+ messages in thread
From: Peter Eisentraut @ 2009-01-09 14:25 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Andrew Dunstan wrote:
> Regression tests have always failed on non-C locales AFAIK. The
> buildfarm goes out of its way to avoid that.
The regression tests should work just fine in non-C locales. If the
buildfarm goes out of its way to avoid non-C locales, then it loses some
significant code coverage, considering that there are several variant
code paths for locales, and considering the amount of users that use them.
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 14:51 Peter Eisentraut <peter_e@gmx.net>
parent: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
1 sibling, 1 reply; 24+ messages in thread
From: Peter Eisentraut @ 2009-01-09 14:51 UTC (permalink / raw)
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; +Cc: pgsql-hackers
Heikki Linnakangas wrote:
> The foreign_data test case is failing when I run "make installcheck"
> against a server that's been initialized with a locale other than C
> (en_GB.UTF-8).
I have removed one of the differences but can't reproduce the other
right now (although it looks consequential). I'll check that on a
different machine.
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 15:21 Andrew Dunstan <andrew@dunslane.net>
parent: Peter Eisentraut <peter_e@gmx.net>
0 siblings, 1 reply; 24+ messages in thread
From: Andrew Dunstan @ 2009-01-09 15:21 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Peter Eisentraut wrote:
> Andrew Dunstan wrote:
>> Regression tests have always failed on non-C locales AFAIK. The
>> buildfarm goes out of its way to avoid that.
>
> The regression tests should work just fine in non-C locales. If the
> buildfarm goes out of its way to avoid non-C locales, then it loses
> some significant code coverage, considering that there are several
> variant code paths for locales, and considering the amount of users
> that use them.
It was discussed here at the time, IIRC, and we put in the check
precisely because other locales broke the buildfarm. Originally
buildfarm just inherited the locale from its environment.
If it is no longer true that other locales break the tests, then I'm
happy to examine alternatives.
cheers
andrew
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 16:24 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andrew Dunstan <andrew@dunslane.net>
0 siblings, 2 replies; 24+ messages in thread
From: Tom Lane @ 2009-01-09 16:24 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Andrew Dunstan <andrew@dunslane.net> writes:
> Peter Eisentraut wrote:
>> The regression tests should work just fine in non-C locales.
> It was discussed here at the time, IIRC, and we put in the check
> precisely because other locales broke the buildfarm. Originally
> buildfarm just inherited the locale from its environment.
I don't think we are prepared to buy into a general policy that the
regression tests should pass in *any* locale; maintaining a large
number of variant expected-files isn't very practical. However, the
de facto policy is that we try to keep them passing in locales that
are used by any of the regular developers. I think it would be useful
to have buildfarm members testing in a few common locales.
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 16:45 Guillaume Smet <guillaume.smet@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 24+ messages in thread
From: Guillaume Smet @ 2009-01-09 16:45 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
On Fri, Jan 9, 2009 at 5:24 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> However, the
> de facto policy is that we try to keep them passing in locales that
> are used by any of the regular developers. I think it would be useful
> to have buildfarm members testing in a few common locales.
If you define common locales, I can set up as many new animals as
needed to cover the locales needed for any branch we'd like to test.
Perhaps we should add a parameter to the buildfarm config file so that
the buildfarm script can check the locale is accepted and set it
directly. Considering that we won't have the locale information in the
animal description, it's a good way to have it in the report.
Just let me know.
--
Guillaume
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-09 17:16 Andrew Dunstan <andrew@dunslane.net>
parent: Guillaume Smet <guillaume.smet@gmail.com>
0 siblings, 1 reply; 24+ messages in thread
From: Andrew Dunstan @ 2009-01-09 17:16 UTC (permalink / raw)
To: Guillaume Smet <guillaume.smet@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Guillaume Smet wrote:
> On Fri, Jan 9, 2009 at 5:24 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
>> However, the
>> de facto policy is that we try to keep them passing in locales that
>> are used by any of the regular developers. I think it would be useful
>> to have buildfarm members testing in a few common locales.
>>
>
> If you define common locales, I can set up as many new animals as
> needed to cover the locales needed for any branch we'd like to test.
>
> Perhaps we should add a parameter to the buildfarm config file so that
> the buildfarm script can check the locale is accepted and set it
> directly. Considering that we won't have the locale information in the
> animal description, it's a good way to have it in the report.
>
>
>
Sure, we can easily have buildfarm's initdb step set any locale (and
encoding, for that matter) we like. That's a simple change.
cheers
andrew
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-11 09:41 Peter Eisentraut <peter_e@gmx.net>
parent: Peter Eisentraut <peter_e@gmx.net>
0 siblings, 0 replies; 24+ messages in thread
From: Peter Eisentraut @ 2009-01-11 09:41 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
On Friday 09 January 2009 16:51:44 Peter Eisentraut wrote:
> Heikki Linnakangas wrote:
> > The foreign_data test case is failing when I run "make installcheck"
> > against a server that's been initialized with a locale other than C
> > (en_GB.UTF-8).
>
> I have removed one of the differences but can't reproduce the other
> right now (although it looks consequential). I'll check that on a
> different machine.
Also fixed now.
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-11 10:54 Peter Eisentraut <peter_e@gmx.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 3 replies; 24+ messages in thread
From: Peter Eisentraut @ 2009-01-11 10:54 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andrew Dunstan <andrew@dunslane.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
On Friday 09 January 2009 18:24:55 Tom Lane wrote:
> I don't think we are prepared to buy into a general policy that the
> regression tests should pass in *any* locale; maintaining a large
> number of variant expected-files isn't very practical. However, the
> de facto policy is that we try to keep them passing in locales that
> are used by any of the regular developers. I think it would be useful
> to have buildfarm members testing in a few common locales.
This called for an extensive test ... :-)
My glibc installation supplies 668 locales (locale -a), which appear to
represent about 225 distinct language/country combinations. (The rest are
encoding variants.)
I ran the regression tests with all of them, and got 95 failures (out of 668).
15 out of the 95 failures are initdb not completing because the encoding
specified by the locale is not supported by PostgreSQL. But it appears that
at least xx_XX.utf8 works for each of these cases, so the language is
supported in some way.
The remaining 80 failures are more-or-less linguistic issues that belong to
the following 26 language/country combinations:
az_AZ sorts k < q < l; Turkish i
br_FR sorts ch separately
crh_UA Turkish i
cs_CZ sorts ch separately; sorts st = s
cy_GB sorts ch separately
da_DK sorts aa = å > z
es_EC sorts ch separately
es_US sorts ch separately
et_EE sorts v = w
fo_FO sorts aa = å > z
ha_NG sorts sh separately
hsb_DE sorts ch separately
ig_NG sorts ch separately; sorts sh separately
ik_CA sorts ch separately
kl_GL sorts aa = å > z
nb_NO sorts aa = å > z
nn_NO sorts aa = å > z
om_ET sorts ch separately (> z); sorts sh separately
om_KE sorts ch separately (> z); sorts sh separately
pl_PL (some other inexplicable sorting regression)
sk_SK sorts ch separately; sorts st = s
sv_SE sorts v = w
tk_TM sorts v = w
tr_CY Turkish i
tr_TR Turkish i
tt_RU sorts k < q < l
The "Turkish i" failures are in the tsearch tests. I'm not completely
comfortable that it's doing the right thing there.
We could easily get rid of the aa, ch, and v/w failures by adjusting the test
data, since the data is completely coincidental anyway. I propose to do
that, and document these issues so that they can be avoided in future tests.
I'm not so worried about the other cases.
Also, considering that some of these alternative sorting rules appear to be
controversial even among users of the language (e.g., we have had actual bug
reports that the es_EC rule is wrong, and the sv_SE rule is also obsolete
according to the language regulators), it might be interesting to write a
small test program that can tell users how their current locale behaves in
known corner cases.
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-11 16:46 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter_e@gmx.net>
2 siblings, 1 reply; 24+ messages in thread
From: Tom Lane @ 2009-01-11 16:46 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: pgsql-hackers@postgresql.org, Andrew Dunstan <andrew@dunslane.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
Peter Eisentraut <peter_e@gmx.net> writes:
> This called for an extensive test ... :-)
> My glibc installation supplies 668 locales (locale -a), which appear to
> represent about 225 distinct language/country combinations. (The rest are
> encoding variants.)
> I ran the regression tests with all of them, and got 95 failures (out of 668).
Fascinating data. I assume you did not remove the existing
locale-variant expected files? IOW this isn't "all the locale
dependencies", but "all the ones we didn't fix previously"?
> We could easily get rid of the aa, ch, and v/w failures by adjusting the test
> data, since the data is completely coincidental anyway. I propose to do
> that, and document these issues so that they can be avoided in future tests.
I have no confidence in the ability of some documentation to keep the
tests clean. However, if we had buildfarm members testing in locales
that exercise each of those cases, it'd be all right.
If we try to fix those cases I think we should try to fix Turkish i
as well ... but I concur that first requires determining if it's
behaving wrong or not. Devrim, or someone?
> Also, considering that some of these alternative sorting rules appear to be
> controversial even among users of the language (e.g., we have had actual bug
> reports that the es_EC rule is wrong, and the sv_SE rule is also obsolete
> according to the language regulators), it might be interesting to write a
> small test program that can tell users how their current locale behaves in
> known corner cases.
Considering the number of people who complain about en_US (expecting C
sort order instead), I'm not sure you should consider this a corner
case.
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-11 19:45 Devrim GÜNDÜZ <devrim@gunduz.org>
parent: Peter Eisentraut <peter_e@gmx.net>
2 siblings, 0 replies; 24+ messages in thread
From: Devrim GÜNDÜZ @ 2009-01-11 19:45 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: pgsql-hackers@postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>; Andrew Dunstan <andrew@dunslane.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
On Sun, 2009-01-11 at 12:54 +0200, Peter Eisentraut wrote:
> The "Turkish i" failures are in the tsearch tests. I'm not completely
> comfortable that it's doing the right thing there.
AFAIK, ISO-8859-9 is broken in a way, and the Turkish maintainers are
not interested in fixing them -- they ask us to move to tr_TR.UTF-8.
--
Devrim GÜNDÜZ, RHCE
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org
Attachments:
signature.asc (application/pgp-signature, 196B) download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-11 20:01 Devrim GÜNDÜZ <devrim@gunduz.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 24+ messages in thread
From: Devrim GÜNDÜZ @ 2009-01-11 20:01 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Peter Eisentraut <peter_e@gmx.net>; pgsql-hackers@postgresql.org, Andrew Dunstan <andrew@dunslane.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
On Sun, 2009-01-11 at 11:46 -0500, Tom Lane wrote:
> If we try to fix those cases I think we should try to fix Turkish i
> as well ... but I concur that first requires determining if it's
> behaving wrong or not. Devrim, or someone?
What exactly do you want to see?
Regards,
--
Devrim GÜNDÜZ, RHCE
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org
Attachments:
signature.asc (application/pgp-signature, 196B) download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-12 10:06 Peter Eisentraut <peter_e@gmx.net>
parent: Devrim GÜNDÜZ <devrim@gunduz.org>
0 siblings, 1 reply; 24+ messages in thread
From: Peter Eisentraut @ 2009-01-12 10:06 UTC (permalink / raw)
To: Devrim GÜNDÜZ <devrim@gunduz.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
Devrim GÜNDÜZ wrote:
> On Sun, 2009-01-11 at 11:46 -0500, Tom Lane wrote:
>
>> If we try to fix those cases I think we should try to fix Turkish i
>> as well ... but I concur that first requires determining if it's
>> behaving wrong or not. Devrim, or someone?
>
> What exactly do you want to see?
Using a glibc system, initdb with --locale=tr_TR (or tr_TR.utf8 or
whatever) and run make installcheck. You should see test failures in
the tsearch and tsdicts tests that appear to relate to issues with
lowercasing the "I" letter correctly. And then use your language skills
to determine what the correct behavior is. ;-)
Note that on Mac OS X with tr_TR locales, the tests do not fail.
I actually suspect that both current answers are wrong.
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-12 10:40 Devrim GÜNDÜZ <devrim@gunduz.org>
parent: Peter Eisentraut <peter_e@gmx.net>
0 siblings, 0 replies; 24+ messages in thread
From: Devrim GÜNDÜZ @ 2009-01-12 10:40 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
Hi,
On Mon, 2009-01-12 at 12:06 +0200, Peter Eisentraut wrote:
> Using a glibc system, initdb with --locale=tr_TR (or tr_TR.utf8 or
> whatever) and run make installcheck. You should see test failures in
> the tsearch and tsdicts tests that appear to relate to issues with
> lowercasing the "I" letter correctly.
Yep, I ran them already, and as you wrote, I'm getting 3 errors (tsearch
tests + foreign_data test).
> And then use your language skills to determine what the correct
> behavior is. ;-)
SKIES would be skıes (dotless i).
Here is the conversion table:
I (capital) <-> ı
İ (capital <-> i
We also have a few more chars, but I did not test them yet:
ş <-> Ş (capital) (S with a tail)
ü <-> Ü (capital) (U with dots)
ç <-> Ç (capital) (C with a tail)
ğ <-> Ğ (capital) (G with a hat)
ö <-> Ö (capital) (O with dots)
Regards,
--
Devrim GÜNDÜZ, RHCE
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org
Attachments:
signature.asc (application/pgp-signature, 196B) download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-19 20:13 Zdenek Kotala <Zdenek.Kotala@Sun.COM>
parent: Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 24+ messages in thread
From: Zdenek Kotala @ 2009-01-19 20:13 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Guillaume Smet <guillaume.smet@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Andrew Dunstan píše v pá 09. 01. 2009 v 12:16 -0500:
>
> Guillaume Smet wrote:
> > On Fri, Jan 9, 2009 at 5:24 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> >> However, the
> >> de facto policy is that we try to keep them passing in locales that
> >> are used by any of the regular developers. I think it would be useful
> >> to have buildfarm members testing in a few common locales.
> >>
> >
> > If you define common locales, I can set up as many new animals as
> > needed to cover the locales needed for any branch we'd like to test.
> >
> > Perhaps we should add a parameter to the buildfarm config file so that
> > the buildfarm script can check the locale is accepted and set it
> > directly. Considering that we won't have the locale information in the
> > animal description, it's a good way to have it in the report.
> >
> >
> >
>
> Sure, we can easily have buildfarm's initdb step set any locale (and
> encoding, for that matter) we like. That's a simple change.
Will be possible to set more locales and run tests without recompilation
on all of them? For example I have installed all Solaris'es locales on
my animal, but currently it means that I need perform whole cycle for
each locale.
Zdenek
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-19 20:39 Zdenek Kotala <Zdenek.Kotala@Sun.COM>
parent: Peter Eisentraut <peter_e@gmx.net>
2 siblings, 1 reply; 24+ messages in thread
From: Zdenek Kotala @ 2009-01-19 20:39 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: pgsql-hackers@postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>; Andrew Dunstan <andrew@dunslane.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
Peter Eisentraut píše v ne 11. 01. 2009 v 12:54 +0200:
> The remaining 80 failures are more-or-less linguistic issues that belong to
> the following 26 language/country combinations:
>
> cs_CZ sorts ch separately; sorts st = s
s < st
Zdenek
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-20 05:44 Peter Eisentraut <peter_e@gmx.net>
parent: Zdenek Kotala <Zdenek.Kotala@Sun.COM>
0 siblings, 0 replies; 24+ messages in thread
From: Peter Eisentraut @ 2009-01-20 05:44 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Zdenek Kotala <Zdenek.Kotala@sun.com>
On Monday 19 January 2009 22:39:30 Zdenek Kotala wrote:
> Peter Eisentraut píše v ne 11. 01. 2009 v 12:54 +0200:
> > The remaining 80 failures are more-or-less linguistic issues that belong
> > to the following 26 language/country combinations:
> >
> >
> > cs_CZ sorts ch separately; sorts st = s
>
> s < st
I had initially misinterpreted the failures. The real difference is that
Czech sorts numbers after letters, most other locales do it the other way
around.
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-24 04:57 Andrew Dunstan <andrew@dunslane.net>
parent: Zdenek Kotala <Zdenek.Kotala@Sun.COM>
0 siblings, 1 reply; 24+ messages in thread
From: Andrew Dunstan @ 2009-01-24 04:57 UTC (permalink / raw)
To: Zdenek Kotala <Zdenek.Kotala@Sun.COM>; +Cc: Guillaume Smet <guillaume.smet@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Zdenek Kotala wrote:
> Andrew Dunstan píše v pá 09. 01. 2009 v 12:16 -0500:
>
>
>> Sure, we can easily have buildfarm's initdb step set any locale (and
>> encoding, for that matter) we like. That's a simple change.
>>
>
> Will be possible to set more locales and run tests without recompilation
> on all of them? For example I have installed all Solaris'es locales on
> my animal, but currently it means that I need perform whole cycle for
> each locale.
>
I'm working on this. Yes, you will be able to specify a list of locales
to check. For each locale the following tests will be run:
installcheck, pl-installcheck, and contrib-installcheck.
However, our tests are still a bit short of working across locales.
PL-check gives the diff below on PLTCL tests under en_US locale. I guess
the simplest answer is to add an alternative result file.
cheers
andrew
select * from T_pkey1 order by key1 using @<, key2;
key1 | key2 | txt
------+----------------------+------------------------------------------
- 1 | KEY1-3 | should work
1 | key1-1 | test key
1 | key1-2 | test key
1 | key1-3 | test key
2 | key2-3 | test key
2 | key2-9 | test key
(6 rows)
--- 166,175 ----
select * from T_pkey1 order by key1 using @<, key2;
key1 | key2 | txt
------+----------------------+------------------------------------------
1 | key1-1 | test key
1 | key1-2 | test key
1 | key1-3 | test key
+ 1 | KEY1-3 | should work
2 | key2-3 | test key
2 | key2-9 | test key
(6 rows)
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-24 11:18 Zdenek Kotala <Zdenek.Kotala@Sun.COM>
parent: Andrew Dunstan <andrew@dunslane.net>
0 siblings, 2 replies; 24+ messages in thread
From: Zdenek Kotala @ 2009-01-24 11:18 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Guillaume Smet <guillaume.smet@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Andrew Dunstan píše v pá 23. 01. 2009 v 23:57 -0500:
>
> Zdenek Kotala wrote:
> > Andrew Dunstan píše v pá 09. 01. 2009 v 12:16 -0500:
> >
> >
> >> Sure, we can easily have buildfarm's initdb step set any locale (and
> >> encoding, for that matter) we like. That's a simple change.
> >>
> >
> > Will be possible to set more locales and run tests without recompilation
> > on all of them? For example I have installed all Solaris'es locales on
> > my animal, but currently it means that I need perform whole cycle for
> > each locale.
> >
>
> I'm working on this. Yes, you will be able to specify a list of locales
> to check. For each locale the following tests will be run:
> installcheck, pl-installcheck, and contrib-installcheck.
thanks
> However, our tests are still a bit short of working across locales.
Yes, they are. Peter cleaned up some of them, but there are still open
issues. And MacOS has broken locale which is different problem.
> PL-check gives the diff below on PLTCL tests under en_US locale. I guess
> the simplest answer is to add an alternative result file.
Yes, I thought about add locale suffix for alternative result file, but
it could be useless overhead.
But some tests can be modified. For example
select * from T_pkey1 order by key1 using @<, key2;
can be rewritten as
select * from T_pkey1 order by key1 using @<, key2::name;
Zdenek
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-26 16:09 Andrew Dunstan <andrew@dunslane.net>
parent: Zdenek Kotala <Zdenek.Kotala@Sun.COM>
1 sibling, 0 replies; 24+ messages in thread
From: Andrew Dunstan @ 2009-01-26 16:09 UTC (permalink / raw)
To: Zdenek Kotala <Zdenek.Kotala@Sun.COM>; +Cc: Guillaume Smet <guillaume.smet@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Zdenek Kotala wrote:
> Andrew Dunstan píše v pá 23. 01. 2009 v 23:57 -0500:
>
>> Zdenek Kotala wrote:
>>
>>> Andrew Dunstan píše v pá 09. 01. 2009 v 12:16 -0500:
>>>
>>>
>>>
>>>> Sure, we can easily have buildfarm's initdb step set any locale (and
>>>> encoding, for that matter) we like. That's a simple change.
>>>>
>>>>
>>> Will be possible to set more locales and run tests without recompilation
>>> on all of them? For example I have installed all Solaris'es locales on
>>> my animal, but currently it means that I need perform whole cycle for
>>> each locale.
>>>
>>>
>> I'm working on this. Yes, you will be able to specify a list of locales
>> to check. For each locale the following tests will be run:
>> installcheck, pl-installcheck, and contrib-installcheck.
>>
>
> thanks
>
>
>
Example run with locales C, en_US.utf8 and french:
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dungbeetle&dt=2009-01-26%2012:44:01
cheers
andrew
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-01-31 22:08 Andrew Dunstan <andrew@dunslane.net>
parent: Zdenek Kotala <Zdenek.Kotala@Sun.COM>
1 sibling, 1 reply; 24+ messages in thread
From: Andrew Dunstan @ 2009-01-31 22:08 UTC (permalink / raw)
To: Zdenek Kotala <Zdenek.Kotala@Sun.COM>; +Cc: Guillaume Smet <guillaume.smet@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Zdenek Kotala wrote:
>> PL-check gives the diff below on PLTCL tests under en_US locale. I guess
>> the simplest answer is to add an alternative result file.
>>
>
> Yes, I thought about add locale suffix for alternative result file, but
> it could be useless overhead.
>
> But some tests can be modified. For example
>
> select * from T_pkey1 order by key1 using @<, key2;
>
> can be rewritten as
>
> select * from T_pkey1 order by key1 using @<, key2::name;
>
>
>
>
Is that the preferred solution? I want to fix this so I can re-enable
building with TCL in dungbeetle.
cheers
andrew
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: foreign_data test fails with non-C locale
@ 2009-02-02 07:51 Zdenek Kotala <Zdenek.Kotala@Sun.COM>
parent: Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 24+ messages in thread
From: Zdenek Kotala @ 2009-02-02 07:51 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Guillaume Smet <guillaume.smet@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter_e@gmx.net>; Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>; pgsql-hackers
Andrew Dunstan píše v so 31. 01. 2009 v 17:08 -0500:
>
> Zdenek Kotala wrote:
> >> PL-check gives the diff below on PLTCL tests under en_US locale. I guess
> >> the simplest answer is to add an alternative result file.
> >>
> >
> > Yes, I thought about add locale suffix for alternative result file, but
> > it could be useless overhead.
> >
> > But some tests can be modified. For example
> >
> > select * from T_pkey1 order by key1 using @<, key2;
> >
> > can be rewritten as
> >
> > select * from T_pkey1 order by key1 using @<, key2::name;
> >
> >
> >
> >
>
> Is that the preferred solution? I want to fix this so I can re-enable
> building with TCL in dungbeetle.
Probably not in all cases.
Zdenek
^ permalink raw reply [nested|flat] 24+ messages in thread
end of thread, other threads:[~2009-02-02 07:51 UTC | newest]
Thread overview: 24+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2009-01-09 14:12 foreign_data test fails with non-C locale Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2009-01-09 14:17 ` Andrew Dunstan <andrew@dunslane.net>
2009-01-09 14:23 ` Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
2009-01-09 14:25 ` Peter Eisentraut <peter_e@gmx.net>
2009-01-09 15:21 ` Andrew Dunstan <andrew@dunslane.net>
2009-01-09 16:24 ` Tom Lane <tgl@sss.pgh.pa.us>
2009-01-09 16:45 ` Guillaume Smet <guillaume.smet@gmail.com>
2009-01-09 17:16 ` Andrew Dunstan <andrew@dunslane.net>
2009-01-19 20:13 ` Zdenek Kotala <Zdenek.Kotala@Sun.COM>
2009-01-24 04:57 ` Andrew Dunstan <andrew@dunslane.net>
2009-01-24 11:18 ` Zdenek Kotala <Zdenek.Kotala@Sun.COM>
2009-01-26 16:09 ` Andrew Dunstan <andrew@dunslane.net>
2009-01-31 22:08 ` Andrew Dunstan <andrew@dunslane.net>
2009-02-02 07:51 ` Zdenek Kotala <Zdenek.Kotala@Sun.COM>
2009-01-11 10:54 ` Peter Eisentraut <peter_e@gmx.net>
2009-01-11 16:46 ` Tom Lane <tgl@sss.pgh.pa.us>
2009-01-11 20:01 ` Devrim GÜNDÜZ <devrim@gunduz.org>
2009-01-12 10:06 ` Peter Eisentraut <peter_e@gmx.net>
2009-01-12 10:40 ` Devrim GÜNDÜZ <devrim@gunduz.org>
2009-01-11 19:45 ` Devrim GÜNDÜZ <devrim@gunduz.org>
2009-01-19 20:39 ` Zdenek Kotala <Zdenek.Kotala@Sun.COM>
2009-01-20 05:44 ` Peter Eisentraut <peter_e@gmx.net>
2009-01-09 14:51 ` Peter Eisentraut <peter_e@gmx.net>
2009-01-11 09:41 ` Peter Eisentraut <peter_e@gmx.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox