pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Uppercase version of ß desired
8+ messages / 6 participants
[nested] [flat]

* Uppercase version of ß desired
@ 2023-03-13 21:38 Celia McInnis <celia.mcinnis@gmail.com>
  2023-03-13 22:45 ` Re: Uppercase version of ß desired Peter J. Holzer <hjp-pgsql@hjp.at>
  2023-03-14 13:12 ` Re: Uppercase version of ß desired Philip Semanchuk <philip@americanefficient.com>
  0 siblings, 2 replies; 8+ messages in thread

From: Celia McInnis @ 2023-03-13 21:38 UTC (permalink / raw)
  To: pgsql-general

HI:

I would be really happy if postgresql  had an upper case version of the ß
german character. The wiki page
https://en.wikipedia.org/wiki/%C3%9F

indicates that the capital (U+1E9E ẞ LATIN CAPITAL LETTER SHARP S) was
encoded <https://en.wikipedia.org/wiki/Codepoint; by ISO 10646
<https://en.wikipedia.org/wiki/ISO_10646; in 2008.

BTW the reason that I'd like upper('ß') to give something different than
'ß'  is because I have written a simple substitution puzzle for a large
number of languages where I show the encrypted lower case words in upper
case and the successful letter substitution submissions in lower case - so
I need the upper and lower case versions of each letter to be different!

Thanks for any assistance! Maybe I can hack what I want in python (which is
what I am using for the puzzle).

Celia McInnis

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Uppercase version of ß desired
  2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
@ 2023-03-13 22:45 ` Peter J. Holzer <hjp-pgsql@hjp.at>
  2023-03-13 22:54   ` Re: Uppercase version of ß desired Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 1 reply; 8+ messages in thread

From: Peter J. Holzer @ 2023-03-13 22:45 UTC (permalink / raw)
  To: pgsql-general@lists.postgresql.org

On 2023-03-13 17:38:51 -0400, Celia McInnis wrote:
> I would be really happy if postgresql  had an upper case version of the ß
> german character. The wiki page
> https://en.wikipedia.org/wiki/%C3%9F
> 
> indicates that the capital (U+1E9E ẞ LATIN CAPITAL LETTER SHARP S) was encoded
> by ISO 10646 in 2008.

The character is there, of course, and lower-casing it works as
expected:

hjp=> select 'ẞ', lower('ẞ');
╔══════════╤═══════╗
║ ?column? │ lower ║
╟──────────┼───────╢
║ ẞ        │ ß     ║
╚══════════╧═══════╝
(1 row)

But the 'ß' is a bit special as it is usually uppercased to 'SS'
(although 'ẞ' is now officially allowed, too).

Apparently your (and my) locale doesn't uppercase ß at all, which isn't
correct according to German spelling rules but was very common in the
last decades.

You can specify an alternate locale:

hjp=> select upper('ß');
╔═══════╗
║ upper ║
╟───────╢
║ ß     ║
╚═══════╝
(1 row)


hjp=> select upper('ß' collate "de-AT-x-icu");
╔═══════╗
║ upper ║
╟───────╢
║ SS    ║
╚═══════╝
(1 row)


The challenge now is to find a locale which uppercases ß to ẞ.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Uppercase version of ß desired
  2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
  2023-03-13 22:45 ` Re: Uppercase version of ß desired Peter J. Holzer <hjp-pgsql@hjp.at>
@ 2023-03-13 22:54   ` Tom Lane <tgl@sss.pgh.pa.us>
  2023-03-14 06:53     ` Re: Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Tom Lane @ 2023-03-13 22:54 UTC (permalink / raw)
  To: Peter J. Holzer <hjp-pgsql@hjp.at>; +Cc: pgsql-general@lists.postgresql.org, Celia McInnis <celia.mcinnis@gmail.com>

"Peter J. Holzer" <hjp-pgsql@hjp.at> writes:
> On 2023-03-13 17:38:51 -0400, Celia McInnis wrote:
>> I would be really happy if postgresql had an upper case version of the ß
>> german character.

> But the 'ß' is a bit special as it is usually uppercased to 'SS'
> (although 'ẞ' is now officially allowed, too).
> Apparently your (and my) locale doesn't uppercase ß at all, which isn't
> correct according to German spelling rules but was very common in the
> last decades.

Our code for libc locales doesn't support upcasing 'ß' to 'SS',
because it uses towlower() which can only manage
one-character-to-one-character transformations.  It should work for
upcasing to 'ẞ', but as you say, you need to find a locale that thinks
that should happen.

You might have better luck if you have a version of Postgres that
supports ICU and you can use an ICU locale.  That code path doesn't
appear to have any hard-wired assumption about how many characters
in convert to how many out.

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Uppercase version of ß desired
  2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
  2023-03-13 22:45 ` Re: Uppercase version of ß desired Peter J. Holzer <hjp-pgsql@hjp.at>
  2023-03-13 22:54   ` Re: Uppercase version of ß desired Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-03-14 06:53     ` Celia McInnis <celia.mcinnis@gmail.com>
  2023-03-14 09:27       ` Re: Uppercase version of ß desired Thorsten Glaser <tg@evolvis.org>
  0 siblings, 1 reply; 8+ messages in thread

From: Celia McInnis @ 2023-03-14 06:53 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Peter J. Holzer <hjp-pgsql@hjp.at>; pgsql-general@lists.postgresql.org

Hmmm. Yes the  unicode rules seem to be a little strict on conforming to
the past! I just made the following fix to my python code in forming the
upper case alphabet from the lower case one:

uc_alphabet = lc_alphabet.replace('ß', 'ẞ').upper()

So far I have only found German to have a lower case letter which has the
same value for its upper cased one.

Thanks,
Celia McInnis

.

On Mon, Mar 13, 2023 at 6:54 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> "Peter J. Holzer" <hjp-pgsql@hjp.at> writes:
> > On 2023-03-13 17:38:51 -0400, Celia McInnis wrote:
> >> I would be really happy if postgresql had an upper case version of the ß
> >> german character.
>
> > But the 'ß' is a bit special as it is usually uppercased to 'SS'
> > (although 'ẞ' is now officially allowed, too).
> > Apparently your (and my) locale doesn't uppercase ß at all, which isn't
> > correct according to German spelling rules but was very common in the
> > last decades.
>
> Our code for libc locales doesn't support upcasing 'ß' to 'SS',
> because it uses towlower() which can only manage
> one-character-to-one-character transformations.  It should work for
> upcasing to 'ẞ', but as you say, you need to find a locale that thinks
> that should happen.
>
> You might have better luck if you have a version of Postgres that
> supports ICU and you can use an ICU locale.  That code path doesn't
> appear to have any hard-wired assumption about how many characters
> in convert to how many out.
>
>                         regards, tom lane
>

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Uppercase version of ß desired
  2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
  2023-03-13 22:45 ` Re: Uppercase version of ß desired Peter J. Holzer <hjp-pgsql@hjp.at>
  2023-03-13 22:54   ` Re: Uppercase version of ß desired Tom Lane <tgl@sss.pgh.pa.us>
  2023-03-14 06:53     ` Re: Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
@ 2023-03-14 09:27       ` Thorsten Glaser <tg@evolvis.org>
  0 siblings, 0 replies; 8+ messages in thread

From: Thorsten Glaser @ 2023-03-14 09:27 UTC (permalink / raw)
  To: Celia McInnis <celia.mcinnis@gmail.com>; +Cc: pgsql-general@lists.postgresql.org

On Tue, 14 Mar 2023, Celia McInnis wrote:

>uc_alphabet = lc_alphabet.replace('ß', 'ẞ').upper()

That’s probably for the best. The uppercase Eszett was only added
to Unicode under the rule that the lowercase Eszett’s case rules
are kept unchanged, and the former’s considered normally only ever
typed manually.

Of course, the grammar rules about uppercasing ß have since changed,
but since there’s two valid ways, choosing is the application’s duty.

bye,
//mirabilos
-- 
15:41⎜<Lo-lan-do:#fusionforge> Somebody write a testsuite for helloworld :-)





^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Uppercase version of ß desired
  2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
@ 2023-03-14 13:12 ` Philip Semanchuk <philip@americanefficient.com>
  2023-03-14 13:20   ` Re: Uppercase version of ß desired Kip Cole <kipcole9@gmail.com>
  2023-03-14 14:19   ` Re: Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
  1 sibling, 2 replies; 8+ messages in thread

From: Philip Semanchuk @ 2023-03-14 13:12 UTC (permalink / raw)
  To: Celia McInnis <celia.mcinnis@gmail.com>; +Cc: pgsql-general



> On Mar 13, 2023, at 5:38 PM, Celia McInnis <celia.mcinnis@gmail.com> wrote:
> 
> HI:
> 
> I would be really happy if postgresql  had an upper case version of the ß german character. The wiki page 
> https://en.wikipedia.org/wiki/%C3%9F
> 
> indicates that the capital (U+1E9E ẞ LATIN CAPITAL LETTER SHARP S) was encoded by ISO 10646 in 2008.
> 
> BTW the reason that I'd like upper('ß') to give something different than 'ß'  is because I have written a simple substitution puzzle for a large number of languages where I show the encrypted lower case words in upper case and the successful letter substitution submissions in lower case - so I need the upper and lower case versions of each letter to be different!
> 
> Thanks for any assistance! Maybe I can hack what I want in python (which is what I am using for the puzzle).

Hi Celia,
I ran into this too back when we were transitioning from Python 2 to 3 (2 behaved differently from 3). While researching it I discovered this Python issue which maybe sheds some additional light on the subject: https://github.com/python/cpython/issues/74993

We ultimately found 90 characters that (under Python 3) grew longer when uppercased. 

python -c "print([c for c in range(0x80, 0x22ff) if len(chr(c)) != len(chr(c).upper())])”


I hope this is at least interesting. :-)

Cheers
Philip







^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Uppercase version of ß desired
  2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
  2023-03-14 13:12 ` Re: Uppercase version of ß desired Philip Semanchuk <philip@americanefficient.com>
@ 2023-03-14 13:20   ` Kip Cole <kipcole9@gmail.com>
  1 sibling, 0 replies; 8+ messages in thread

From: Kip Cole @ 2023-03-14 13:20 UTC (permalink / raw)
  To: Philip Semanchuk <philip@americanefficient.com>; +Cc: Celia McInnis <celia.mcinnis@gmail.com>; pgsql-general

The relevant Unicode reference is https://unicode.org/faq/casemap_charprop.html#11

Which basically says that since Unicode 5.0 (its now at Unicode 15.0) stability is guaranteed and the upper-casing to  (U+1E9E ẞ LATIN CAPITAL LETTER SHARP S)  is optional.

> On 14 Mar 2023, at 9:12 pm, Philip Semanchuk <philip@americanefficient.com> wrote:
> 
> 
> 
>> On Mar 13, 2023, at 5:38 PM, Celia McInnis <celia.mcinnis@gmail.com> wrote:
>> 
>> HI:
>> 
>> I would be really happy if postgresql  had an upper case version of the ß german character. The wiki page 
>> https://en.wikipedia.org/wiki/%C3%9F
>> 
>> indicates that the capital (U+1E9E ẞ LATIN CAPITAL LETTER SHARP S) was encoded by ISO 10646 in 2008.
>> 
>> BTW the reason that I'd like upper('ß') to give something different than 'ß'  is because I have written a simple substitution puzzle for a large number of languages where I show the encrypted lower case words in upper case and the successful letter substitution submissions in lower case - so I need the upper and lower case versions of each letter to be different!
>> 
>> Thanks for any assistance! Maybe I can hack what I want in python (which is what I am using for the puzzle).
> 
> Hi Celia,
> I ran into this too back when we were transitioning from Python 2 to 3 (2 behaved differently from 3). While researching it I discovered this Python issue which maybe sheds some additional light on the subject: https://github.com/python/cpython/issues/74993 <https://github.com/python/cpython/issues/74993;
> 
> We ultimately found 90 characters that (under Python 3) grew longer when uppercased. 
> 
> python -c "print([c for c in range(0x80, 0x22ff) if len(chr(c)) != len(chr(c).upper())])”
> 
> 
> I hope this is at least interesting. :-)
> 
> Cheers
> Philip

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Uppercase version of ß desired
  2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
  2023-03-14 13:12 ` Re: Uppercase version of ß desired Philip Semanchuk <philip@americanefficient.com>
@ 2023-03-14 14:19   ` Celia McInnis <celia.mcinnis@gmail.com>
  1 sibling, 0 replies; 8+ messages in thread

From: Celia McInnis @ 2023-03-14 14:19 UTC (permalink / raw)
  To: Philip Semanchuk <philip@americanefficient.com>; +Cc: pgsql-general

Thanks Philip. Certainly interesting, As long as postgresql and python
return something different for upper and lower case versions of these
letters and python indexing of the string picks out the characters
"properly" then it might not require a python fix for me.

What a can of worms!  But at least postgresql and python do far better with
unicode than mysql and perl did! :-)

Celia McInnis

On Tue, Mar 14, 2023 at 9:12 AM Philip Semanchuk <
philip@americanefficient.com> wrote:

>
>
> > On Mar 13, 2023, at 5:38 PM, Celia McInnis <celia.mcinnis@gmail.com>
> wrote:
> >
> > HI:
> >
> > I would be really happy if postgresql  had an upper case version of the
> ß german character. The wiki page
> > https://en.wikipedia.org/wiki/%C3%9F
> >
> > indicates that the capital (U+1E9E ẞ LATIN CAPITAL LETTER SHARP S) was
> encoded by ISO 10646 in 2008.
> >
> > BTW the reason that I'd like upper('ß') to give something different than
> 'ß'  is because I have written a simple substitution puzzle for a large
> number of languages where I show the encrypted lower case words in upper
> case and the successful letter substitution submissions in lower case - so
> I need the upper and lower case versions of each letter to be different!
> >
> > Thanks for any assistance! Maybe I can hack what I want in python (which
> is what I am using for the puzzle).
>
> Hi Celia,
> I ran into this too back when we were transitioning from Python 2 to 3 (2
> behaved differently from 3). While researching it I discovered this Python
> issue which maybe sheds some additional light on the subject:
> https://github.com/python/cpython/issues/74993
>
> We ultimately found 90 characters that (under Python 3) grew longer when
> uppercased.
>
> python -c "print([c for c in range(0x80, 0x22ff) if len(chr(c)) !=
> len(chr(c).upper())])”
>
>
> I hope this is at least interesting. :-)
>
> Cheers
> Philip
>
>
>

^ permalink  raw  reply  [nested|flat] 8+ messages in thread


end of thread, other threads:[~2023-03-14 14:19 UTC | newest]

Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-03-13 21:38 Uppercase version of ß desired Celia McInnis <celia.mcinnis@gmail.com>
2023-03-13 22:45 ` Peter J. Holzer <hjp-pgsql@hjp.at>
2023-03-13 22:54   ` Tom Lane <tgl@sss.pgh.pa.us>
2023-03-14 06:53     ` Celia McInnis <celia.mcinnis@gmail.com>
2023-03-14 09:27       ` Thorsten Glaser <tg@evolvis.org>
2023-03-14 13:12 ` Philip Semanchuk <philip@americanefficient.com>
2023-03-14 13:20   ` Kip Cole <kipcole9@gmail.com>
2023-03-14 14:19   ` Celia McInnis <celia.mcinnis@gmail.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox