From: Tom Lane <tgl@sss.pgh.pa.us>
To: =?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?=
 <juanjo.santamaria@gmail.com>
cc: Alvaro Herrera <alvherre@2ndquadrant.com>,
        Arthur Zakirov <zaartur@gmail.com>,
        Tomas Vondra <tomas.vondra@2ndquadrant.com>,
        PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Allow to_date() and to_timestamp() to accept localized names
In-reply-to: 
 <CAC+AXB1fQtg-iWH=4qZnV0xYm08WR6W7AhoApKA3tkFsWgf90A@mail.gmail.com>
References: <20200123164955.GA23074@alvherre.pgsql>
 <20326.1579816853@sss.pgh.pa.us>
 <CAC+AXB1fQtg-iWH=4qZnV0xYm08WR6W7AhoApKA3tkFsWgf90A@mail.gmail.com>
Comments: In-reply-to =?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?=
 <juanjo.santamaria@gmail.com>
	message dated "Fri, 24 Jan 2020 10:48:19 +0100"
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-ID: <28022.1579879230.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Fri, 24 Jan 2020 10:20:30 -0500
Message-ID: <28023.1579879230@sss.pgh.pa.us>
Precedence: bulk

=3D?UTF-8?Q?Juan_Jos=3DC3=3DA9_Santamar=3DC3=3DADa_Flecha?=3D <juanjo.sant=
amaria@gmail.com> writes:
> On Thu, Jan 23, 2020 at 11:00 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> * It's not exactly apparent to me why this code should be concerned
>> about non-normalized characters when noplace else in the backend is.

> There is an open patch that will make the normalization functionality us=
er
> visible [1]. So, if a user can call to_date(normalize('01 =C5=9EUB 2010'=
), 'DD
> TMMON YYYY') I would vote to drop the normalization logic inside this pa=
tch
> altogether.

Works for me.

> * I have no faith in this calculation that decides how long the match
>> length was:
>> 	*len =3D element_len + name_len - norm_len;

> The proper logic would come from do_to_timestamp() receiving a normalize=
d
> "date_txt" input, so we do not operate with unnormalize and normalize
> strings at the same time.

No, that only solves half the problem, because the downcasing
transformation can change the string length too.  Two easy examples:

* In German, I believe "=C3=9F" downcases to "ss".  In Latin-1 encoding th=
at's
a length change, though I think it might accidentally not be in UTF8.

* The Turks distinguish dotted and dotless "i", so that "=C4=B0" downcases=
 to
"i", and conversely "I" downcases to "=C4=B1".  Those are length changes i=
n
UTF8, though not in whichever Latin-N encoding works for Turkish.

Even if these cases happen not to apply to any month or day name of
the relevant language, we still have a problem, arising from the fact
that we're downcasing the whole remaining string --- so length changes
after the match could occur anyway.

> I would like to rise a couple of questions myself:

> * When compiled with DEBUG_TO_FROM_CHAR, there is a warning "=E2=80=98du=
mp_node=E2=80=99
> defined but not used". Should we drop this function or uncomment its usa=
ge?

Maybe, but I don't think it belongs in this patch.

> * Would it be worth moving str_tolower(localized_name)
> from seq_search_localized() into cache_locale_time()?

I think it'd complicate tracking when that cache has to be invalidated
(i.e. it now depends on more than just LC_TIME).  On the whole I wouldn't
bother unless someone does the measurements to show there'd be a useful
speedup.

			regards, tom lane