pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedOn non-Windows, hard depend on uselocale(3)
49+ messages / 11 participants
[nested] [flat]
Commits from this thread:
2025-03-28 3c8e463b0d Revert "Tidy up locale thread safety in ECPG library."
2025-03-28 8e993bff53 Tidy up locale thread safety in ECPG library.
2024-11-27 a62d90f2e5 Revert "Blind attempt to fix _configthreadlocale() failures on MinGW."
2024-11-27 1758d42446 Require ucrt if using MinGW.
2024-11-27 f1da075d9a Remove configure check for _configthreadlocale().
2024-10-01 9c2a6c5a5f Simplify checking for xlocale.h
* On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 10:27 Tristan Partin <tristan@neon.tech>
0 siblings, 1 reply; 49+ messages in thread
From: Tristan Partin @ 2023-11-15 10:27 UTC (permalink / raw)
To: pgsql-hackers
I have been working on adding using thread-safe locale APIs within
Postgres where appropriate[0]. The patch that I originally submitted
crashed during initdb (whoops!), so I worked on fixing the crash, which
led me to having to touch some code in chklocale.c, which became
a frustrating experience because chklocale.c is compiled in 3 different
configurations.
> pgport_variants = {
> '_srv': internal_lib_args + {
> 'dependencies': [backend_port_code],
> },
> '': default_lib_args + {
> 'dependencies': [frontend_port_code],
> },
> '_shlib': default_lib_args + {
> 'pic': true,
> 'dependencies': [frontend_port_code],
> },
> }
This means that some APIs I added or changed in pg_locale.c, can't be
used without conditional compilation depending on what variant is being
compiled. Additionally, I also have conditional compilation based on
HAVE_USELOCALE and WIN32.
I would like to propose removing HAVE_USELOCALE, and just have WIN32,
which means that Postgres would require uselocale(3) on anything that
isn't WIN32.
[0]: https://www.postgresql.org/message-id/CWMW5OZBWJ10.1YFLQWSUE5RE9@neon.tech
--
Tristan Partin
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 17:45 Tom Lane <tgl@sss.pgh.pa.us>
parent: Tristan Partin <tristan@neon.tech>
0 siblings, 2 replies; 49+ messages in thread
From: Tom Lane @ 2023-11-15 17:45 UTC (permalink / raw)
To: Tristan Partin <tristan@neon.tech>; +Cc: pgsql-hackers
"Tristan Partin" <tristan@neon.tech> writes:
> I would like to propose removing HAVE_USELOCALE, and just have WIN32,
> which means that Postgres would require uselocale(3) on anything that
> isn't WIN32.
You would need to do some research and try to prove that that won't
be a problem on any modern platform. Presumably it once was a problem,
or we'd not have bothered with a configure check.
(Some git archaeology might yield useful info about when and why
we added the check.)
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 18:38 Thomas Munro <thomas.munro@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2023-11-15 18:38 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Thu, Nov 16, 2023 at 6:45 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> "Tristan Partin" <tristan@neon.tech> writes:
> > I would like to propose removing HAVE_USELOCALE, and just have WIN32,
> > which means that Postgres would require uselocale(3) on anything that
> > isn't WIN32.
>
> You would need to do some research and try to prove that that won't
> be a problem on any modern platform. Presumably it once was a problem,
> or we'd not have bothered with a configure check.
>
> (Some git archaeology might yield useful info about when and why
> we added the check.)
According to data I scraped from the build farm, the last two systems
we had that didn't have uselocale() were curculio (OpenBSD 5.9) and
wrasse (Solaris 11.3), but those were both shut down (though wrasse
still runs old branches) as they were well out of support. OpenBSD
gained uselocale() in 6.2, and Solaris in 11.4, as part of the same
suite of POSIX changes that we already required in commit 8d9a9f03.
+1 for the change.
https://man.openbsd.org/uselocale.3
https://docs.oracle.com/cd/E88353_01/html/E37843/uselocale-3c.html
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 18:42 Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 2 replies; 49+ messages in thread
From: Dagfinn Ilmari Mannsåker @ 2023-11-15 18:42 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
Tom Lane <tgl@sss.pgh.pa.us> writes:
> "Tristan Partin" <tristan@neon.tech> writes:
>> I would like to propose removing HAVE_USELOCALE, and just have WIN32,
>> which means that Postgres would require uselocale(3) on anything that
>> isn't WIN32.
>
> You would need to do some research and try to prove that that won't
> be a problem on any modern platform. Presumably it once was a problem,
> or we'd not have bothered with a configure check.
>
> (Some git archaeology might yield useful info about when and why
> we added the check.)
For reference, the Perl effort to use the POSIX.1-2008 thread-safe
locale APIs have revealed several platform-specific bugs that cause it
to disable them on FreeBSD and macOS:
https://github.com/perl/perl5/commit/9cbc12c368981c56d4d8e40cc9417ac26bec2c35
https://github.com/perl/perl5/commit/dd4eb78c55aab441aec1639b1dd49f88bd960831
and work around bugs on others (e.g. OpenBSD):
https://github.com/perl/perl5/commit/0f3830f3997cf7ef1531bad26d2e0f13220dd862
But Perl actually makes use of per-thread locales, because it has a
separate interpereer per thread, each of which can have a different
locale active. Since Postgres isn't actually multi-threaded (yet),
these concerns might not apply to the same degree.
> regards, tom lane
- ilmari
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 19:04 Tom Lane <tgl@sss.pgh.pa.us>
parent: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
1 sibling, 0 replies; 49+ messages in thread
From: Tom Lane @ 2023-11-15 19:04 UTC (permalink / raw)
To: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= <ilmari@ilmari.org> writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> "Tristan Partin" <tristan@neon.tech> writes:
>>> I would like to propose removing HAVE_USELOCALE, and just have WIN32,
>>> which means that Postgres would require uselocale(3) on anything that
>>> isn't WIN32.
>> You would need to do some research and try to prove that that won't
>> be a problem on any modern platform. Presumably it once was a problem,
>> or we'd not have bothered with a configure check.
> For reference, the Perl effort to use the POSIX.1-2008 thread-safe
> locale APIs have revealed several platform-specific bugs that cause it
> to disable them on FreeBSD and macOS:
> https://github.com/perl/perl5/commit/9cbc12c368981c56d4d8e40cc9417ac26bec2c35
> https://github.com/perl/perl5/commit/dd4eb78c55aab441aec1639b1dd49f88bd960831
> and work around bugs on others (e.g. OpenBSD):
> https://github.com/perl/perl5/commit/0f3830f3997cf7ef1531bad26d2e0f13220dd862
> But Perl actually makes use of per-thread locales, because it has a
> separate interpereer per thread, each of which can have a different
> locale active. Since Postgres isn't actually multi-threaded (yet),
> these concerns might not apply to the same degree.
Interesting. That need not stop us from dropping the configure
check for uselocale(), but it might be a problem for Tristan's
larger ambitions.
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 19:16 Thomas Munro <thomas.munro@gmail.com>
parent: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
1 sibling, 0 replies; 49+ messages in thread
From: Thomas Munro @ 2023-11-15 19:16 UTC (permalink / raw)
To: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Thu, Nov 16, 2023 at 7:42 AM Dagfinn Ilmari Mannsåker
<ilmari@ilmari.org> wrote:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>
> > "Tristan Partin" <tristan@neon.tech> writes:
> >> I would like to propose removing HAVE_USELOCALE, and just have WIN32,
> >> which means that Postgres would require uselocale(3) on anything that
> >> isn't WIN32.
> >
> > You would need to do some research and try to prove that that won't
> > be a problem on any modern platform. Presumably it once was a problem,
> > or we'd not have bothered with a configure check.
> >
> > (Some git archaeology might yield useful info about when and why
> > we added the check.)
>
> For reference, the Perl effort to use the POSIX.1-2008 thread-safe
> locale APIs have revealed several platform-specific bugs that cause it
> to disable them on FreeBSD and macOS:
>
> https://github.com/perl/perl5/commit/9cbc12c368981c56d4d8e40cc9417ac26bec2c35
Interesting that C vs C.UTF-8 has come up there, something that has
also confused us and others (in fact I still owe Daniel Vérité a
response to his complaint about how we treat the latter; I got stuck
on a logical problem with the proposal and then dumped core...). The
idea of C.UTF-8 is relatively new, and seems to have shaken a few bugs
out in a few places. Anyway, that in particular is a brand new
FreeBSD bug report and I am sure it will be addressed soon.
> https://github.com/perl/perl5/commit/dd4eb78c55aab441aec1639b1dd49f88bd960831
As for macOS, one thing I noticed is that the FreeBSD -> macOS
pipeline appears to have re-awoken after many years of slumber. I
don't know anything about that other than that when I recently
upgraded my Mac to 14.1, suddenly a few userspace tools are now
running the recentish FreeBSD versions of certain userland tools (tar,
grep, ...?), instead of something from the Jurassic. Whether that
might apply to libc, who can say... they seemed to have quite ancient
BSD locale code last time I checked.
> https://github.com/perl/perl5/commit/0f3830f3997cf7ef1531bad26d2e0f13220dd862
That linked issue appears to be fixed already.
> But Perl actually makes use of per-thread locales, because it has a
> separate interpereer per thread, each of which can have a different
> locale active. Since Postgres isn't actually multi-threaded (yet),
> these concerns might not apply to the same degree.
ECPG might use them in multi-threaded code. I'm not sure if it's a
problem and whose problem it is.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 20:51 Tom Lane <tgl@sss.pgh.pa.us>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Tom Lane @ 2023-11-15 20:51 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
Thomas Munro <thomas.munro@gmail.com> writes:
> On Thu, Nov 16, 2023 at 6:45 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> You would need to do some research and try to prove that that won't
>> be a problem on any modern platform. Presumably it once was a problem,
>> or we'd not have bothered with a configure check.
> According to data I scraped from the build farm, the last two systems
> we had that didn't have uselocale() were curculio (OpenBSD 5.9) and
> wrasse (Solaris 11.3), but those were both shut down (though wrasse
> still runs old branches) as they were well out of support.
AFAICS, NetBSD still doesn't have it. They have no on-line man page
for it, and my animal mamba shows it as not found.
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 21:08 Thomas Munro <thomas.munro@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2023-11-15 21:08 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Thu, Nov 16, 2023 at 9:51 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > On Thu, Nov 16, 2023 at 6:45 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> You would need to do some research and try to prove that that won't
> >> be a problem on any modern platform. Presumably it once was a problem,
> >> or we'd not have bothered with a configure check.
>
> > According to data I scraped from the build farm, the last two systems
> > we had that didn't have uselocale() were curculio (OpenBSD 5.9) and
> > wrasse (Solaris 11.3), but those were both shut down (though wrasse
> > still runs old branches) as they were well out of support.
>
> AFAICS, NetBSD still doesn't have it. They have no on-line man page
> for it, and my animal mamba shows it as not found.
Oh :-( I see that but had missed that sidewinder was NetBSD and my
scraped data predates mamba. Sorry for the wrong info.
Currently pg_locale.c requires systems to have *either* uselocale() or
mbstowcs_l()/wcstombs_l(), but NetBSD satisfies the second
requirement. The other uses of uselocale() are in ECPG code that must
be falling back to the setlocale() path. In other words, isn't it the
case that we don't require uselocale() to compile ECPG stuff, but it'll
probably crash or corrupt itself or give wrong answers if you push it
on NetBSD, so... uhh, really we do require it?
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 21:17 Tom Lane <tgl@sss.pgh.pa.us>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Tom Lane @ 2023-11-15 21:17 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
Thomas Munro <thomas.munro@gmail.com> writes:
> Currently pg_locale.c requires systems to have *either* uselocale() or
> mbstowcs_l()/wcstombs_l(), but NetBSD satisfies the second
> requirement.
Check.
> The other uses of uselocale() are in ECPG code that must
> be falling back to the setlocale() path. In other words, isn't it the
> case that we don't require uselocale() to compile ECPG stuff, but it'll
> probably crash or corrupt itself or give wrong answers if you push it
> on NetBSD, so... uhh, really we do require it?
Dunno. mamba is getting through the ecpg regression tests okay,
but we all know that doesn't prove a lot. (AFAICS, ecpg only
cares about this to the extent of not wanting an LC_NUMERIC
locale where the decimal point isn't '.'. I'm not sure that
NetBSD supports any such locale anyway --- I think they're like
OpenBSD in having only pro-forma locale support.)
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 22:40 Thomas Munro <thomas.munro@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2023-11-15 22:40 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Thu, Nov 16, 2023 at 10:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > The other uses of uselocale() are in ECPG code that must
> > be falling back to the setlocale() path. In other words, isn't it the
> > case that we don't require uselocale() to compile ECPG stuff, but it'll
> > probably crash or corrupt itself or give wrong answers if you push it
> > on NetBSD, so... uhh, really we do require it?
>
> Dunno. mamba is getting through the ecpg regression tests okay,
> but we all know that doesn't prove a lot. (AFAICS, ecpg only
> cares about this to the extent of not wanting an LC_NUMERIC
> locale where the decimal point isn't '.'. I'm not sure that
> NetBSD supports any such locale anyway --- I think they're like
> OpenBSD in having only pro-forma locale support.)
Idea #1
For output, which happens with sprintf(ptr, "%.15g%s", ...) in
execute.c, perhaps we could use our in-tree Ryu routine instead?
For input, which happens with strtod() in data.c, rats, we don't have
a parser and I understand that it is not for the faint of heart (naive
implementation gets subtle things wrong, cf "How to read floating
point numbers accurately" by W D Clinger + whatever improvements have
happened in this space since 1990).
Idea #2
Perhaps we could use snprintf_l() and strtod_l() where available.
They're not standard, but they are obvious extensions that NetBSD and
Windows have, and those are the two systems for which we are doing
grotty things in that code. That would amount to extending
pg_locale.c's philosophy: either you must have uselocale(), or the
full set of _l() functions (that POSIX failed to standardise, dunno
what the history is behind that, seems weird).
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-15 23:06 Tom Lane <tgl@sss.pgh.pa.us>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Tom Lane @ 2023-11-15 23:06 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
Thomas Munro <thomas.munro@gmail.com> writes:
> Idea #1
> For output, which happens with sprintf(ptr, "%.15g%s", ...) in
> execute.c, perhaps we could use our in-tree Ryu routine instead?
> For input, which happens with strtod() in data.c, rats, we don't have
> a parser and I understand that it is not for the faint of heart
Yeah. Getting rid of ecpg's use of uselocale() would certainly be
nice, but I'm not ready to add our own implementation of strtod()
to get there.
> Idea #2
> Perhaps we could use snprintf_l() and strtod_l() where available.
> They're not standard, but they are obvious extensions that NetBSD and
> Windows have, and those are the two systems for which we are doing
> grotty things in that code.
Oooh, shiny. I do not see any man page for strtod_l, but I do see
that it's declared on mamba's host. I wonder how long they've had it?
The man page for snprintf_l appears to be quite ancient, so we could
hope that strtod_l is available on all versions anyone cares about.
> That would amount to extending
> pg_locale.c's philosophy: either you must have uselocale(), or the
> full set of _l() functions (that POSIX failed to standardise, dunno
> what the history is behind that, seems weird).
Yeah. I'd say the _l functions should be preferred over uselocale()
if available, but sadly they're not there on common systems. (It
looks like glibc has strtod_l but not snprintf_l, which is odd.)
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-16 19:57 Thomas Munro <thomas.munro@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 49+ messages in thread
From: Thomas Munro @ 2023-11-16 19:57 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Thu, Nov 16, 2023 at 12:06 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > Perhaps we could use snprintf_l() and strtod_l() where available.
> > They're not standard, but they are obvious extensions that NetBSD and
> > Windows have, and those are the two systems for which we are doing
> > grotty things in that code.
>
> Oooh, shiny. I do not see any man page for strtod_l, but I do see
> that it's declared on mamba's host. I wonder how long they've had it?
> The man page for snprintf_l appears to be quite ancient, so we could
> hope that strtod_l is available on all versions anyone cares about.
A decade[1]. And while I'm doing archeology, I noticed that POSIX has
agreed[2] in principle that *all* functions affected by the thread's
current locale should have a _l() variant, it's just that no one has
sent in the patch.
> > That would amount to extending
> > pg_locale.c's philosophy: either you must have uselocale(), or the
> > full set of _l() functions (that POSIX failed to standardise, dunno
> > what the history is behind that, seems weird).
>
> Yeah. I'd say the _l functions should be preferred over uselocale()
> if available, but sadly they're not there on common systems. (It
> looks like glibc has strtod_l but not snprintf_l, which is odd.)
Here is a first attempt. In this version, new functions are exported
by pgtypeslib. I realised that I had to do it in there because ECPG's
uselocale() jiggery-pokery is clearly intended to affect the
conversions happening in there too, and we probably don't want
circular dependencies between pgtypeslib and ecpglib. I think this
means that pgtypeslib is actually subtly b0rked if you use it
independently without an ECPG connection (is that a thing people do?),
because all that code copied-and-pasted from the backend when run in
frontend code with eg a French locale will produce eg "0,42"; this
patch doesn't change that.
I also had a go[3] at doing it with static inlined functions, to avoid
creating a load of new exported functions and associated function call
overheads. It worked fine, except on Windows: I needed a global
variable PGTYPESclocale that all the inlined functions can see when
called from ecpglib or pgtypeslib code, but if I put that in the
exports list then on that platform it seems to contain garbage; there
is probably some other magic needed to export non-function symbols
from the DLL or something like that, I didn't look into it. See CI
failure + crash dumps.
[1] https://github.com/NetBSD/src/commit/c99aac45e540bc210cc660619a6b5323cbb5c17f
[2] https://www.austingroupbugs.net/view.php?id=1004
[3] https://github.com/macdice/postgres/tree/strtod_l_inline
Attachments:
0001-ecpg-Use-thread-safe-_l-functions-if-possible.patch (application/octet-stream, 24.0K) download
From 512110458d6916efb7515770d6c4f4b7477b1d22 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Thu, 16 Nov 2023 15:53:51 +1300
Subject: [PATCH] ecpg: Use thread-safe _l() functions if possible.
In order to use snprintf("%g") and strtod() with the C locale in certain
parts of the ECPG code, we call uselocale() around those sections.
Two operating systems don't have uselocale(), but our fallbacks were
non-ideal. For NetBSD we'd use setlocale(), which is not thread-safe
and dangerous, and for Windows we'd either use setlocale() or a Windows
API that is egregiously different and hard to maintain, depending on the
compiler tool chain.
We can remove all of that and use not-yet-standardized [v]s[n]printf_l()
and strtod_l(), if available. Those two operating systems, along with
the other BSDs and macOS have them. Glibc has strtod_l() but not yet
the other.
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
configure | 13 +-
configure.ac | 5 +-
meson.build | 5 +-
src/include/pg_config.h.in | 15 +-
src/interfaces/ecpg/ecpglib/connect.c | 55 ++++----
src/interfaces/ecpg/ecpglib/data.c | 3 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 29 +---
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 11 --
src/interfaces/ecpg/ecpglib/execute.c | 54 +-------
src/interfaces/ecpg/include/pgtypes.h | 6 +
src/interfaces/ecpg/include/pgtypes_format.h | 26 ++++
src/interfaces/ecpg/pgtypeslib/common.c | 138 ++++++++++++++++++-
src/interfaces/ecpg/pgtypeslib/exports.txt | 6 +
src/interfaces/ecpg/pgtypeslib/interval.c | 5 +-
src/interfaces/ecpg/pgtypeslib/numeric.c | 3 +-
src/tools/msvc/Solution.pm | 5 +-
16 files changed, 233 insertions(+), 146 deletions(-)
create mode 100644 src/interfaces/ecpg/include/pgtypes_format.h
diff --git a/configure b/configure
index c064115038..91c9865ae3 100755
--- a/configure
+++ b/configure
@@ -15539,7 +15539,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal strtod_l syncfs sync_file_range vsnprintf_l vsprintf_l wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -16309,17 +16309,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index f220b379b3..67ecb83447 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1782,9 +1782,11 @@ AC_CHECK_FUNCS(m4_normalize([
setproctitle_fast
strchrnul
strsignal
+ strtod_l
syncfs
sync_file_range
- uselocale
+ vsnprintf_l
+ vsprintf_l
wcstombs_l
]))
@@ -1868,7 +1870,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index 286d7e4269..e24277b473 100644
--- a/meson.build
+++ b/meson.build
@@ -2417,7 +2417,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
@@ -2456,9 +2455,11 @@ func_checks = [
['strlcpy'],
['strnlen'],
['strsignal'],
+ ['strtod_l'],
['sync_file_range'],
['syncfs'],
- ['uselocale'],
+ ['vsnprintf_l'],
+ ['vsprintf_l'],
['wcstombs_l'],
]
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index d8a2985567..ea047e85cf 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -429,6 +429,9 @@
/* Define to 1 if you have the `strsignal' function. */
#undef HAVE_STRSIGNAL
+/* Define to 1 if you have the `strtod_l' function. */
+#undef HAVE_STRTOD_L
+
/* Define to 1 if the system has the type `struct option'. */
#undef HAVE_STRUCT_OPTION
@@ -495,9 +498,6 @@
/* Define to 1 if you have the <unistd.h> header file. */
#undef HAVE_UNISTD_H
-/* Define to 1 if you have the `uselocale' function. */
-#undef HAVE_USELOCALE
-
/* Define to 1 if you have BSD UUID support. */
#undef HAVE_UUID_BSD
@@ -516,6 +516,12 @@
/* Define to 1 if your compiler knows the visibility("hidden") attribute. */
#undef HAVE_VISIBILITY_ATTRIBUTE
+/* Define to 1 if you have the `vsnprintf_l' function. */
+#undef HAVE_VSNPRINTF_L
+
+/* Define to 1 if you have the `vsprintf_l' function. */
+#undef HAVE_VSPRINTF_L
+
/* Define to 1 if you have the `wcstombs_l' function. */
#undef HAVE_WCSTOMBS_L
@@ -561,9 +567,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 8afb1f0a26..cf2083713c 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -8,12 +8,9 @@
#include "ecpglib.h"
#include "ecpglib_extern.h"
#include "ecpgtype.h"
+#include "pgtypes.h"
#include "sqlca.h"
-#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale = (locale_t) 0;
-#endif
-
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
@@ -484,37 +481,31 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
pthread_mutex_lock(&connections_mutex);
/*
- * ... but first, make certain we have created ecpg_clocale. Rely on
- * holding connections_mutex to ensure this is done by only one thread.
+ * ... but first, make certain pgtypes' locale is set up. Rely on holding
+ * connections_mutex to ensure this is done by only one thread.
*/
-#ifdef HAVE_USELOCALE
- if (!ecpg_clocale)
+ if (PGTYPESinit() < 0)
{
- ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (!ecpg_clocale)
- {
- pthread_mutex_unlock(&connections_mutex);
- ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
- ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
- if (host)
- ecpg_free(host);
- if (port)
- ecpg_free(port);
- if (options)
- ecpg_free(options);
- if (realname)
- ecpg_free(realname);
- if (dbname)
- ecpg_free(dbname);
- if (conn_keywords)
- ecpg_free(conn_keywords);
- if (conn_values)
- ecpg_free(conn_values);
- free(this);
- return false;
- }
+ pthread_mutex_unlock(&connections_mutex);
+ ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
+ ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
+ if (host)
+ ecpg_free(host);
+ if (port)
+ ecpg_free(port);
+ if (options)
+ ecpg_free(options);
+ if (realname)
+ ecpg_free(realname);
+ if (dbname)
+ ecpg_free(dbname);
+ if (conn_keywords)
+ ecpg_free(conn_keywords);
+ if (conn_values)
+ ecpg_free(conn_values);
+ free(this);
+ return false;
}
-#endif
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index fa56276758..0f3ab36575 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -10,6 +10,7 @@
#include "ecpglib_extern.h"
#include "ecpgtype.h"
#include "pgtypes_date.h"
+#include "pgtypes_format.h"
#include "pgtypes_interval.h"
#include "pgtypes_numeric.h"
#include "pgtypes_timestamp.h"
@@ -466,7 +467,7 @@ ecpg_get_data(const PGresult *results, int act_tuple, int act_field, int lineno,
pval++;
if (!check_special_value(pval, &dres, &scan_length))
- dres = strtod(pval, &scan_length);
+ dres = PGTYPESstrtod(pval, &scan_length);
if (isarray && *scan_length == '"')
scan_length++;
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c..e7d4cdf971 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -12,6 +12,7 @@
#include "ecpglib.h"
#include "ecpglib_extern.h"
#include "ecpgtype.h"
+#include "pgtypes_format.h"
#include "sql3types.h"
#include "sqlca.h"
#include "sqlda.h"
@@ -478,43 +479,21 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
/* Make sure we do NOT honor the locale for numeric input */
/* since the database gives the standard decimal point */
/* (see comments in execute.c) */
-#ifdef HAVE_USELOCALE
/*
* To get here, the above PQnfields() test must have found nonzero
* fields. One needs a connection to create such a descriptor. (EXEC
* SQL SET DESCRIPTOR can populate the descriptor's "items", but it
* can't change the descriptor's PQnfields().) Any successful
- * connection initializes ecpg_clocale.
+ * connection calls PGTYPESinit().
*/
- Assert(ecpg_clocale);
- stmt.oldlocale = uselocale(ecpg_clocale);
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- setlocale(LC_NUMERIC, "C");
-#endif
+ PGTYPESbegin_clocale(&stmt.oldlocale);
/* desperate try to guess something sensible */
stmt.connection = ecpg_get_connection(NULL);
ecpg_store_result(ECPGresult, index, &stmt, &data_var);
-#ifdef HAVE_USELOCALE
- if (stmt.oldlocale != (locale_t) 0)
- uselocale(stmt.oldlocale);
-#else
- if (stmt.oldlocale)
- {
- setlocale(LC_NUMERIC, stmt.oldlocale);
- ecpg_free(stmt.oldlocale);
- }
-#ifdef HAVE__CONFIGTHREADLOCALE
- if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
-#endif
-#endif
+ PGTYPESend_clocale(stmt.oldlocale);
}
else if (data_var.ind_type != ECPGt_NO_INDICATOR && data_var.ind_pointer != NULL)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 01b4309a71..ae4c49d87e 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -59,10 +59,6 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
-#ifdef HAVE_USELOCALE
-extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
-#endif
-
/* structure to store one statement */
struct statement
{
@@ -76,14 +72,7 @@ struct statement
bool questionmarks;
struct variable *inlist;
struct variable *outlist;
-#ifdef HAVE_USELOCALE
locale_t oldlocale;
-#else
- char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
- int oldthreadlocale;
-#endif
-#endif
int nparams;
char **paramvalues;
int *paramlengths;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 04d0b40c53..a9afc88a61 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -24,6 +24,7 @@
#include "ecpglib_extern.h"
#include "ecpgtype.h"
#include "pgtypes_date.h"
+#include "pgtypes_format.h"
#include "pgtypes_interval.h"
#include "pgtypes_numeric.h"
#include "pgtypes_timestamp.h"
@@ -101,9 +102,6 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifndef HAVE_USELOCALE
- ecpg_free(stmt->oldlocale);
-#endif
ecpg_free(stmt);
}
@@ -465,7 +463,7 @@ sprintf_double_value(char *ptr, double value, const char *delim)
sprintf(ptr, "%s%s", "Infinity", delim);
}
else
- sprintf(ptr, "%.15g%s", value, delim);
+ PGTYPESsprintf(ptr, "%.15g%s", value, delim);
}
static void
@@ -481,7 +479,7 @@ sprintf_float_value(char *ptr, float value, const char *delim)
sprintf(ptr, "%s%s", "Infinity", delim);
}
else
- sprintf(ptr, "%.15g%s", value, delim);
+ PGTYPESsprintf(ptr, "%.15g%s", value, delim);
}
static char *
@@ -1975,37 +1973,13 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
/*
* Make sure we do NOT honor the locale for numeric input/output since the
- * database wants the standard decimal point. If available, use
- * uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
+ * database wants the standard decimal point.
*/
-#ifdef HAVE_USELOCALE
-
- /*
- * Since ecpg_init() succeeded, we have a connection. Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt->oldlocale = uselocale(ecpg_clocale);
- if (stmt->oldlocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- if (stmt->oldlocale == NULL)
+ if (PGTYPESbegin_clocale(&stmt->oldlocale) < 0)
{
ecpg_do_epilogue(stmt);
return false;
}
- setlocale(LC_NUMERIC, "C");
-#endif
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
@@ -2213,23 +2187,7 @@ ecpg_do_epilogue(struct statement *stmt)
if (stmt == NULL)
return;
-#ifdef HAVE_USELOCALE
- if (stmt->oldlocale != (locale_t) 0)
- uselocale(stmt->oldlocale);
-#else
- if (stmt->oldlocale)
- setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
-#endif
-#endif
+ PGTYPESend_clocale(stmt->oldlocale);
free_statement(stmt);
}
diff --git a/src/interfaces/ecpg/include/pgtypes.h b/src/interfaces/ecpg/include/pgtypes.h
index dbf759b45f..18251d74c9 100644
--- a/src/interfaces/ecpg/include/pgtypes.h
+++ b/src/interfaces/ecpg/include/pgtypes.h
@@ -10,6 +10,12 @@ extern "C"
extern void PGTYPESchar_free(char *ptr);
+/*
+ * Initialize. Not thread-safe and should only be called in one thread at a
+ * time, but idempotent. Returns 0 on success, -1 on failure and sets errno.
+ */
+extern int PGTYPESinit(void);
+
#ifdef __cplusplus
}
#endif
diff --git a/src/interfaces/ecpg/include/pgtypes_format.h b/src/interfaces/ecpg/include/pgtypes_format.h
new file mode 100644
index 0000000000..d6dd06d361
--- /dev/null
+++ b/src/interfaces/ecpg/include/pgtypes_format.h
@@ -0,0 +1,26 @@
+/*
+ * We would like to be able to convert between strings and doubles using the C
+ * locale. We can set the thread's current locale with uselocale() and use
+ * snprintf() and strtod() according to the POSIX standard, but some systems
+ * haven't implemented uselocale() yet. We can use snprintf_l() and
+ * strtod_l() on some systems, but POSIX hasn't standardized those yet. All
+ * systems we target can do one or the other, so provide a simple abstraction.
+ */
+
+#ifndef PGTYPES_FORMAT_H
+#define PGTYPES_FORMAT_H
+
+#if defined(LOCALE_T_IN_XLOCALE)
+#include <xlocale.h>
+#else
+#include <locale.h>
+#endif
+
+extern int PGTYPESbegin_clocale(locale_t *old_locale);
+extern void PGTYPESend_clocale(locale_t old_locale);
+
+extern double PGTYPESstrtod(const char *str, char **endptr);
+extern int PGTYPESsprintf(char *str, const char *format, ...) pg_attribute_printf(2, 3);
+extern int PGTYPESsnprintf(char *str, size_t size, const char *format, ...) pg_attribute_printf(3, 4);
+
+#endif
diff --git a/src/interfaces/ecpg/pgtypeslib/common.c b/src/interfaces/ecpg/pgtypeslib/common.c
index 8972229ca2..37ba35bc7a 100644
--- a/src/interfaces/ecpg/pgtypeslib/common.c
+++ b/src/interfaces/ecpg/pgtypeslib/common.c
@@ -3,8 +3,142 @@
#include "postgres_fe.h"
#include "pgtypes.h"
+#include "pgtypes_format.h"
#include "pgtypeslib_extern.h"
+/*
+ * Use wcstombs_l's header as a clue about where to find the other extra _l
+ * functions.
+ */
+#if defined(LOCALE_T_IN_XLOCALE) || defined(WCSTOMBS_L_IN_XLOCALE)
+#include <xlocale.h>
+#else
+#include <locale.h>
+#endif
+
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+static locale_t PGTYPESclocale = (locale_t) 0;
+
+int
+PGTYPESinit(void)
+{
+ /* Already called? */
+ if (PGTYPESclocale != (locale_t) 0)
+ return 0;
+
+#ifdef WIN32
+ PGTYPESclocale = _create_locale(LC_ALL, "C");
+#else
+ PGTYPESclocale = newlocale(LC_ALL_MASK, "C", (locale_t) 0);
+#endif
+ if (PGTYPESclocale == (locale_t) 0)
+ return -1;
+ return -0;
+}
+
+/*
+ * If any of these _l() functions are missing, we need to mess with the
+ * thread-local locale.
+ */
+#if !defined(HAVE_STRTOD_L) || !defined(HAVE_VSPRINTF_L) || !defined(HAVE_VSNPRINTF_L)
+#if defined(WIN32)
+/* Windows has all of these in slightly scrambled form. */
+#else
+/* At least one of these functions is missing. We'll do the save/restore. */
+#define PGTYPES_MUST_SAVE_RESTORE_THREAD_LOCALE
+#endif
+#endif
+
+/*
+ * Before running code that might do conversions, call this to change the
+ * thread's current locale on platforms that need it. This is done as a
+ * separate function rather than inside individual conversion wrappers for
+ * some batching effect, avoiding repeated save/restore.
+ *
+ * PGTYPESinit() must have been called, or this will have no effect.
+ */
+int
+PGTYPESbegin_clocale(locale_t *old_locale)
+{
+#ifdef PGTYPES_MUST_SAVE_RESTORE_THREAD_LOCALE
+ /*
+ * On this platform, at least one of the functions below expects us to
+ * have changed the thread's locale. The caller provides space for us to
+ * store the current locale, so we can change it back later.
+ */
+ *old_locale = uselocale(PGTYPESclocale);
+ return (*old_locale == (locale_t) 0) ? -1 : 0;
+#else
+ /*
+ * Dummy value. We have _l() variants of the functions we need, and we
+ * might not even have uselocale() on this platform.
+ */
+ *old_locale = (locale_t) 0;
+ return 0;
+#endif
+}
+
+/*
+ * Restore the current thread's locale. Call with the value returned by
+ * PGTYPESbegin_clocale(). Does nothing on platforms with all the required
+ * _l() functions.
+ */
+void
+PGTYPESend_clocale(locale_t old_locale)
+{
+#ifdef PGTYPES_MUST_SAVE_RESTORE_THREAD_LOCALE
+ /* Put it back. */
+ uselocale(old_locale);
+#endif
+}
+
+double
+PGTYPESstrtod(const char *str, char **endptr)
+{
+#if defined(WIN32)
+ return _strtod_l(str, endptr, PGTYPESclocale);
+#elif defined(HAVE_STRTOD_L)
+ return strtod_l(str, endptr, PGTYPESclocale);
+#else
+ return strtod(str, endptr);
+#endif
+}
+
+int
+PGTYPESsprintf(char *str, const char *format, ...)
+{
+ va_list args;
+
+ va_start(args, format);
+
+#if defined(WIN32)
+ return _vsprintf_l(str, format, PGTYPESclocale, args);
+#elif defined(HAVE_VSPRINTF_L)
+ return vsprintf_l(str, PGTYPESclocale, format, args);
+#else
+ return vsprintf(str, format, args);
+#endif
+}
+
+int
+PGTYPESsnprintf(char *str, size_t size, const char *format, ...)
+{
+ va_list args;
+
+ va_start(args, format);
+
+#if defined(WIN32)
+ return _vsnprintf_l(str, size, format, PGTYPESclocale, args);
+#elif defined(HAVE_VSPRINTF_L)
+ return vsnprintf_l(str, size, PGTYPESclocale, format, args);
+#else
+ return vsnprintf(str, size, format, args);
+#endif
+}
+
/* Return value is zero-filled. */
char *
pgtypes_alloc(long size)
@@ -81,8 +215,8 @@ pgtypes_fmt_replace(union un_fmt_comb replace_val, int replace_type, char **outp
switch (replace_type)
{
case PGTYPES_TYPE_DOUBLE_NF:
- i = snprintf(t, PGTYPES_FMT_NUM_MAX_DIGITS,
- "%0.0g", replace_val.double_val);
+ i = PGTYPESsnprintf(t, PGTYPES_FMT_NUM_MAX_DIGITS,
+ "%0.0g", replace_val.double_val);
break;
case PGTYPES_TYPE_INT64:
i = snprintf(t, PGTYPES_FMT_NUM_MAX_DIGITS,
diff --git a/src/interfaces/ecpg/pgtypeslib/exports.txt b/src/interfaces/ecpg/pgtypeslib/exports.txt
index 2d5ec17656..3fe02bc60a 100644
--- a/src/interfaces/ecpg/pgtypeslib/exports.txt
+++ b/src/interfaces/ecpg/pgtypeslib/exports.txt
@@ -46,3 +46,9 @@ PGTYPEStimestamp_sub 43
PGTYPEStimestamp_sub_interval 44
PGTYPEStimestamp_to_asc 45
PGTYPESchar_free 46
+PGTYPESinit 47
+PGTYPESsprintf 48
+PGTYPESsnprintf 49
+PGTYPESstrtod 50
+PGTYPESbegin_clocale 51
+PGTYPESend_clocale 52
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 936a688381..124ed5e1c2 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -13,6 +13,7 @@
#include "common/string.h"
#include "dt.h"
#include "pgtypes_error.h"
+#include "pgtypes_format.h"
#include "pgtypes_interval.h"
#include "pgtypeslib_extern.h"
@@ -60,7 +61,7 @@ ParseISO8601Number(const char *str, char **endptr, int *ipart, double *fpart)
if (!(isdigit((unsigned char) *str) || *str == '-' || *str == '.'))
return DTERR_BAD_FORMAT;
errno = 0;
- val = strtod(str, endptr);
+ val = PGTYPESstrtod(str, endptr);
/* did we not see anything that looks like a double? */
if (*endptr == str || errno != 0)
return DTERR_BAD_FORMAT;
@@ -455,7 +456,7 @@ DecodeInterval(char **field, int *ftype, int nf, /* int range, */
else if (*cp == '.')
{
errno = 0;
- fval = strtod(cp, &cp);
+ fval = PGTYPESstrtod(cp, &cp);
if (*cp != '\0' || errno != 0)
return DTERR_BAD_FORMAT;
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 35e7b92da4..a8bc19cc8f 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -7,6 +7,7 @@
#include <limits.h>
#include "pgtypes_error.h"
+#include "pgtypes_format.h"
#include "pgtypes_numeric.h"
#include "pgtypeslib_extern.h"
@@ -1414,7 +1415,7 @@ PGTYPESnumeric_from_double(double d, numeric *dst)
numeric *tmp;
int i;
- if (sprintf(buffer, "%.*g", DBL_DIG, d) <= 0)
+ if (PGTYPESsprintf(buffer, "%.*g", DBL_DIG, d) <= 0)
return -1;
if ((tmp = PGTYPESnumeric_from_asc(buffer, NULL)) == NULL)
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 98a5b5d872..6c550f3d37 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -339,6 +339,7 @@ sub GenerateFiles
HAVE_STRLCPY => undef,
HAVE_STRNLEN => 1,
HAVE_STRSIGNAL => undef,
+ HAVE_STRTOD_L => undef,
HAVE_STRUCT_OPTION => undef,
HAVE_STRUCT_SOCKADDR_SA_LEN => undef,
HAVE_STRUCT_TM_TM_ZONE => undef,
@@ -361,7 +362,6 @@ sub GenerateFiles
HAVE_UINT8 => undef,
HAVE_UNION_SEMUN => undef,
HAVE_UNISTD_H => 1,
- HAVE_USELOCALE => undef,
HAVE_UUID_BSD => undef,
HAVE_UUID_E2FS => undef,
HAVE_UUID_OSSP => undef,
@@ -369,6 +369,8 @@ sub GenerateFiles
HAVE_UUID_UUID_H => undef,
HAVE_WCSTOMBS_L => undef,
HAVE_VISIBILITY_ATTRIBUTE => undef,
+ HAVE_VSNPRINTF_L => undef,
+ HAVE_VSPRINTF_L => undef,
HAVE_X509_GET_SIGNATURE_INFO => undef,
HAVE_X86_64_POPCNTQ => undef,
HAVE__BOOL => undef,
@@ -383,7 +385,6 @@ sub GenerateFiles
HAVE__BUILTIN_POPCOUNT => undef,
HAVE__BUILTIN_TYPES_COMPATIBLE_P => undef,
HAVE__BUILTIN_UNREACHABLE => undef,
- HAVE__CONFIGTHREADLOCALE => 1,
HAVE__CPUID => 1,
HAVE__GET_CPUID => undef,
HAVE__STATIC_ASSERT => undef,
--
2.39.3 (Apple Git-145)
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-17 18:18 Tom Lane <tgl@sss.pgh.pa.us>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 1 reply; 49+ messages in thread
From: Tom Lane @ 2023-11-17 18:18 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
Thomas Munro <thomas.munro@gmail.com> writes:
> On Thu, Nov 16, 2023 at 12:06 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Thomas Munro <thomas.munro@gmail.com> writes:
>>> Perhaps we could use snprintf_l() and strtod_l() where available.
>>> They're not standard, but they are obvious extensions that NetBSD and
>>> Windows have, and those are the two systems for which we are doing
>>> grotty things in that code.
>> Yeah. I'd say the _l functions should be preferred over uselocale()
>> if available, but sadly they're not there on common systems. (It
>> looks like glibc has strtod_l but not snprintf_l, which is odd.)
> Here is a first attempt.
I've not reviewed this closely, but I did try it on mamba's host.
It compiles and passes regression testing, but I see two warnings:
common.c: In function 'PGTYPESsprintf':
common.c:120:2: warning: function 'PGTYPESsprintf' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
120 | return vsprintf_l(str, PGTYPESclocale, format, args);
| ^~~~~~
common.c: In function 'PGTYPESsnprintf':
common.c:136:2: warning: function 'PGTYPESsnprintf' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
136 | return vsnprintf_l(str, size, PGTYPESclocale, format, args);
| ^~~~~~
That happens because on NetBSD, we define PG_PRINTF_ATTRIBUTE as
"__syslog__" so that the compiler will not warn about use of %m
(apparently, they support %m in syslog() but not printf(), sigh).
I think this is telling us about an actual problem: these new
functions are based on libc's printf not what we have in snprintf.c,
and therefore we really shouldn't be assuming that they will support
any format specs beyond what POSIX requires for printf. If somebody
tried to use %m in one of these calls, we'd like to get warnings about
that.
I experimented with the attached delta patch and it does silence
these warnings. I suspect that ecpg_log() should be marked as
pg_attribute_std_printf() too, because it has the same issue,
but I didn't try that. (Probably, we see no warning for that
because the compiler isn't quite bright enough to connect the
format argument with the string that gets passed to vfprintf().)
regards, tom lane
Attachments:
0002-use-correct-printf-attribute.patch (text/x-diff, 1.7K) download
diff --git a/src/include/c.h b/src/include/c.h
index 82f8e9d4c7..98e3bbf386 100644
--- a/src/include/c.h
+++ b/src/include/c.h
@@ -171,13 +171,19 @@
#define PG_USED_FOR_ASSERTS_ONLY pg_attribute_unused()
#endif
-/* GCC and XLC support format attributes */
+/*
+ * GCC and XLC support format attributes. Use pg_attribute_printf()
+ * for our src/port/snprintf.c implementation and functions based on it.
+ * Use pg_attribute_std_printf() for functions based on libc's printf.
+ */
#if defined(__GNUC__) || defined(__IBMC__)
#define pg_attribute_format_arg(a) __attribute__((format_arg(a)))
#define pg_attribute_printf(f,a) __attribute__((format(PG_PRINTF_ATTRIBUTE, f, a)))
+#define pg_attribute_std_printf(f,a) __attribute__((format(printf, f, a)))
#else
#define pg_attribute_format_arg(a)
#define pg_attribute_printf(f,a)
+#define pg_attribute_std_printf(f,a)
#endif
/* GCC, Sunpro and XLC support aligned, packed and noreturn */
diff --git a/src/interfaces/ecpg/include/pgtypes_format.h b/src/interfaces/ecpg/include/pgtypes_format.h
index d6dd06d361..87160fab59 100644
--- a/src/interfaces/ecpg/include/pgtypes_format.h
+++ b/src/interfaces/ecpg/include/pgtypes_format.h
@@ -20,7 +20,7 @@ extern int PGTYPESbegin_clocale(locale_t *old_locale);
extern void PGTYPESend_clocale(locale_t old_locale);
extern double PGTYPESstrtod(const char *str, char **endptr);
-extern int PGTYPESsprintf(char *str, const char *format, ...) pg_attribute_printf(2, 3);
-extern int PGTYPESsnprintf(char *str, size_t size, const char *format, ...) pg_attribute_printf(3, 4);
+extern int PGTYPESsprintf(char *str, const char *format, ...) pg_attribute_std_printf(2, 3);
+extern int PGTYPESsnprintf(char *str, size_t size, const char *format, ...) pg_attribute_std_printf(3, 4);
#endif
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-17 22:58 Tom Lane <tgl@sss.pgh.pa.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 49+ messages in thread
From: Tom Lane @ 2023-11-17 22:58 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
I wrote:
> I've not reviewed this closely, but I did try it on mamba's host.
> It compiles and passes regression testing, but I see two warnings:
> common.c: In function 'PGTYPESsprintf':
> common.c:120:2: warning: function 'PGTYPESsprintf' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
> 120 | return vsprintf_l(str, PGTYPESclocale, format, args);
> | ^~~~~~
> common.c: In function 'PGTYPESsnprintf':
> common.c:136:2: warning: function 'PGTYPESsnprintf' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
> 136 | return vsnprintf_l(str, size, PGTYPESclocale, format, args);
> | ^~~~~~
> I think this is telling us about an actual problem: these new
> functions are based on libc's printf not what we have in snprintf.c,
> and therefore we really shouldn't be assuming that they will support
> any format specs beyond what POSIX requires for printf.
Wait, I just realized that there's more to this. ecpglib *does*
rely on our snprintf.c functions:
$ nm --ext --undef src/interfaces/ecpg/ecpglib/*.o | grep printf
U pg_snprintf
U pg_fprintf
U pg_snprintf
U pg_printf
U pg_snprintf
U pg_sprintf
U pg_fprintf
U pg_snprintf
U pg_vfprintf
U pg_snprintf
U pg_sprintf
U pg_sprintf
We are getting these warnings because vsprintf_l and
vsnprintf_l don't have snprintf.c implementations, so the
compiler sees the attributes attached to them by stdio.h.
This raises the question of whether changing snprintf.c
could be part of the solution. I'm not sure that we want
to try to emulate vs[n]printf_l directly, but perhaps there's
another way?
In any case, my concern about ecpg_log() is misplaced.
That is really using pg_vfprintf, so it's correctly marked.
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-18 00:03 Andres Freund <andres@anarazel.de>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 0 replies; 49+ messages in thread
From: Andres Freund @ 2023-11-18 00:03 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Tristan Partin <tristan@neon.tech>; pgsql-hackers
Hi,
On 2023-11-17 08:57:47 +1300, Thomas Munro wrote:
> I also had a go[3] at doing it with static inlined functions, to avoid
> creating a load of new exported functions and associated function call
> overheads. It worked fine, except on Windows: I needed a global
> variable PGTYPESclocale that all the inlined functions can see when
> called from ecpglib or pgtypeslib code, but if I put that in the
> exports list then on that platform it seems to contain garbage; there
> is probably some other magic needed to export non-function symbols
> from the DLL or something like that, I didn't look into it. See CI
> failure + crash dumps.
I suspect you'd need __declspec(dllimport) on the variable to make that work.
I.e. use PGDLLIMPORT and define BUILDING_DLL while building the libraries, so
they see __declspec (dllexport). I luckily forgot the details, but functions
just call into some thunk that does necessary magic, but that option doesn't
exist for variables, so the compiler/linker have to do stuff, hence needing
__declspec(dllimport).
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-19 22:00 Thomas Munro <thomas.munro@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2023-11-19 22:00 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Sat, Nov 18, 2023 at 11:58 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I wrote:
> > I've not reviewed this closely, but I did try it on mamba's host.
> > It compiles and passes regression testing, but I see two warnings:
>
> > common.c: In function 'PGTYPESsprintf':
> > common.c:120:2: warning: function 'PGTYPESsprintf' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
> > 120 | return vsprintf_l(str, PGTYPESclocale, format, args);
> > | ^~~~~~
> > common.c: In function 'PGTYPESsnprintf':
> > common.c:136:2: warning: function 'PGTYPESsnprintf' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
> > 136 | return vsnprintf_l(str, size, PGTYPESclocale, format, args);
> > | ^~~~~~
>
> > I think this is telling us about an actual problem: these new
> > functions are based on libc's printf not what we have in snprintf.c,
> > and therefore we really shouldn't be assuming that they will support
> > any format specs beyond what POSIX requires for printf.
Right, thanks.
> We are getting these warnings because vsprintf_l and
> vsnprintf_l don't have snprintf.c implementations, so the
> compiler sees the attributes attached to them by stdio.h.
>
> This raises the question of whether changing snprintf.c
> could be part of the solution. I'm not sure that we want
> to try to emulate vs[n]printf_l directly, but perhaps there's
> another way?
Yeah, I have been wondering about that too.
The stuff I posted so far was just about how to remove some gross and
incorrect code from ecpg, a somewhat niche frontend part of
PostgreSQL. I guess Tristan is thinking bigger: removing obstacles to
going multi-threaded in the backend. Clearly locales are one of the
places where global state will bite us, so we either need to replace
setlocale() with uselocale() for the database default locale, or use
explicit locale arguments with _l() functions everywhere and pass in
the right locale. Due to incompleteness of (a) libc implementations
and (b) the standard, we can't directly do either, so we'll need to
cope with that.
Thought experiment: If we supplied our own fallback _l() replacement
functions where missing, and those did uselocale() save/restore, many
systems wouldn't need them, for example glibc has strtod_l() as you
noted, and several other systems have systematically added them for
all sorts of stuff. The case of the *printf* family is quite
interesting, because there we already have our own implement for other
reasons, so it might make sense to add the _l() variants to our
snprintf.c implementations. On glibc, snprintf.c would have to do a
uselocale() save/restore where it punts %g to the system snprintf, but
if that offends some instruction cycle bean counter, perhaps we could
replace that bit with Ryu anyway (or is it not general enough to
handle all the stuff %g et al can do? I haven't looked).
I am not sure how you would ever figure out what other stuff is
affected by the global locale in general, for example code hiding in
extensions etc, but, I mean, that's what's wrong with global state in
a nutshell and it has often been speculated that multi-threaded
PostgreSQL might have a way to say 'I still want one process per
session because my extensions don't all identify themselves as
thread-safe yet'.
BTW is this comment in snprintf.c true?
* 1. No locale support: the radix character is always '.' and the '
* (single quote) format flag is ignored.
It is in the backend but only because we nail down LC_NUMERIC early
on, not because of any property of snprintf.c, no?
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-19 22:36 Tom Lane <tgl@sss.pgh.pa.us>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Tom Lane @ 2023-11-19 22:36 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
Thomas Munro <thomas.munro@gmail.com> writes:
> BTW is this comment in snprintf.c true?
> * 1. No locale support: the radix character is always '.' and the '
> * (single quote) format flag is ignored.
> It is in the backend but only because we nail down LC_NUMERIC early
> on, not because of any property of snprintf.c, no?
Hmm, the second part of it is true. But given that we punt float
formatting to libc, I think you are right that the first part
depends on LC_NUMERIC being frozen.
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-20 00:00 Thomas Munro <thomas.munro@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2023-11-20 00:00 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Mon, Nov 20, 2023 at 11:36 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > BTW is this comment in snprintf.c true?
>
> > * 1. No locale support: the radix character is always '.' and the '
> > * (single quote) format flag is ignored.
>
> > It is in the backend but only because we nail down LC_NUMERIC early
> > on, not because of any property of snprintf.c, no?
>
> Hmm, the second part of it is true. But given that we punt float
> formatting to libc, I think you are right that the first part
> depends on LC_NUMERIC being frozen.
If we are sure that we'll *never* want locale-aware printf-family
functions (ie we *always* want "C" locale), then in the thought
experiment above where I suggested we supply replacement _l()
functions, we could just skip that for the printf family, but make
that above comment actually true. Perhaps with Ryu, but otherwise by
punting to libc _l() or uselocale() save/restore.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2023-11-20 16:40 Tom Lane <tgl@sss.pgh.pa.us>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Tom Lane @ 2023-11-20 16:40 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
Thomas Munro <thomas.munro@gmail.com> writes:
> If we are sure that we'll *never* want locale-aware printf-family
> functions (ie we *always* want "C" locale), then in the thought
> experiment above where I suggested we supply replacement _l()
> functions, we could just skip that for the printf family, but make
> that above comment actually true. Perhaps with Ryu, but otherwise by
> punting to libc _l() or uselocale() save/restore.
It is pretty annoying that we've got that shiny Ryu code and can't
use it here. From memory, we did look into that and concluded that
Ryu wasn't amenable to providing "exactly this many digits" as is
required by most variants of printf's conversion specs. But maybe
somebody should go try harder. (Worst case, you could do rounding
off by hand on the produced digit string, but that's ugly...)
regards, tom lane
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-08-10 01:29 Thomas Munro <thomas.munro@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2024-08-10 01:29 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Tue, Nov 21, 2023 at 5:40 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > If we are sure that we'll *never* want locale-aware printf-family
> > functions (ie we *always* want "C" locale), then in the thought
> > experiment above where I suggested we supply replacement _l()
> > functions, we could just skip that for the printf family, but make
> > that above comment actually true. Perhaps with Ryu, but otherwise by
> > punting to libc _l() or uselocale() save/restore.
Here is a new attempt at this can of portability worms. This time:
* pg_get_c_locale() is available to anyone who needs a "C" locale_t
* ECPG uses strtod_l(..., pg_get_c_locale()) for parsing
* snprintf.c always uses "C" for floats, so it conforms to its own
documented behaviour, and ECPG doesn't have to do anything special
I'm not trying to offer a working *printf_l() family to the whole tree
because it seems like really we only ever care about "C" for this
purpose. So snprintf.c internally uses pg_get_c_locale() with
snprintf_l(), _snprintf_l() or uselocale()/snprintf()/uselocale()
depending on platform.
> It is pretty annoying that we've got that shiny Ryu code and can't
> use it here. From memory, we did look into that and concluded that
> Ryu wasn't amenable to providing "exactly this many digits" as is
> required by most variants of printf's conversion specs. But maybe
> somebody should go try harder. (Worst case, you could do rounding
> off by hand on the produced digit string, but that's ugly...)
Yeah it does seem like a promising idea, but I haven't looked into it myself.
Attachments:
v2-0001-Improve-locale-thread-safety-of-ECPG.patch (text/x-patch, 29.6K) download
From bc64f2db61b3e2cf7a0cd00b1deaa260d91e2b3c Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 10 Aug 2024 11:01:21 +1200
Subject: [PATCH v2] Improve locale thread safety of ECPG.
Remove the use of setlocale() as a fallback strategy on systems that
don't have uselocale(), where ECPG tries to control LC_NUMERIC
formatting on input and output of floating point numbers.
Instead, create a new function pg_get_c_locale() that can be used to get
a locale_t object for use with thread-safe parsing and formatting
functions. Then:
1. Use strtod_l() for parsing, and supply an implementation using
uselocale() if it is missing. (All systems have one of strtod_l(),
_strtod_l() or uselocale() for a replacement.)
2. Inside our own snprintf.c, use snprintf_l() or _snprintf_l() where
we punt floating point numbers to the system snprintf(), or wrap it in
uselocale() if those are not available.
Since the non-standard _l() functions require <xlocale.h> on *BSD/macOS
systems, simplify the configure probes: instead of XXX_IN_XLOCALE_H for
several features XXX, let's just include <xlocale.h> if HAVE_XLOCALE_H.
The reason for the extra complication was apparently that some old glibc
systems also had an <xlocale.h>, and you weren't supposed to include it
directly, but it's gone now (as far as I can tell it was harmless to do
so anyway).
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
config/c-library.m4 | 55 ---------
configure | 115 +------------------
configure.ac | 6 +-
meson.build | 37 +-----
src/include/pg_config.h.in | 15 +--
src/include/port.h | 29 +++++
src/include/utils/pg_locale.h | 2 +-
src/interfaces/ecpg/ecpglib/connect.c | 37 ------
src/interfaces/ecpg/ecpglib/data.c | 2 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 37 ------
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 15 ---
src/interfaces/ecpg/ecpglib/execute.c | 55 ---------
src/interfaces/ecpg/pgtypeslib/dt_common.c | 6 +-
src/interfaces/ecpg/pgtypeslib/interval.c | 4 +-
src/interfaces/ecpg/pgtypeslib/numeric.c | 2 +-
src/port/Makefile | 1 +
src/port/locale.c | 73 ++++++++++++
src/port/meson.build | 1 +
src/port/snprintf.c | 33 ++++++
19 files changed, 157 insertions(+), 368 deletions(-)
create mode 100644 src/port/locale.c
diff --git a/config/c-library.m4 b/config/c-library.m4
index aa8223d2ef..421bc612b2 100644
--- a/config/c-library.m4
+++ b/config/c-library.m4
@@ -81,58 +81,3 @@ AC_DEFUN([PGAC_STRUCT_SOCKADDR_SA_LEN],
[#include <sys/types.h>
#include <sys/socket.h>
])])# PGAC_STRUCT_SOCKADDR_MEMBERS
-
-
-# PGAC_TYPE_LOCALE_T
-# ------------------
-# Check for the locale_t type and find the right header file. macOS
-# needs xlocale.h; standard is locale.h, but glibc <= 2.25 also had an
-# xlocale.h file that we should not use, so we check the standard
-# header first.
-AC_DEFUN([PGAC_TYPE_LOCALE_T],
-[AC_CACHE_CHECK([for locale_t], pgac_cv_type_locale_t,
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <locale.h>
-locale_t x;],
-[])],
-[pgac_cv_type_locale_t=yes],
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <xlocale.h>
-locale_t x;],
-[])],
-[pgac_cv_type_locale_t='yes (in xlocale.h)'],
-[pgac_cv_type_locale_t=no])])])
-if test "$pgac_cv_type_locale_t" = 'yes (in xlocale.h)'; then
- AC_DEFINE(LOCALE_T_IN_XLOCALE, 1,
- [Define to 1 if `locale_t' requires <xlocale.h>.])
-fi])# PGAC_TYPE_LOCALE_T
-
-
-# PGAC_FUNC_WCSTOMBS_L
-# --------------------
-# Try to find a declaration for wcstombs_l(). It might be in stdlib.h
-# (following the POSIX requirement for wcstombs()), or in locale.h, or in
-# xlocale.h. If it's in the latter, define WCSTOMBS_L_IN_XLOCALE.
-#
-AC_DEFUN([PGAC_FUNC_WCSTOMBS_L],
-[AC_CACHE_CHECK([for wcstombs_l declaration], pgac_cv_func_wcstombs_l,
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <stdlib.h>
-#include <locale.h>],
-[#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif])],
-[pgac_cv_func_wcstombs_l='yes'],
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <stdlib.h>
-#include <locale.h>
-#include <xlocale.h>],
-[#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif])],
-[pgac_cv_func_wcstombs_l='yes (in xlocale.h)'],
-[pgac_cv_func_wcstombs_l='no'])])])
-if test "$pgac_cv_func_wcstombs_l" = 'yes (in xlocale.h)'; then
- AC_DEFINE(WCSTOMBS_L_IN_XLOCALE, 1,
- [Define to 1 if `wcstombs_l' requires <xlocale.h>.])
-fi])# PGAC_FUNC_WCSTOMBS_L
diff --git a/configure b/configure
index 4f3aa44756..298492ec25 100755
--- a/configure
+++ b/configure
@@ -14635,55 +14635,6 @@ _ACEOF
fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for locale_t" >&5
-$as_echo_n "checking for locale_t... " >&6; }
-if ${pgac_cv_type_locale_t+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <locale.h>
-locale_t x;
-int
-main ()
-{
-
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_type_locale_t=yes
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <xlocale.h>
-locale_t x;
-int
-main ()
-{
-
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_type_locale_t='yes (in xlocale.h)'
-else
- pgac_cv_type_locale_t=no
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_type_locale_t" >&5
-$as_echo "$pgac_cv_type_locale_t" >&6; }
-if test "$pgac_cv_type_locale_t" = 'yes (in xlocale.h)'; then
-
-$as_echo "#define LOCALE_T_IN_XLOCALE 1" >>confdefs.h
-
-fi
-
# MSVC doesn't cope well with defining restrict to __restrict, the
# spelling it understands, because it conflicts with
# __declspec(restrict). Therefore we define pg_restrict to the
@@ -15170,59 +15121,6 @@ if test x"$pgac_cv_var_int_timezone" = xyes ; then
$as_echo "#define HAVE_INT_TIMEZONE 1" >>confdefs.h
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for wcstombs_l declaration" >&5
-$as_echo_n "checking for wcstombs_l declaration... " >&6; }
-if ${pgac_cv_func_wcstombs_l+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <stdlib.h>
-#include <locale.h>
-int
-main ()
-{
-#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_func_wcstombs_l='yes'
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <stdlib.h>
-#include <locale.h>
-#include <xlocale.h>
-int
-main ()
-{
-#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_func_wcstombs_l='yes (in xlocale.h)'
-else
- pgac_cv_func_wcstombs_l='no'
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_func_wcstombs_l" >&5
-$as_echo "$pgac_cv_func_wcstombs_l" >&6; }
-if test "$pgac_cv_func_wcstombs_l" = 'yes (in xlocale.h)'; then
-
-$as_echo "#define WCSTOMBS_L_IN_XLOCALE 1" >>confdefs.h
-
fi
# Some versions of libedit contain strlcpy(), setproctitle(), and other
@@ -15232,7 +15130,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strtod_l strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -16005,17 +15903,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index 049bc01491..302aadd427 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1620,8 +1620,6 @@ PGAC_UNION_SEMUN
AC_CHECK_TYPES(socklen_t, [], [], [#include <sys/socket.h>])
PGAC_STRUCT_SOCKADDR_SA_LEN
-PGAC_TYPE_LOCALE_T
-
# MSVC doesn't cope well with defining restrict to __restrict, the
# spelling it understands, because it conflicts with
# __declspec(restrict). Therefore we define pg_restrict to the
@@ -1720,7 +1718,6 @@ fi
##
PGAC_VAR_INT_TIMEZONE
-PGAC_FUNC_WCSTOMBS_L
# Some versions of libedit contain strlcpy(), setproctitle(), and other
# symbols that that library has no business exposing to the world. Pending
@@ -1744,6 +1741,8 @@ AC_CHECK_FUNCS(m4_normalize([
pthread_is_threaded_np
setproctitle
setproctitle_fast
+ snprintf_l
+ strtod_l
strchrnul
strsignal
syncfs
@@ -1833,7 +1832,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index cc176f11b5..932334ed63 100644
--- a/meson.build
+++ b/meson.build
@@ -2407,6 +2407,7 @@ header_checks = [
'sys/ucred.h',
'termios.h',
'ucred.h',
+ 'xlocale.h',
]
foreach header : header_checks
@@ -2550,15 +2551,6 @@ else
cdata.set('STRERROR_R_INT', false)
endif
-# Find the right header file for the locale_t type. macOS needs xlocale.h;
-# standard is locale.h, but glibc <= 2.25 also had an xlocale.h file that
-# we should not use so we check the standard header first. MSVC has a
-# replacement defined in src/include/port/win32_port.h.
-if not cc.has_type('locale_t', prefix: '#include <locale.h>') and \
- cc.has_type('locale_t', prefix: '#include <xlocale.h>')
- cdata.set('LOCALE_T_IN_XLOCALE', 1)
-endif
-
# Check if the C compiler understands typeof or a variant. Define
# HAVE_TYPEOF if so, and define 'typeof' to the actual key word.
foreach kw : ['typeof', '__typeof__', 'decltype']
@@ -2583,30 +2575,6 @@ int main(void)
endif
endforeach
-
-# Try to find a declaration for wcstombs_l(). It might be in stdlib.h
-# (following the POSIX requirement for wcstombs()), or in locale.h, or in
-# xlocale.h. If it's in the latter, define WCSTOMBS_L_IN_XLOCALE.
-wcstombs_l_test = '''
-#include <stdlib.h>
-#include <locale.h>
-@0@
-
-void main(void)
-{
-#ifndef wcstombs_l
- (void) wcstombs_l;
-#endif
-}
-'''
-if (not cc.compiles(wcstombs_l_test.format(''),
- name: 'wcstombs_l') and
- cc.compiles(wcstombs_l_test.format('#include <xlocale.h>'),
- name: 'wcstombs_l in xlocale.h'))
- cdata.set('WCSTOMBS_L_IN_XLOCALE', 1)
-endif
-
-
# MSVC doesn't cope well with defining restrict to __restrict, the spelling it
# understands, because it conflicts with __declspec(restrict). Therefore we
# define pg_restrict to the appropriate definition, which presumably won't
@@ -2658,7 +2626,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
@@ -2690,6 +2657,8 @@ func_checks = [
['shm_open', {'dependencies': [rt_dep], 'define': false}],
['shm_unlink', {'dependencies': [rt_dep], 'define': false}],
['shmget', {'dependencies': [cygipc_dep], 'define': false}],
+ ['snprintf_l'],
+ ['strtod_l'],
['socket', {'dependencies': [socket_dep], 'define': false}],
['strchrnul'],
['strerror_r', {'dependencies': [thread_dep]}],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 0e9b108e66..c0586edab1 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -376,6 +376,9 @@
/* Define to 1 if you have the `setproctitle_fast' function. */
#undef HAVE_SETPROCTITLE_FAST
+/* Define to 1 if you have the `snprintf_l' function. */
+#undef HAVE_SNPRINTF_L
+
/* Define to 1 if the system has the type `socklen_t'. */
#undef HAVE_SOCKLEN_T
@@ -421,6 +424,9 @@
/* Define to 1 if you have the `strsignal' function. */
#undef HAVE_STRSIGNAL
+/* Define to 1 if you have the `strtod_l' function. */
+#undef HAVE_STRTOD_L
+
/* Define to 1 if the system has the type `struct option'. */
#undef HAVE_STRUCT_OPTION
@@ -556,9 +562,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
@@ -577,9 +580,6 @@
/* Define to the appropriate printf length modifier for 64-bit ints. */
#undef INT64_MODIFIER
-/* Define to 1 if `locale_t' requires <xlocale.h>. */
-#undef LOCALE_T_IN_XLOCALE
-
/* Define as the maximum alignment requirement of any C data type. */
#undef MAXIMUM_ALIGNOF
@@ -766,9 +766,6 @@
/* Define to 1 to build with ZSTD support. (--with-zstd) */
#undef USE_ZSTD
-/* Define to 1 if `wcstombs_l' requires <xlocale.h>. */
-#undef WCSTOMBS_L_IN_XLOCALE
-
/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
significant byte first (like Motorola and SPARC, unlike Intel). */
#if defined AC_APPLE_UNIVERSAL_BUILD
diff --git a/src/include/port.h b/src/include/port.h
index c740005267..26a78320f8 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -15,6 +15,11 @@
#include <ctype.h>
+#include <locale.h>
+#ifdef HAVE_XLOCALE_H
+#include <xlocale.h>
+#endif
+
/*
* Windows has enough specialized port stuff that we push most of it off
* into another file.
@@ -217,6 +222,30 @@ extern int pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2,
extern int pg_vprintf(const char *fmt, va_list args) pg_attribute_printf(1, 0);
extern int pg_printf(const char *fmt,...) pg_attribute_printf(1, 2);
+/* Acessor for a thread-safe process-lifetime "C" locale. */
+extern locale_t pg_get_c_locale(void);
+
+#ifndef HAVE_STRTOD_L
+/*
+ * POSIX doesn't define this function, but we can implement it with thread-safe
+ * save-and-restore. Not all systems have uselocale(), but the ones that don't
+ * have strtod_l().
+ */
+static inline double
+strtod_l(const char *nptr, char **endptr, locale_t loc)
+{
+#ifdef WIN32
+ return _strtod_l(nptr, endptr, loc);
+#else
+ locale_t save = uselocale(loc);
+ double result = strtod(nptr, endptr);
+
+ uselocale(save);
+ return result;
+#endif
+}
+#endif
+
#ifndef WIN32
/*
* We add a pg_ prefix as a warning that the Windows implementations have the
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index f41d33975b..9651f3c3ab 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -12,7 +12,7 @@
#ifndef _PG_LOCALE_
#define _PG_LOCALE_
-#if defined(LOCALE_T_IN_XLOCALE) || defined(WCSTOMBS_L_IN_XLOCALE)
+#ifdef HAVE_XLOCALE_H
#include <xlocale.h>
#endif
#ifdef USE_ICU
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 8afb1f0a26..eac1a2e9b5 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,10 +10,6 @@
#include "ecpgtype.h"
#include "sqlca.h"
-#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale = (locale_t) 0;
-#endif
-
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
@@ -483,39 +479,6 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
/* add connection to our list */
pthread_mutex_lock(&connections_mutex);
- /*
- * ... but first, make certain we have created ecpg_clocale. Rely on
- * holding connections_mutex to ensure this is done by only one thread.
- */
-#ifdef HAVE_USELOCALE
- if (!ecpg_clocale)
- {
- ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (!ecpg_clocale)
- {
- pthread_mutex_unlock(&connections_mutex);
- ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
- ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
- if (host)
- ecpg_free(host);
- if (port)
- ecpg_free(port);
- if (options)
- ecpg_free(options);
- if (realname)
- ecpg_free(realname);
- if (dbname)
- ecpg_free(dbname);
- if (conn_keywords)
- ecpg_free(conn_keywords);
- if (conn_values)
- ecpg_free(conn_values);
- free(this);
- return false;
- }
- }
-#endif
-
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index fa56276758..228df49052 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -466,7 +466,7 @@ ecpg_get_data(const PGresult *results, int act_tuple, int act_field, int lineno,
pval++;
if (!check_special_value(pval, &dres, &scan_length))
- dres = strtod(pval, &scan_length);
+ dres = strtod_l(pval, &scan_length, pg_get_c_locale());
if (isarray && *scan_length == '"')
scan_length++;
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c..855322e037 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -475,46 +475,9 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
memset(&stmt, 0, sizeof stmt);
stmt.lineno = lineno;
- /* Make sure we do NOT honor the locale for numeric input */
- /* since the database gives the standard decimal point */
- /* (see comments in execute.c) */
-#ifdef HAVE_USELOCALE
-
- /*
- * To get here, the above PQnfields() test must have found nonzero
- * fields. One needs a connection to create such a descriptor. (EXEC
- * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
- * can't change the descriptor's PQnfields().) Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt.oldlocale = uselocale(ecpg_clocale);
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- setlocale(LC_NUMERIC, "C");
-#endif
-
/* desperate try to guess something sensible */
stmt.connection = ecpg_get_connection(NULL);
ecpg_store_result(ECPGresult, index, &stmt, &data_var);
-
-#ifdef HAVE_USELOCALE
- if (stmt.oldlocale != (locale_t) 0)
- uselocale(stmt.oldlocale);
-#else
- if (stmt.oldlocale)
- {
- setlocale(LC_NUMERIC, stmt.oldlocale);
- ecpg_free(stmt.oldlocale);
- }
-#ifdef HAVE__CONFIGTHREADLOCALE
- if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
-#endif
-#endif
}
else if (data_var.ind_type != ECPGt_NO_INDICATOR && data_var.ind_pointer != NULL)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 01b4309a71..40988d5357 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -13,9 +13,6 @@
#ifndef CHAR_BIT
#include <limits.h>
#endif
-#ifdef LOCALE_T_IN_XLOCALE
-#include <xlocale.h>
-#endif
enum COMPAT_MODE
{
@@ -59,10 +56,6 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
-#ifdef HAVE_USELOCALE
-extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
-#endif
-
/* structure to store one statement */
struct statement
{
@@ -76,14 +69,6 @@ struct statement
bool questionmarks;
struct variable *inlist;
struct variable *outlist;
-#ifdef HAVE_USELOCALE
- locale_t oldlocale;
-#else
- char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
- int oldthreadlocale;
-#endif
-#endif
int nparams;
char **paramvalues;
int *paramlengths;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 04d0b40c53..806d4779fa 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,9 +101,6 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifndef HAVE_USELOCALE
- ecpg_free(stmt->oldlocale);
-#endif
ecpg_free(stmt);
}
@@ -1973,40 +1970,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
if (stmt == NULL)
return false;
- /*
- * Make sure we do NOT honor the locale for numeric input/output since the
- * database wants the standard decimal point. If available, use
- * uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
- */
-#ifdef HAVE_USELOCALE
-
- /*
- * Since ecpg_init() succeeded, we have a connection. Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt->oldlocale = uselocale(ecpg_clocale);
- if (stmt->oldlocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- if (stmt->oldlocale == NULL)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- setlocale(LC_NUMERIC, "C");
-#endif
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
@@ -2213,24 +2176,6 @@ ecpg_do_epilogue(struct statement *stmt)
if (stmt == NULL)
return;
-#ifdef HAVE_USELOCALE
- if (stmt->oldlocale != (locale_t) 0)
- uselocale(stmt->oldlocale);
-#else
- if (stmt->oldlocale)
- setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
-#endif
-#endif
-
free_statement(stmt);
}
diff --git a/src/interfaces/ecpg/pgtypeslib/dt_common.c b/src/interfaces/ecpg/pgtypeslib/dt_common.c
index ed08088bfe..6aff989e0f 100644
--- a/src/interfaces/ecpg/pgtypeslib/dt_common.c
+++ b/src/interfaces/ecpg/pgtypeslib/dt_common.c
@@ -1216,7 +1216,7 @@ DecodeNumber(int flen, char *str, int fmask,
return DecodeNumberField(flen, str, (fmask | DTK_DATE_M),
tmask, tm, fsec, is2digits);
- *fsec = strtod(cp, &cp);
+ *fsec = strtod_l(cp, &cp, pg_get_c_locale());
if (*cp != '\0')
return -1;
}
@@ -2028,7 +2028,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double frac;
- frac = strtod(cp, &cp);
+ frac = strtod_l(cp, &cp, pg_get_c_locale());
if (*cp != '\0')
return -1;
*fsec = frac * 1000000;
@@ -2052,7 +2052,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double time;
- time = strtod(cp, &cp);
+ time = strtod_l(cp, &cp, pg_get_c_locale());
if (*cp != '\0')
return -1;
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 936a688381..8fc34acf68 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -60,7 +60,7 @@ ParseISO8601Number(const char *str, char **endptr, int *ipart, double *fpart)
if (!(isdigit((unsigned char) *str) || *str == '-' || *str == '.'))
return DTERR_BAD_FORMAT;
errno = 0;
- val = strtod(str, endptr);
+ val = strtod_l(str, endptr, pg_get_c_locale());
/* did we not see anything that looks like a double? */
if (*endptr == str || errno != 0)
return DTERR_BAD_FORMAT;
@@ -455,7 +455,7 @@ DecodeInterval(char **field, int *ftype, int nf, /* int range, */
else if (*cp == '.')
{
errno = 0;
- fval = strtod(cp, &cp);
+ fval = strtod_l(cp, &cp, pg_get_c_locale());
if (*cp != '\0' || errno != 0)
return DTERR_BAD_FORMAT;
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 35e7b92da4..f53fdb9f10 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -1455,7 +1455,7 @@ numericvar_to_double(numeric *var, double *dp)
* strtod does not reset errno to 0 in case of success.
*/
errno = 0;
- val = strtod(tmp, &endptr);
+ val = strtod_l(tmp, &endptr, pg_get_c_locale());
if (errno == ERANGE)
{
free(tmp);
diff --git a/src/port/Makefile b/src/port/Makefile
index db7c02117b..03b55bd26b 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -42,6 +42,7 @@ OBJS = \
bsearch_arg.o \
chklocale.o \
inet_net_ntop.o \
+ locale.o \
noblock.o \
path.o \
pg_bitutils.o \
diff --git a/src/port/locale.c b/src/port/locale.c
new file mode 100644
index 0000000000..ca463257ef
--- /dev/null
+++ b/src/port/locale.c
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * locale.c
+ * Helper routines for thread-safe system locale usage.
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/locale.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#include <locale.h>
+#ifdef HAVE_XLOCALE_H
+#include <xlocale.h>
+#endif
+
+#ifndef WIN32
+#include <pthread.h>
+#else
+#include <synchapi.h>
+#endif
+
+static locale_t c_locale;
+
+#ifndef WIN32
+static void
+init_c_locale_once(void)
+{
+ c_locale = newlocale(LC_ALL, "C", NULL);
+}
+#else
+static BOOL
+init_c_locale_once(PINIT_ONCE once, PVOID parameter, PVOID *context)
+{
+ c_locale = _create_locale(LC_ALL, "C");
+ return true;
+}
+#endif
+
+locale_t
+pg_get_c_locale(void)
+{
+ /*
+ * Fast path if already initialized. This assumes that we can read a
+ * locale_t (in practice, a pointer) without tearing in a multi-threaded
+ * program.
+ */
+ if (c_locale != (locale_t) 0)
+ return c_locale;
+
+ /* Make a locale_t. It will live until process exit. */
+ {
+#ifndef WIN32
+ static pthread_once_t once = PTHREAD_ONCE_INIT;
+
+ pthread_once(&once, init_c_locale_once);
+#else
+ static INIT_ONCE once;
+ InitOnceExecuteOnce(&once, init_c_locale_once, NULL, NULL);
+#endif
+ }
+
+ Assert(c_locale != (locale_t) 0);
+
+ return c_locale;
+}
diff --git a/src/port/meson.build b/src/port/meson.build
index ff54b7b53e..e21961fd73 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -5,6 +5,7 @@ pgport_sources = [
'chklocale.c',
'inet_net_ntop.c',
'noblock.c',
+ 'locale.c',
'path.c',
'pg_bitutils.c',
'pg_strong_random.c',
diff --git a/src/port/snprintf.c b/src/port/snprintf.c
index 884f0262dd..c122e0deb7 100644
--- a/src/port/snprintf.c
+++ b/src/port/snprintf.c
@@ -35,6 +35,7 @@
#include <math.h>
+
/*
* We used to use the platform's NL_ARGMAX here, but that's a bad idea,
* first because the point of this module is to remove platform dependencies
@@ -109,6 +110,15 @@
#undef vprintf
#undef printf
+#ifdef WIN32
+/*
+ * This is only good enough to claim support for the usage we make in this
+ * module, so keep this claim of HAVE_SNPRINTF_L local.
+ */
+#define snprintf_l(str, size, loc, format, ...) _snprintf_l(str, size, format, loc, __VA_ARGS__)
+#define HAVE_SNPRINTF_L
+#endif
+
/*
* Info about where the formatted output is going.
*
@@ -1184,6 +1194,9 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
* according to == but not according to memcmp.
*/
static const double dzero = 0.0;
+#ifndef HAVE_SNPRINTF_L
+ locale_t save_locale = uselocale(pg_get_c_locale());
+#endif
if (adjust_sign((value < 0.0 ||
(value == 0.0 &&
@@ -1205,7 +1218,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '*';
fmt[3] = type;
fmt[4] = '\0';
+#ifdef HAVE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), pg_get_c_locale(), fmt, prec, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, prec, value);
+#endif
}
else
{
@@ -1214,6 +1231,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '\0';
vallen = snprintf(convert, sizeof(convert), fmt, value);
}
+
+#ifndef HAVE_SNPRINTF_L
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
goto fail;
@@ -1336,12 +1358,23 @@ pg_strfromd(char *str, size_t count, int precision, double value)
}
else
{
+#ifndef HAVE_SNPRINTF_L
+ locale_t save_locale;
+#endif
+
fmt[0] = '%';
fmt[1] = '.';
fmt[2] = '*';
fmt[3] = 'g';
fmt[4] = '\0';
+#ifdef HAVE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), pg_get_c_locale(), fmt, precision, value);
+#else
+ save_locale = uselocale(pg_get_c_locale());
vallen = snprintf(convert, sizeof(convert), fmt, precision, value);
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
{
target.failed = true;
--
2.39.2
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-08-10 03:48 Thomas Munro <thomas.munro@gmail.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2024-08-10 03:48 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Sat, Aug 10, 2024 at 1:29 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> Here is a new attempt at this can of portability worms.
Slightly better version:
* it's OK to keep relying on the global locale in the backend; for
now, we know that LC_NUMERIC is set in main(), and in the
multi-threaded future calling setlocale() even transiently will be
banned, so it seems it'll be OK to just keep doing that, right?
* we could use LC_C_LOCALE to get a "C" locale slightly more
efficiently on those; we could define it ourselves for other systems,
using pg_get_c_locale()
Attachments:
v3-0001-Improve-locale-thread-safety-of-ECPG.patch (application/octet-stream, 30.5K) download
From bf222d4a1261daccf499c24ba786b836092769ed Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 10 Aug 2024 11:01:21 +1200
Subject: [PATCH v3] Improve locale thread safety of ECPG.
Remove the use of setlocale() as a fallback strategy on systems that
don't have uselocale(), where ECPG tries to control LC_NUMERIC
formatting on input and output of floating point numbers.
Instead, request the C locale explicitly. Use the special locale_t
value LC_C_LOCALE, which is a non-standard extension in macOS and
NetBSD, and provide our own fallback implementation for every other
system that will allocate a reusable locale_t for the lifetime of the
process. Then:
1. Use strtod_l() for parsing, and supply an implementation using
uselocale() if it is missing.
2. Inside our own snprintf.c, when used in frontend code, use
snprintf_l() or _snprintf_l() where we punt floating point numbers to
the system snprintf(), or wrap it in uselocale() if those are not
available.
Since the non-standard _l() functions require <xlocale.h> on *BSD/macOS
systems, simplify the configure probes: instead of XXX_IN_XLOCALE_H for
several features XXX, let's just include <xlocale.h> if HAVE_XLOCALE_H.
The reason for the extra complication was apparently that some old glibc
systems also had an <xlocale.h>, and you weren't supposed to include it
directly, but it's gone now (as far as I can tell it was harmless to do
so anyway).
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
config/c-library.m4 | 55 ---------
configure | 115 +------------------
configure.ac | 6 +-
meson.build | 37 +-----
src/include/pg_config.h.in | 15 +--
src/include/port.h | 37 ++++++
src/include/utils/pg_locale.h | 2 +-
src/interfaces/ecpg/ecpglib/connect.c | 37 ------
src/interfaces/ecpg/ecpglib/data.c | 2 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 37 ------
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 15 ---
src/interfaces/ecpg/ecpglib/execute.c | 55 ---------
src/interfaces/ecpg/pgtypeslib/dt_common.c | 6 +-
src/interfaces/ecpg/pgtypeslib/interval.c | 4 +-
src/interfaces/ecpg/pgtypeslib/numeric.c | 2 +-
src/port/Makefile | 1 +
src/port/locale.c | 73 ++++++++++++
src/port/meson.build | 1 +
src/port/snprintf.c | 53 +++++++++
19 files changed, 185 insertions(+), 368 deletions(-)
create mode 100644 src/port/locale.c
diff --git a/config/c-library.m4 b/config/c-library.m4
index aa8223d2ef0..421bc612b27 100644
--- a/config/c-library.m4
+++ b/config/c-library.m4
@@ -81,58 +81,3 @@ AC_DEFUN([PGAC_STRUCT_SOCKADDR_SA_LEN],
[#include <sys/types.h>
#include <sys/socket.h>
])])# PGAC_STRUCT_SOCKADDR_MEMBERS
-
-
-# PGAC_TYPE_LOCALE_T
-# ------------------
-# Check for the locale_t type and find the right header file. macOS
-# needs xlocale.h; standard is locale.h, but glibc <= 2.25 also had an
-# xlocale.h file that we should not use, so we check the standard
-# header first.
-AC_DEFUN([PGAC_TYPE_LOCALE_T],
-[AC_CACHE_CHECK([for locale_t], pgac_cv_type_locale_t,
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <locale.h>
-locale_t x;],
-[])],
-[pgac_cv_type_locale_t=yes],
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <xlocale.h>
-locale_t x;],
-[])],
-[pgac_cv_type_locale_t='yes (in xlocale.h)'],
-[pgac_cv_type_locale_t=no])])])
-if test "$pgac_cv_type_locale_t" = 'yes (in xlocale.h)'; then
- AC_DEFINE(LOCALE_T_IN_XLOCALE, 1,
- [Define to 1 if `locale_t' requires <xlocale.h>.])
-fi])# PGAC_TYPE_LOCALE_T
-
-
-# PGAC_FUNC_WCSTOMBS_L
-# --------------------
-# Try to find a declaration for wcstombs_l(). It might be in stdlib.h
-# (following the POSIX requirement for wcstombs()), or in locale.h, or in
-# xlocale.h. If it's in the latter, define WCSTOMBS_L_IN_XLOCALE.
-#
-AC_DEFUN([PGAC_FUNC_WCSTOMBS_L],
-[AC_CACHE_CHECK([for wcstombs_l declaration], pgac_cv_func_wcstombs_l,
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <stdlib.h>
-#include <locale.h>],
-[#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif])],
-[pgac_cv_func_wcstombs_l='yes'],
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <stdlib.h>
-#include <locale.h>
-#include <xlocale.h>],
-[#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif])],
-[pgac_cv_func_wcstombs_l='yes (in xlocale.h)'],
-[pgac_cv_func_wcstombs_l='no'])])])
-if test "$pgac_cv_func_wcstombs_l" = 'yes (in xlocale.h)'; then
- AC_DEFINE(WCSTOMBS_L_IN_XLOCALE, 1,
- [Define to 1 if `wcstombs_l' requires <xlocale.h>.])
-fi])# PGAC_FUNC_WCSTOMBS_L
diff --git a/configure b/configure
index 4f3aa447566..298492ec251 100755
--- a/configure
+++ b/configure
@@ -14635,55 +14635,6 @@ _ACEOF
fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for locale_t" >&5
-$as_echo_n "checking for locale_t... " >&6; }
-if ${pgac_cv_type_locale_t+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <locale.h>
-locale_t x;
-int
-main ()
-{
-
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_type_locale_t=yes
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <xlocale.h>
-locale_t x;
-int
-main ()
-{
-
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_type_locale_t='yes (in xlocale.h)'
-else
- pgac_cv_type_locale_t=no
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_type_locale_t" >&5
-$as_echo "$pgac_cv_type_locale_t" >&6; }
-if test "$pgac_cv_type_locale_t" = 'yes (in xlocale.h)'; then
-
-$as_echo "#define LOCALE_T_IN_XLOCALE 1" >>confdefs.h
-
-fi
-
# MSVC doesn't cope well with defining restrict to __restrict, the
# spelling it understands, because it conflicts with
# __declspec(restrict). Therefore we define pg_restrict to the
@@ -15170,59 +15121,6 @@ if test x"$pgac_cv_var_int_timezone" = xyes ; then
$as_echo "#define HAVE_INT_TIMEZONE 1" >>confdefs.h
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for wcstombs_l declaration" >&5
-$as_echo_n "checking for wcstombs_l declaration... " >&6; }
-if ${pgac_cv_func_wcstombs_l+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <stdlib.h>
-#include <locale.h>
-int
-main ()
-{
-#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_func_wcstombs_l='yes'
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <stdlib.h>
-#include <locale.h>
-#include <xlocale.h>
-int
-main ()
-{
-#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_func_wcstombs_l='yes (in xlocale.h)'
-else
- pgac_cv_func_wcstombs_l='no'
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_func_wcstombs_l" >&5
-$as_echo "$pgac_cv_func_wcstombs_l" >&6; }
-if test "$pgac_cv_func_wcstombs_l" = 'yes (in xlocale.h)'; then
-
-$as_echo "#define WCSTOMBS_L_IN_XLOCALE 1" >>confdefs.h
-
fi
# Some versions of libedit contain strlcpy(), setproctitle(), and other
@@ -15232,7 +15130,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strtod_l strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -16005,17 +15903,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index 049bc014911..302aadd427d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1620,8 +1620,6 @@ PGAC_UNION_SEMUN
AC_CHECK_TYPES(socklen_t, [], [], [#include <sys/socket.h>])
PGAC_STRUCT_SOCKADDR_SA_LEN
-PGAC_TYPE_LOCALE_T
-
# MSVC doesn't cope well with defining restrict to __restrict, the
# spelling it understands, because it conflicts with
# __declspec(restrict). Therefore we define pg_restrict to the
@@ -1720,7 +1718,6 @@ fi
##
PGAC_VAR_INT_TIMEZONE
-PGAC_FUNC_WCSTOMBS_L
# Some versions of libedit contain strlcpy(), setproctitle(), and other
# symbols that that library has no business exposing to the world. Pending
@@ -1744,6 +1741,8 @@ AC_CHECK_FUNCS(m4_normalize([
pthread_is_threaded_np
setproctitle
setproctitle_fast
+ snprintf_l
+ strtod_l
strchrnul
strsignal
syncfs
@@ -1833,7 +1832,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index cc176f11b5d..932334ed634 100644
--- a/meson.build
+++ b/meson.build
@@ -2407,6 +2407,7 @@ header_checks = [
'sys/ucred.h',
'termios.h',
'ucred.h',
+ 'xlocale.h',
]
foreach header : header_checks
@@ -2550,15 +2551,6 @@ else
cdata.set('STRERROR_R_INT', false)
endif
-# Find the right header file for the locale_t type. macOS needs xlocale.h;
-# standard is locale.h, but glibc <= 2.25 also had an xlocale.h file that
-# we should not use so we check the standard header first. MSVC has a
-# replacement defined in src/include/port/win32_port.h.
-if not cc.has_type('locale_t', prefix: '#include <locale.h>') and \
- cc.has_type('locale_t', prefix: '#include <xlocale.h>')
- cdata.set('LOCALE_T_IN_XLOCALE', 1)
-endif
-
# Check if the C compiler understands typeof or a variant. Define
# HAVE_TYPEOF if so, and define 'typeof' to the actual key word.
foreach kw : ['typeof', '__typeof__', 'decltype']
@@ -2583,30 +2575,6 @@ int main(void)
endif
endforeach
-
-# Try to find a declaration for wcstombs_l(). It might be in stdlib.h
-# (following the POSIX requirement for wcstombs()), or in locale.h, or in
-# xlocale.h. If it's in the latter, define WCSTOMBS_L_IN_XLOCALE.
-wcstombs_l_test = '''
-#include <stdlib.h>
-#include <locale.h>
-@0@
-
-void main(void)
-{
-#ifndef wcstombs_l
- (void) wcstombs_l;
-#endif
-}
-'''
-if (not cc.compiles(wcstombs_l_test.format(''),
- name: 'wcstombs_l') and
- cc.compiles(wcstombs_l_test.format('#include <xlocale.h>'),
- name: 'wcstombs_l in xlocale.h'))
- cdata.set('WCSTOMBS_L_IN_XLOCALE', 1)
-endif
-
-
# MSVC doesn't cope well with defining restrict to __restrict, the spelling it
# understands, because it conflicts with __declspec(restrict). Therefore we
# define pg_restrict to the appropriate definition, which presumably won't
@@ -2658,7 +2626,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
@@ -2690,6 +2657,8 @@ func_checks = [
['shm_open', {'dependencies': [rt_dep], 'define': false}],
['shm_unlink', {'dependencies': [rt_dep], 'define': false}],
['shmget', {'dependencies': [cygipc_dep], 'define': false}],
+ ['snprintf_l'],
+ ['strtod_l'],
['socket', {'dependencies': [socket_dep], 'define': false}],
['strchrnul'],
['strerror_r', {'dependencies': [thread_dep]}],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 0e9b108e667..c0586edab13 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -376,6 +376,9 @@
/* Define to 1 if you have the `setproctitle_fast' function. */
#undef HAVE_SETPROCTITLE_FAST
+/* Define to 1 if you have the `snprintf_l' function. */
+#undef HAVE_SNPRINTF_L
+
/* Define to 1 if the system has the type `socklen_t'. */
#undef HAVE_SOCKLEN_T
@@ -421,6 +424,9 @@
/* Define to 1 if you have the `strsignal' function. */
#undef HAVE_STRSIGNAL
+/* Define to 1 if you have the `strtod_l' function. */
+#undef HAVE_STRTOD_L
+
/* Define to 1 if the system has the type `struct option'. */
#undef HAVE_STRUCT_OPTION
@@ -556,9 +562,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
@@ -577,9 +580,6 @@
/* Define to the appropriate printf length modifier for 64-bit ints. */
#undef INT64_MODIFIER
-/* Define to 1 if `locale_t' requires <xlocale.h>. */
-#undef LOCALE_T_IN_XLOCALE
-
/* Define as the maximum alignment requirement of any C data type. */
#undef MAXIMUM_ALIGNOF
@@ -766,9 +766,6 @@
/* Define to 1 to build with ZSTD support. (--with-zstd) */
#undef USE_ZSTD
-/* Define to 1 if `wcstombs_l' requires <xlocale.h>. */
-#undef WCSTOMBS_L_IN_XLOCALE
-
/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
significant byte first (like Motorola and SPARC, unlike Intel). */
#if defined AC_APPLE_UNIVERSAL_BUILD
diff --git a/src/include/port.h b/src/include/port.h
index c7400052675..5ff464a0609 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -15,6 +15,11 @@
#include <ctype.h>
+#include <locale.h>
+#ifdef HAVE_XLOCALE_H
+#include <xlocale.h>
+#endif
+
/*
* Windows has enough specialized port stuff that we push most of it off
* into another file.
@@ -217,6 +222,38 @@ extern int pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2,
extern int pg_vprintf(const char *fmt, va_list args) pg_attribute_printf(1, 0);
extern int pg_printf(const char *fmt,...) pg_attribute_printf(1, 2);
+/* Accessor for a thread-safe process-lifetime "C" locale. */
+extern locale_t pg_get_c_locale(void);
+
+/*
+ * A couple of systems offer a fast way to access a "C" locale. We can use
+ * that name in our tree, but fall back to the above function.
+ */
+#ifndef LC_C_LOCALE
+#define LC_C_LOCALE pg_get_c_locale()
+#endif
+
+#ifndef HAVE_STRTOD_L
+/*
+ * POSIX doesn't define this function, but we can implement it with thread-safe
+ * save-and-restore. Not all systems have uselocale(), but the ones that don't
+ * have strtod_l().
+ */
+static inline double
+strtod_l(const char *nptr, char **endptr, locale_t loc)
+{
+#ifdef WIN32
+ return _strtod_l(nptr, endptr, loc);
+#else
+ locale_t save = uselocale(loc);
+ double result = strtod(nptr, endptr);
+
+ uselocale(save);
+ return result;
+#endif
+}
+#endif
+
#ifndef WIN32
/*
* We add a pg_ prefix as a warning that the Windows implementations have the
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index f41d33975be..9651f3c3ab5 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -12,7 +12,7 @@
#ifndef _PG_LOCALE_
#define _PG_LOCALE_
-#if defined(LOCALE_T_IN_XLOCALE) || defined(WCSTOMBS_L_IN_XLOCALE)
+#ifdef HAVE_XLOCALE_H
#include <xlocale.h>
#endif
#ifdef USE_ICU
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 8afb1f0a26f..eac1a2e9b53 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,10 +10,6 @@
#include "ecpgtype.h"
#include "sqlca.h"
-#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale = (locale_t) 0;
-#endif
-
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
@@ -483,39 +479,6 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
/* add connection to our list */
pthread_mutex_lock(&connections_mutex);
- /*
- * ... but first, make certain we have created ecpg_clocale. Rely on
- * holding connections_mutex to ensure this is done by only one thread.
- */
-#ifdef HAVE_USELOCALE
- if (!ecpg_clocale)
- {
- ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (!ecpg_clocale)
- {
- pthread_mutex_unlock(&connections_mutex);
- ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
- ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
- if (host)
- ecpg_free(host);
- if (port)
- ecpg_free(port);
- if (options)
- ecpg_free(options);
- if (realname)
- ecpg_free(realname);
- if (dbname)
- ecpg_free(dbname);
- if (conn_keywords)
- ecpg_free(conn_keywords);
- if (conn_values)
- ecpg_free(conn_values);
- free(this);
- return false;
- }
- }
-#endif
-
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index fa562767585..fbbfd0613fd 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -466,7 +466,7 @@ ecpg_get_data(const PGresult *results, int act_tuple, int act_field, int lineno,
pval++;
if (!check_special_value(pval, &dres, &scan_length))
- dres = strtod(pval, &scan_length);
+ dres = strtod_l(pval, &scan_length, LC_C_LOCALE);
if (isarray && *scan_length == '"')
scan_length++;
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c4..855322e037f 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -475,46 +475,9 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
memset(&stmt, 0, sizeof stmt);
stmt.lineno = lineno;
- /* Make sure we do NOT honor the locale for numeric input */
- /* since the database gives the standard decimal point */
- /* (see comments in execute.c) */
-#ifdef HAVE_USELOCALE
-
- /*
- * To get here, the above PQnfields() test must have found nonzero
- * fields. One needs a connection to create such a descriptor. (EXEC
- * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
- * can't change the descriptor's PQnfields().) Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt.oldlocale = uselocale(ecpg_clocale);
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- setlocale(LC_NUMERIC, "C");
-#endif
-
/* desperate try to guess something sensible */
stmt.connection = ecpg_get_connection(NULL);
ecpg_store_result(ECPGresult, index, &stmt, &data_var);
-
-#ifdef HAVE_USELOCALE
- if (stmt.oldlocale != (locale_t) 0)
- uselocale(stmt.oldlocale);
-#else
- if (stmt.oldlocale)
- {
- setlocale(LC_NUMERIC, stmt.oldlocale);
- ecpg_free(stmt.oldlocale);
- }
-#ifdef HAVE__CONFIGTHREADLOCALE
- if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
-#endif
-#endif
}
else if (data_var.ind_type != ECPGt_NO_INDICATOR && data_var.ind_pointer != NULL)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 01b4309a710..40988d53575 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -13,9 +13,6 @@
#ifndef CHAR_BIT
#include <limits.h>
#endif
-#ifdef LOCALE_T_IN_XLOCALE
-#include <xlocale.h>
-#endif
enum COMPAT_MODE
{
@@ -59,10 +56,6 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
-#ifdef HAVE_USELOCALE
-extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
-#endif
-
/* structure to store one statement */
struct statement
{
@@ -76,14 +69,6 @@ struct statement
bool questionmarks;
struct variable *inlist;
struct variable *outlist;
-#ifdef HAVE_USELOCALE
- locale_t oldlocale;
-#else
- char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
- int oldthreadlocale;
-#endif
-#endif
int nparams;
char **paramvalues;
int *paramlengths;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 04d0b40c537..806d4779fa9 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,9 +101,6 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifndef HAVE_USELOCALE
- ecpg_free(stmt->oldlocale);
-#endif
ecpg_free(stmt);
}
@@ -1973,40 +1970,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
if (stmt == NULL)
return false;
- /*
- * Make sure we do NOT honor the locale for numeric input/output since the
- * database wants the standard decimal point. If available, use
- * uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
- */
-#ifdef HAVE_USELOCALE
-
- /*
- * Since ecpg_init() succeeded, we have a connection. Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt->oldlocale = uselocale(ecpg_clocale);
- if (stmt->oldlocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- if (stmt->oldlocale == NULL)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- setlocale(LC_NUMERIC, "C");
-#endif
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
@@ -2213,24 +2176,6 @@ ecpg_do_epilogue(struct statement *stmt)
if (stmt == NULL)
return;
-#ifdef HAVE_USELOCALE
- if (stmt->oldlocale != (locale_t) 0)
- uselocale(stmt->oldlocale);
-#else
- if (stmt->oldlocale)
- setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
-#endif
-#endif
-
free_statement(stmt);
}
diff --git a/src/interfaces/ecpg/pgtypeslib/dt_common.c b/src/interfaces/ecpg/pgtypeslib/dt_common.c
index ed08088bfe1..f1f57f15a5e 100644
--- a/src/interfaces/ecpg/pgtypeslib/dt_common.c
+++ b/src/interfaces/ecpg/pgtypeslib/dt_common.c
@@ -1216,7 +1216,7 @@ DecodeNumber(int flen, char *str, int fmask,
return DecodeNumberField(flen, str, (fmask | DTK_DATE_M),
tmask, tm, fsec, is2digits);
- *fsec = strtod(cp, &cp);
+ *fsec = strtod_l(cp, &cp, LC_C_LOCALE);
if (*cp != '\0')
return -1;
}
@@ -2028,7 +2028,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double frac;
- frac = strtod(cp, &cp);
+ frac = strtod_l(cp, &cp, LC_C_LOCALE);
if (*cp != '\0')
return -1;
*fsec = frac * 1000000;
@@ -2052,7 +2052,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double time;
- time = strtod(cp, &cp);
+ time = strtod_l(cp, &cp, LC_C_LOCALE);
if (*cp != '\0')
return -1;
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 936a6883816..4d4d9a8f045 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -60,7 +60,7 @@ ParseISO8601Number(const char *str, char **endptr, int *ipart, double *fpart)
if (!(isdigit((unsigned char) *str) || *str == '-' || *str == '.'))
return DTERR_BAD_FORMAT;
errno = 0;
- val = strtod(str, endptr);
+ val = strtod_l(str, endptr, LC_C_LOCALE);
/* did we not see anything that looks like a double? */
if (*endptr == str || errno != 0)
return DTERR_BAD_FORMAT;
@@ -455,7 +455,7 @@ DecodeInterval(char **field, int *ftype, int nf, /* int range, */
else if (*cp == '.')
{
errno = 0;
- fval = strtod(cp, &cp);
+ fval = strtod_l(cp, &cp, LC_C_LOCALE);
if (*cp != '\0' || errno != 0)
return DTERR_BAD_FORMAT;
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 35e7b92da40..4d39fb6c409 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -1455,7 +1455,7 @@ numericvar_to_double(numeric *var, double *dp)
* strtod does not reset errno to 0 in case of success.
*/
errno = 0;
- val = strtod(tmp, &endptr);
+ val = strtod_l(tmp, &endptr, LC_C_LOCALE);
if (errno == ERANGE)
{
free(tmp);
diff --git a/src/port/Makefile b/src/port/Makefile
index db7c02117b0..03b55bd26b0 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -42,6 +42,7 @@ OBJS = \
bsearch_arg.o \
chklocale.o \
inet_net_ntop.o \
+ locale.o \
noblock.o \
path.o \
pg_bitutils.o \
diff --git a/src/port/locale.c b/src/port/locale.c
new file mode 100644
index 00000000000..ca463257ef6
--- /dev/null
+++ b/src/port/locale.c
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * locale.c
+ * Helper routines for thread-safe system locale usage.
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/locale.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#include <locale.h>
+#ifdef HAVE_XLOCALE_H
+#include <xlocale.h>
+#endif
+
+#ifndef WIN32
+#include <pthread.h>
+#else
+#include <synchapi.h>
+#endif
+
+static locale_t c_locale;
+
+#ifndef WIN32
+static void
+init_c_locale_once(void)
+{
+ c_locale = newlocale(LC_ALL, "C", NULL);
+}
+#else
+static BOOL
+init_c_locale_once(PINIT_ONCE once, PVOID parameter, PVOID *context)
+{
+ c_locale = _create_locale(LC_ALL, "C");
+ return true;
+}
+#endif
+
+locale_t
+pg_get_c_locale(void)
+{
+ /*
+ * Fast path if already initialized. This assumes that we can read a
+ * locale_t (in practice, a pointer) without tearing in a multi-threaded
+ * program.
+ */
+ if (c_locale != (locale_t) 0)
+ return c_locale;
+
+ /* Make a locale_t. It will live until process exit. */
+ {
+#ifndef WIN32
+ static pthread_once_t once = PTHREAD_ONCE_INIT;
+
+ pthread_once(&once, init_c_locale_once);
+#else
+ static INIT_ONCE once;
+ InitOnceExecuteOnce(&once, init_c_locale_once, NULL, NULL);
+#endif
+ }
+
+ Assert(c_locale != (locale_t) 0);
+
+ return c_locale;
+}
diff --git a/src/port/meson.build b/src/port/meson.build
index ff54b7b53e9..e21961fd739 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -5,6 +5,7 @@ pgport_sources = [
'chklocale.c',
'inet_net_ntop.c',
'noblock.c',
+ 'locale.c',
'path.c',
'pg_bitutils.c',
'pg_strong_random.c',
diff --git a/src/port/snprintf.c b/src/port/snprintf.c
index 884f0262dd1..fce619eb673 100644
--- a/src/port/snprintf.c
+++ b/src/port/snprintf.c
@@ -109,6 +109,34 @@
#undef vprintf
#undef printf
+#if defined(FRONTEND)
+/*
+ * Frontend code might be multi-threaded. When calling the system snprintf()
+ * for floats, we have to use either the snprintf_l() variant, or
+ * save-and-restore the calling thread's locale using uselocale().
+ */
+#if defined(HAVE_SNPRINTF_L)
+/* At least macOS and the BSDs have the snprintf_l() extension. */
+#define USE_SNPRINTF_L
+#elif defined(WIN32)
+/* Windows has a version with a different name and argument order. */
+#define snprintf_l(str, size, loc, format, ...) _snprintf_l(str, size, format, loc, __VA_ARGS__)
+#define USE_SNPRINTF_L
+#else
+/* We have to do a thread-safe save-and-restore around snprintf(). */
+#define USE_USE_LOCALE
+#endif
+#else
+/*
+ * Backend code doesn't need to worry about the locales, because LC_NUMERIC is
+ * set to "C" in main() and doesn't change. Plain old snprintf() is always OK.
+ *
+ * XXX If the backend were multithreaded, it would have to be 100% certain that
+ * no one is calling setlocale() concurrently, even transiently, to be able to
+ * keep this small optimization.
+ */
+#endif
+
/*
* Info about where the formatted output is going.
*
@@ -1184,6 +1212,9 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
* according to == but not according to memcmp.
*/
static const double dzero = 0.0;
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(LC_C_LOCALE);
+#endif
if (adjust_sign((value < 0.0 ||
(value == 0.0 &&
@@ -1205,7 +1236,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '*';
fmt[3] = type;
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), LC_C_LOCALE, fmt, prec, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, prec, value);
+#endif
}
else
{
@@ -1214,6 +1249,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '\0';
vallen = snprintf(convert, sizeof(convert), fmt, value);
}
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
goto fail;
@@ -1336,12 +1376,25 @@ pg_strfromd(char *str, size_t count, int precision, double value)
}
else
{
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(LC_C_LOCALE);
+#endif
+
fmt[0] = '%';
fmt[1] = '.';
fmt[2] = '*';
fmt[3] = 'g';
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), LC_C_LOCALE, fmt, precision, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, precision, value);
+#endif
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
{
target.failed = true;
--
2.39.3 (Apple Git-146)
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-08-10 03:52 Thomas Munro <thomas.munro@gmail.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2024-08-10 03:52 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
On Sat, Aug 10, 2024 at 3:48 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> * we could use LC_C_LOCALE to get a "C" locale slightly more
> efficiently on those
Oops, lost some words, I meant "on those systems that have them (macOS
and NetBSD AFAIK)"
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-08-10 22:11 Thomas Munro <thomas.munro@gmail.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 2 replies; 49+ messages in thread
From: Thomas Munro @ 2024-08-10 22:11 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tristan Partin <tristan@neon.tech>; pgsql-hackers
v4 adds error handling, in case newlocale("C") fails. I created CF
entry #5166 for this.
Attachments:
v4-0001-Improve-locale-thread-safety-of-ECPG.patch (text/x-patch, 31.4K) download
From 3543cc04b0d66c7f83bdccbb247eedd93cdea4af Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 10 Aug 2024 11:01:21 +1200
Subject: [PATCH v4] Improve locale thread safety of ECPG.
Remove the use of setlocale() as a fallback strategy on systems that
don't have uselocale(), where ECPG tries to control LC_NUMERIC
formatting on input and output of floating point numbers.
Instead, use the C locale explicitly. Provide the name PG_C_LOCALE for
this purpose. On a couple of systems this maps to a system-provided
LC_C_LOCALE (a non-standard extension to POSIX), and otherwise we
provide a thread-safe singleton constructor that allocates a locale_t
for (LC_ALL, "C") for the lifetime of the process.
1. Use strtod_l(..., PG_C_LOCALE) for parsing floats, and supply an
implementation of that common but non-standard extension with
standard uselocale() + strtod() if it is missing.
2. Inside our own snprintf.c, in front-end code only, use non-standard
snprintf_l() where we punt floating point numbers to the system
snprintf(), or wrap it in uselocale() if not available.
Since the non-standard _l() functions require <xlocale.h> on *BSD/macOS
systems, simplify the configure probes: instead of XXX_IN_XLOCALE_H for
several features XXX, let's just include <xlocale.h> if HAVE_XLOCALE_H.
The reason for the extra complication was apparently that some old glibc
systems also had an <xlocale.h>, and you weren't supposed to include it
directly, but it's gone now (as far as I can tell it was harmless to do
so anyway).
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
config/c-library.m4 | 55 ---------
configure | 115 +------------------
configure.ac | 6 +-
meson.build | 37 +-----
src/include/pg_config.h.in | 15 +--
src/include/port.h | 42 +++++++
src/include/utils/pg_locale.h | 2 +-
src/interfaces/ecpg/ecpglib/connect.c | 39 +------
src/interfaces/ecpg/ecpglib/data.c | 2 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 37 ------
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 15 ---
src/interfaces/ecpg/ecpglib/execute.c | 55 ---------
src/interfaces/ecpg/pgtypeslib/dt_common.c | 6 +-
src/interfaces/ecpg/pgtypeslib/interval.c | 4 +-
src/interfaces/ecpg/pgtypeslib/numeric.c | 2 +-
src/port/Makefile | 1 +
src/port/locale.c | 78 +++++++++++++
src/port/meson.build | 1 +
src/port/snprintf.c | 53 +++++++++
19 files changed, 196 insertions(+), 369 deletions(-)
create mode 100644 src/port/locale.c
diff --git a/config/c-library.m4 b/config/c-library.m4
index aa8223d2ef0..421bc612b27 100644
--- a/config/c-library.m4
+++ b/config/c-library.m4
@@ -81,58 +81,3 @@ AC_DEFUN([PGAC_STRUCT_SOCKADDR_SA_LEN],
[#include <sys/types.h>
#include <sys/socket.h>
])])# PGAC_STRUCT_SOCKADDR_MEMBERS
-
-
-# PGAC_TYPE_LOCALE_T
-# ------------------
-# Check for the locale_t type and find the right header file. macOS
-# needs xlocale.h; standard is locale.h, but glibc <= 2.25 also had an
-# xlocale.h file that we should not use, so we check the standard
-# header first.
-AC_DEFUN([PGAC_TYPE_LOCALE_T],
-[AC_CACHE_CHECK([for locale_t], pgac_cv_type_locale_t,
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <locale.h>
-locale_t x;],
-[])],
-[pgac_cv_type_locale_t=yes],
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <xlocale.h>
-locale_t x;],
-[])],
-[pgac_cv_type_locale_t='yes (in xlocale.h)'],
-[pgac_cv_type_locale_t=no])])])
-if test "$pgac_cv_type_locale_t" = 'yes (in xlocale.h)'; then
- AC_DEFINE(LOCALE_T_IN_XLOCALE, 1,
- [Define to 1 if `locale_t' requires <xlocale.h>.])
-fi])# PGAC_TYPE_LOCALE_T
-
-
-# PGAC_FUNC_WCSTOMBS_L
-# --------------------
-# Try to find a declaration for wcstombs_l(). It might be in stdlib.h
-# (following the POSIX requirement for wcstombs()), or in locale.h, or in
-# xlocale.h. If it's in the latter, define WCSTOMBS_L_IN_XLOCALE.
-#
-AC_DEFUN([PGAC_FUNC_WCSTOMBS_L],
-[AC_CACHE_CHECK([for wcstombs_l declaration], pgac_cv_func_wcstombs_l,
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <stdlib.h>
-#include <locale.h>],
-[#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif])],
-[pgac_cv_func_wcstombs_l='yes'],
-[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
-[#include <stdlib.h>
-#include <locale.h>
-#include <xlocale.h>],
-[#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif])],
-[pgac_cv_func_wcstombs_l='yes (in xlocale.h)'],
-[pgac_cv_func_wcstombs_l='no'])])])
-if test "$pgac_cv_func_wcstombs_l" = 'yes (in xlocale.h)'; then
- AC_DEFINE(WCSTOMBS_L_IN_XLOCALE, 1,
- [Define to 1 if `wcstombs_l' requires <xlocale.h>.])
-fi])# PGAC_FUNC_WCSTOMBS_L
diff --git a/configure b/configure
index 4f3aa447566..298492ec251 100755
--- a/configure
+++ b/configure
@@ -14635,55 +14635,6 @@ _ACEOF
fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for locale_t" >&5
-$as_echo_n "checking for locale_t... " >&6; }
-if ${pgac_cv_type_locale_t+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <locale.h>
-locale_t x;
-int
-main ()
-{
-
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_type_locale_t=yes
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <xlocale.h>
-locale_t x;
-int
-main ()
-{
-
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_type_locale_t='yes (in xlocale.h)'
-else
- pgac_cv_type_locale_t=no
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_type_locale_t" >&5
-$as_echo "$pgac_cv_type_locale_t" >&6; }
-if test "$pgac_cv_type_locale_t" = 'yes (in xlocale.h)'; then
-
-$as_echo "#define LOCALE_T_IN_XLOCALE 1" >>confdefs.h
-
-fi
-
# MSVC doesn't cope well with defining restrict to __restrict, the
# spelling it understands, because it conflicts with
# __declspec(restrict). Therefore we define pg_restrict to the
@@ -15170,59 +15121,6 @@ if test x"$pgac_cv_var_int_timezone" = xyes ; then
$as_echo "#define HAVE_INT_TIMEZONE 1" >>confdefs.h
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for wcstombs_l declaration" >&5
-$as_echo_n "checking for wcstombs_l declaration... " >&6; }
-if ${pgac_cv_func_wcstombs_l+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <stdlib.h>
-#include <locale.h>
-int
-main ()
-{
-#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_func_wcstombs_l='yes'
-else
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-#include <stdlib.h>
-#include <locale.h>
-#include <xlocale.h>
-int
-main ()
-{
-#ifndef wcstombs_l
-(void) wcstombs_l;
-#endif
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_compile "$LINENO"; then :
- pgac_cv_func_wcstombs_l='yes (in xlocale.h)'
-else
- pgac_cv_func_wcstombs_l='no'
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_func_wcstombs_l" >&5
-$as_echo "$pgac_cv_func_wcstombs_l" >&6; }
-if test "$pgac_cv_func_wcstombs_l" = 'yes (in xlocale.h)'; then
-
-$as_echo "#define WCSTOMBS_L_IN_XLOCALE 1" >>confdefs.h
-
fi
# Some versions of libedit contain strlcpy(), setproctitle(), and other
@@ -15232,7 +15130,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strtod_l strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -16005,17 +15903,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index 049bc014911..302aadd427d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1620,8 +1620,6 @@ PGAC_UNION_SEMUN
AC_CHECK_TYPES(socklen_t, [], [], [#include <sys/socket.h>])
PGAC_STRUCT_SOCKADDR_SA_LEN
-PGAC_TYPE_LOCALE_T
-
# MSVC doesn't cope well with defining restrict to __restrict, the
# spelling it understands, because it conflicts with
# __declspec(restrict). Therefore we define pg_restrict to the
@@ -1720,7 +1718,6 @@ fi
##
PGAC_VAR_INT_TIMEZONE
-PGAC_FUNC_WCSTOMBS_L
# Some versions of libedit contain strlcpy(), setproctitle(), and other
# symbols that that library has no business exposing to the world. Pending
@@ -1744,6 +1741,8 @@ AC_CHECK_FUNCS(m4_normalize([
pthread_is_threaded_np
setproctitle
setproctitle_fast
+ snprintf_l
+ strtod_l
strchrnul
strsignal
syncfs
@@ -1833,7 +1832,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index cc176f11b5d..932334ed634 100644
--- a/meson.build
+++ b/meson.build
@@ -2407,6 +2407,7 @@ header_checks = [
'sys/ucred.h',
'termios.h',
'ucred.h',
+ 'xlocale.h',
]
foreach header : header_checks
@@ -2550,15 +2551,6 @@ else
cdata.set('STRERROR_R_INT', false)
endif
-# Find the right header file for the locale_t type. macOS needs xlocale.h;
-# standard is locale.h, but glibc <= 2.25 also had an xlocale.h file that
-# we should not use so we check the standard header first. MSVC has a
-# replacement defined in src/include/port/win32_port.h.
-if not cc.has_type('locale_t', prefix: '#include <locale.h>') and \
- cc.has_type('locale_t', prefix: '#include <xlocale.h>')
- cdata.set('LOCALE_T_IN_XLOCALE', 1)
-endif
-
# Check if the C compiler understands typeof or a variant. Define
# HAVE_TYPEOF if so, and define 'typeof' to the actual key word.
foreach kw : ['typeof', '__typeof__', 'decltype']
@@ -2583,30 +2575,6 @@ int main(void)
endif
endforeach
-
-# Try to find a declaration for wcstombs_l(). It might be in stdlib.h
-# (following the POSIX requirement for wcstombs()), or in locale.h, or in
-# xlocale.h. If it's in the latter, define WCSTOMBS_L_IN_XLOCALE.
-wcstombs_l_test = '''
-#include <stdlib.h>
-#include <locale.h>
-@0@
-
-void main(void)
-{
-#ifndef wcstombs_l
- (void) wcstombs_l;
-#endif
-}
-'''
-if (not cc.compiles(wcstombs_l_test.format(''),
- name: 'wcstombs_l') and
- cc.compiles(wcstombs_l_test.format('#include <xlocale.h>'),
- name: 'wcstombs_l in xlocale.h'))
- cdata.set('WCSTOMBS_L_IN_XLOCALE', 1)
-endif
-
-
# MSVC doesn't cope well with defining restrict to __restrict, the spelling it
# understands, because it conflicts with __declspec(restrict). Therefore we
# define pg_restrict to the appropriate definition, which presumably won't
@@ -2658,7 +2626,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
@@ -2690,6 +2657,8 @@ func_checks = [
['shm_open', {'dependencies': [rt_dep], 'define': false}],
['shm_unlink', {'dependencies': [rt_dep], 'define': false}],
['shmget', {'dependencies': [cygipc_dep], 'define': false}],
+ ['snprintf_l'],
+ ['strtod_l'],
['socket', {'dependencies': [socket_dep], 'define': false}],
['strchrnul'],
['strerror_r', {'dependencies': [thread_dep]}],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 0e9b108e667..c0586edab13 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -376,6 +376,9 @@
/* Define to 1 if you have the `setproctitle_fast' function. */
#undef HAVE_SETPROCTITLE_FAST
+/* Define to 1 if you have the `snprintf_l' function. */
+#undef HAVE_SNPRINTF_L
+
/* Define to 1 if the system has the type `socklen_t'. */
#undef HAVE_SOCKLEN_T
@@ -421,6 +424,9 @@
/* Define to 1 if you have the `strsignal' function. */
#undef HAVE_STRSIGNAL
+/* Define to 1 if you have the `strtod_l' function. */
+#undef HAVE_STRTOD_L
+
/* Define to 1 if the system has the type `struct option'. */
#undef HAVE_STRUCT_OPTION
@@ -556,9 +562,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
@@ -577,9 +580,6 @@
/* Define to the appropriate printf length modifier for 64-bit ints. */
#undef INT64_MODIFIER
-/* Define to 1 if `locale_t' requires <xlocale.h>. */
-#undef LOCALE_T_IN_XLOCALE
-
/* Define as the maximum alignment requirement of any C data type. */
#undef MAXIMUM_ALIGNOF
@@ -766,9 +766,6 @@
/* Define to 1 to build with ZSTD support. (--with-zstd) */
#undef USE_ZSTD
-/* Define to 1 if `wcstombs_l' requires <xlocale.h>. */
-#undef WCSTOMBS_L_IN_XLOCALE
-
/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
significant byte first (like Motorola and SPARC, unlike Intel). */
#if defined AC_APPLE_UNIVERSAL_BUILD
diff --git a/src/include/port.h b/src/include/port.h
index c7400052675..8e814700e7f 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -15,6 +15,11 @@
#include <ctype.h>
+#include <locale.h>
+#ifdef HAVE_XLOCALE_H
+#include <xlocale.h>
+#endif
+
/*
* Windows has enough specialized port stuff that we push most of it off
* into another file.
@@ -217,6 +222,43 @@ extern int pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2,
extern int pg_vprintf(const char *fmt, va_list args) pg_attribute_printf(1, 0);
extern int pg_printf(const char *fmt,...) pg_attribute_printf(1, 2);
+/* Thread-safe singleton constructor for process-lifetime "C" locale. */
+extern locale_t pg_get_c_locale(void);
+
+/*
+ * A couple of systems offer a pre-defined locale_t value for the "C" locale.
+ * Otherwise fall back to the above function. Provide a way to check that a
+ * "C" locale has been allocated at a convenient time for error reporting.
+ */
+#ifdef LC_C_LOCALE
+#define PG_C_LOCALE LC_C_LOCALE
+#define pg_ensure_c_locale() true
+#else
+#define PG_C_LOCALE pg_get_c_locale()
+#define pg_ensure_c_locale() (pg_get_c_locale() != (locale_t) 0)
+#endif
+
+#ifndef HAVE_STRTOD_L
+/*
+ * POSIX doesn't define this function, but we can implement it with thread-safe
+ * save-and-restore. Not all systems have uselocale(), but the ones that don't
+ * have strtod_l().
+ */
+static inline double
+strtod_l(const char *nptr, char **endptr, locale_t loc)
+{
+#ifdef WIN32
+ return _strtod_l(nptr, endptr, loc);
+#else
+ locale_t save = uselocale(loc);
+ double result = strtod(nptr, endptr);
+
+ uselocale(save);
+ return result;
+#endif
+}
+#endif
+
#ifndef WIN32
/*
* We add a pg_ prefix as a warning that the Windows implementations have the
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index f41d33975be..9651f3c3ab5 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -12,7 +12,7 @@
#ifndef _PG_LOCALE_
#define _PG_LOCALE_
-#if defined(LOCALE_T_IN_XLOCALE) || defined(WCSTOMBS_L_IN_XLOCALE)
+#ifdef HAVE_XLOCALE_H
#include <xlocale.h>
#endif
#ifdef USE_ICU
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 8afb1f0a26f..f4ba5b722eb 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,10 +10,6 @@
#include "ecpgtype.h"
#include "sqlca.h"
-#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale = (locale_t) 0;
-#endif
-
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
@@ -268,7 +264,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
const char **conn_keywords;
const char **conn_values;
- if (sqlca == NULL)
+ if (sqlca == NULL || !pg_ensure_c_locale())
{
ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
@@ -483,39 +479,6 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
/* add connection to our list */
pthread_mutex_lock(&connections_mutex);
- /*
- * ... but first, make certain we have created ecpg_clocale. Rely on
- * holding connections_mutex to ensure this is done by only one thread.
- */
-#ifdef HAVE_USELOCALE
- if (!ecpg_clocale)
- {
- ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (!ecpg_clocale)
- {
- pthread_mutex_unlock(&connections_mutex);
- ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
- ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
- if (host)
- ecpg_free(host);
- if (port)
- ecpg_free(port);
- if (options)
- ecpg_free(options);
- if (realname)
- ecpg_free(realname);
- if (dbname)
- ecpg_free(dbname);
- if (conn_keywords)
- ecpg_free(conn_keywords);
- if (conn_values)
- ecpg_free(conn_values);
- free(this);
- return false;
- }
- }
-#endif
-
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index fa562767585..856f4c9472d 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -466,7 +466,7 @@ ecpg_get_data(const PGresult *results, int act_tuple, int act_field, int lineno,
pval++;
if (!check_special_value(pval, &dres, &scan_length))
- dres = strtod(pval, &scan_length);
+ dres = strtod_l(pval, &scan_length, PG_C_LOCALE);
if (isarray && *scan_length == '"')
scan_length++;
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c4..855322e037f 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -475,46 +475,9 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
memset(&stmt, 0, sizeof stmt);
stmt.lineno = lineno;
- /* Make sure we do NOT honor the locale for numeric input */
- /* since the database gives the standard decimal point */
- /* (see comments in execute.c) */
-#ifdef HAVE_USELOCALE
-
- /*
- * To get here, the above PQnfields() test must have found nonzero
- * fields. One needs a connection to create such a descriptor. (EXEC
- * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
- * can't change the descriptor's PQnfields().) Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt.oldlocale = uselocale(ecpg_clocale);
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- setlocale(LC_NUMERIC, "C");
-#endif
-
/* desperate try to guess something sensible */
stmt.connection = ecpg_get_connection(NULL);
ecpg_store_result(ECPGresult, index, &stmt, &data_var);
-
-#ifdef HAVE_USELOCALE
- if (stmt.oldlocale != (locale_t) 0)
- uselocale(stmt.oldlocale);
-#else
- if (stmt.oldlocale)
- {
- setlocale(LC_NUMERIC, stmt.oldlocale);
- ecpg_free(stmt.oldlocale);
- }
-#ifdef HAVE__CONFIGTHREADLOCALE
- if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
-#endif
-#endif
}
else if (data_var.ind_type != ECPGt_NO_INDICATOR && data_var.ind_pointer != NULL)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 01b4309a710..40988d53575 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -13,9 +13,6 @@
#ifndef CHAR_BIT
#include <limits.h>
#endif
-#ifdef LOCALE_T_IN_XLOCALE
-#include <xlocale.h>
-#endif
enum COMPAT_MODE
{
@@ -59,10 +56,6 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
-#ifdef HAVE_USELOCALE
-extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
-#endif
-
/* structure to store one statement */
struct statement
{
@@ -76,14 +69,6 @@ struct statement
bool questionmarks;
struct variable *inlist;
struct variable *outlist;
-#ifdef HAVE_USELOCALE
- locale_t oldlocale;
-#else
- char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
- int oldthreadlocale;
-#endif
-#endif
int nparams;
char **paramvalues;
int *paramlengths;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 04d0b40c537..806d4779fa9 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,9 +101,6 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifndef HAVE_USELOCALE
- ecpg_free(stmt->oldlocale);
-#endif
ecpg_free(stmt);
}
@@ -1973,40 +1970,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
if (stmt == NULL)
return false;
- /*
- * Make sure we do NOT honor the locale for numeric input/output since the
- * database wants the standard decimal point. If available, use
- * uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
- */
-#ifdef HAVE_USELOCALE
-
- /*
- * Since ecpg_init() succeeded, we have a connection. Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt->oldlocale = uselocale(ecpg_clocale);
- if (stmt->oldlocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- if (stmt->oldlocale == NULL)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- setlocale(LC_NUMERIC, "C");
-#endif
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
@@ -2213,24 +2176,6 @@ ecpg_do_epilogue(struct statement *stmt)
if (stmt == NULL)
return;
-#ifdef HAVE_USELOCALE
- if (stmt->oldlocale != (locale_t) 0)
- uselocale(stmt->oldlocale);
-#else
- if (stmt->oldlocale)
- setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
-#endif
-#endif
-
free_statement(stmt);
}
diff --git a/src/interfaces/ecpg/pgtypeslib/dt_common.c b/src/interfaces/ecpg/pgtypeslib/dt_common.c
index ed08088bfe1..92459728bf4 100644
--- a/src/interfaces/ecpg/pgtypeslib/dt_common.c
+++ b/src/interfaces/ecpg/pgtypeslib/dt_common.c
@@ -1216,7 +1216,7 @@ DecodeNumber(int flen, char *str, int fmask,
return DecodeNumberField(flen, str, (fmask | DTK_DATE_M),
tmask, tm, fsec, is2digits);
- *fsec = strtod(cp, &cp);
+ *fsec = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
}
@@ -2028,7 +2028,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double frac;
- frac = strtod(cp, &cp);
+ frac = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
*fsec = frac * 1000000;
@@ -2052,7 +2052,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double time;
- time = strtod(cp, &cp);
+ time = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 936a6883816..155c6cc7770 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -60,7 +60,7 @@ ParseISO8601Number(const char *str, char **endptr, int *ipart, double *fpart)
if (!(isdigit((unsigned char) *str) || *str == '-' || *str == '.'))
return DTERR_BAD_FORMAT;
errno = 0;
- val = strtod(str, endptr);
+ val = strtod_l(str, endptr, PG_C_LOCALE);
/* did we not see anything that looks like a double? */
if (*endptr == str || errno != 0)
return DTERR_BAD_FORMAT;
@@ -455,7 +455,7 @@ DecodeInterval(char **field, int *ftype, int nf, /* int range, */
else if (*cp == '.')
{
errno = 0;
- fval = strtod(cp, &cp);
+ fval = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0' || errno != 0)
return DTERR_BAD_FORMAT;
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 35e7b92da40..49938543d03 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -1455,7 +1455,7 @@ numericvar_to_double(numeric *var, double *dp)
* strtod does not reset errno to 0 in case of success.
*/
errno = 0;
- val = strtod(tmp, &endptr);
+ val = strtod_l(tmp, &endptr, PG_C_LOCALE);
if (errno == ERANGE)
{
free(tmp);
diff --git a/src/port/Makefile b/src/port/Makefile
index db7c02117b0..03b55bd26b0 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -42,6 +42,7 @@ OBJS = \
bsearch_arg.o \
chklocale.o \
inet_net_ntop.o \
+ locale.o \
noblock.o \
path.o \
pg_bitutils.o \
diff --git a/src/port/locale.c b/src/port/locale.c
new file mode 100644
index 00000000000..a5b0648bb8b
--- /dev/null
+++ b/src/port/locale.c
@@ -0,0 +1,78 @@
+/*-------------------------------------------------------------------------
+ *
+ * locale.c
+ * Helper routines for thread-safe system locale usage.
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/locale.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#include <locale.h>
+#ifdef HAVE_XLOCALE_H
+#include <xlocale.h>
+#endif
+
+#ifndef WIN32
+#include <pthread.h>
+#else
+#include <synchapi.h>
+#endif
+
+static locale_t c_locale;
+
+#ifndef WIN32
+static void
+init_c_locale_once(void)
+{
+ c_locale = newlocale(LC_ALL, "C", NULL);
+}
+#else
+static BOOL
+init_c_locale_once(PINIT_ONCE once, PVOID parameter, PVOID *context)
+{
+ c_locale = _create_locale(LC_ALL, "C");
+ return true;
+}
+#endif
+
+locale_t
+pg_get_c_locale(void)
+{
+ /*
+ * Fast path if already initialized. This assumes that we can read a
+ * locale_t (in practice, a pointer) without tearing in a multi-threaded
+ * program.
+ */
+ if (c_locale != (locale_t) 0)
+ return c_locale;
+
+ /* Make a locale_t. It will live until process exit. */
+ {
+#ifndef WIN32
+ static pthread_once_t once = PTHREAD_ONCE_INIT;
+
+ pthread_once(&once, init_c_locale_once);
+#else
+ static INIT_ONCE once;
+ InitOnceExecuteOnce(&once, init_c_locale_once, NULL, NULL);
+#endif
+ }
+
+ /*
+ * It is remotely possible that the locale couldn't be allocated, due to
+ * lack of memory. In that case (locale_t) 0 will be returned. A caller
+ * can test for that condition with pg_ensure_c_locale() at a convenient
+ * time for error reporting.
+ */
+
+ return c_locale;
+}
diff --git a/src/port/meson.build b/src/port/meson.build
index ff54b7b53e9..e21961fd739 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -5,6 +5,7 @@ pgport_sources = [
'chklocale.c',
'inet_net_ntop.c',
'noblock.c',
+ 'locale.c',
'path.c',
'pg_bitutils.c',
'pg_strong_random.c',
diff --git a/src/port/snprintf.c b/src/port/snprintf.c
index 884f0262dd1..983d39a7729 100644
--- a/src/port/snprintf.c
+++ b/src/port/snprintf.c
@@ -109,6 +109,34 @@
#undef vprintf
#undef printf
+#if defined(FRONTEND)
+/*
+ * Frontend code might be multi-threaded. When calling the system snprintf()
+ * for floats, we have to use either the snprintf_l() variant, or
+ * save-and-restore the calling thread's locale using uselocale().
+ */
+#if defined(HAVE_SNPRINTF_L)
+/* At least macOS and the BSDs have the snprintf_l() extension. */
+#define USE_SNPRINTF_L
+#elif defined(WIN32)
+/* Windows has a version with a different name and argument order. */
+#define snprintf_l(str, size, loc, format, ...) _snprintf_l(str, size, format, loc, __VA_ARGS__)
+#define USE_SNPRINTF_L
+#else
+/* We have to do a thread-safe save-and-restore around snprintf(). */
+#define NEED_USE_LOCALE
+#endif
+#else
+/*
+ * Backend code doesn't need to worry about the locales, because LC_NUMERIC is
+ * set to "C" in main() and doesn't change. Plain old snprintf() is always OK.
+ *
+ * XXX If the backend were multithreaded, it would have to be 100% certain that
+ * no one is calling setlocale() concurrently, even transiently, to be able to
+ * keep this small optimization.
+ */
+#endif
+
/*
* Info about where the formatted output is going.
*
@@ -1184,6 +1212,9 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
* according to == but not according to memcmp.
*/
static const double dzero = 0.0;
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
if (adjust_sign((value < 0.0 ||
(value == 0.0 &&
@@ -1205,7 +1236,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '*';
fmt[3] = type;
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, prec, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, prec, value);
+#endif
}
else
{
@@ -1214,6 +1249,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '\0';
vallen = snprintf(convert, sizeof(convert), fmt, value);
}
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
goto fail;
@@ -1336,12 +1376,25 @@ pg_strfromd(char *str, size_t count, int precision, double value)
}
else
{
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
+
fmt[0] = '%';
fmt[1] = '.';
fmt[2] = '*';
fmt[3] = 'g';
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, precision, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, precision, value);
+#endif
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
{
target.failed = true;
--
2.46.0
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-08-13 23:17 Tristan Partin <tristan@partin.io>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 1 reply; 49+ messages in thread
From: Tristan Partin @ 2024-08-13 23:17 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers
Hey Thomas,
Thanks for picking this up. I think your patch looks really good. Are
you familiar with gcc's function poisoning?
#include <stdio.h>
#pragma GCC poison puts
int main(){
#pragma GCC bless begin puts
puts("a");
#pragma GCC bless end puts
}
I wonder if we could use function poisoning to our advantage. For
instance in ecpg, it looks like you got all of the strtod() invocations
and replaced them with strtod_l(). Here is a patch with an example of
what I'm talking about.
--
Tristan Partin
Neon (https://neon.tech)
Attachments:
v1-0001-Poison-strtod-in-ecpg.patch (text/x-patch, 2.5K) download
From a76a3f2577d098f7ce5a3ed1768aca4293aedfcc Mon Sep 17 00:00:00 2001
From: Tristan Partin <tristan@partin.io>
Date: Tue, 13 Aug 2024 18:06:20 -0500
Subject: [PATCH v1] Poison strtod in ecpg
ecpg exclusively uses strtod_l now.
Signed-off-by: Tristan Partin <tristan@partin.io>
---
src/interfaces/ecpg/ecpglib/data.c | 1 +
src/interfaces/ecpg/include/ecpg_poison.h | 6 ++++++
src/interfaces/ecpg/pgtypeslib/dt_common.c | 1 +
src/interfaces/ecpg/pgtypeslib/interval.c | 1 +
src/interfaces/ecpg/pgtypeslib/numeric.c | 1 +
5 files changed, 10 insertions(+)
create mode 100644 src/interfaces/ecpg/include/ecpg_poison.h
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index 856f4c9472d..13023fc21eb 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -6,6 +6,7 @@
#include <math.h>
#include "ecpgerrno.h"
+#include "ecpg_poison.h"
#include "ecpglib.h"
#include "ecpglib_extern.h"
#include "ecpgtype.h"
diff --git a/src/interfaces/ecpg/include/ecpg_poison.h b/src/interfaces/ecpg/include/ecpg_poison.h
new file mode 100644
index 00000000000..97ae9d7a143
--- /dev/null
+++ b/src/interfaces/ecpg/include/ecpg_poison.h
@@ -0,0 +1,6 @@
+#ifndef _ECPG_POISON_H
+#define _ECPG_POISON_H
+
+#pragma GCC poison strtod
+
+#endif /* !_ECPG_POISON_H */
diff --git a/src/interfaces/ecpg/pgtypeslib/dt_common.c b/src/interfaces/ecpg/pgtypeslib/dt_common.c
index 92459728bf4..2a9c5ec32c0 100644
--- a/src/interfaces/ecpg/pgtypeslib/dt_common.c
+++ b/src/interfaces/ecpg/pgtypeslib/dt_common.c
@@ -8,6 +8,7 @@
#include "common/string.h"
#include "dt.h"
+#include "ecpg_poison.h"
#include "pgtypes_timestamp.h"
#include "pgtypeslib_extern.h"
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 155c6cc7770..bfcc7233999 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -12,6 +12,7 @@
#include "common/string.h"
#include "dt.h"
+#include "ecpg_poison.h"
#include "pgtypes_error.h"
#include "pgtypes_interval.h"
#include "pgtypeslib_extern.h"
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 49938543d03..2623f362565 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -6,6 +6,7 @@
#include <float.h>
#include <limits.h>
+#include "ecpg_poison.h"
#include "pgtypes_error.h"
#include "pgtypes_numeric.h"
#include "pgtypeslib_extern.h"
--
Tristan Partin
https://tristan.partin.io
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-08-15 08:49 Thomas Munro <thomas.munro@gmail.com>
parent: Tristan Partin <tristan@partin.io>
0 siblings, 0 replies; 49+ messages in thread
From: Thomas Munro @ 2024-08-15 08:49 UTC (permalink / raw)
To: Tristan Partin <tristan@partin.io>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Aug 14, 2024 at 11:17 AM Tristan Partin <tristan@partin.io> wrote:
> Thanks for picking this up. I think your patch looks really good.
Thanks for looking!
> Are
> you familiar with gcc's function poisoning?
>
> #include <stdio.h>
> #pragma GCC poison puts
>
> int main(){
> #pragma GCC bless begin puts
> puts("a");
> #pragma GCC bless end puts
> }
>
> I wonder if we could use function poisoning to our advantage. For
> instance in ecpg, it looks like you got all of the strtod() invocations
> and replaced them with strtod_l(). Here is a patch with an example of
> what I'm talking about.
Thanks, this looks very useful.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-08-28 18:50 Peter Eisentraut <peter@eisentraut.org>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 3 replies; 49+ messages in thread
From: Peter Eisentraut @ 2024-08-28 18:50 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 11.08.24 00:11, Thomas Munro wrote:
> v4 adds error handling, in case newlocale("C") fails. I created CF
> entry #5166 for this.
I took a look at this. It was quite a complicated discussion that led
to this, but I agree with the solution that was arrived at.
I suggest that the simplification of the xlocale.h configure tests could
be committed separately. This would also be useful independent of this,
and it's a sizeable chunk of this patch.
Also, you're removing the configure test for _configthreadlocale().
Presumably because you're removing all the uses. But wouldn't we need
that back later in the backend maybe? Or is that test even relevant
anymore, that is, are there Windows versions that don't have it?
Adding global includes to port.h doesn't seem great. That's not a place
one would normally look. We already include <locale.h> in c.h anyway,
so it would probably be even better overall if you just added a
conditional #include <xlocale.h> to c.h as well.
For Windows, we already have things like
#define strcoll_l _strcoll_l
in src/include/port/win32_port.h, so it would seem more sensible to add
strtod_l to that list, instead of in port.h.
The error handling with pg_ensure_c_locale(), that's the sort of thing
I'm afraid will be hard to test or even know how it will behave. And it
creates this weird coupling between pgtypeslib and ecpglib that you
mentioned earlier. And if there are other users of PG_C_LOCALE in the
future, there will be similar questions about the proper initialization
and error handling sequence.
I would consider instead making a local static variable in each function
that needs this. For example, numericvar_to_double() might do
{
static locale_t c_locale;
if (!c_locale)
{
c_locale = pg_get_c_locale();
if (!c_locale)
return -1; /* local error reporting convention */
}
...
}
This is a bit more code in total, but then you only initialize what you
need and you can handle errors locally.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-10-01 12:04 Peter Eisentraut <peter@eisentraut.org>
parent: Peter Eisentraut <peter@eisentraut.org>
2 siblings, 1 reply; 49+ messages in thread
From: Peter Eisentraut @ 2024-10-01 12:04 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 28.08.24 20:50, Peter Eisentraut wrote:
> I suggest that the simplification of the xlocale.h configure tests could
> be committed separately. This would also be useful independent of this,
> and it's a sizeable chunk of this patch.
To keep this moving along a bit, I have extracted this part and
committed it separately. I had to make a few small tweaks, e.g., there
was no check for xlocale.h in configure.ac, and the old
xlocale.h-including stanza could be removed from chklocale.h. Let's see
how this goes.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-14 01:54 Thomas Munro <thomas.munro@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 0 replies; 49+ messages in thread
From: Thomas Munro @ 2024-11-14 01:54 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Oct 2, 2024 at 1:04 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> On 28.08.24 20:50, Peter Eisentraut wrote:
> > I suggest that the simplification of the xlocale.h configure tests could
> > be committed separately. This would also be useful independent of this,
> > and it's a sizeable chunk of this patch.
>
> To keep this moving along a bit, I have extracted this part and
> committed it separately. I had to make a few small tweaks, e.g., there
> was no check for xlocale.h in configure.ac, and the old
> xlocale.h-including stanza could be removed from chklocale.h. Let's see
> how this goes.
Thanks, Peter. That seemed to work fine, and we shouldn't have to
think about xlocale.h as a special case too much again.
I am back to working on all these cleanup patches. Here is a rebase
of this one, with a few minor style adjustments and a new attempt to
explain the overall approach in the commit message.
Attachments:
v5-0001-Tidy-up-locale-thread-safety-in-ECPG-library.patch (text/x-patch, 24.0K) download
From a5eebfdd0b946f64310f60fab68e67b84bb474ae Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 10 Aug 2024 11:01:21 +1200
Subject: [PATCH v5] Tidy up locale thread safety in ECPG library.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Remove setlocale() and _configthreadlocal() as fallback strategy on
systems that don't have uselocale(), where ECPG tries to control
LC_NUMERIC formatting on input and output of floating point numbers. It
was probably broken on some systems (NetBSD), and the code was also
quite messy and complicated, with obsolete configure tests (Windows).
It was also arguably broken, or at least had unstated environmental
requirements, if pgtypeslib code was called directly.
Instead, introduce PG_C_LOCALE to refer to the "C" locale as a locale_t
value. It maps to the special constant LC_C_LOCALE when defined by libc
(macOS, NetBSD), or otherwise uses a process-lifetime locale_t that is
allocated on first use, just as ECPG previously did itself. The new
replacement might be more widely useful. Then change the float parsing
and printing code to pass that to _l() functions where appropriate.
Unfortunately the portability of those functions is a bit complicated.
First, many obvious and useful _l() functions are missing from POSIX,
though most standard libraries define some of them anyway. Second,
although the thread-safe save/restore technique can be used to replace
the missing ones, Windows and NetBSD refused to implement standard
uselocale(). They might have a point: "wide scope" uselocale() is hard
to combine with other code and error-prone, especially in library code.
Luckily they have the _l() functions we want so far anyway. So we have
to be prepared for both ways of doing things:
1. In ECPG, use strtod_l() for parsing, and supply a port.h replacement
using uselocale() over a limited scope if missing.
2. Inside our own snprintf.c, use three different approaches to format
floats. For frontend code, call libc's snprintf_l(), or wrap libc's
snprintf() in uselocale() if it's missing. For backend code, snprintf.c
can keep assuming that the global locale's LC_NUMERIC is "C" and call
libc's snprintf() without change, for now.
(It might eventually be possible to call our in-tree Ryū routines to
display floats in snprintf.c, given the C-locale-always remit of our
in-tree snprintf(), but this patch doesn't risk changing anything that
complicated.)
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Tristan Partin <tristan@partin.io>
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
configure | 13 +---
configure.ac | 3 +-
meson.build | 3 +-
src/include/pg_config.h.in | 9 ++-
src/include/port.h | 36 +++++++++
src/interfaces/ecpg/ecpglib/connect.c | 39 +---------
src/interfaces/ecpg/ecpglib/data.c | 2 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 37 ---------
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 12 ---
src/interfaces/ecpg/ecpglib/execute.c | 55 --------------
src/interfaces/ecpg/pgtypeslib/dt_common.c | 6 +-
src/interfaces/ecpg/pgtypeslib/interval.c | 4 +-
src/interfaces/ecpg/pgtypeslib/numeric.c | 2 +-
src/port/Makefile | 1 +
src/port/locale.c | 80 ++++++++++++++++++++
src/port/meson.build | 1 +
src/port/snprintf.c | 55 ++++++++++++++
17 files changed, 192 insertions(+), 166 deletions(-)
create mode 100644 src/port/locale.c
diff --git a/configure b/configure
index f58eae1baa8..6a133c09d3b 100755
--- a/configure
+++ b/configure
@@ -15039,7 +15039,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strtod_l strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -15812,17 +15812,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index 82c5009e3e8..e73e0985862 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1727,6 +1727,8 @@ AC_CHECK_FUNCS(m4_normalize([
pthread_is_threaded_np
setproctitle
setproctitle_fast
+ snprintf_l
+ strtod_l
strchrnul
strsignal
syncfs
@@ -1816,7 +1818,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index 5b0510cef78..dc557a317df 100644
--- a/meson.build
+++ b/meson.build
@@ -2614,7 +2614,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
@@ -2646,6 +2645,8 @@ func_checks = [
['shm_open', {'dependencies': [rt_dep], 'define': false}],
['shm_unlink', {'dependencies': [rt_dep], 'define': false}],
['shmget', {'dependencies': [cygipc_dep], 'define': false}],
+ ['snprintf_l'],
+ ['strtod_l'],
['socket', {'dependencies': [socket_dep], 'define': false}],
['strchrnul'],
['strerror_r', {'dependencies': [thread_dep]}],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index cdd9a6e9355..10c05775ff2 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -349,6 +349,9 @@
/* Define to 1 if you have the `setproctitle_fast' function. */
#undef HAVE_SETPROCTITLE_FAST
+/* Define to 1 if you have the `snprintf_l' function. */
+#undef HAVE_SNPRINTF_L
+
/* Define to 1 if the system has the type `socklen_t'. */
#undef HAVE_SOCKLEN_T
@@ -397,6 +400,9 @@
/* Define to 1 if you have the `strsignal' function. */
#undef HAVE_STRSIGNAL
+/* Define to 1 if you have the `strtod_l' function. */
+#undef HAVE_STRTOD_L
+
/* Define to 1 if the system has the type `struct option'. */
#undef HAVE_STRUCT_OPTION
@@ -529,9 +535,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
diff --git a/src/include/port.h b/src/include/port.h
index ba9ab0d34f4..f8344a1eb3d 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -217,6 +217,42 @@ extern int pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2,
extern int pg_vprintf(const char *fmt, va_list args) pg_attribute_printf(1, 0);
extern int pg_printf(const char *fmt,...) pg_attribute_printf(1, 2);
+/*
+ * A couple of systems offer a fast constant locale_t value representing the
+ * "C" locale. We use that if possible, but fall back to creating a singleton
+ * object otherwise. To check that it is available, call pg_ensure_c_locale()
+ * and assume out of memory if it returns false.
+ */
+#ifdef LC_C_LOCALE
+#define PG_C_LOCALE LC_C_LOCALE
+#define pg_ensure_c_locale() true
+#else
+extern locale_t pg_get_c_locale(void);
+#define PG_C_LOCALE pg_get_c_locale()
+#define pg_ensure_c_locale() (PG_C_LOCALE != 0)
+#endif
+
+#ifndef HAVE_STRTOD_L
+/*
+ * POSIX doesn't define this function, but we can implement it with thread-safe
+ * save-and-restore. Not all systems have uselocale(), but the ones that don't
+ * have strtod_l().
+ */
+static inline double
+strtod_l(const char *nptr, char **endptr, locale_t loc)
+{
+#ifdef WIN32
+ return _strtod_l(nptr, endptr, loc);
+#else
+ locale_t save = uselocale(loc);
+ double result = strtod(nptr, endptr);
+
+ uselocale(save);
+ return result;
+#endif
+}
+#endif
+
#ifndef WIN32
/*
* We add a pg_ prefix as a warning that the Windows implementations have the
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index b24b310ce59..78e501a2656 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,10 +10,6 @@
#include "ecpgtype.h"
#include "sqlca.h"
-#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale = (locale_t) 0;
-#endif
-
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
@@ -268,7 +264,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
const char **conn_keywords;
const char **conn_values;
- if (sqlca == NULL)
+ if (sqlca == NULL || !pg_ensure_c_locale())
{
ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
@@ -483,39 +479,6 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
/* add connection to our list */
pthread_mutex_lock(&connections_mutex);
- /*
- * ... but first, make certain we have created ecpg_clocale. Rely on
- * holding connections_mutex to ensure this is done by only one thread.
- */
-#ifdef HAVE_USELOCALE
- if (!ecpg_clocale)
- {
- ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (!ecpg_clocale)
- {
- pthread_mutex_unlock(&connections_mutex);
- ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
- ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
- if (host)
- ecpg_free(host);
- if (port)
- ecpg_free(port);
- if (options)
- ecpg_free(options);
- if (realname)
- ecpg_free(realname);
- if (dbname)
- ecpg_free(dbname);
- if (conn_keywords)
- ecpg_free(conn_keywords);
- if (conn_values)
- ecpg_free(conn_values);
- free(this);
- return false;
- }
- }
-#endif
-
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index fa562767585..856f4c9472d 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -466,7 +466,7 @@ ecpg_get_data(const PGresult *results, int act_tuple, int act_field, int lineno,
pval++;
if (!check_special_value(pval, &dres, &scan_length))
- dres = strtod(pval, &scan_length);
+ dres = strtod_l(pval, &scan_length, PG_C_LOCALE);
if (isarray && *scan_length == '"')
scan_length++;
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c4..855322e037f 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -475,46 +475,9 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
memset(&stmt, 0, sizeof stmt);
stmt.lineno = lineno;
- /* Make sure we do NOT honor the locale for numeric input */
- /* since the database gives the standard decimal point */
- /* (see comments in execute.c) */
-#ifdef HAVE_USELOCALE
-
- /*
- * To get here, the above PQnfields() test must have found nonzero
- * fields. One needs a connection to create such a descriptor. (EXEC
- * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
- * can't change the descriptor's PQnfields().) Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt.oldlocale = uselocale(ecpg_clocale);
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- setlocale(LC_NUMERIC, "C");
-#endif
-
/* desperate try to guess something sensible */
stmt.connection = ecpg_get_connection(NULL);
ecpg_store_result(ECPGresult, index, &stmt, &data_var);
-
-#ifdef HAVE_USELOCALE
- if (stmt.oldlocale != (locale_t) 0)
- uselocale(stmt.oldlocale);
-#else
- if (stmt.oldlocale)
- {
- setlocale(LC_NUMERIC, stmt.oldlocale);
- ecpg_free(stmt.oldlocale);
- }
-#ifdef HAVE__CONFIGTHREADLOCALE
- if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
-#endif
-#endif
}
else if (data_var.ind_type != ECPGt_NO_INDICATOR && data_var.ind_pointer != NULL)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index bad3cd9920b..40988d53575 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -56,10 +56,6 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
-#ifdef HAVE_USELOCALE
-extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
-#endif
-
/* structure to store one statement */
struct statement
{
@@ -73,14 +69,6 @@ struct statement
bool questionmarks;
struct variable *inlist;
struct variable *outlist;
-#ifdef HAVE_USELOCALE
- locale_t oldlocale;
-#else
- char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
- int oldthreadlocale;
-#endif
-#endif
int nparams;
char **paramvalues;
int *paramlengths;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index c578c21cf66..ec939d0f63a 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,9 +101,6 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifndef HAVE_USELOCALE
- ecpg_free(stmt->oldlocale);
-#endif
ecpg_free(stmt);
}
@@ -1973,40 +1970,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
if (stmt == NULL)
return false;
- /*
- * Make sure we do NOT honor the locale for numeric input/output since the
- * database wants the standard decimal point. If available, use
- * uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
- */
-#ifdef HAVE_USELOCALE
-
- /*
- * Since ecpg_init() succeeded, we have a connection. Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt->oldlocale = uselocale(ecpg_clocale);
- if (stmt->oldlocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- if (stmt->oldlocale == NULL)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- setlocale(LC_NUMERIC, "C");
-#endif
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
@@ -2213,24 +2176,6 @@ ecpg_do_epilogue(struct statement *stmt)
if (stmt == NULL)
return;
-#ifdef HAVE_USELOCALE
- if (stmt->oldlocale != (locale_t) 0)
- uselocale(stmt->oldlocale);
-#else
- if (stmt->oldlocale)
- setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
-#endif
-#endif
-
free_statement(stmt);
}
diff --git a/src/interfaces/ecpg/pgtypeslib/dt_common.c b/src/interfaces/ecpg/pgtypeslib/dt_common.c
index c4119ab7932..f8349b66ce9 100644
--- a/src/interfaces/ecpg/pgtypeslib/dt_common.c
+++ b/src/interfaces/ecpg/pgtypeslib/dt_common.c
@@ -1218,7 +1218,7 @@ DecodeNumber(int flen, char *str, int fmask,
return DecodeNumberField(flen, str, (fmask | DTK_DATE_M),
tmask, tm, fsec, is2digits);
- *fsec = strtod(cp, &cp);
+ *fsec = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
}
@@ -2030,7 +2030,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double frac;
- frac = strtod(cp, &cp);
+ frac = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
*fsec = frac * 1000000;
@@ -2054,7 +2054,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double time;
- time = strtod(cp, &cp);
+ time = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 936a6883816..155c6cc7770 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -60,7 +60,7 @@ ParseISO8601Number(const char *str, char **endptr, int *ipart, double *fpart)
if (!(isdigit((unsigned char) *str) || *str == '-' || *str == '.'))
return DTERR_BAD_FORMAT;
errno = 0;
- val = strtod(str, endptr);
+ val = strtod_l(str, endptr, PG_C_LOCALE);
/* did we not see anything that looks like a double? */
if (*endptr == str || errno != 0)
return DTERR_BAD_FORMAT;
@@ -455,7 +455,7 @@ DecodeInterval(char **field, int *ftype, int nf, /* int range, */
else if (*cp == '.')
{
errno = 0;
- fval = strtod(cp, &cp);
+ fval = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0' || errno != 0)
return DTERR_BAD_FORMAT;
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 35e7b92da40..49938543d03 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -1455,7 +1455,7 @@ numericvar_to_double(numeric *var, double *dp)
* strtod does not reset errno to 0 in case of success.
*/
errno = 0;
- val = strtod(tmp, &endptr);
+ val = strtod_l(tmp, &endptr, PG_C_LOCALE);
if (errno == ERANGE)
{
free(tmp);
diff --git a/src/port/Makefile b/src/port/Makefile
index 366c814bd92..70d63963611 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -41,6 +41,7 @@ OBJS = \
bsearch_arg.o \
chklocale.o \
inet_net_ntop.o \
+ locale.o \
noblock.o \
path.o \
pg_bitutils.o \
diff --git a/src/port/locale.c b/src/port/locale.c
new file mode 100644
index 00000000000..387fb4a107a
--- /dev/null
+++ b/src/port/locale.c
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * locale.c
+ * Helper routines for thread-safe system locale usage.
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/locale.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#ifndef WIN32
+#include <pthread.h>
+#else
+#include <synchapi.h>
+#endif
+
+/* A process-lifetime singleton, allocated on first need. */
+static locale_t c_locale;
+
+#ifndef WIN32
+static void
+init_c_locale_once(void)
+{
+ c_locale = newlocale(LC_ALL, "C", NULL);
+}
+#else
+static BOOL
+init_c_locale_once(PINIT_ONCE once, PVOID parameter, PVOID *context)
+{
+ c_locale = _create_locale(LC_ALL, "C");
+ return true;
+}
+#endif
+
+/*
+ * Access a process-lifetime singleton locale_t object. Use the macro
+ * PG_C_LOCALE instead of calling this directly, as it can skip the function
+ * call on some systems.
+ */
+locale_t
+pg_get_c_locale(void)
+{
+ /*
+ * Fast path if already initialized. This assumes that we can read a
+ * locale_t (in practice, a pointer) without tearing in a multi-threaded
+ * program.
+ */
+ if (c_locale != (locale_t) 0)
+ return c_locale;
+
+ /* Make a locale_t. It will live until process exit. */
+ {
+#ifndef WIN32
+ static pthread_once_t once = PTHREAD_ONCE_INIT;
+
+ pthread_once(&once, init_c_locale_once);
+#else
+ static INIT_ONCE once;
+ InitOnceExecuteOnce(&once, init_c_locale_once, NULL, NULL);
+#endif
+ }
+
+ /*
+ * It's possible that the allocation of the locale failed due to low
+ * memory, and then (locale_t) 0 will be returned. Users of PG_C_LOCALE
+ * should defend against that by checking pg_ensure_c_locale() at a
+ * convenient time, so that they can treat it as a simple constant after
+ * that.
+ */
+
+ return c_locale;
+}
diff --git a/src/port/meson.build b/src/port/meson.build
index 83a06325209..93f6158c307 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -5,6 +5,7 @@ pgport_sources = [
'chklocale.c',
'inet_net_ntop.c',
'noblock.c',
+ 'locale.c',
'path.c',
'pg_bitutils.c',
'pg_popcount_avx512.c',
diff --git a/src/port/snprintf.c b/src/port/snprintf.c
index 884f0262dd1..3cc49f01ea6 100644
--- a/src/port/snprintf.c
+++ b/src/port/snprintf.c
@@ -109,6 +109,36 @@
#undef vprintf
#undef printf
+#if defined(FRONTEND)
+/*
+ * Frontend code might be multi-threaded. When calling the system snprintf()
+ * for floats, we have to use either the non-standard snprintf_l() variant, or
+ * save-and-restore the calling thread's locale using uselocale(), depending on
+ * availability.
+ */
+#if defined(HAVE_SNPRINTF_L)
+/* At least macOS and the BSDs have the snprintf_l() extension. */
+#define USE_SNPRINTF_L
+#elif defined(WIN32)
+/* Windows has a version with a different name and argument order. */
+#define snprintf_l(str, size, loc, format, ...) _snprintf_l(str, size, format, loc, __VA_ARGS__)
+#define USE_SNPRINTF_L
+#else
+/* We have to do a thread-safe save-and-restore around snprintf(). */
+#define NEED_USE_LOCALE
+#endif
+#else
+/*
+ * Backend code doesn't need to worry about locales here, because LC_NUMERIC is
+ * set to "C" in main() and doesn't change. Plain old snprintf() is always OK
+ * without uselocale().
+ *
+ * XXX If the backend were multithreaded, we would have to be 100% certain that
+ * no one is calling setlocale() concurrently, even transiently, to be able to
+ * keep this small optimization.
+ */
+#endif
+
/*
* Info about where the formatted output is going.
*
@@ -1184,6 +1214,9 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
* according to == but not according to memcmp.
*/
static const double dzero = 0.0;
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
if (adjust_sign((value < 0.0 ||
(value == 0.0 &&
@@ -1205,7 +1238,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '*';
fmt[3] = type;
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, prec, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, prec, value);
+#endif
}
else
{
@@ -1214,6 +1251,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '\0';
vallen = snprintf(convert, sizeof(convert), fmt, value);
}
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
goto fail;
@@ -1336,12 +1378,25 @@ pg_strfromd(char *str, size_t count, int precision, double value)
}
else
{
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
+
fmt[0] = '%';
fmt[1] = '.';
fmt[2] = '*';
fmt[3] = 'g';
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, precision, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, precision, value);
+#endif
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
{
target.failed = true;
--
2.47.0
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-14 07:48 Thomas Munro <thomas.munro@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
2 siblings, 2 replies; 49+ messages in thread
From: Thomas Munro @ 2024-11-14 07:48 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Aug 29, 2024 at 6:50 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> I took a look at this. It was quite a complicated discussion that led
> to this, but I agree with the solution that was arrived at.
Sorry I missed/forgot this prior feedback when posting earlier today.
Responses to the points you raised here:
> Also, you're removing the configure test for _configthreadlocale().
> Presumably because you're removing all the uses. But wouldn't we need
> that back later in the backend maybe? Or is that test even relevant
> anymore, that is, are there Windows versions that don't have it?
Yeah, this removes the uses. As it happens, in the thread about
localeconv() I am also proposing to add a new user of it[1], but even
then I don't think we'd need the configure test, because all relevant
systems have it. It's in ucrt, the modern C runtime that replaced the
old msvcrt, as used by Visual Studio 2015+ and Windows 10+, and also
by MinGW in recent years. I think it was also in some versions of
msvcrt, but MinGW had issues with that apparently.
It's pretty confusing, so I went looking for proof and clues about the
history. Here's a thread about 2019 animal jacana lacking the
function:
https://www.postgresql.org/message-id/flat/31420.1547783697%40sss.pgh.pa.us
Once jacana was upgraded, it started finding the function, but next
failed with -1 at runtime, cf 2cf91ccb. Grovelling through
MinGW-w64's github, it looks like they added a dummy function that
fails with -1 in 2016:
https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-crt/misc/_configthreadlocale.c#L11
They only seem to interpose the dummy function when building against msvcrt.
https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-crt/Makefile.am#L298
The default and recommended runtime library as of a few years ago is ucrt:
https://www.msys2.org/docs/environments/#msvcrt-vs-ucrt
The three MinGW environments we test today are using ucrt, and
configure detects the symbol on all. Namely: fairwren
(msys2/mingw64), the CI mingw64 task and the mingw cross-build that
runs on Linux in the CI CompilerWarnings task. As far as I know these
are the reasons for, and mechanism by which, we keep MinGW support
working. We have no policy requiring arbitrary old MinGW systems
work, and we wouldn't know anyway.
I was paranoid that it might still be a dummy function, so I tried
running a simple program on our CI with the MinGW compiler to confirm
it's reaching the real library:
int
main()
{
printf("hello world\n");
printf("got %d\n", _configthreadlocale(_ENABLE_PER_THREAD_LOCALE));
printf("got %d\n", _configthreadlocale(_DISABLE_PER_THREAD_LOCALE));
}
[06:16:39.580] c:\cirrus>call C:\msys64\usr\bin\bash.exe -l -c "gcc
doit.c -o c:/doit"
...
[06:16:41.474] c:\cirrus>call c:\doit.exe
[06:16:41.482] hello world
[06:16:41.482] got 2
[06:16:41.482] got 1
It's not clear to me whether or when we might have made any changes
already that prevented PostgreSQL from working with msvcrt, of course,
but I'm pretty sure if someone proposed that we *should* support it,
we'd reject the proposal. That'd effectively be a different, obsolete
one corresponding to Windows versions we've dropped, which wouldn't
make any sense.
Other mentions of msvcrt in win32env.c and pg_locale.c may also be
effectively obsolete now too.
> For Windows, we already have things like
>
> #define strcoll_l _strcoll_l
>
> in src/include/port/win32_port.h, so it would seem more sensible to add
> strtod_l to that list, instead of in port.h.
OK, I moved that line into that list. I left the implementation for
Unixen (at a guess: probably just Solaris?) there.
> The error handling with pg_ensure_c_locale(), that's the sort of thing
> I'm afraid will be hard to test or even know how it will behave. And it
> creates this weird coupling between pgtypeslib and ecpglib that you
> mentioned earlier. And if there are other users of PG_C_LOCALE in the
> future, there will be similar questions about the proper initialization
> and error handling sequence.
Ack. The coupling does become at least less weird (currently it must
be capable of giving the wrong answers reliably if called directly I
think, no?) and weaker, but I acknowledge that it's still non-ideal
that out-of-memory would be handled nicely only if you used ecpg
first, and subtle misbehaviour would ensure otherwise, and future
users face the same sort of problem unless they design in a reasonable
initialisation place that could check pg_ensure_c_locale() nicely.
Classic retro-fitting problem, though.
> I would consider instead making a local static variable in each function
> that needs this. For example, numericvar_to_double() might do
>
> {
> static locale_t c_locale;
>
> if (!c_locale)
> {
> c_locale = pg_get_c_locale();
> if (!c_locale)
> return -1; /* local error reporting convention */
> }
>
> ...
> }
>
> This is a bit more code in total, but then you only initialize what you
> need and you can handle errors locally.
Hmm. Seems like quite a lot of duplication, and also assumes that we
can actually unwind and handle the error in a reasonable way at all
callsites. Let me think about that some more...
[1] https://www.postgresql.org/message-id/flat/CA+hUKGJqVe0+Pv9dvC9dSums_PXxGo9SWcxYAMBguWJUGbWz-A@mail....
Attachments:
v6-0001-Tidy-up-locale-thread-safety-in-ECPG-library.patch (text/x-patch, 24.4K) download
From 2965979a7180265a8df91adb45c1ce32f05b190f Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 10 Aug 2024 11:01:21 +1200
Subject: [PATCH v6] Tidy up locale thread safety in ECPG library.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Remove setlocale() and _configthreadlocal() as fallback strategy on
systems that don't have uselocale(), where ECPG tries to control
LC_NUMERIC formatting on input and output of floating point numbers. It
was probably broken on some systems (NetBSD), and the code was also
quite messy and complicated, with obsolete configure tests (Windows).
It was also arguably broken, or at least had unstated environmental
requirements, if pgtypeslib code was called directly.
Instead, introduce PG_C_LOCALE to refer to the "C" locale as a locale_t
value. It maps to the special constant LC_C_LOCALE when defined by libc
(macOS, NetBSD), or otherwise uses a process-lifetime locale_t that is
allocated on first use, just as ECPG previously did itself. The new
replacement might be more widely useful. Then change the float parsing
and printing code to pass that to _l() functions where appropriate.
Unfortunately the portability of those functions is a bit complicated.
First, many obvious and useful _l() functions are missing from POSIX,
though most standard libraries define some of them anyway. Second,
although the thread-safe save/restore technique can be used to replace
the missing ones, Windows and NetBSD refused to implement standard
uselocale(). They might have a point: "wide scope" uselocale() is hard
to combine with other code and error-prone, especially in library code.
Luckily they have the _l() functions we want so far anyway. So we have
to be prepared for both ways of doing things:
1. In ECPG, use strtod_l() for parsing, and supply a port.h replacement
using uselocale() over a limited scope if missing.
2. Inside our own snprintf.c, use three different approaches to format
floats. For frontend code, call libc's snprintf_l(), or wrap libc's
snprintf() in uselocale() if it's missing. For backend code, snprintf.c
can keep assuming that the global locale's LC_NUMERIC is "C" and call
libc's snprintf() without change, for now.
(It might eventually be possible to call our in-tree Ryū routines to
display floats in snprintf.c, given the C-locale-always remit of our
in-tree snprintf(), but this patch doesn't risk changing anything that
complicated.)
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Tristan Partin <tristan@partin.io>
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
configure | 13 +---
configure.ac | 3 +-
meson.build | 3 +-
src/include/pg_config.h.in | 9 ++-
src/include/port.h | 31 ++++++++
src/include/port/win32_port.h | 1 +
src/interfaces/ecpg/ecpglib/connect.c | 39 +---------
src/interfaces/ecpg/ecpglib/data.c | 2 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 37 ---------
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 12 ---
src/interfaces/ecpg/ecpglib/execute.c | 55 --------------
src/interfaces/ecpg/pgtypeslib/dt_common.c | 6 +-
src/interfaces/ecpg/pgtypeslib/interval.c | 4 +-
src/interfaces/ecpg/pgtypeslib/numeric.c | 2 +-
src/port/Makefile | 1 +
src/port/locale.c | 80 ++++++++++++++++++++
src/port/meson.build | 1 +
src/port/snprintf.c | 55 ++++++++++++++
18 files changed, 188 insertions(+), 166 deletions(-)
create mode 100644 src/port/locale.c
diff --git a/configure b/configure
index f58eae1baa8..6a133c09d3b 100755
--- a/configure
+++ b/configure
@@ -15039,7 +15039,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strtod_l strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -15812,17 +15812,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index 82c5009e3e8..e73e0985862 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1727,6 +1727,8 @@ AC_CHECK_FUNCS(m4_normalize([
pthread_is_threaded_np
setproctitle
setproctitle_fast
+ snprintf_l
+ strtod_l
strchrnul
strsignal
syncfs
@@ -1816,7 +1818,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index 5b0510cef78..dc557a317df 100644
--- a/meson.build
+++ b/meson.build
@@ -2614,7 +2614,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
@@ -2646,6 +2645,8 @@ func_checks = [
['shm_open', {'dependencies': [rt_dep], 'define': false}],
['shm_unlink', {'dependencies': [rt_dep], 'define': false}],
['shmget', {'dependencies': [cygipc_dep], 'define': false}],
+ ['snprintf_l'],
+ ['strtod_l'],
['socket', {'dependencies': [socket_dep], 'define': false}],
['strchrnul'],
['strerror_r', {'dependencies': [thread_dep]}],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index cdd9a6e9355..10c05775ff2 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -349,6 +349,9 @@
/* Define to 1 if you have the `setproctitle_fast' function. */
#undef HAVE_SETPROCTITLE_FAST
+/* Define to 1 if you have the `snprintf_l' function. */
+#undef HAVE_SNPRINTF_L
+
/* Define to 1 if the system has the type `socklen_t'. */
#undef HAVE_SOCKLEN_T
@@ -397,6 +400,9 @@
/* Define to 1 if you have the `strsignal' function. */
#undef HAVE_STRSIGNAL
+/* Define to 1 if you have the `strtod_l' function. */
+#undef HAVE_STRTOD_L
+
/* Define to 1 if the system has the type `struct option'. */
#undef HAVE_STRUCT_OPTION
@@ -529,9 +535,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
diff --git a/src/include/port.h b/src/include/port.h
index ba9ab0d34f4..48292cb28f8 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -217,6 +217,37 @@ extern int pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2,
extern int pg_vprintf(const char *fmt, va_list args) pg_attribute_printf(1, 0);
extern int pg_printf(const char *fmt,...) pg_attribute_printf(1, 2);
+/*
+ * A couple of systems offer a fast constant locale_t value representing the
+ * "C" locale. We use that if possible, but fall back to creating a singleton
+ * object otherwise. To check that it is available, call pg_ensure_c_locale()
+ * and assume out of memory if it returns false.
+ */
+#ifdef LC_C_LOCALE
+#define PG_C_LOCALE LC_C_LOCALE
+#define pg_ensure_c_locale() true
+#else
+extern locale_t pg_get_c_locale(void);
+#define PG_C_LOCALE pg_get_c_locale()
+#define pg_ensure_c_locale() (PG_C_LOCALE != 0)
+#endif
+
+#if !defined(HAVE_STRTOD_L) && !defined(WIN32)
+/*
+ * POSIX doesn't define this function, but we can implement it with thread-safe
+ * save-and-restore.
+ */
+static inline double
+strtod_l(const char *nptr, char **endptr, locale_t loc)
+{
+ locale_t save = uselocale(loc);
+ double result = strtod(nptr, endptr);
+
+ uselocale(save);
+ return result;
+}
+#endif
+
#ifndef WIN32
/*
* We add a pg_ prefix as a warning that the Windows implementations have the
diff --git a/src/include/port/win32_port.h b/src/include/port/win32_port.h
index 7789e0431aa..8dbde7da954 100644
--- a/src/include/port/win32_port.h
+++ b/src/include/port/win32_port.h
@@ -463,6 +463,7 @@ extern int _pglstat64(const char *name, struct stat *buf);
#define isspace_l _isspace_l
#define iswspace_l _iswspace_l
#define strcoll_l _strcoll_l
+#define strtod_l _strtod_l
#define strxfrm_l _strxfrm_l
#define wcscoll_l _wcscoll_l
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index b24b310ce59..78e501a2656 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,10 +10,6 @@
#include "ecpgtype.h"
#include "sqlca.h"
-#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale = (locale_t) 0;
-#endif
-
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
@@ -268,7 +264,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
const char **conn_keywords;
const char **conn_values;
- if (sqlca == NULL)
+ if (sqlca == NULL || !pg_ensure_c_locale())
{
ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
@@ -483,39 +479,6 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
/* add connection to our list */
pthread_mutex_lock(&connections_mutex);
- /*
- * ... but first, make certain we have created ecpg_clocale. Rely on
- * holding connections_mutex to ensure this is done by only one thread.
- */
-#ifdef HAVE_USELOCALE
- if (!ecpg_clocale)
- {
- ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (!ecpg_clocale)
- {
- pthread_mutex_unlock(&connections_mutex);
- ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
- ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
- if (host)
- ecpg_free(host);
- if (port)
- ecpg_free(port);
- if (options)
- ecpg_free(options);
- if (realname)
- ecpg_free(realname);
- if (dbname)
- ecpg_free(dbname);
- if (conn_keywords)
- ecpg_free(conn_keywords);
- if (conn_values)
- ecpg_free(conn_values);
- free(this);
- return false;
- }
- }
-#endif
-
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index fa562767585..856f4c9472d 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -466,7 +466,7 @@ ecpg_get_data(const PGresult *results, int act_tuple, int act_field, int lineno,
pval++;
if (!check_special_value(pval, &dres, &scan_length))
- dres = strtod(pval, &scan_length);
+ dres = strtod_l(pval, &scan_length, PG_C_LOCALE);
if (isarray && *scan_length == '"')
scan_length++;
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c4..855322e037f 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -475,46 +475,9 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
memset(&stmt, 0, sizeof stmt);
stmt.lineno = lineno;
- /* Make sure we do NOT honor the locale for numeric input */
- /* since the database gives the standard decimal point */
- /* (see comments in execute.c) */
-#ifdef HAVE_USELOCALE
-
- /*
- * To get here, the above PQnfields() test must have found nonzero
- * fields. One needs a connection to create such a descriptor. (EXEC
- * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
- * can't change the descriptor's PQnfields().) Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt.oldlocale = uselocale(ecpg_clocale);
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- setlocale(LC_NUMERIC, "C");
-#endif
-
/* desperate try to guess something sensible */
stmt.connection = ecpg_get_connection(NULL);
ecpg_store_result(ECPGresult, index, &stmt, &data_var);
-
-#ifdef HAVE_USELOCALE
- if (stmt.oldlocale != (locale_t) 0)
- uselocale(stmt.oldlocale);
-#else
- if (stmt.oldlocale)
- {
- setlocale(LC_NUMERIC, stmt.oldlocale);
- ecpg_free(stmt.oldlocale);
- }
-#ifdef HAVE__CONFIGTHREADLOCALE
- if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
-#endif
-#endif
}
else if (data_var.ind_type != ECPGt_NO_INDICATOR && data_var.ind_pointer != NULL)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index bad3cd9920b..40988d53575 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -56,10 +56,6 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
-#ifdef HAVE_USELOCALE
-extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
-#endif
-
/* structure to store one statement */
struct statement
{
@@ -73,14 +69,6 @@ struct statement
bool questionmarks;
struct variable *inlist;
struct variable *outlist;
-#ifdef HAVE_USELOCALE
- locale_t oldlocale;
-#else
- char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
- int oldthreadlocale;
-#endif
-#endif
int nparams;
char **paramvalues;
int *paramlengths;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index c578c21cf66..ec939d0f63a 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,9 +101,6 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifndef HAVE_USELOCALE
- ecpg_free(stmt->oldlocale);
-#endif
ecpg_free(stmt);
}
@@ -1973,40 +1970,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
if (stmt == NULL)
return false;
- /*
- * Make sure we do NOT honor the locale for numeric input/output since the
- * database wants the standard decimal point. If available, use
- * uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
- */
-#ifdef HAVE_USELOCALE
-
- /*
- * Since ecpg_init() succeeded, we have a connection. Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt->oldlocale = uselocale(ecpg_clocale);
- if (stmt->oldlocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#else
-#ifdef HAVE__CONFIGTHREADLOCALE
- stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- if (stmt->oldlocale == NULL)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- setlocale(LC_NUMERIC, "C");
-#endif
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
@@ -2213,24 +2176,6 @@ ecpg_do_epilogue(struct statement *stmt)
if (stmt == NULL)
return;
-#ifdef HAVE_USELOCALE
- if (stmt->oldlocale != (locale_t) 0)
- uselocale(stmt->oldlocale);
-#else
- if (stmt->oldlocale)
- setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
-#endif
-#endif
-
free_statement(stmt);
}
diff --git a/src/interfaces/ecpg/pgtypeslib/dt_common.c b/src/interfaces/ecpg/pgtypeslib/dt_common.c
index c4119ab7932..f8349b66ce9 100644
--- a/src/interfaces/ecpg/pgtypeslib/dt_common.c
+++ b/src/interfaces/ecpg/pgtypeslib/dt_common.c
@@ -1218,7 +1218,7 @@ DecodeNumber(int flen, char *str, int fmask,
return DecodeNumberField(flen, str, (fmask | DTK_DATE_M),
tmask, tm, fsec, is2digits);
- *fsec = strtod(cp, &cp);
+ *fsec = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
}
@@ -2030,7 +2030,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double frac;
- frac = strtod(cp, &cp);
+ frac = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
*fsec = frac * 1000000;
@@ -2054,7 +2054,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double time;
- time = strtod(cp, &cp);
+ time = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 936a6883816..155c6cc7770 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -60,7 +60,7 @@ ParseISO8601Number(const char *str, char **endptr, int *ipart, double *fpart)
if (!(isdigit((unsigned char) *str) || *str == '-' || *str == '.'))
return DTERR_BAD_FORMAT;
errno = 0;
- val = strtod(str, endptr);
+ val = strtod_l(str, endptr, PG_C_LOCALE);
/* did we not see anything that looks like a double? */
if (*endptr == str || errno != 0)
return DTERR_BAD_FORMAT;
@@ -455,7 +455,7 @@ DecodeInterval(char **field, int *ftype, int nf, /* int range, */
else if (*cp == '.')
{
errno = 0;
- fval = strtod(cp, &cp);
+ fval = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0' || errno != 0)
return DTERR_BAD_FORMAT;
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 35e7b92da40..49938543d03 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -1455,7 +1455,7 @@ numericvar_to_double(numeric *var, double *dp)
* strtod does not reset errno to 0 in case of success.
*/
errno = 0;
- val = strtod(tmp, &endptr);
+ val = strtod_l(tmp, &endptr, PG_C_LOCALE);
if (errno == ERANGE)
{
free(tmp);
diff --git a/src/port/Makefile b/src/port/Makefile
index 366c814bd92..70d63963611 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -41,6 +41,7 @@ OBJS = \
bsearch_arg.o \
chklocale.o \
inet_net_ntop.o \
+ locale.o \
noblock.o \
path.o \
pg_bitutils.o \
diff --git a/src/port/locale.c b/src/port/locale.c
new file mode 100644
index 00000000000..387fb4a107a
--- /dev/null
+++ b/src/port/locale.c
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * locale.c
+ * Helper routines for thread-safe system locale usage.
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/locale.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#ifndef WIN32
+#include <pthread.h>
+#else
+#include <synchapi.h>
+#endif
+
+/* A process-lifetime singleton, allocated on first need. */
+static locale_t c_locale;
+
+#ifndef WIN32
+static void
+init_c_locale_once(void)
+{
+ c_locale = newlocale(LC_ALL, "C", NULL);
+}
+#else
+static BOOL
+init_c_locale_once(PINIT_ONCE once, PVOID parameter, PVOID *context)
+{
+ c_locale = _create_locale(LC_ALL, "C");
+ return true;
+}
+#endif
+
+/*
+ * Access a process-lifetime singleton locale_t object. Use the macro
+ * PG_C_LOCALE instead of calling this directly, as it can skip the function
+ * call on some systems.
+ */
+locale_t
+pg_get_c_locale(void)
+{
+ /*
+ * Fast path if already initialized. This assumes that we can read a
+ * locale_t (in practice, a pointer) without tearing in a multi-threaded
+ * program.
+ */
+ if (c_locale != (locale_t) 0)
+ return c_locale;
+
+ /* Make a locale_t. It will live until process exit. */
+ {
+#ifndef WIN32
+ static pthread_once_t once = PTHREAD_ONCE_INIT;
+
+ pthread_once(&once, init_c_locale_once);
+#else
+ static INIT_ONCE once;
+ InitOnceExecuteOnce(&once, init_c_locale_once, NULL, NULL);
+#endif
+ }
+
+ /*
+ * It's possible that the allocation of the locale failed due to low
+ * memory, and then (locale_t) 0 will be returned. Users of PG_C_LOCALE
+ * should defend against that by checking pg_ensure_c_locale() at a
+ * convenient time, so that they can treat it as a simple constant after
+ * that.
+ */
+
+ return c_locale;
+}
diff --git a/src/port/meson.build b/src/port/meson.build
index 83a06325209..93f6158c307 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -5,6 +5,7 @@ pgport_sources = [
'chklocale.c',
'inet_net_ntop.c',
'noblock.c',
+ 'locale.c',
'path.c',
'pg_bitutils.c',
'pg_popcount_avx512.c',
diff --git a/src/port/snprintf.c b/src/port/snprintf.c
index 884f0262dd1..3cc49f01ea6 100644
--- a/src/port/snprintf.c
+++ b/src/port/snprintf.c
@@ -109,6 +109,36 @@
#undef vprintf
#undef printf
+#if defined(FRONTEND)
+/*
+ * Frontend code might be multi-threaded. When calling the system snprintf()
+ * for floats, we have to use either the non-standard snprintf_l() variant, or
+ * save-and-restore the calling thread's locale using uselocale(), depending on
+ * availability.
+ */
+#if defined(HAVE_SNPRINTF_L)
+/* At least macOS and the BSDs have the snprintf_l() extension. */
+#define USE_SNPRINTF_L
+#elif defined(WIN32)
+/* Windows has a version with a different name and argument order. */
+#define snprintf_l(str, size, loc, format, ...) _snprintf_l(str, size, format, loc, __VA_ARGS__)
+#define USE_SNPRINTF_L
+#else
+/* We have to do a thread-safe save-and-restore around snprintf(). */
+#define NEED_USE_LOCALE
+#endif
+#else
+/*
+ * Backend code doesn't need to worry about locales here, because LC_NUMERIC is
+ * set to "C" in main() and doesn't change. Plain old snprintf() is always OK
+ * without uselocale().
+ *
+ * XXX If the backend were multithreaded, we would have to be 100% certain that
+ * no one is calling setlocale() concurrently, even transiently, to be able to
+ * keep this small optimization.
+ */
+#endif
+
/*
* Info about where the formatted output is going.
*
@@ -1184,6 +1214,9 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
* according to == but not according to memcmp.
*/
static const double dzero = 0.0;
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
if (adjust_sign((value < 0.0 ||
(value == 0.0 &&
@@ -1205,7 +1238,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '*';
fmt[3] = type;
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, prec, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, prec, value);
+#endif
}
else
{
@@ -1214,6 +1251,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '\0';
vallen = snprintf(convert, sizeof(convert), fmt, value);
}
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
goto fail;
@@ -1336,12 +1378,25 @@ pg_strfromd(char *str, size_t count, int precision, double value)
}
else
{
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
+
fmt[0] = '%';
fmt[1] = '.';
fmt[2] = '*';
fmt[3] = 'g';
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, precision, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, precision, value);
+#endif
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
{
target.failed = true;
--
2.47.0
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-14 09:54 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 0 replies; 49+ messages in thread
From: Heikki Linnakangas @ 2024-11-14 09:54 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; Peter Eisentraut <peter@eisentraut.org>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 14/11/2024 09:48, Thomas Munro wrote:
> On Thu, Aug 29, 2024 at 6:50 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> The error handling with pg_ensure_c_locale(), that's the sort of thing
>> I'm afraid will be hard to test or even know how it will behave. And it
>> creates this weird coupling between pgtypeslib and ecpglib that you
>> mentioned earlier. And if there are other users of PG_C_LOCALE in the
>> future, there will be similar questions about the proper initialization
>> and error handling sequence.
>
> Ack. The coupling does become at least less weird (currently it must
> be capable of giving the wrong answers reliably if called directly I
> think, no?) and weaker, but I acknowledge that it's still non-ideal
> that out-of-memory would be handled nicely only if you used ecpg
> first, and subtle misbehaviour would ensure otherwise, and future
> users face the same sort of problem unless they design in a reasonable
> initialisation place that could check pg_ensure_c_locale() nicely.
> Classic retro-fitting problem, though.
Hmm, so should we add calls to pg_ensure_c_locale() in pgtypeslib too,
before each call to strtod_l()?
Looking at the callers of strtod() in ecpg, all but one of them actually
readily convert the returned value to integer, with some multiplication
or division with constants. It would be nice to replace those with a
function that would avoid going through double in the first place. That
still leaves one caller in numericvar_to_double() which really wants a
double, though
Overall, the patch looks good to me.
--
Heikki Linnakangas
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-14 12:53 Peter Eisentraut <peter@eisentraut.org>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 1 reply; 49+ messages in thread
From: Peter Eisentraut @ 2024-11-14 12:53 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 14.11.24 08:48, Thomas Munro wrote:
> The three MinGW environments we test today are using ucrt, and
> configure detects the symbol on all. Namely: fairwren
> (msys2/mingw64), the CI mingw64 task and the mingw cross-build that
> runs on Linux in the CI CompilerWarnings task. As far as I know these
> are the reasons for, and mechanism by which, we keep MinGW support
> working. We have no policy requiring arbitrary old MinGW systems
> work, and we wouldn't know anyway.
Right. So I think we could unwind this in steps. First, remove the
configure test for _configthreadlocale() and all the associated #ifdefs
in the existing ecpg code. This seems totally safe, it would just leave
behind MinGW older than 2016 and MSVC older than 2015, the latter of
which is already the current threshold.
Then the question whether we want to re-enable the error checking on
_configthreadlocale() that was reverted by 2cf91ccb, or at least
something similar. This should also be okay based on your description
of the different Windows runtimes. I think it would also be good to do
this to make sure this works before we employ _configthreadlocale() in
higher-stakes situations.
I suggest doing these two steps as separate patches, so this doesn't get
confused between the various thread-related threads that want to
variously add or remove uses of this function.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-20 09:00 Thomas Munro <thomas.munro@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 2 replies; 49+ messages in thread
From: Thomas Munro @ 2024-11-20 09:00 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Fri, Nov 15, 2024 at 1:53 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> On 14.11.24 08:48, Thomas Munro wrote:
> > The three MinGW environments we test today are using ucrt, and
> > configure detects the symbol on all. Namely: fairwren
> > (msys2/mingw64), the CI mingw64 task and the mingw cross-build that
> > runs on Linux in the CI CompilerWarnings task. As far as I know these
> > are the reasons for, and mechanism by which, we keep MinGW support
> > working. We have no policy requiring arbitrary old MinGW systems
> > work, and we wouldn't know anyway.
>
> Right. So I think we could unwind this in steps. First, remove the
> configure test for _configthreadlocale() and all the associated #ifdefs
> in the existing ecpg code. This seems totally safe, it would just leave
> behind MinGW older than 2016 and MSVC older than 2015, the latter of
> which is already the current threshold.
>
> Then the question whether we want to re-enable the error checking on
> _configthreadlocale() that was reverted by 2cf91ccb, or at least
> something similar. This should also be okay based on your description
> of the different Windows runtimes. I think it would also be good to do
> this to make sure this works before we employ _configthreadlocale() in
> higher-stakes situations.
>
> I suggest doing these two steps as separate patches, so this doesn't get
> confused between the various thread-related threads that want to
> variously add or remove uses of this function.
OK, do you think these three patches tell the _configthreadlocale()
story properly? (Then after that we can get back to getting rid of
it...)
Attachments: download all (3 files, .tar.gz)
0001-Remove-configure-check-for-_configthreadlocale.patch (application/octet-stream, 5.0K) download
From 4fbeb868b1acf6c760a5ad23b3be3cc70c6afdb6 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 20 Nov 2024 15:22:23 +1300
Subject: [PATCH 1/3] Remove configure check for _configthreadlocale().
All modern Windows systems have _configthreadlocale(). It was
introduced in msvcrt80.dll as used by Visual Studio 2005. Historically,
MinGW was stuck on even older msvcrt.dll, but added its own dummy
implementation years ago anyway, rendering the configure test useless.
We can handle MinGW's dummy implementation, but in practice we don't
encounter it anymore because MinGW uses ucrt by default.
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
configure | 11 -----------
configure.ac | 1 -
meson.build | 1 -
src/include/pg_config.h.in | 3 ---
src/interfaces/ecpg/ecpglib/descriptor.c | 4 ++--
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 2 +-
src/interfaces/ecpg/ecpglib/execute.c | 4 ++--
7 files changed, 5 insertions(+), 21 deletions(-)
diff --git a/configure b/configure
index f58eae1baa8..728deeb30ff 100755
--- a/configure
+++ b/configure
@@ -15812,17 +15812,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index 82c5009e3e8..b7aef8ac11e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1816,7 +1816,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index 5b0510cef78..2accff2c992 100644
--- a/meson.build
+++ b/meson.build
@@ -2614,7 +2614,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index cdd9a6e9355..130d0216d7d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -529,9 +529,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c4..56e2bc41531 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -490,7 +490,7 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
Assert(ecpg_clocale);
stmt.oldlocale = uselocale(ecpg_clocale);
#else
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
#endif
stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
@@ -510,7 +510,7 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
setlocale(LC_NUMERIC, stmt.oldlocale);
ecpg_free(stmt.oldlocale);
}
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
if (stmt.oldthreadlocale != -1)
(void) _configthreadlocale(stmt.oldthreadlocale);
#endif
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index bad3cd9920b..75cc68275bd 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -77,7 +77,7 @@ struct statement
locale_t oldlocale;
#else
char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
int oldthreadlocale;
#endif
#endif
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index c578c21cf66..466d5600f9b 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1995,7 +1995,7 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
return false;
}
#else
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
#endif
stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
@@ -2219,7 +2219,7 @@ ecpg_do_epilogue(struct statement *stmt)
#else
if (stmt->oldlocale)
setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
/*
* This is a bit trickier than it looks: if we failed partway through
--
2.39.5 (Apple Git-154)
0002-Formally-require-ucrt-on-Windows.patch (application/octet-stream, 3.0K) download
From 330767ee9aa7081b9b4fc6ca82c337bb0c2a89fc Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 20 Nov 2024 15:56:46 +1300
Subject: [PATCH 2/3] Formally require ucrt on Windows.
Historically we tolerated the absence of various Windows features when
using the MinGW tool chain, because it used ancient msvcrt.dll for a
long period of time. It now uses ucrt by default (like Windows 10,
Visual Studio 2015).
It was difficult or impossible to use MinGW with the various
msvcrXXX.dll versions that came between the original msvcrt.dll and
ucrt, so this represents a sudden jump in API from about 1996 to about
2015. In fact we already effectively required ucrt in PostgreSQL 17:
commit 8d9a9f03 required _create_locale and friends, which arrived in
msvcr120.dll (Visual Studio 2013, the last of the pre-ucrt series of
runtimes), and for MinGW users that practically meant ucrt; meanwhile
for Visual Studio we currently require at least 2015, which uses ucrt.
That may not have been the first such case, but msvrt.dll and Visual
Studio 2013 builds had already dropped off our testing radar so we
weren't paying attention if so.
This commit formalizes the requirement. It removes obsolete comments
that discussed msvcrt.dll limitations, and some tests of
!defined(_MSC_VER) to imply msvcrt.dll. Later commits will remove more
anachronisms resulting from this policy decision.
---
src/backend/utils/adt/pg_locale.c | 19 -------------------
1 file changed, 19 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index d4e89663ec1..d1b392c248c 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1017,13 +1017,6 @@ cache_locale_time(void)
* get ISO Locale name directly by using GetLocaleInfoEx() with LCType as
* LOCALE_SNAME.
*
- * MinGW headers declare _create_locale(), but msvcrt.dll lacks that symbol in
- * releases before Windows 8. IsoLocaleName() always fails in a MinGW-built
- * postgres.exe, so only Unix-style values of the lc_messages GUC can elicit
- * localized messages. In particular, every lc_messages setting that initdb
- * can select automatically will yield only C-locale messages. XXX This could
- * be fixed by running the fully-qualified locale name through a lookup table.
- *
* This function returns a pointer to a static buffer bearing the converted
* name or NULL if conversion fails.
*
@@ -1031,8 +1024,6 @@ cache_locale_time(void)
* [2] https://docs.microsoft.com/en-us/windows/win32/intl/locale-names
*/
-#if defined(_MSC_VER)
-
/*
* Callback function for EnumSystemLocalesEx() in get_iso_localename().
*
@@ -1201,16 +1192,6 @@ IsoLocaleName(const char *winlocname)
return get_iso_localename(winlocname);
}
-#else /* !defined(_MSC_VER) */
-
-static char *
-IsoLocaleName(const char *winlocname)
-{
- return NULL; /* Not supported on MinGW */
-}
-
-#endif /* defined(_MSC_VER) */
-
#endif /* WIN32 && LC_MESSAGES */
--
2.39.5 (Apple Git-154)
0003-Revert-Blind-attempt-to-fix-_configthreadlocale-fail.patch (application/octet-stream, 3.4K) download
From 9e4ee5b625992fa2e0c2a9778942c0ec1e02bfbc Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 20 Nov 2024 17:45:45 +1300
Subject: [PATCH 3/3] Revert "Blind attempt to fix _configthreadlocale()
failures on MinGW."
This reverts commit 2cf91ccb73ce888c44e3751548fb7c77e87335f2.
When using the old msvcrt.dll, MinGW would supply its own dummy version
of _configthreadlocale() that just returns -1 if you try to use it. For
a time we tolerated that to shut the build farm up. We would fall back
to code that was enough for the tests to pass. It would certainly crash
a multithreaded program though, so it was bogus.
We don't need that kludge anymore, because we require ucrt (the modern
Windows C runtime). We expect the real _configthreadlocale() to be
present, and the ECPG tests will now fail if it isn't. The workaround
was dead code and it's time to revert it.
Discussion: https://postgr.es/m/d9e7731c-ca1b-477c-9298-fa51e135574a%40eisentraut.org
---
src/interfaces/ecpg/ecpglib/descriptor.c | 2 +-
src/interfaces/ecpg/ecpglib/execute.c | 20 +++++++++-----------
2 files changed, 10 insertions(+), 12 deletions(-)
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index 56e2bc41531..aee888432f0 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -512,7 +512,7 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
}
#ifdef WIN32
if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
+ _configthreadlocale(stmt.oldthreadlocale);
#endif
#endif
}
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 466d5600f9b..b5089eac787 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1977,9 +1977,7 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
* Make sure we do NOT honor the locale for numeric input/output since the
* database wants the standard decimal point. If available, use
* uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
+ * that, but it does have _configthreadlocale().
*/
#ifdef HAVE_USELOCALE
@@ -1997,6 +1995,11 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
#else
#ifdef WIN32
stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
+ if (stmt->oldthreadlocale == -1)
+ {
+ ecpg_do_epilogue(stmt);
+ return false;
+ }
#endif
stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
if (stmt->oldlocale == NULL)
@@ -2218,17 +2221,12 @@ ecpg_do_epilogue(struct statement *stmt)
uselocale(stmt->oldlocale);
#else
if (stmt->oldlocale)
+ {
setlocale(LC_NUMERIC, stmt->oldlocale);
#ifdef WIN32
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
+ _configthreadlocale(stmt->oldthreadlocale);
#endif
+ }
#endif
free_statement(stmt);
--
2.39.5 (Apple Git-154)
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-20 23:30 Thomas Munro <thomas.munro@gmail.com>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 0 replies; 49+ messages in thread
From: Thomas Munro @ 2024-11-20 23:30 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Nov 20, 2024 at 10:00 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> OK, do you think these three patches tell the _configthreadlocale()
> story properly? (Then after that we can get back to getting rid of
> it...)
Just by the way, here's another interesting thing I have learned about
the msvcrt->ucrt evolution: ucrt introduced UTF-8 support (ie locales
can use UTF-8 encoding, and all the standard library char functions
work with it just fine like on Unix), but PostgreSQL still believes
that Windows can't do that and has a lot of special code paths to work
with wchar_t and perform extra conversions. I started nibbling at
that[1], but at the time I was still a bit fuzzy on whether we could
really rip all that old stuff out yet and harmonise with Unix, as I
didn't understand the MinGW/runtime/history situation. It seems the
coast is now clear, and that would be a satisfying cleanup. (There's
still non-trivial work to do for that though: we allowed weird
mismatched encoding scenarios just on that OS, and would need to stop
that, which might create some upgrade path problems, needs some
thought, see that thread.)
[1] https://www.postgresql.org/message-id/CA%2BhUKGJ%3Dca39Cg%3Dy%3DS89EaCYvvCF8NrZRO%3Duog-cnz0VzC6Kfg%...
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-21 07:38 Peter Eisentraut <peter@eisentraut.org>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 1 reply; 49+ messages in thread
From: Peter Eisentraut @ 2024-11-21 07:38 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 20.11.24 10:00, Thomas Munro wrote:
> On Fri, Nov 15, 2024 at 1:53 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> On 14.11.24 08:48, Thomas Munro wrote:
>>> The three MinGW environments we test today are using ucrt, and
>>> configure detects the symbol on all. Namely: fairwren
>>> (msys2/mingw64), the CI mingw64 task and the mingw cross-build that
>>> runs on Linux in the CI CompilerWarnings task. As far as I know these
>>> are the reasons for, and mechanism by which, we keep MinGW support
>>> working. We have no policy requiring arbitrary old MinGW systems
>>> work, and we wouldn't know anyway.
>>
>> Right. So I think we could unwind this in steps. First, remove the
>> configure test for _configthreadlocale() and all the associated #ifdefs
>> in the existing ecpg code. This seems totally safe, it would just leave
>> behind MinGW older than 2016 and MSVC older than 2015, the latter of
>> which is already the current threshold.
>>
>> Then the question whether we want to re-enable the error checking on
>> _configthreadlocale() that was reverted by 2cf91ccb, or at least
>> something similar. This should also be okay based on your description
>> of the different Windows runtimes. I think it would also be good to do
>> this to make sure this works before we employ _configthreadlocale() in
>> higher-stakes situations.
>>
>> I suggest doing these two steps as separate patches, so this doesn't get
>> confused between the various thread-related threads that want to
>> variously add or remove uses of this function.
>
> OK, do you think these three patches tell the _configthreadlocale()
> story properly? (Then after that we can get back to getting rid of
> it...)
Yes, this is very clear and helpful. Thanks.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-22 21:32 Thomas Munro <thomas.munro@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2024-11-22 21:32 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; Andrew Dunstan <andrew@dunslane.net>; +Cc: Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Nov 21, 2024 at 8:38 PM Peter Eisentraut <peter@eisentraut.org> wrote:
> On 20.11.24 10:00, Thomas Munro wrote:
> > OK, do you think these three patches tell the _configthreadlocale()
> > story properly? (Then after that we can get back to getting rid of
> > it...)
>
> Yes, this is very clear and helpful. Thanks.
I realised that there is another aspect to this: it must be impossible
to build PostgreSQL with the original MinGW/MSYS project by now. I
don't understand the history of the MinGW/MinGW-w64 fork, but if
they're both still live projects out there adding to the general
confusion about the frankenwindows multiverse, we should clarify our
situation. As far as I know, we're only testing the second thing, and
only the second thing can use UCRT, and only the second thing is a
viable alternative toolchain for software that is primarily targeting
current Visual Studio, which I think is something we can say about our
project. Right?
I was looking at the installation.sgml file and wondering where to
write "UCRT required" in the 0002 patch. Perhaps we don't even need
to mention it if it's the default on MinGW-w64, but I think we do need
to remove the discussion of the original MinGW. It's even pointing to
a cybersquatted URL. I also suspect that MinGW-w64 shouldn't be
described as for "building 64 bit binaries" (maybe it started that way
but by now it's has other goals, modern libraries/APIs etc keeping up
with VC; and AFAICS it can build 32 bit binaries too, if anyone still
cares about that; wow, the naming of all this stuff is so confusing to
a non-Windows person, 32 and 64 get thrown around all over the place
with apparently no real meaning, but I digress...). Andrew, would you
be interested in updating this section to talk only about the modern
installation and build steps for MinGW-w64/MSYS2 et al? There are
several confusingly related projects and I'm not sure I'd get it right
if I tried, I've never used any of it...
Attachments: download all (3 files, .tar.gz)
v2-0001-Remove-configure-check-for-_configthreadlocale.patch (text/x-patch, 5.0K) download
From fbe7be0c0c1d4e2b047d2dcfb0ba5ccd9ce0994a Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Sat, 23 Nov 2024 08:44:02 +1300
Subject: [PATCH v2 1/3] Remove configure check for _configthreadlocale().
All modern Windows systems have _configthreadlocale(). It was first
introduced in msvcr80.dll from Visual Studio 2005. Historically, MinGW
was stuck on even older msvcrt.dll, but added its own dummy
implementation of the function when using msvcrt.dll years ago anyway,
effectively rendering the configure test useless. In practice we don't
encounter the dummy anymore because modern MinGW uses ucrt.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
configure | 11 -----------
configure.ac | 1 -
meson.build | 1 -
src/include/pg_config.h.in | 3 ---
src/interfaces/ecpg/ecpglib/descriptor.c | 4 ++--
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 2 +-
src/interfaces/ecpg/ecpglib/execute.c | 4 ++--
7 files changed, 5 insertions(+), 21 deletions(-)
diff --git a/configure b/configure
index 28719ed30c..07dd2dec1b 100755
--- a/configure
+++ b/configure
@@ -15917,17 +15917,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- for ac_func in _configthreadlocale
-do :
- ac_fn_c_check_func "$LINENO" "_configthreadlocale" "ac_cv_func__configthreadlocale"
-if test "x$ac_cv_func__configthreadlocale" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE__CONFIGTHREADLOCALE 1
-_ACEOF
-
-fi
-done
-
case " $LIBOBJS " in
*" dirmod.$ac_objext "* ) ;;
*) LIBOBJS="$LIBOBJS dirmod.$ac_objext"
diff --git a/configure.ac b/configure.ac
index 533f4ab78a..184b920e3e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1825,7 +1825,6 @@ fi
# Win32 (really MinGW) support
if test "$PORTNAME" = "win32"; then
- AC_CHECK_FUNCS(_configthreadlocale)
AC_LIBOBJ(dirmod)
AC_LIBOBJ(kill)
AC_LIBOBJ(open)
diff --git a/meson.build b/meson.build
index b64d253fe4..a56f3311ea 100644
--- a/meson.build
+++ b/meson.build
@@ -2614,7 +2614,6 @@ endif
# XXX: Might be worth conditioning some checks on the OS, to avoid doing
# unnecessary checks over and over, particularly on windows.
func_checks = [
- ['_configthreadlocale', {'skip': host_system != 'windows'}],
['backtrace_symbols', {'dependencies': [execinfo_dep]}],
['clock_gettime', {'dependencies': [rt_dep], 'define': false}],
['copyfile'],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index a903c60a3a..e53560c87f 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -535,9 +535,6 @@
/* Define to 1 if your compiler understands __builtin_unreachable. */
#undef HAVE__BUILTIN_UNREACHABLE
-/* Define to 1 if you have the `_configthreadlocale' function. */
-#undef HAVE__CONFIGTHREADLOCALE
-
/* Define to 1 if you have __cpuid. */
#undef HAVE__CPUID
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c..56e2bc4153 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -490,7 +490,7 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
Assert(ecpg_clocale);
stmt.oldlocale = uselocale(ecpg_clocale);
#else
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
#endif
stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
@@ -510,7 +510,7 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
setlocale(LC_NUMERIC, stmt.oldlocale);
ecpg_free(stmt.oldlocale);
}
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
if (stmt.oldthreadlocale != -1)
(void) _configthreadlocale(stmt.oldthreadlocale);
#endif
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index bad3cd9920..75cc68275b 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -77,7 +77,7 @@ struct statement
locale_t oldlocale;
#else
char *oldlocale;
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
int oldthreadlocale;
#endif
#endif
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index c578c21cf6..466d5600f9 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1995,7 +1995,7 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
return false;
}
#else
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
#endif
stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
@@ -2219,7 +2219,7 @@ ecpg_do_epilogue(struct statement *stmt)
#else
if (stmt->oldlocale)
setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef HAVE__CONFIGTHREADLOCALE
+#ifdef WIN32
/*
* This is a bit trickier than it looks: if we failed partway through
--
2.47.0
v2-0002-Formally-require-ucrt-on-Windows.patch (text/x-patch, 3.3K) download
From a5e4b17a10be22e20d81252783094753ff0afda3 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Sat, 23 Nov 2024 09:14:48 +1300
Subject: [PATCH v2 2/3] Formally require ucrt on Windows.
Historically we tolerated the absence of various C runtime library
features for the benefit of the MinGW tool chain, because it used
ancient msvcrt.dll for a long period of time. It now uses ucrt by
default (like Windows 10+, Visual Studio 2015+), and that's the only
configuration we're testing.
In practice, we effectively required ucrt already in PostgreSQL 17, when
commit 8d9a9f03 required _create_locale etc, first available in
msvcr120.dll (Visual Studio 2013, the last of the pre-ucrt series of
runtimes), and for MinGW users that practically meant ucrt because it
was difficult or impossible to use msvcr120.dll. That may even not have
been the first such case, but old MinGW configurations had already
dropped off our testing radar so we weren't paying much attention.
This commit formalizes the requirement. It also removes a couple of
obsolete comments that discussed msvcrt.dll limitations, and some tests
of !defined(_MSC_VER) to imply msvcrt.dll. There are many more
anachronisms, but it'll take some time to figure out how to remove them
all.
XXX Help needed to rewrite doc/src/sgml/installation.sgml, to clarify
that we now only support https://en.wikipedia.org/wiki/Mingw-w64 and
MSYS2, not https://en.wikipedia.org/wiki/MinGW and MSYS (no one is
testing the latter, and it probably can't build PostgreSQL 17, since it
can't use UCRT?)
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/d9e7731c-ca1b-477c-9298-fa51e135574a%40eisentraut.org
---
src/backend/utils/adt/pg_locale.c | 19 -------------------
1 file changed, 19 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index d4e89663ec..d1b392c248 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1017,13 +1017,6 @@ cache_locale_time(void)
* get ISO Locale name directly by using GetLocaleInfoEx() with LCType as
* LOCALE_SNAME.
*
- * MinGW headers declare _create_locale(), but msvcrt.dll lacks that symbol in
- * releases before Windows 8. IsoLocaleName() always fails in a MinGW-built
- * postgres.exe, so only Unix-style values of the lc_messages GUC can elicit
- * localized messages. In particular, every lc_messages setting that initdb
- * can select automatically will yield only C-locale messages. XXX This could
- * be fixed by running the fully-qualified locale name through a lookup table.
- *
* This function returns a pointer to a static buffer bearing the converted
* name or NULL if conversion fails.
*
@@ -1031,8 +1024,6 @@ cache_locale_time(void)
* [2] https://docs.microsoft.com/en-us/windows/win32/intl/locale-names
*/
-#if defined(_MSC_VER)
-
/*
* Callback function for EnumSystemLocalesEx() in get_iso_localename().
*
@@ -1201,16 +1192,6 @@ IsoLocaleName(const char *winlocname)
return get_iso_localename(winlocname);
}
-#else /* !defined(_MSC_VER) */
-
-static char *
-IsoLocaleName(const char *winlocname)
-{
- return NULL; /* Not supported on MinGW */
-}
-
-#endif /* defined(_MSC_VER) */
-
#endif /* WIN32 && LC_MESSAGES */
--
2.47.0
v2-0003-Revert-Blind-attempt-to-fix-_configthreadlocale-f.patch (text/x-patch, 3.4K) download
From 4c94af123a38c8c68178497f7b59e1d948f2d9ca Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Sat, 23 Nov 2024 09:43:16 +1300
Subject: [PATCH v2 3/3] Revert "Blind attempt to fix _configthreadlocale()
failures on MinGW."
This reverts commit 2cf91ccb73ce888c44e3751548fb7c77e87335f2.
When using the old msvcrt.dll, MinGW would supply its own dummy version
of _configthreadlocale() that just returns -1 if you try to use it. For
a time we tolerated that to shut the build farm up. We would fall back
to code that was enough for the tests to pass, but it would have crashed
a real multithreaded program though, so it was bogus.
We don't need that kludge anymore, because we require ucrt (the modern
Windows C runtime). We expect the real _configthreadlocale() to be
present, and the ECPG tests will now fail if it isn't. The workaround
was dead code and it's time to revert it.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/d9e7731c-ca1b-477c-9298-fa51e135574a%40eisentraut.org
---
src/interfaces/ecpg/ecpglib/descriptor.c | 2 +-
src/interfaces/ecpg/ecpglib/execute.c | 20 +++++++++-----------
2 files changed, 10 insertions(+), 12 deletions(-)
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index 56e2bc4153..aee888432f 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -512,7 +512,7 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
}
#ifdef WIN32
if (stmt.oldthreadlocale != -1)
- (void) _configthreadlocale(stmt.oldthreadlocale);
+ _configthreadlocale(stmt.oldthreadlocale);
#endif
#endif
}
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 466d5600f9..b5089eac78 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1977,9 +1977,7 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
* Make sure we do NOT honor the locale for numeric input/output since the
* database wants the standard decimal point. If available, use
* uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it usually does have _configthreadlocale(). In some versions
- * of MinGW, _configthreadlocale() exists but always returns -1 --- so
- * treat that situation as if the function doesn't exist.
+ * that, but it does have _configthreadlocale().
*/
#ifdef HAVE_USELOCALE
@@ -1997,6 +1995,11 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
#else
#ifdef WIN32
stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
+ if (stmt->oldthreadlocale == -1)
+ {
+ ecpg_do_epilogue(stmt);
+ return false;
+ }
#endif
stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
if (stmt->oldlocale == NULL)
@@ -2218,17 +2221,12 @@ ecpg_do_epilogue(struct statement *stmt)
uselocale(stmt->oldlocale);
#else
if (stmt->oldlocale)
+ {
setlocale(LC_NUMERIC, stmt->oldlocale);
#ifdef WIN32
-
- /*
- * This is a bit trickier than it looks: if we failed partway through
- * statement initialization, oldthreadlocale could still be 0. But that's
- * okay because a call with 0 is defined to be a no-op.
- */
- if (stmt->oldthreadlocale != -1)
- (void) _configthreadlocale(stmt->oldthreadlocale);
+ _configthreadlocale(stmt->oldthreadlocale);
#endif
+ }
#endif
free_statement(stmt);
--
2.47.0
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-25 00:42 Michael Paquier <michael@paquier.xyz>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Michael Paquier @ 2024-11-25 00:42 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Peter Eisentraut <peter@eisentraut.org>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Sat, Nov 23, 2024 at 10:32:31AM +1300, Thomas Munro wrote:
> I realised that there is another aspect to this: it must be impossible
> to build PostgreSQL with the original MinGW/MSYS project by now. I
> don't understand the history of the MinGW/MinGW-w64 fork, but if
> they're both still live projects out there adding to the general
> confusion about the frankenwindows multiverse, we should clarify our
> situation. As far as I know, we're only testing the second thing, and
> only the second thing can use UCRT, and only the second thing is a
> viable alternative toolchain for software that is primarily targeting
> current Visual Studio, which I think is something we can say about our
> project. Right?
FWIW, I am not seeing any advantage in mentioning MinGW at all at this
stage, just extra maintenance burden. As far as I know, MinGW is a
gcc port that has only a 32b implementation. MinGW-w64 is built on
top of it and it includes *both* 32b and 64b implementations, as you
say, with more WIN32 APIs than the former.
So +1 to simplify a bit that stuff.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-25 00:57 Thomas Munro <thomas.munro@gmail.com>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2024-11-25 00:57 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Peter Eisentraut <peter@eisentraut.org>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Mon, Nov 25, 2024 at 1:43 PM Michael Paquier <michael@paquier.xyz> wrote:
> On Sat, Nov 23, 2024 at 10:32:31AM +1300, Thomas Munro wrote:
> > I realised that there is another aspect to this: it must be impossible
> > to build PostgreSQL with the original MinGW/MSYS project by now. I
> > don't understand the history of the MinGW/MinGW-w64 fork, but if
> > they're both still live projects out there adding to the general
> > confusion about the frankenwindows multiverse, we should clarify our
> > situation. As far as I know, we're only testing the second thing, and
> > only the second thing can use UCRT, and only the second thing is a
> > viable alternative toolchain for software that is primarily targeting
> > current Visual Studio, which I think is something we can say about our
> > project. Right?
>
> FWIW, I am not seeing any advantage in mentioning MinGW at all at this
> stage, just extra maintenance burden. As far as I know, MinGW is a
> gcc port that has only a 32b implementation. MinGW-w64 is built on
> top of it and it includes *both* 32b and 64b implementations, as you
> say, with more WIN32 APIs than the former.
>
> So +1 to simplify a bit that stuff.
Thanks. I'm going to have a go at adjusting the docs myself so I can
get this committed. Invitation remains open for someone closer to the
topic to rewrite in a later commit as required for maximum utility to
the reader (I'm never going to install MSYS2, or Windows, I just want
to blow away as much dead code as possible here as it's in the way of
multithreading and other modernisation projects).
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-26 16:23 Peter Eisentraut <peter@eisentraut.org>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 49+ messages in thread
From: Peter Eisentraut @ 2024-11-26 16:23 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 25.11.24 01:57, Thomas Munro wrote:
> Thanks. I'm going to have a go at adjusting the docs myself so I can
> get this committed. Invitation remains open for someone closer to the
> topic to rewrite in a later commit as required for maximum utility to
> the reader (I'm never going to install MSYS2, or Windows, I just want
> to blow away as much dead code as possible here as it's in the way of
> multithreading and other modernisation projects).
Attached is a simple proposal. The section about MinGW can be replaced
mostly by "use MSYS2". That's also what CI and the buildfarm uses.
Anyone who strays from that can figure it out themselves.
In the Visual Studio section, there was some text that recommended
getting flex and bison via MinGW, which seemed clearly outdated. I put
in the URL that the CI images use. I don't know if there are other
common sources, but that one seems good enough.
Attachments:
0001-doc-Update-some-outdated-installation-info-related-t.patch (text/plain, 3.7K) download
From 939a94b524b498cab3a86f6a2e001f61a2e520cd Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Tue, 26 Nov 2024 17:18:36 +0100
Subject: [PATCH] doc: Update some outdated installation info related to MinGW
---
doc/src/sgml/installation.sgml | 60 +++-------------------------------
1 file changed, 5 insertions(+), 55 deletions(-)
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 5621606f59a..d6986dbd885 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -3582,33 +3582,9 @@ <title>MinGW</title>
<para>
PostgreSQL for Windows can be built using MinGW, a Unix-like build
- environment for Microsoft operating systems.
- The MinGW build procedure uses the normal build system described in
- this chapter.
- </para>
-
- <para>
- MinGW, the Unix-like build tools, and MSYS, a collection
- of Unix tools required to run shell scripts
- like <command>configure</command>, can be downloaded
- from <ulink url="http://www.mingw.org/"></ulink>. Neither is
- required to run the resulting binaries; they are needed only for
- creating the binaries.
- </para>
-
- <para>
- To build 64 bit binaries using MinGW, install the 64 bit tool set
- from <ulink url="https://mingw-w64.org/"></ulink>, put its bin
- directory in the <envar>PATH</envar>, and run
- <command>configure</command> with the
- <command>--host=x86_64-w64-mingw32</command> option.
- </para>
-
- <para>
- After you have everything installed, it is suggested that you
- run <application>psql</application>
- under <command>CMD.EXE</command>, as the MSYS console has
- buffering issues.
+ environment for Windows. It is recommended to use the <ulink
+ url="https://www.msys2.org/">MSYS2</ulink> environment for this and also
+ use that to install any prerequisite packages.
</para>
<sect3 id="mingw-crash-dumps">
@@ -3838,35 +3814,9 @@ <title>Requirements</title>
<productname>Bison</productname> and <productname>Flex</productname> are
required. Only <productname>Bison</productname> versions 2.3 and later
will work. <productname>Flex</productname> must be version 2.5.35 or later.
+ Binaries can be downloaded from <ulink
+ url="https://github.com/lexxmark/winflexbison"></ulink>.
</para>
-
- <para>
- Both <productname>Bison</productname> and <productname>Flex</productname>
- are included in the <productname>msys</productname> tool suite, available
- from <ulink url="http://www.mingw.org/wiki/MSYS"></ulink> as part of the
- <productname>MinGW</productname> compiler suite.
- </para>
-
- <para>
- You will need to add the directory containing
- <filename>flex.exe</filename> and <filename>bison.exe</filename> to the
- PATH environment variable. In the case of MinGW, the directory is the
- <filename>\msys\1.0\bin</filename> subdirectory of your MinGW
- installation directory.
- </para>
-
- <note>
- <para>
- The Bison distribution from GnuWin32 appears to have a bug that
- causes Bison to malfunction when installed in a directory with
- spaces in the name, such as the default location on English
- installations <filename>C:\Program Files\GnuWin32</filename>.
- Consider installing into <filename>C:\GnuWin32</filename> or use the
- NTFS short name path to GnuWin32 in your PATH environment setting
- (e.g., <filename>C:\PROGRA~1\GnuWin32</filename>).
- </para>
- </note>
-
</listitem>
</varlistentry>
</variablelist>
--
2.47.0
* Re: On non-Windows, hard depend on uselocale(3)
@ 2024-11-26 20:24 Thomas Munro <thomas.munro@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 49+ messages in thread
From: Thomas Munro @ 2024-11-26 20:24 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Nov 27, 2024 at 5:23 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Attached is a simple proposal. The section about MinGW can be replaced
> mostly by "use MSYS2". That's also what CI and the buildfarm uses.
> Anyone who strays from that can figure it out themselves.
Thanks! I'll take it.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2025-02-09 07:32 Peter Eisentraut <peter@eisentraut.org>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 2 replies; 49+ messages in thread
From: Peter Eisentraut @ 2025-02-09 07:32 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
Checking the status of this thread ...
The patches that removed the configure checks for _configthreadlocale(),
and related cleanup, have been committed.
The original patch to "Tidy up locale thread safety in ECPG library" is
still outstanding.
Attached is a rebased version, based on the posted v6, with a couple of
small fixups from me.
I haven't re-reviewed it yet, but from scanning the discussion, it looks
close to done.
Attachments: download all (3 files, .tar.gz)
v7-0001-Tidy-up-locale-thread-safety-in-ECPG-library.patch (text/plain, 22.8K) download
From 842a5837898a4a47c8c79d07a2d14f3a82514a36 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 10 Aug 2024 11:01:21 +1200
Subject: [PATCH v7 1/3] Tidy up locale thread safety in ECPG library.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Remove setlocale() and _configthreadlocal() as fallback strategy on
systems that don't have uselocale(), where ECPG tries to control
LC_NUMERIC formatting on input and output of floating point numbers. It
was probably broken on some systems (NetBSD), and the code was also
quite messy and complicated, with obsolete configure tests (Windows).
It was also arguably broken, or at least had unstated environmental
requirements, if pgtypeslib code was called directly.
Instead, introduce PG_C_LOCALE to refer to the "C" locale as a locale_t
value. It maps to the special constant LC_C_LOCALE when defined by libc
(macOS, NetBSD), or otherwise uses a process-lifetime locale_t that is
allocated on first use, just as ECPG previously did itself. The new
replacement might be more widely useful. Then change the float parsing
and printing code to pass that to _l() functions where appropriate.
Unfortunately the portability of those functions is a bit complicated.
First, many obvious and useful _l() functions are missing from POSIX,
though most standard libraries define some of them anyway. Second,
although the thread-safe save/restore technique can be used to replace
the missing ones, Windows and NetBSD refused to implement standard
uselocale(). They might have a point: "wide scope" uselocale() is hard
to combine with other code and error-prone, especially in library code.
Luckily they have the _l() functions we want so far anyway. So we have
to be prepared for both ways of doing things:
1. In ECPG, use strtod_l() for parsing, and supply a port.h replacement
using uselocale() over a limited scope if missing.
2. Inside our own snprintf.c, use three different approaches to format
floats. For frontend code, call libc's snprintf_l(), or wrap libc's
snprintf() in uselocale() if it's missing. For backend code, snprintf.c
can keep assuming that the global locale's LC_NUMERIC is "C" and call
libc's snprintf() without change, for now.
(It might eventually be possible to call our in-tree Ryū routines to
display floats in snprintf.c, given the C-locale-always remit of our
in-tree snprintf(), but this patch doesn't risk changing anything that
complicated.)
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Tristan Partin <tristan@partin.io>
Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech
---
configure | 2 +-
configure.ac | 2 +
meson.build | 2 +
src/include/pg_config.h.in | 6 ++
src/include/port.h | 31 ++++++++
src/include/port/win32_port.h | 1 +
src/interfaces/ecpg/ecpglib/connect.c | 39 +---------
src/interfaces/ecpg/ecpglib/data.c | 2 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 37 ---------
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 12 ---
src/interfaces/ecpg/ecpglib/execute.c | 53 -------------
src/interfaces/ecpg/pgtypeslib/dt_common.c | 6 +-
src/interfaces/ecpg/pgtypeslib/interval.c | 4 +-
src/interfaces/ecpg/pgtypeslib/numeric.c | 2 +-
src/port/Makefile | 1 +
src/port/locale.c | 80 ++++++++++++++++++++
src/port/meson.build | 1 +
src/port/snprintf.c | 55 ++++++++++++++
18 files changed, 188 insertions(+), 148 deletions(-)
create mode 100644 src/port/locale.c
diff --git a/configure b/configure
index 0ffcaeb4367..017e63598c1 100755
--- a/configure
+++ b/configure
@@ -14934,7 +14934,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range elf_aux_info getauxval getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range elf_aux_info getauxval getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strtod_l strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.ac b/configure.ac
index f56681e0d91..b06a9cd1080 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1718,6 +1718,8 @@ AC_CHECK_FUNCS(m4_normalize([
pthread_is_threaded_np
setproctitle
setproctitle_fast
+ snprintf_l
+ strtod_l
strchrnul
strsignal
syncfs
diff --git a/meson.build b/meson.build
index 1ceadb9a830..e03b5083f27 100644
--- a/meson.build
+++ b/meson.build
@@ -2648,6 +2648,8 @@ func_checks = [
['shm_open', {'dependencies': [rt_dep], 'define': false}],
['shm_unlink', {'dependencies': [rt_dep], 'define': false}],
['shmget', {'dependencies': [cygipc_dep], 'define': false}],
+ ['snprintf_l'],
+ ['strtod_l'],
['socket', {'dependencies': [socket_dep], 'define': false}],
['strchrnul'],
['strerror_r', {'dependencies': [thread_dep]}],
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 07b2f798abd..ae530506190 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -349,6 +349,9 @@
/* Define to 1 if you have the `setproctitle_fast' function. */
#undef HAVE_SETPROCTITLE_FAST
+/* Define to 1 if you have the `snprintf_l' function. */
+#undef HAVE_SNPRINTF_L
+
/* Define to 1 if the system has the type `socklen_t'. */
#undef HAVE_SOCKLEN_T
@@ -394,6 +397,9 @@
/* Define to 1 if you have the `strsignal' function. */
#undef HAVE_STRSIGNAL
+/* Define to 1 if you have the `strtod_l' function. */
+#undef HAVE_STRTOD_L
+
/* Define to 1 if the system has the type `struct option'. */
#undef HAVE_STRUCT_OPTION
diff --git a/src/include/port.h b/src/include/port.h
index 703cad868ba..30109bfbc95 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -218,6 +218,37 @@ extern int pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2,
extern int pg_vprintf(const char *fmt, va_list args) pg_attribute_printf(1, 0);
extern int pg_printf(const char *fmt,...) pg_attribute_printf(1, 2);
+/*
+ * A couple of systems offer a fast constant locale_t value representing the
+ * "C" locale. We use that if possible, but fall back to creating a singleton
+ * object otherwise. To check that it is available, call pg_ensure_c_locale()
+ * and assume out of memory if it returns false.
+ */
+#ifdef LC_C_LOCALE
+#define PG_C_LOCALE LC_C_LOCALE
+#define pg_ensure_c_locale() true
+#else
+extern locale_t pg_get_c_locale(void);
+#define PG_C_LOCALE pg_get_c_locale()
+#define pg_ensure_c_locale() (PG_C_LOCALE != 0)
+#endif
+
+#if !defined(HAVE_STRTOD_L) && !defined(WIN32)
+/*
+ * POSIX doesn't define this function, but we can implement it with thread-safe
+ * save-and-restore.
+ */
+static inline double
+strtod_l(const char *nptr, char **endptr, locale_t loc)
+{
+ locale_t save = uselocale(loc);
+ double result = strtod(nptr, endptr);
+
+ uselocale(save);
+ return result;
+}
+#endif
+
#ifndef WIN32
/*
* We add a pg_ prefix as a warning that the Windows implementations have the
diff --git a/src/include/port/win32_port.h b/src/include/port/win32_port.h
index ff7028bdc81..ff0b7a0de18 100644
--- a/src/include/port/win32_port.h
+++ b/src/include/port/win32_port.h
@@ -453,6 +453,7 @@ extern int _pglstat64(const char *name, struct stat *buf);
#define isspace_l _isspace_l
#define iswspace_l _iswspace_l
#define strcoll_l _strcoll_l
+#define strtod_l _strtod_l
#define strxfrm_l _strxfrm_l
#define wcscoll_l _wcscoll_l
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 2bbb70333dc..475f0215603 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,10 +10,6 @@
#include "ecpgtype.h"
#include "sqlca.h"
-#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale = (locale_t) 0;
-#endif
-
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
@@ -268,7 +264,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
const char **conn_keywords;
const char **conn_values;
- if (sqlca == NULL)
+ if (sqlca == NULL || !pg_ensure_c_locale())
{
ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
@@ -483,39 +479,6 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
/* add connection to our list */
pthread_mutex_lock(&connections_mutex);
- /*
- * ... but first, make certain we have created ecpg_clocale. Rely on
- * holding connections_mutex to ensure this is done by only one thread.
- */
-#ifdef HAVE_USELOCALE
- if (!ecpg_clocale)
- {
- ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (!ecpg_clocale)
- {
- pthread_mutex_unlock(&connections_mutex);
- ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
- ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
- if (host)
- ecpg_free(host);
- if (port)
- ecpg_free(port);
- if (options)
- ecpg_free(options);
- if (realname)
- ecpg_free(realname);
- if (dbname)
- ecpg_free(dbname);
- if (conn_keywords)
- ecpg_free(conn_keywords);
- if (conn_values)
- ecpg_free(conn_values);
- free(this);
- return false;
- }
- }
-#endif
-
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/data.c b/src/interfaces/ecpg/ecpglib/data.c
index fa562767585..856f4c9472d 100644
--- a/src/interfaces/ecpg/ecpglib/data.c
+++ b/src/interfaces/ecpg/ecpglib/data.c
@@ -466,7 +466,7 @@ ecpg_get_data(const PGresult *results, int act_tuple, int act_field, int lineno,
pval++;
if (!check_special_value(pval, &dres, &scan_length))
- dres = strtod(pval, &scan_length);
+ dres = strtod_l(pval, &scan_length, PG_C_LOCALE);
if (isarray && *scan_length == '"')
scan_length++;
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index 8525a6812f2..fc5e1c9dd89 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -475,46 +475,9 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
memset(&stmt, 0, sizeof stmt);
stmt.lineno = lineno;
- /* Make sure we do NOT honor the locale for numeric input */
- /* since the database gives the standard decimal point */
- /* (see comments in execute.c) */
-#ifdef HAVE_USELOCALE
-
- /*
- * To get here, the above PQnfields() test must have found nonzero
- * fields. One needs a connection to create such a descriptor. (EXEC
- * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
- * can't change the descriptor's PQnfields().) Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt.oldlocale = uselocale(ecpg_clocale);
-#else
-#ifdef WIN32
- stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
-#endif
- stmt.oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- setlocale(LC_NUMERIC, "C");
-#endif
-
/* desperate try to guess something sensible */
stmt.connection = ecpg_get_connection(NULL);
ecpg_store_result(ECPGresult, index, &stmt, &data_var);
-
-#ifdef HAVE_USELOCALE
- if (stmt.oldlocale != (locale_t) 0)
- uselocale(stmt.oldlocale);
-#else
- if (stmt.oldlocale)
- {
- setlocale(LC_NUMERIC, stmt.oldlocale);
- ecpg_free(stmt.oldlocale);
- }
-#ifdef WIN32
- if (stmt.oldthreadlocale != -1)
- _configthreadlocale(stmt.oldthreadlocale);
-#endif
-#endif
}
else if (data_var.ind_type != ECPGt_NO_INDICATOR && data_var.ind_pointer != NULL)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 75cc68275bd..40988d53575 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -56,10 +56,6 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
-#ifdef HAVE_USELOCALE
-extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
-#endif
-
/* structure to store one statement */
struct statement
{
@@ -73,14 +69,6 @@ struct statement
bool questionmarks;
struct variable *inlist;
struct variable *outlist;
-#ifdef HAVE_USELOCALE
- locale_t oldlocale;
-#else
- char *oldlocale;
-#ifdef WIN32
- int oldthreadlocale;
-#endif
-#endif
int nparams;
char **paramvalues;
int *paramlengths;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index b5089eac787..ec939d0f63a 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,9 +101,6 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifndef HAVE_USELOCALE
- ecpg_free(stmt->oldlocale);
-#endif
ecpg_free(stmt);
}
@@ -1973,43 +1970,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
if (stmt == NULL)
return false;
- /*
- * Make sure we do NOT honor the locale for numeric input/output since the
- * database wants the standard decimal point. If available, use
- * uselocale() for this because it's thread-safe. Windows doesn't have
- * that, but it does have _configthreadlocale().
- */
-#ifdef HAVE_USELOCALE
-
- /*
- * Since ecpg_init() succeeded, we have a connection. Any successful
- * connection initializes ecpg_clocale.
- */
- Assert(ecpg_clocale);
- stmt->oldlocale = uselocale(ecpg_clocale);
- if (stmt->oldlocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#else
-#ifdef WIN32
- stmt->oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
- if (stmt->oldthreadlocale == -1)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-#endif
- stmt->oldlocale = ecpg_strdup(setlocale(LC_NUMERIC, NULL), lineno);
- if (stmt->oldlocale == NULL)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- setlocale(LC_NUMERIC, "C");
-#endif
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
@@ -2216,19 +2176,6 @@ ecpg_do_epilogue(struct statement *stmt)
if (stmt == NULL)
return;
-#ifdef HAVE_USELOCALE
- if (stmt->oldlocale != (locale_t) 0)
- uselocale(stmt->oldlocale);
-#else
- if (stmt->oldlocale)
- {
- setlocale(LC_NUMERIC, stmt->oldlocale);
-#ifdef WIN32
- _configthreadlocale(stmt->oldthreadlocale);
-#endif
- }
-#endif
-
free_statement(stmt);
}
diff --git a/src/interfaces/ecpg/pgtypeslib/dt_common.c b/src/interfaces/ecpg/pgtypeslib/dt_common.c
index c4119ab7932..f8349b66ce9 100644
--- a/src/interfaces/ecpg/pgtypeslib/dt_common.c
+++ b/src/interfaces/ecpg/pgtypeslib/dt_common.c
@@ -1218,7 +1218,7 @@ DecodeNumber(int flen, char *str, int fmask,
return DecodeNumberField(flen, str, (fmask | DTK_DATE_M),
tmask, tm, fsec, is2digits);
- *fsec = strtod(cp, &cp);
+ *fsec = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
}
@@ -2030,7 +2030,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double frac;
- frac = strtod(cp, &cp);
+ frac = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
*fsec = frac * 1000000;
@@ -2054,7 +2054,7 @@ DecodeDateTime(char **field, int *ftype, int nf,
{
double time;
- time = strtod(cp, &cp);
+ time = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0')
return -1;
diff --git a/src/interfaces/ecpg/pgtypeslib/interval.c b/src/interfaces/ecpg/pgtypeslib/interval.c
index 936a6883816..155c6cc7770 100644
--- a/src/interfaces/ecpg/pgtypeslib/interval.c
+++ b/src/interfaces/ecpg/pgtypeslib/interval.c
@@ -60,7 +60,7 @@ ParseISO8601Number(const char *str, char **endptr, int *ipart, double *fpart)
if (!(isdigit((unsigned char) *str) || *str == '-' || *str == '.'))
return DTERR_BAD_FORMAT;
errno = 0;
- val = strtod(str, endptr);
+ val = strtod_l(str, endptr, PG_C_LOCALE);
/* did we not see anything that looks like a double? */
if (*endptr == str || errno != 0)
return DTERR_BAD_FORMAT;
@@ -455,7 +455,7 @@ DecodeInterval(char **field, int *ftype, int nf, /* int range, */
else if (*cp == '.')
{
errno = 0;
- fval = strtod(cp, &cp);
+ fval = strtod_l(cp, &cp, PG_C_LOCALE);
if (*cp != '\0' || errno != 0)
return DTERR_BAD_FORMAT;
diff --git a/src/interfaces/ecpg/pgtypeslib/numeric.c b/src/interfaces/ecpg/pgtypeslib/numeric.c
index 35e7b92da40..49938543d03 100644
--- a/src/interfaces/ecpg/pgtypeslib/numeric.c
+++ b/src/interfaces/ecpg/pgtypeslib/numeric.c
@@ -1455,7 +1455,7 @@ numericvar_to_double(numeric *var, double *dp)
* strtod does not reset errno to 0 in case of success.
*/
errno = 0;
- val = strtod(tmp, &endptr);
+ val = strtod_l(tmp, &endptr, PG_C_LOCALE);
if (errno == ERANGE)
{
free(tmp);
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..8b864cccce3 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -41,6 +41,7 @@ OBJS = \
bsearch_arg.o \
chklocale.o \
inet_net_ntop.o \
+ locale.o \
noblock.o \
path.o \
pg_bitutils.o \
diff --git a/src/port/locale.c b/src/port/locale.c
new file mode 100644
index 00000000000..387fb4a107a
--- /dev/null
+++ b/src/port/locale.c
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * locale.c
+ * Helper routines for thread-safe system locale usage.
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/locale.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#ifndef WIN32
+#include <pthread.h>
+#else
+#include <synchapi.h>
+#endif
+
+/* A process-lifetime singleton, allocated on first need. */
+static locale_t c_locale;
+
+#ifndef WIN32
+static void
+init_c_locale_once(void)
+{
+ c_locale = newlocale(LC_ALL, "C", NULL);
+}
+#else
+static BOOL
+init_c_locale_once(PINIT_ONCE once, PVOID parameter, PVOID *context)
+{
+ c_locale = _create_locale(LC_ALL, "C");
+ return true;
+}
+#endif
+
+/*
+ * Access a process-lifetime singleton locale_t object. Use the macro
+ * PG_C_LOCALE instead of calling this directly, as it can skip the function
+ * call on some systems.
+ */
+locale_t
+pg_get_c_locale(void)
+{
+ /*
+ * Fast path if already initialized. This assumes that we can read a
+ * locale_t (in practice, a pointer) without tearing in a multi-threaded
+ * program.
+ */
+ if (c_locale != (locale_t) 0)
+ return c_locale;
+
+ /* Make a locale_t. It will live until process exit. */
+ {
+#ifndef WIN32
+ static pthread_once_t once = PTHREAD_ONCE_INIT;
+
+ pthread_once(&once, init_c_locale_once);
+#else
+ static INIT_ONCE once;
+ InitOnceExecuteOnce(&once, init_c_locale_once, NULL, NULL);
+#endif
+ }
+
+ /*
+ * It's possible that the allocation of the locale failed due to low
+ * memory, and then (locale_t) 0 will be returned. Users of PG_C_LOCALE
+ * should defend against that by checking pg_ensure_c_locale() at a
+ * convenient time, so that they can treat it as a simple constant after
+ * that.
+ */
+
+ return c_locale;
+}
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..4ba8f63323e 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -5,6 +5,7 @@ pgport_sources = [
'chklocale.c',
'inet_net_ntop.c',
'noblock.c',
+ 'locale.c',
'path.c',
'pg_bitutils.c',
'pg_popcount_avx512.c',
diff --git a/src/port/snprintf.c b/src/port/snprintf.c
index f8f2018ea0c..376d5c906a0 100644
--- a/src/port/snprintf.c
+++ b/src/port/snprintf.c
@@ -109,6 +109,36 @@
#undef vprintf
#undef printf
+#if defined(FRONTEND)
+/*
+ * Frontend code might be multi-threaded. When calling the system snprintf()
+ * for floats, we have to use either the non-standard snprintf_l() variant, or
+ * save-and-restore the calling thread's locale using uselocale(), depending on
+ * availability.
+ */
+#if defined(HAVE_SNPRINTF_L)
+/* At least macOS and the BSDs have the snprintf_l() extension. */
+#define USE_SNPRINTF_L
+#elif defined(WIN32)
+/* Windows has a version with a different name and argument order. */
+#define snprintf_l(str, size, loc, format, ...) _snprintf_l(str, size, format, loc, __VA_ARGS__)
+#define USE_SNPRINTF_L
+#else
+/* We have to do a thread-safe save-and-restore around snprintf(). */
+#define NEED_USE_LOCALE
+#endif
+#else
+/*
+ * Backend code doesn't need to worry about locales here, because LC_NUMERIC is
+ * set to "C" in main() and doesn't change. Plain old snprintf() is always OK
+ * without uselocale().
+ *
+ * XXX If the backend were multithreaded, we would have to be 100% certain that
+ * no one is calling setlocale() concurrently, even transiently, to be able to
+ * keep this small optimization.
+ */
+#endif
+
/*
* Info about where the formatted output is going.
*
@@ -1220,6 +1250,9 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
* according to == but not according to memcmp.
*/
static const double dzero = 0.0;
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
if (adjust_sign((value < 0.0 ||
(value == 0.0 &&
@@ -1241,7 +1274,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '*';
fmt[3] = type;
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, prec, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, prec, value);
+#endif
}
else
{
@@ -1250,6 +1287,11 @@ fmtfloat(double value, char type, int forcesign, int leftjust,
fmt[2] = '\0';
vallen = snprintf(convert, sizeof(convert), fmt, value);
}
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
goto fail;
@@ -1372,12 +1414,25 @@ pg_strfromd(char *str, size_t count, int precision, double value)
}
else
{
+#ifdef NEED_USE_LOCALE
+ locale_t save_locale = uselocale(PG_C_LOCALE);
+#endif
+
fmt[0] = '%';
fmt[1] = '.';
fmt[2] = '*';
fmt[3] = 'g';
fmt[4] = '\0';
+#ifdef USE_SNPRINTF_L
+ vallen = snprintf_l(convert, sizeof(convert), PG_C_LOCALE, fmt, precision, value);
+#else
vallen = snprintf(convert, sizeof(convert), fmt, precision, value);
+#endif
+
+#ifdef NEED_USE_LOCALE
+ uselocale(save_locale);
+#endif
+
if (vallen < 0)
{
target.failed = true;
base-commit: b92c03342dcd69d259262b06b5c290e27249cb11
--
2.48.1
v7-0002-Fixup-fix-compiler-warnings-if-LC_C_LOCALE-exists.patch (text/plain, 652B) download
From d61d358997ce6c15a891588953b76e174e3041ac Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Sun, 9 Feb 2025 07:49:00 +0100
Subject: [PATCH v7 2/3] Fixup: fix compiler warnings if LC_C_LOCALE exists
---
src/port/locale.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/port/locale.c b/src/port/locale.c
index 387fb4a107a..b3d48628915 100644
--- a/src/port/locale.c
+++ b/src/port/locale.c
@@ -16,6 +16,8 @@
#include "c.h"
+#ifndef LC_C_LOCALE
+
#ifndef WIN32
#include <pthread.h>
#else
@@ -78,3 +80,5 @@ pg_get_c_locale(void)
return c_locale;
}
+
+#endif /* not LC_C_LOCALE */
--
2.48.1
v7-0003-Fixup-sort-some-lists-better.patch (text/plain, 2.5K) download
From edf9e527e82f1997396cd3fde873e82c25aade37 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Sun, 9 Feb 2025 07:52:45 +0100
Subject: [PATCH v7 3/3] Fixup: sort some lists better
---
configure | 2 +-
configure.ac | 2 +-
meson.build | 2 +-
src/port/meson.build | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 017e63598c1..bbe4731bcf5 100755
--- a/configure
+++ b/configure
@@ -14934,7 +14934,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range elf_aux_info getauxval getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strtod_l strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range elf_aux_info getauxval getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast snprintf_l strchrnul strsignal strtod_l syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.ac b/configure.ac
index b06a9cd1080..d636d4678b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1719,9 +1719,9 @@ AC_CHECK_FUNCS(m4_normalize([
setproctitle
setproctitle_fast
snprintf_l
- strtod_l
strchrnul
strsignal
+ strtod_l
syncfs
sync_file_range
uselocale
diff --git a/meson.build b/meson.build
index e03b5083f27..73ccbd4cee2 100644
--- a/meson.build
+++ b/meson.build
@@ -2649,7 +2649,6 @@ func_checks = [
['shm_unlink', {'dependencies': [rt_dep], 'define': false}],
['shmget', {'dependencies': [cygipc_dep], 'define': false}],
['snprintf_l'],
- ['strtod_l'],
['socket', {'dependencies': [socket_dep], 'define': false}],
['strchrnul'],
['strerror_r', {'dependencies': [thread_dep]}],
@@ -2658,6 +2657,7 @@ func_checks = [
['strnlen'],
['strsep'],
['strsignal'],
+ ['strtod_l'],
['sync_file_range'],
['syncfs'],
['uselocale'],
diff --git a/src/port/meson.build b/src/port/meson.build
index 4ba8f63323e..1656328f210 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -4,8 +4,8 @@ pgport_sources = [
'bsearch_arg.c',
'chklocale.c',
'inet_net_ntop.c',
- 'noblock.c',
'locale.c',
+ 'noblock.c',
'path.c',
'pg_bitutils.c',
'pg_popcount_avx512.c',
--
2.48.1
* Re: On non-Windows, hard depend on uselocale(3)
@ 2025-03-28 15:30 Peter Eisentraut <peter@eisentraut.org>
parent: Peter Eisentraut <peter@eisentraut.org>
1 sibling, 1 reply; 49+ messages in thread
From: Peter Eisentraut @ 2025-03-28 15:30 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 09.02.25 08:32, Peter Eisentraut wrote:
> Checking the status of this thread ...
>
> The patches that removed the configure checks for _configthreadlocale(),
> and related cleanup, have been committed.
>
> The original patch to "Tidy up locale thread safety in ECPG library" is
> still outstanding.
>
> Attached is a rebased version, based on the posted v6, with a couple of
> small fixups from me.
>
> I haven't re-reviewed it yet, but from scanning the discussion, it looks
> close to done.
After staring at this a few more times, I figured it was ready enough
and I committed it.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2025-03-28 16:14 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 49+ messages in thread
From: Masahiko Sawada @ 2025-03-28 16:14 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Fri, Mar 28, 2025 at 8:30 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>
> On 09.02.25 08:32, Peter Eisentraut wrote:
> > Checking the status of this thread ...
> >
> > The patches that removed the configure checks for _configthreadlocale(),
> > and related cleanup, have been committed.
> >
> > The original patch to "Tidy up locale thread safety in ECPG library" is
> > still outstanding.
> >
> > Attached is a rebased version, based on the posted v6, with a couple of
> > small fixups from me.
> >
> > I haven't re-reviewed it yet, but from scanning the discussion, it looks
> > close to done.
>
> After staring at this a few more times, I figured it was ready enough
> and I committed it.
It seems that some bf animals such as jackdaw are unhappy with this
commit[0][1]. I also got the same 'undefined reference to symbol
error' locally when building test_json_parser.
Regards,
[0] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=snakefly&dt=2025-03-28%2015%3A29%3A04
[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jackdaw&dt=2025-03-28%2015%3A30%3A44
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2025-03-28 16:32 Peter Eisentraut <peter@eisentraut.org>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 2 replies; 49+ messages in thread
From: Peter Eisentraut @ 2025-03-28 16:32 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 28.03.25 17:14, Masahiko Sawada wrote:
> On Fri, Mar 28, 2025 at 8:30 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>>
>> On 09.02.25 08:32, Peter Eisentraut wrote:
>>> Checking the status of this thread ...
>>>
>>> The patches that removed the configure checks for _configthreadlocale(),
>>> and related cleanup, have been committed.
>>>
>>> The original patch to "Tidy up locale thread safety in ECPG library" is
>>> still outstanding.
>>>
>>> Attached is a rebased version, based on the posted v6, with a couple of
>>> small fixups from me.
>>>
>>> I haven't re-reviewed it yet, but from scanning the discussion, it looks
>>> close to done.
>>
>> After staring at this a few more times, I figured it was ready enough
>> and I committed it.
>
> It seems that some bf animals such as jackdaw are unhappy with this
> commit[0][1]. I also got the same 'undefined reference to symbol
> error' locally when building test_json_parser.
Yeah, looks like we'll have to revert this for now. But I'm confused,
because I don't see any clear pattern for which platforms or
configurations it's failing and for which not.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2025-03-28 20:34 Peter Eisentraut <peter@eisentraut.org>
parent: Peter Eisentraut <peter@eisentraut.org>
1 sibling, 0 replies; 49+ messages in thread
From: Peter Eisentraut @ 2025-03-28 20:34 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 28.03.25 17:32, Peter Eisentraut wrote:
> On 28.03.25 17:14, Masahiko Sawada wrote:
>> On Fri, Mar 28, 2025 at 8:30 AM Peter Eisentraut
>> <peter@eisentraut.org> wrote:
>>>
>>> On 09.02.25 08:32, Peter Eisentraut wrote:
>>>> Checking the status of this thread ...
>>>>
>>>> The patches that removed the configure checks for
>>>> _configthreadlocale(),
>>>> and related cleanup, have been committed.
>>>>
>>>> The original patch to "Tidy up locale thread safety in ECPG library" is
>>>> still outstanding.
>>>>
>>>> Attached is a rebased version, based on the posted v6, with a couple of
>>>> small fixups from me.
>>>>
>>>> I haven't re-reviewed it yet, but from scanning the discussion, it
>>>> looks
>>>> close to done.
>>>
>>> After staring at this a few more times, I figured it was ready enough
>>> and I committed it.
>>
>> It seems that some bf animals such as jackdaw are unhappy with this
>> commit[0][1]. I also got the same 'undefined reference to symbol
>> error' locally when building test_json_parser.
>
> Yeah, looks like we'll have to revert this for now. But I'm confused,
> because I don't see any clear pattern for which platforms or
> configurations it's failing and for which not.
reverted
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2025-03-29 00:01 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
1 sibling, 0 replies; 49+ messages in thread
From: Masahiko Sawada @ 2025-03-29 00:01 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Fri, Mar 28, 2025 at 9:32 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>
> On 28.03.25 17:14, Masahiko Sawada wrote:
> > On Fri, Mar 28, 2025 at 8:30 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> >>
> >> On 09.02.25 08:32, Peter Eisentraut wrote:
> >>> Checking the status of this thread ...
> >>>
> >>> The patches that removed the configure checks for _configthreadlocale(),
> >>> and related cleanup, have been committed.
> >>>
> >>> The original patch to "Tidy up locale thread safety in ECPG library" is
> >>> still outstanding.
> >>>
> >>> Attached is a rebased version, based on the posted v6, with a couple of
> >>> small fixups from me.
> >>>
> >>> I haven't re-reviewed it yet, but from scanning the discussion, it looks
> >>> close to done.
> >>
> >> After staring at this a few more times, I figured it was ready enough
> >> and I committed it.
> >
> > It seems that some bf animals such as jackdaw are unhappy with this
> > commit[0][1]. I also got the same 'undefined reference to symbol
> > error' locally when building test_json_parser.
>
> Yeah, looks like we'll have to revert this for now. But I'm confused,
> because I don't see any clear pattern for which platforms or
> configurations it's failing and for which not.
>
Not sure it would help the investigation but I got the linker error
when building with 'make' but not with 'meson'. Looking at the build
logs, when building test_json_parser with meson, it adds -lpthread as
follows:
% /home/masahiko/work/gcc/12.2.0/bin/gcc -v -o
src/test/modules/test_json_parser/test_json_parser_incremental_shlib
src/test/modules/test_json_parser/test_json_parser_incremental_shlib.p/test_json_parser_incremental.c.o
-Wl,--as-needed -Wl,--no-undefined
'-Wl,-rpath,$ORIGIN/../../../interfaces/libpq:/lib/../lib64'
-Wl,-rpath-link,/lib/../lib64
-Wl,-rpath-link,/home/masahiko/pgsql/source/dev_master/build/src/interfaces/libpq
-Wl,--start-group src/common/libpgcommon_excluded_shlib.a
src/common/libpgcommon_shlib.a
src/common/libpgcommon_shlib_config_info.a
src/common/libpgcommon_shlib_ryu.a src/port/libpgport_shlib.a
src/interfaces/libpq/libpq.so.5.18 -lm -ldl -pthread -lrt
/usr/lib64/libz.so /lib/../lib64/libzstd.so /usr/lib64/liblz4.so
/usr/lib64/libssl.so /usr/lib64/libcrypto.so -Wl,--end-group
whereas with 'make' it doesn't:
% gcc -v -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Werror=vla -Wendif-labels
-Wmissing-format-attribute -Wimplicit-fallthrough=3
-Wcast-function-type -Wshadow=compatible-local -Wformat-security
-fno-strict-aliasing -fwrapv -fexcess-precision=standard
-Wno-format-truncation -Wno-stringop-truncation -g -g -O0
test_json_parser_incremental.o -L../../../../src/port
-L../../../../src/common -Wl,--as-needed
-Wl,-rpath,'/home/masahiko/pgsql/master/lib',--enable-new-dtags
-lpgcommon_excluded_shlib -L../../../../src/common -lpgcommon_shlib
-L../../../../src/port -lpgport_shlib
-L../../../../src/interfaces/libpq -lpq -o
test_json_parser_incremental_shlib
FYI the following change fixed the issue in my local env:
inline patch 1: src/test/modules/test_json_parser/Makefile (+0/-0 lines) download inline-1.patch
--- a/src/test/modules/test_json_parser/Makefile
+++ b/src/test/modules/test_json_parser/Makefile
@@ -27,7 +27,7 @@ test_json_parser_incremental$(X):
* Re: On non-Windows, hard depend on uselocale(3)
@ 2025-12-12 18:07 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter@eisentraut.org>
2 siblings, 0 replies; 49+ messages in thread
From: Tom Lane @ 2025-12-12 18:07 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Tristan Partin <tristan@neon.tech>; pgsql-hackers
Peter Eisentraut <peter@eisentraut.org> writes:
> On 28.08.24 20:50, Peter Eisentraut wrote:
>> I suggest that the simplification of the xlocale.h configure tests could
>> be committed separately. This would also be useful independent of this,
>> and it's a sizeable chunk of this patch.
> To keep this moving along a bit, I have extracted this part and
> committed it separately. I had to make a few small tweaks, e.g., there
> was no check for xlocale.h in configure.ac, and the old
> xlocale.h-including stanza could be removed from chklocale.h. Let's see
> how this goes.
For the archives' sake --- I discovered today during a "git bisect"
session that commits between 35eeea623 ("Use thread-safe
nl_langinfo_l(), not nl_langinfo()") and 9c2a6c5a5 ("Simplify checking
for xlocale.h", the commit Peter refers to here) fail to build on
current macOS (26/Tahoe):
chklocale.c:330:8: error: call to undeclared function 'nl_langinfo_l'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
330 | sys = nl_langinfo_l(CODESET, loc);
| ^
chklocale.c:330:8: note: did you mean 'nl_langinfo'?
/Library/Developer/CommandLineTools/SDKs/MacOSX26.1.sdk/usr/include/_langinfo.h:116:20: note: 'nl_langinfo' declared here
116 | char *_LIBC_CSTR nl_langinfo(nl_item);
| ^
chklocale.c:330:6: error: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion]
330 | sys = nl_langinfo_l(CODESET, loc);
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
This happens because nl_langinfo_l() is declared in xlocale.h,
but chklocale.c elects not to #include that, evidently because
it's no longer needed to obtain typedef locale_t.
AFAICS there's nothing we can do about this retroactively;
it'll be a more or less permanent landmine for bisecting on macOS.
Fortunately it's not too difficult to work around, you can just do
inline patch 1: src/port/chklocale.c (+0/-2 lines) download inline-1.patch
diff --git a/src/port/chklocale.c b/src/port/chklocale.c
index 9506cd87ed8..0e35e0cf55f 100644
--- a/src/port/chklocale.c
+++ b/src/port/chklocale.c
@@ -23,9 +23,7 @@
#include <langinfo.h>
#endif
-#ifdef LOCALE_T_IN_XLOCALE
#include <xlocale.h>
-#endif
#include "mb/pg_wchar.h"
* Re: On non-Windows, hard depend on uselocale(3)
@ 2026-02-04 09:49 Álvaro Herrera <alvherre@kurilemu.de>
parent: Peter Eisentraut <peter@eisentraut.org>
1 sibling, 1 reply; 49+ messages in thread
From: Álvaro Herrera @ 2026-02-04 09:49 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2025-Feb-09, Peter Eisentraut wrote:
> Checking the status of this thread ...
>
> The patches that removed the configure checks for _configthreadlocale(), and
> related cleanup, have been committed.
>
> The original patch to "Tidy up locale thread safety in ECPG library" is
> still outstanding.
>
> Attached is a rebased version, based on the posted v6, with a couple of
> small fixups from me.
>
> I haven't re-reviewed it yet, but from scanning the discussion, it looks
> close to done.
This was committed and shortly reverted (8e993bff5326 and 3c8e463b0d88)
but apparently the problem is just some Make/meson trickery that needs
to be addressed, per Sawada-san comments downthread. Is this on
anybody's plate?
It corresponds to this stale CF entry,
https://commitfest.postgresql.org/patch/5166/
Thanks
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2026-02-04 13:58 Peter Eisentraut <peter@eisentraut.org>
parent: Álvaro Herrera <alvherre@kurilemu.de>
0 siblings, 1 reply; 49+ messages in thread
From: Peter Eisentraut @ 2026-02-04 13:58 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 04.02.26 10:49, Álvaro Herrera wrote:
> On 2025-Feb-09, Peter Eisentraut wrote:
>
>> Checking the status of this thread ...
>>
>> The patches that removed the configure checks for _configthreadlocale(), and
>> related cleanup, have been committed.
>>
>> The original patch to "Tidy up locale thread safety in ECPG library" is
>> still outstanding.
>>
>> Attached is a rebased version, based on the posted v6, with a couple of
>> small fixups from me.
>>
>> I haven't re-reviewed it yet, but from scanning the discussion, it looks
>> close to done.
>
> This was committed and shortly reverted (8e993bff5326 and 3c8e463b0d88)
> but apparently the problem is just some Make/meson trickery that needs
> to be addressed, per Sawada-san comments downthread. Is this on
> anybody's plate?
>
> It corresponds to this stale CF entry,
> https://commitfest.postgresql.org/patch/5166/
My recollection is that I didn't at the time find any common pattern
among the failing buildfarm members. So the only way to move forward
here would be to commit it again and then debug it live via the buildfarm.
^ permalink raw reply [nested|flat] 49+ messages in thread
* Re: On non-Windows, hard depend on uselocale(3)
@ 2026-02-04 19:51 Álvaro Herrera <alvherre@kurilemu.de>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 0 replies; 49+ messages in thread
From: Álvaro Herrera @ 2026-02-04 19:51 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; Andrew Dunstan <andrew@dunslane.net>; Tristan Partin <tristan@neon.tech>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2026-Feb-04, Peter Eisentraut wrote:
> > It corresponds to this stale CF entry,
> > https://commitfest.postgresql.org/patch/5166/
>
> My recollection is that I didn't at the time find any common pattern among
> the failing buildfarm members. So the only way to move forward here would
> be to commit it again and then debug it live via the buildfarm.
Did you try adding $(PTHREAD_CFLAGS) or $(PTHREAD_LIBS) to the compile
line for test_json_parser?
I see that the failure in jackdaw (snapshot 2025-03-28 15:30:44)
was
ccache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -O2 -I. -I. -I../../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -c -o test_json_parser_incremental.o test_json_parser_incremental.c
ccache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -O2 test_json_parser_incremental.o -L../../../../src/port -L../../../../src/common -Wl,--as-needed -Wl,-rpath,'/home/debian/17-jackdaw/buildroot/HEAD/inst/lib',--enable-new-dtags -lpgcommon -lpgport -lxslt -lxml2 -lssl -lcrypto -lgssapi_krb5 -lz -lreadline -lpthread -lrt -ldl -lm -o test_json_parser_incremental
ccache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -O2 test_json_parser_incremental.o -L../../../../src/port -L../../../../src/common -Wl,--as-needed -Wl,-rpath,'/home/debian/17-jackdaw/buildroot/HEAD/inst/lib',--enable-new-dtags -lpgcommon_excluded_shlib -L../../../../src/common -lpgcommon_shlib -L../../../../src/port -lpgport_shlib -L../../../../src/interfaces/libpq -lpq -o test_json_parser_incremental_shlib
/usr/bin/ld: ../../../../src/port/libpgport_shlib.a(locale_shlib.o): undefined reference to symbol 'pthread_once@@GLIBC_2.17'
/usr/bin/ld: /lib/aarch64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:30: test_json_parser_incremental_shlib] Error 1
But if you look at how jackdaw compiles, say, ecpglib, there's
ccache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -O2 -pthread -D_REENTRANT -D_THREAD_SAFE -I../include -I../../../../src/interfaces/ecpg/include -I. -I. -I../../../../src/interfaces/ecpg/ecpglib -I../../../../src/interfaces/libpq -I../../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -c -o keywords.o keywords.c
note that it has "-pthread", which is not present in the
test_json_parser compile/link. Does this matter because
libpgport_shlib.a needs that? Did maybe test_json_parser not previously
need this locale thingy?
I suspect this is a problem in test_json_parser that somehow got tickled
by your patch, not a problem in your patch.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
¡Ay, ay, ay! Con lo mucho que yo lo quería (bis)
se fue de mi vera ... se fue para siempre, pa toíta ... pa toíta la vida
¡Ay Camarón! ¡Ay Camarón! (Paco de Lucía)
^ permalink raw reply [nested|flat] 49+ messages in thread
end of thread, other threads:[~2026-02-04 19:51 UTC | newest]
Thread overview: 49+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-11-15 10:27 On non-Windows, hard depend on uselocale(3) Tristan Partin <tristan@neon.tech>
2023-11-15 17:45 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-15 18:38 ` Thomas Munro <thomas.munro@gmail.com>
2023-11-15 20:51 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-15 21:08 ` Thomas Munro <thomas.munro@gmail.com>
2023-11-15 21:17 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-15 22:40 ` Thomas Munro <thomas.munro@gmail.com>
2023-11-15 23:06 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-16 19:57 ` Thomas Munro <thomas.munro@gmail.com>
2023-11-17 18:18 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-17 22:58 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-19 22:00 ` Thomas Munro <thomas.munro@gmail.com>
2023-11-19 22:36 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-20 00:00 ` Thomas Munro <thomas.munro@gmail.com>
2023-11-20 16:40 ` Tom Lane <tgl@sss.pgh.pa.us>
2024-08-10 01:29 ` Thomas Munro <thomas.munro@gmail.com>
2024-08-10 03:48 ` Thomas Munro <thomas.munro@gmail.com>
2024-08-10 03:52 ` Thomas Munro <thomas.munro@gmail.com>
2024-08-10 22:11 ` Thomas Munro <thomas.munro@gmail.com>
2024-08-13 23:17 ` Tristan Partin <tristan@partin.io>
2024-08-15 08:49 ` Thomas Munro <thomas.munro@gmail.com>
2024-08-28 18:50 ` Peter Eisentraut <peter@eisentraut.org>
2024-10-01 12:04 ` Peter Eisentraut <peter@eisentraut.org>
2024-11-14 01:54 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-14 07:48 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-14 09:54 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-11-14 12:53 ` Peter Eisentraut <peter@eisentraut.org>
2024-11-20 09:00 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-20 23:30 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-21 07:38 ` Peter Eisentraut <peter@eisentraut.org>
2024-11-22 21:32 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-25 00:42 ` Michael Paquier <michael@paquier.xyz>
2024-11-25 00:57 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-26 16:23 ` Peter Eisentraut <peter@eisentraut.org>
2024-11-26 20:24 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-09 07:32 ` Peter Eisentraut <peter@eisentraut.org>
2025-03-28 15:30 ` Peter Eisentraut <peter@eisentraut.org>
2025-03-28 16:14 ` Masahiko Sawada <sawada.mshk@gmail.com>
2025-03-28 16:32 ` Peter Eisentraut <peter@eisentraut.org>
2025-03-28 20:34 ` Peter Eisentraut <peter@eisentraut.org>
2025-03-29 00:01 ` Masahiko Sawada <sawada.mshk@gmail.com>
2026-02-04 09:49 ` Álvaro Herrera <alvherre@kurilemu.de>
2026-02-04 13:58 ` Peter Eisentraut <peter@eisentraut.org>
2026-02-04 19:51 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-12-12 18:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-18 00:03 ` Andres Freund <andres@anarazel.de>
2023-11-15 18:42 ` Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
2023-11-15 19:04 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-11-15 19:16 ` Thomas Munro <thomas.munro@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox