pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedpltcl crash on recent macOS
11+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2022-06-23 41a2b56ae8 PL/Tcl: Don't link with -lc explicitly
* pltcl crash on recent macOS
@ 2022-06-13 06:53 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 1 reply; 11+ messages in thread
From: Peter Eisentraut @ 2022-06-13 06:53 UTC (permalink / raw)
To: pgsql-hackers
A little while ago, the pltcl tests starting crashing for me on macOS.
I don't know what had changed, but I suspect it was either an operating
system update or something like an xcode update.
Here is a backtrace:
* frame #0: 0x00007ff7b0e61853
frame #1: 0x00007ff803a28751 libsystem_c.dylib`hash_search + 215
frame #2: 0x0000000110357700
pltcl.so`compile_pltcl_function(fn_oid=16418, tgreloid=0,
is_event_trigger=false, pltrusted=true) at pltcl.c:1418:13
frame #3: 0x0000000110355d50
pltcl.so`pltcl_func_handler(fcinfo=0x00007fb6f1817028,
call_state=0x00007ff7b0e61b80, pltrusted=true) at pltcl.c:814:12
...
Note that the hash_search call goes into some system library, not postgres.
The command to link pltcl is:
gcc ... -ltcl8.6 -lz -lpthread -framework CoreFoundation -lc
-bundle_loader ../../../src/backend/postgres
Notice the -lc in there. If I remove that, it works again.
The -lc is explicitly added in src/pl/tcl/Makefile, so it's our own
doing. I tracked this back, and it's been moved and rearranged in that
makefile a number of time. The original addition was
commit e3909672f12e0ddf3e202b824fda068ad2195ef2
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Dec 14 00:46:49 1998
Build pltcl.so correctly on platforms that want dependent
shared libraries to be listed in the link command.
Has anyone else seen this?
Note, I'm using the tcl-tk package from Homebrew. The tcl installation
provided by macOS itself no longer appears to work for linking against.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-13 11:27 Thomas Munro <thomas.munro@gmail.com>
parent: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 2 replies; 11+ messages in thread
From: Thomas Munro @ 2022-06-13 11:27 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: pgsql-hackers
On Mon, Jun 13, 2022 at 6:53 PM Peter Eisentraut
<peter.eisentraut@enterprisedb.com> wrote:
> frame #1: 0x00007ff803a28751 libsystem_c.dylib`hash_search + 215
> frame #2: 0x0000000110357700
> pltcl.so`compile_pltcl_function(fn_oid=16418, tgreloid=0,
Hmm, I can’t reproduce that…. although that symbol is present in my
libSystem.B.dylib according to dlsym() and callable from a simple
program not linked to anything else, pltcl.so is apparently reaching
postgres’s hash_search for me, based on the fact that make -C
src/pl/tcl check succeeds and nm -m on pltcl.so shows it as "from
executable". It would be interesting to see what nm -m shows for you.
Archeological note: That hash_search stuff, header <strhash.h>, seems
to have been copied from ancient FreeBSD before it was dropped
upstream for the crime of polluting the global symbol namespace with
junk[1]. It's been languishing in Apple's libc for at least 19
years[2], though, so I'm not sure why it's showing up suddenly as a
problem for you now.
> Note, I'm using the tcl-tk package from Homebrew. The tcl installation
> provided by macOS itself no longer appears to work for linking against.
I’m using tcl 8.6.12 installed by MacPorts on macOS 12.4, though, hmm,
SDK 12.3. I see the explicit -lc when building pltcl.so, and I see
that libSystem.B.dylib is explicitly mentioned here, whether or not I
have -lc:
% otool -L ./tmp_install/Users/tmunro/install/lib/postgresql/pltcl.so
./tmp_install/Users/tmunro/install/lib/postgresql/pltcl.so:
/opt/local/lib/libtcl8.6.dylib (compatibility version 8.6.0, current
version 8.6.12)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1311.100.3)
Here’s the complete link line:
ccache cc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Werror=vla
-Werror=unguarded-availability-new -Wendif-labels
-Wmissing-format-attribute -Wcast-function-type -Wformat-security
-fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument
-Wno-compound-token-split-by-macro -g -O0 -bundle -multiply_defined
suppress -o pltcl.so pltcl.o -L../../../src/port
-L../../../src/common -isysroot
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk
-Wl,-dead_strip_dylibs -L/opt/local/lib -ltcl8.6 -lz -lpthread
-framework CoreFoundation -lc -bundle_loader
../../../src/backend/postgres
[1] https://github.com/freebsd/freebsd-src/commit/dc196afb2e58dd05cd66e2da44872bb3d619910f
[2] https://github.com/apple-open-source-mirror/Libc/blame/master/stdlib/FreeBSD/strhash.c
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-13 16:01 Tom Lane <tgl@sss.pgh.pa.us>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 1 reply; 11+ messages in thread
From: Tom Lane @ 2022-06-13 16:01 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; pgsql-hackers
Thomas Munro <thomas.munro@gmail.com> writes:
> On Mon, Jun 13, 2022 at 6:53 PM Peter Eisentraut
> <peter.eisentraut@enterprisedb.com> wrote:
>> frame #1: 0x00007ff803a28751 libsystem_c.dylib`hash_search + 215
>> frame #2: 0x0000000110357700
>> pltcl.so`compile_pltcl_function(fn_oid=16418, tgreloid=0,
> Hmm, I can’t reproduce that….
I can't either, although I'm using the macOS-provided Tcl code,
which still works fine for me. (I grant that Apple might desupport
that someday, but they haven't yet.) sifaka and longfin aren't
unhappy either; although sifaka is close to identical to my laptop.
Having said that, I wonder whether the position of the -bundle_loader
switch in the command line is relevant to which way the hash_search
reference is resolved. Seems like we could put it in front of the
various -l options if that'd help.
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-13 20:21 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
parent: Thomas Munro <thomas.munro@gmail.com>
1 sibling, 2 replies; 11+ messages in thread
From: Peter Eisentraut @ 2022-06-13 20:21 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers
On 13.06.22 13:27, Thomas Munro wrote:
> On Mon, Jun 13, 2022 at 6:53 PM Peter Eisentraut
> <peter.eisentraut@enterprisedb.com> wrote:
>> frame #1: 0x00007ff803a28751 libsystem_c.dylib`hash_search + 215
>> frame #2: 0x0000000110357700
>> pltcl.so`compile_pltcl_function(fn_oid=16418, tgreloid=0,
>
> Hmm, I can’t reproduce that…. although that symbol is present in my
> libSystem.B.dylib according to dlsym() and callable from a simple
> program not linked to anything else, pltcl.so is apparently reaching
> postgres’s hash_search for me, based on the fact that make -C
> src/pl/tcl check succeeds and nm -m on pltcl.so shows it as "from
> executable". It would be interesting to see what nm -m shows for you.
...
(undefined) external _get_call_result_type (from executable)
(undefined) external _getmissingattr (from executable)
(undefined) external _hash_create (from libSystem)
(undefined) external _hash_search (from libSystem)
...
> I’m using tcl 8.6.12 installed by MacPorts on macOS 12.4, though, hmm,
> SDK 12.3. I see the explicit -lc when building pltcl.so, and I see
> that libSystem.B.dylib is explicitly mentioned here, whether or not I
> have -lc:
>
> % otool -L ./tmp_install/Users/tmunro/install/lib/postgresql/pltcl.so
> ./tmp_install/Users/tmunro/install/lib/postgresql/pltcl.so:
> /opt/local/lib/libtcl8.6.dylib (compatibility version 8.6.0, current
> version 8.6.12)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1311.100.3)
Looks the same here:
pltcl.so:
/usr/local/opt/tcl-tk/lib/libtcl8.6.dylib (compatibility version 8.6.0,
current version 8.6.12)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1311.100.3)
> Here’s the complete link line:
>
> ccache cc -Wall -Wmissing-prototypes -Wpointer-arith
> -Wdeclaration-after-statement -Werror=vla
> -Werror=unguarded-availability-new -Wendif-labels
> -Wmissing-format-attribute -Wcast-function-type -Wformat-security
> -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument
> -Wno-compound-token-split-by-macro -g -O0 -bundle -multiply_defined
> suppress -o pltcl.so pltcl.o -L../../../src/port
> -L../../../src/common -isysroot
> /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk
> -Wl,-dead_strip_dylibs -L/opt/local/lib -ltcl8.6 -lz -lpthread
> -framework CoreFoundation -lc -bundle_loader
> ../../../src/backend/postgres
The difference is that I use CC=gcc-11. I have change to CC=cc, then it
works (nm output shows "from executable"). So it's gcc that gets thrown
off by the -lc.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-13 20:24 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
1 sibling, 0 replies; 11+ messages in thread
From: Tom Lane @ 2022-06-13 20:24 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: Thomas Munro <thomas.munro@gmail.com>; pgsql-hackers
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> The difference is that I use CC=gcc-11. I have change to CC=cc, then it
> works (nm output shows "from executable"). So it's gcc that gets thrown
> off by the -lc.
Hah, that makes sense. So does changing the option order help?
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-13 21:05 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 11+ messages in thread
From: Peter Eisentraut @ 2022-06-13 21:05 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers
On 13.06.22 18:01, Tom Lane wrote:
> Having said that, I wonder whether the position of the -bundle_loader
> switch in the command line is relevant to which way the hash_search
> reference is resolved. Seems like we could put it in front of the
> various -l options if that'd help.
Switching the order of -bundle_loader and -lc did not help.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-13 21:32 Thomas Munro <thomas.munro@gmail.com>
parent: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
1 sibling, 1 reply; 11+ messages in thread
From: Thomas Munro @ 2022-06-13 21:32 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: pgsql-hackers
On Tue, Jun 14, 2022 at 8:21 AM Peter Eisentraut
<peter.eisentraut@enterprisedb.com> wrote:
> The difference is that I use CC=gcc-11. I have change to CC=cc, then it
> works (nm output shows "from executable"). So it's gcc that gets thrown
> off by the -lc.
Hrmph, I changed my CC to "ccache gcc-mp-11" (what MacPorts calls GCC
11), and I still can't reproduce the problem. I still get "(from
executable)". In your original quote you showed "gcc", not "gcc-11",
which (assuming it is found as /usr/bin/gcc) is just a little binary
that redirects to clang... trying that, this time without ccache in
the mix... and still no cigar. So something is different about GCC 11
from homebrew, or the linker invocation it produces under the covers,
or the linker it's using?
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-14 03:05 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 1 reply; 11+ messages in thread
From: Tom Lane @ 2022-06-14 03:05 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: Thomas Munro <thomas.munro@gmail.com>; pgsql-hackers
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> Switching the order of -bundle_loader and -lc did not help.
Meh. Well, it was worth a try.
I'd be okay with just dropping the -lc from pl/tcl/Makefile and seeing
what the buildfarm says. The fact that we needed it in 1998 doesn't
mean that we still need it on supported versions of Tcl; nor was it
ever anything but a hack for us to be overriding what TCL_LIBS says.
As a quick check, I tried it on prairiedog's host (which has the oldest
Tcl installation I still have in captivity), and it seemed fine.
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-14 17:58 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 0 replies; 11+ messages in thread
From: Peter Eisentraut @ 2022-06-14 17:58 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers
On 13.06.22 23:32, Thomas Munro wrote:
> Hrmph, I changed my CC to "ccache gcc-mp-11" (what MacPorts calls GCC
> 11), and I still can't reproduce the problem. I still get "(from
> executable)". In your original quote you showed "gcc", not "gcc-11",
> which (assuming it is found as /usr/bin/gcc) is just a little binary
> that redirects to clang... trying that, this time without ccache in
> the mix... and still no cigar. So something is different about GCC 11
> from homebrew, or the linker invocation it produces under the covers,
> or the linker it's using?
The original quote said "gcc" but that just me attempting to simplify.
I have now also figured out that it works with gcc-10 but not with
gcc-11 and gcc-12. For example, below are the underlying linker
invocations from gcc-10 and gcc-11. Note that some of the options are
ordered quite differently. I don't know what all of that means yet, but
it surely points to something in gcc or its packaging being the cause.
However, I think ultimately the use of -lc is an error and we should get
rid of it. This episode shows that it's very fragile in any case.
"/usr/local/Cellar/gcc@10/10.3.0/libexec/gcc/x86_64-apple-darwin20/10.3.0/collect2"
-dynamic -arch x86_64 -bundle -bundle_loader
../../../src/backend/postgres -macosx_version_min 11.4.0
-multiply_defined suppress -syslibroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk
-weak_reference_mismatches non-weak -o pltcl.so -L../../../src/port
-L../../../src/common -L/usr/local/lib -L/usr/local/opt/openldap/lib
"-L/usr/local/opt/openssl@1.1/lib" -L/usr/local/opt/readline/lib
-L/usr/local/opt/krb5/lib -L/usr/local/opt/icu4c/lib
-L/usr/local/opt/tcl-tk/lib -L/usr/local/Cellar/libxml2/2.9.14/lib
-L/usr/local/Cellar/lz4/1.9.3/lib -L/usr/local/Cellar/zstd/1.5.2/lib
-L/usr/local/Cellar/tcl-tk/8.6.12_1/lib
"-L/usr/local/Cellar/gcc@10/10.3.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.3.0"
"-L/usr/local/Cellar/gcc@10/10.3.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.3.0/../../.."
pltcl.o -dead_strip_dylibs -ltcl8.6 -lz -framework CoreFoundation -lc
-lSystem -lgcc_ext.10.5 -lgcc -lSystem -no_compact_unwind -idsym
/usr/local/Cellar/gcc/11.3.0_1/bin/../libexec/gcc/x86_64-apple-darwin21/11/collect2
-dynamic -arch x86_64 -syslibroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk
-macosx_version_min 12.4.0 -o pltcl.so -L../../../src/port
-L../../../src/common -L/usr/local/lib -L/usr/local/opt/openldap/lib
"-L/usr/local/opt/openssl@1.1/lib" -L/usr/local/opt/readline/lib
-L/usr/local/opt/krb5/lib -L/usr/local/opt/icu4c/lib
-L/usr/local/opt/tcl-tk/lib -L/usr/local/Cellar/libxml2/2.9.14/lib
-L/usr/local/Cellar/lz4/1.9.3/lib -L/usr/local/Cellar/zstd/1.5.2/lib
-L/usr/local/Cellar/tcl-tk/8.6.12_1/lib
-L/usr/local/Cellar/gcc/11.3.0_1/bin/../lib/gcc/11/gcc/x86_64-apple-darwin21/11
-L/usr/local/Cellar/gcc/11.3.0_1/bin/../lib/gcc/11/gcc
-L/usr/local/Cellar/gcc/11.3.0_1/bin/../lib/gcc/11/gcc/x86_64-apple-darwin21/11/../../..
pltcl.o -dead_strip_dylibs -ltcl8.6 -lz -lc -bundle_loader
../../../src/backend/postgres -bundle -framework CoreFoundation
-multiply_defined suppress -lemutls_w -lgcc -lSystem -no_compact_unwind
-idsym
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-20 10:36 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 11+ messages in thread
From: Peter Eisentraut @ 2022-06-20 10:36 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Thomas Munro <thomas.munro@gmail.com>; pgsql-hackers
On 14.06.22 05:05, Tom Lane wrote:
> I'd be okay with just dropping the -lc from pl/tcl/Makefile and seeing
> what the buildfarm says. The fact that we needed it in 1998 doesn't
> mean that we still need it on supported versions of Tcl; nor was it
> ever anything but a hack for us to be overriding what TCL_LIBS says.
Ok, I propose to proceed with the attached patch (with a bit more
explanation added) for the master branch (for now) and see how it goes.
Attachments:
0001-PL-Tcl-Don-t-link-with-lc-explicitly.patch (text/plain, 887B) download
From 394ab358d7437768d2c2570381f2cdcef51dc2c4 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Mon, 20 Jun 2022 12:34:13 +0200
Subject: [PATCH] PL/Tcl: Don't link with -lc explicitly
Discussion: https://www.postgresql.org/message-id/flat/a78c847a-4f79-9286-be99-e819e9e4139e%40enterprisedb.com
---
src/pl/tcl/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/pl/tcl/Makefile b/src/pl/tcl/Makefile
index 25e65189b6..314f9b2eec 100644
--- a/src/pl/tcl/Makefile
+++ b/src/pl/tcl/Makefile
@@ -15,7 +15,7 @@ override CPPFLAGS := -I. -I$(srcdir) $(TCL_INCLUDE_SPEC) $(CPPFLAGS)
# On Windows, we don't link directly with the Tcl library; see below
ifneq ($(PORTNAME), win32)
-SHLIB_LINK = $(TCL_LIB_SPEC) $(TCL_LIBS) -lc
+SHLIB_LINK = $(TCL_LIB_SPEC) $(TCL_LIBS)
endif
PGFILEDESC = "PL/Tcl - procedural language"
--
2.36.1
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pltcl crash on recent macOS
@ 2022-06-23 07:54 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
parent: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 0 replies; 11+ messages in thread
From: Peter Eisentraut @ 2022-06-23 07:54 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Thomas Munro <thomas.munro@gmail.com>; pgsql-hackers
On 20.06.22 12:36, Peter Eisentraut wrote:
> On 14.06.22 05:05, Tom Lane wrote:
>> I'd be okay with just dropping the -lc from pl/tcl/Makefile and seeing
>> what the buildfarm says. The fact that we needed it in 1998 doesn't
>> mean that we still need it on supported versions of Tcl; nor was it
>> ever anything but a hack for us to be overriding what TCL_LIBS says.
>
> Ok, I propose to proceed with the attached patch (with a bit more
> explanation added) for the master branch (for now) and see how it goes.
done
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2022-06-23 07:54 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-06-13 06:53 pltcl crash on recent macOS Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-06-13 11:27 ` Thomas Munro <thomas.munro@gmail.com>
2022-06-13 16:01 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-13 21:05 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-06-14 03:05 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-20 10:36 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-06-23 07:54 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-06-13 20:21 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-06-13 20:24 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-06-13 21:32 ` Thomas Munro <thomas.munro@gmail.com>
2022-06-14 17:58 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox