pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved
6+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2024-11-22 15ab513fed 6de14dbb36 8a9a515188 78c09bd9f9 7ca388fd0d 8aefd8b6f6 jit: Use -mno-outline-atomics for bitcode on ARM.
* BUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved
@ 2024-09-11 07:00 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 6+ messages in thread
From: PG Bug reporting form @ 2024-09-11 07:00 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: a.kozhemyakin@postgrespro.ru
The following bug has been logged on the website:
Bug reference: 18610
Logged by: Alexander Kozhemyakin
Email address: a.kozhemyakin@postgrespro.ru
PostgreSQL version: 17rc1
Operating system: ubuntu-24.04
Description:
Hi, On the master branch (3beb945d) built with llvm, the following request
fails with an error
CREATE SCHEMA addr_nsp;
CREATE FOREIGN DATA WRAPPER addr_fdw;
CREATE SERVER addr_fserv FOREIGN DATA WRAPPER addr_fdw;
CREATE TEXT SEARCH DICTIONARY addr_ts_dict (template=simple);
CREATE FOREIGN TABLE addr_nsp.genftable (a int) SERVER addr_fserv;
CREATE PUBLICATION addr_pub FOR TABLE addr_nsp.gentable;
select
pg_last_wal_receive_lsn()
from
pg_publication as ref_1
inner join pg_foreign_table as ref_2
inner join pg_aggregate as ref_4
on ( (select partclass from pg_partitioned_table limit 1) < (select
indcollation from pg_index limit 1))
on ( (select 1 from pg_stat_bgwriter limit 1) <> (select total_time from
pg_stat_xact_user_functions limit 1) )
right join pg_aggregate as ref_5 on ( ref_5.aggtransspace <= (9764) );
ERROR: relation "addr_nsp.gentable" does not exist
WARNING: failed to resolve name __aarch64_swp4_acq_rel
FATAL: fatal llvm error: Program used external function
'__aarch64_swp4_acq_rel' which could not be resolved!
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: BUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved
@ 2024-09-11 09:55 Thomas Munro <thomas.munro@gmail.com>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Munro @ 2024-09-11 09:55 UTC (permalink / raw)
To: a.kozhemyakin@postgrespro.ru; pgsql-bugs@lists.postgresql.org
On Wed, Sep 11, 2024 at 7:58 PM PG Bug reporting form
<noreply@postgresql.org> wrote:
> FATAL: fatal llvm error: Program used external function
> '__aarch64_swp4_acq_rel' which could not be resolved!
Hmm, I think it is inlining spinlock code from
pg_last_wal_receive_lsn()'s call to GetWalRcvFlushRecPtr(), and then
failing to find fallbacks for pre-ARMv8.1 systems that didn't have the
LSE atomic instructions. I wonder if there could be some mismatch in
the default -march for parts of your toolchain, so that it doesn't
link the library that has that stuff, but then the clang that built
walreceiverfuncs.bc expects it to be found. I wonder if it goes away
if you add "-mno-outline-atomics" or "-march=armv8.1-a" or
"-march=armv8-a+lse" to BITCODE_CXXFLAGS (not saying that's a fix,
just trying to understand what's happening...). Assuming you're using
GCC, maybe "gcc -Q --help=target | grep march" could show what Ubuntu
24 has set as the baseline, and "clang -### -c -x c /dev/null" might
show what clang is selecting... just ideas, I'm not sure, I don't have
such a system, I just noticed a few distros cranking up the baseline
instruction sets recently...
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: BUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved
@ 2024-09-12 02:30 a.kozhemyakin <a.kozhemyakin@postgrespro.ru>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: a.kozhemyakin @ 2024-09-12 02:30 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; pgsql-bugs@lists.postgresql.org
Thank you, the error no longer occurs after adding -mno-outline-atomics
or -march=armv8.1-a
I installed packages on debian-12 (arm64) from
https://apt.postgresql.org/pub/repos/apt, the error is repeated.
11.09.2024 16:55, Thomas Munro пишет:
> Hmm, I think it is inlining spinlock code from
> pg_last_wal_receive_lsn()'s call to GetWalRcvFlushRecPtr(), and then
> failing to find fallbacks for pre-ARMv8.1 systems that didn't have the
> LSE atomic instructions. I wonder if there could be some mismatch in
> the default -march for parts of your toolchain, so that it doesn't
> link the library that has that stuff, but then the clang that built
> walreceiverfuncs.bc expects it to be found. I wonder if it goes away
> if you add "-mno-outline-atomics" or "-march=armv8.1-a" or
> "-march=armv8-a+lse" to BITCODE_CXXFLAGS (not saying that's a fix,
> just trying to understand what's happening...). Assuming you're using
> GCC, maybe "gcc -Q --help=target | grep march" could show what Ubuntu
> 24 has set as the baseline, and "clang -### -c -x c /dev/null" might
> show what clang is selecting... just ideas, I'm not sure, I don't have
> such a system, I just noticed a few distros cranking up the baseline
> instruction sets recently...
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: BUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved
@ 2024-09-12 22:33 Thomas Munro <thomas.munro@gmail.com>
parent: a.kozhemyakin <a.kozhemyakin@postgrespro.ru>
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Munro @ 2024-09-12 22:33 UTC (permalink / raw)
To: a.kozhemyakin <a.kozhemyakin@postgrespro.ru>; +Cc: pgsql-bugs@lists.postgresql.org
On Thu, Sep 12, 2024 at 2:30 PM a.kozhemyakin
<a.kozhemyakin@postgrespro.ru> wrote:
> Thank you, the error no longer occurs after adding -mno-outline-atomics or -march=armv8.1-a
>
> I installed packages on debian-12 (arm64) from https://apt.postgresql.org/pub/repos/apt, the error is repeated.
Ahh, I see that we already have a thread proposing to add
-moutline-atomics to our main executable, over here:
https://www.postgresql.org/message-id/flat/099F69EE-51D3-4214-934A-1F28C0A1A7A7%40amazon.com
If clang has changed to assuming -moutline-atomics now (perhaps you
can see which version of clang starting adding that switch by default
the command I showed earlier, if you have a few versions of clang
around, perhaps from apt.llvm.org), then we either need to stop it
from doing that with -mno-outline-atomics, or compile our executable
(or at least llvmjit.so?) to use that too so the helper library that
defines those functions is linked in. The people who developed that
stuff are, I think, interested in using the faster LSE stuff on modern
phones, while still being able to run apps on the older phones that
are still in circulation, but of course we have the same issue: we
want our stuff to use LSE on Ampere/Grativon etc server chips, while
still being able to run on Raspberry Pi 4 etc. And that'll speed up
many things related to locking in the server itself, not only in this
fairly obscure JIT-inlined thing.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: BUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved
@ 2024-11-08 05:15 Thomas Munro <thomas.munro@gmail.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Munro @ 2024-11-08 05:15 UTC (permalink / raw)
To: a.kozhemyakin <a.kozhemyakin@postgrespro.ru>; +Cc: pgsql-bugs@lists.postgresql.org
While hacking on ARM problems this week, I remembered this one.
I see now that this is not new: all Debian clang versions at least as
far back as 14, including the default one on Debian stable, have it on
by default. That is out of sync with their gcc package. So unless
you also set CC to clang, you get:
postgres=# set jit_inline_above_cost = 0;
SET
postgres=# set jit_above_cost = 0;
SET
postgres=# select pg_last_wal_receive_lsn();
WARNING: failed to resolve name __aarch64_swp4_acq_rel
FATAL: fatal llvm error: Program used external function
'__aarch64_swp4_acq_rel' which could not be resolved!
I suspect some other distros might have turned this on in their gcc
too but I am not sure which ones.
Some options:
1. Somehow figure out if $CC is using -moutline-atomics, then make
BITCODE_CFLAGS agree. I haven't thought of a nice way to do it...
2. Tell $CC to use it. That was already kinda rejected in that other
thread, leaving this up to the toolchain people. If we ever change
our minds on that, it I guess it would be master only, so not help
with this problem.
3. Disable it in BITCODE_CFLAGS, whether or not $CC is using it. It
might be fairly rare for inlined functions to use relevant atomic ops,
so perhaps it's not such a big deal to be conservative here for now.
Here's a patch for approach #3. This is low-urgency and a code freeze
begins in a day or two, so I'm *not* proposing it for next week's
release.
Attachments:
0001-jit-Use-mno-outline-atomics-for-bitcode.patch (application/octet-stream, 6.8K) download
From 6d038308cf73a4281611b5ce07417e8993cce349 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 8 Nov 2024 02:31:15 +0000
Subject: [PATCH] jit: Use -mno-outline-atomics for bitcode.
If postgres was built by gcc not using -moutline-atomics, and the
equivalent bitcode was built by clang using -moutput-atomics, then the
bitcode could contain references to outline atomics symbols that are not
available in the executable or anywhere else.
At least Debian and Ubuntu are known to ship gcc and clang compilers
that differ on the use of outline atomics by default.
Example of resulting failure, when trying to inline .bc code:
postgres=# set jit_inline_above_cost = 0;
SET
postgres=# set jit_above_cost = 0;
SET
postgres=# select pg_last_wal_receive_lsn();
WARNING: failed to resolve name __aarch64_swp4_acq_rel
FATAL: fatal llvm error: Program used external function
'__aarch64_swp4_acq_rel' which could not be resolved!
For now, explicitly disable outline atomics in bitcode, even if it would
be OK to use them on this system. This is done only in the configure
scripts, because the meson scripts don't yet generate bitcode.
This doesn't affect builds using -march higher than armv8-a (eg macOS),
because then LSE instructions are used directly inline.
Back-patch to all supported releases.
Reported-by: Alexander Kozhemyakin <a.kozhemyakin@postgrespro.ru>
Discussion: https://postgr.es/m/18610-37bf303f904fede3%40postgresql.org
---
configure | 105 +++++++++++++++++++++++++++++++++++++++++++++++++++
configure.ac | 7 ++++
2 files changed, 112 insertions(+)
diff --git a/configure b/configure
index 4b01b682b15..ba5a4c1af73 100755
--- a/configure
+++ b/configure
@@ -7284,6 +7284,111 @@ if test x"$pgac_cv_prog_CLANGXX_cxxflags__fexcess_precision_standard" = x"yes";
fi
+ # Ideally bitcode would match $CC's use, or not, of outline atomic functions,
+ # a way to access fast LSE lock instructions or fallback code on when
+ # targeting the basic armv8-a instruction set (which lacked them) but for now
+ # err on the side of not assuming they are not available in inlined code
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CLANG} supports -mno-outline-atomics, for BITCODE_CFLAGS" >&5
+$as_echo_n "checking whether ${CLANG} supports -mno-outline-atomics, for BITCODE_CFLAGS... " >&6; }
+if ${pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CXXFLAGS=$CXXFLAGS
+pgac_save_CXX=$CXX
+CXX=${CLANG}
+CXXFLAGS="${BITCODE_CFLAGS} -mno-outline-atomics"
+ac_save_cxx_werror_flag=$ac_cxx_werror_flag
+ac_cxx_werror_flag=yes
+ac_ext=cpp
+ac_cpp='$CXXCPP $CPPFLAGS'
+ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'
+ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'
+ac_compiler_gnu=$ac_cv_cxx_compiler_gnu
+
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_cxx_try_compile "$LINENO"; then :
+ pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics=yes
+else
+ pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics=no
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
+ac_ext=c
+ac_cpp='$CPP $CPPFLAGS'
+ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'
+ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'
+ac_compiler_gnu=$ac_cv_c_compiler_gnu
+
+ac_cxx_werror_flag=$ac_save_cxx_werror_flag
+CXXFLAGS="$pgac_save_CXXFLAGS"
+CXX="$pgac_save_CXX"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics" >&5
+$as_echo "$pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics" >&6; }
+if test x"$pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics" = x"yes"; then
+ BITCODE_CFLAGS="${BITCODE_CFLAGS} -mno-outline-atomics"
+fi
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CLANG} supports -mno-outline-atomics, for BITCODE_CXXFLAGS" >&5
+$as_echo_n "checking whether ${CLANG} supports -mno-outline-atomics, for BITCODE_CXXFLAGS... " >&6; }
+if ${pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CXXFLAGS=$CXXFLAGS
+pgac_save_CXX=$CXX
+CXX=${CLANG}
+CXXFLAGS="${BITCODE_CXXFLAGS} -mno-outline-atomics"
+ac_save_cxx_werror_flag=$ac_cxx_werror_flag
+ac_cxx_werror_flag=yes
+ac_ext=cpp
+ac_cpp='$CXXCPP $CPPFLAGS'
+ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'
+ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'
+ac_compiler_gnu=$ac_cv_cxx_compiler_gnu
+
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_cxx_try_compile "$LINENO"; then :
+ pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics=yes
+else
+ pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics=no
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
+ac_ext=c
+ac_cpp='$CPP $CPPFLAGS'
+ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'
+ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'
+ac_compiler_gnu=$ac_cv_c_compiler_gnu
+
+ac_cxx_werror_flag=$ac_save_cxx_werror_flag
+CXXFLAGS="$pgac_save_CXXFLAGS"
+CXX="$pgac_save_CXX"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics" >&5
+$as_echo "$pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics" >&6; }
+if test x"$pgac_cv_prog_CLANG_cxxflags__mno_outline_atomics" = x"yes"; then
+ BITCODE_CXXFLAGS="${BITCODE_CXXFLAGS} -mno-outline-atomics"
+fi
+
+
NOT_THE_CFLAGS=""
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CLANG} supports -Wunused-command-line-argument, for NOT_THE_CFLAGS" >&5
$as_echo_n "checking whether ${CLANG} supports -Wunused-command-line-argument, for NOT_THE_CFLAGS... " >&6; }
diff --git a/configure.ac b/configure.ac
index e7f4f0fc22f..f28ddc5e583 100644
--- a/configure.ac
+++ b/configure.ac
@@ -633,6 +633,13 @@ if test "$with_llvm" = yes ; then
PGAC_PROG_VARCC_VARFLAGS_OPT(CLANG, BITCODE_CFLAGS, [-fexcess-precision=standard])
PGAC_PROG_VARCXX_VARFLAGS_OPT(CLANGXX, BITCODE_CXXFLAGS, [-fexcess-precision=standard])
+ # Ideally bitcode would match $CC's use, or not, of outline atomic functions,
+ # a way to access fast LSE lock instructions or fallback code on when
+ # targeting the basic armv8-a instruction set (which lacked them) but for now
+ # err on the side of not assuming they are not available in inlined code
+ PGAC_PROG_VARCXX_VARFLAGS_OPT(CLANG, BITCODE_CFLAGS, [-mno-outline-atomics])
+ PGAC_PROG_VARCXX_VARFLAGS_OPT(CLANG, BITCODE_CXXFLAGS, [-mno-outline-atomics])
+
NOT_THE_CFLAGS=""
PGAC_PROG_VARCC_VARFLAGS_OPT(CLANG, NOT_THE_CFLAGS, [-Wunused-command-line-argument])
if test -n "$NOT_THE_CFLAGS"; then
--
2.39.5 (Apple Git-154)
* Re: BUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved
@ 2024-11-22 02:51 Thomas Munro <thomas.munro@gmail.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 0 replies; 6+ messages in thread
From: Thomas Munro @ 2024-11-22 02:51 UTC (permalink / raw)
To: a.kozhemyakin <a.kozhemyakin@postgrespro.ru>; +Cc: pgsql-bugs@lists.postgresql.org
On Fri, Nov 8, 2024 at 6:15 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> Here's a patch for approach #3. This is low-urgency and a code freeze
> begins in a day or two, so I'm *not* proposing it for next week's
> release.
Hearing no objections, I pushed this.
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2024-11-22 02:51 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-09-11 07:00 BUG #18610: llvm error: __aarch64_swp4_acq_rel which could not be resolved PG Bug reporting form <noreply@postgresql.org>
2024-09-11 09:55 ` Thomas Munro <thomas.munro@gmail.com>
2024-09-12 02:30 ` a.kozhemyakin <a.kozhemyakin@postgrespro.ru>
2024-09-12 22:33 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-08 05:15 ` Thomas Munro <thomas.munro@gmail.com>
2024-11-22 02:51 ` Thomas Munro <thomas.munro@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox