agora inbox for pgsql-bugs@postgresql.org
help / color / mirror / Atom feedBUG #19727: pg-combinebackup fails to link
8+ messages / 3 participants
[nested] [flat]
* BUG #19727: pg-combinebackup fails to link
@ 2026-09-29 14:01 PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: PG Bug reporting form @ 2026-09-29 14:01 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: ross@burtonini.com
The following bug has been logged on the website:
Bug reference: 19727
Logged by: Ross Burton
Email address: ross@burtonini.com
PostgreSQL version: 18.6
Operating system: Linux
Description:
When I build pg-combineback from postgress 18.6 (in Yocto, cross-compiled on
aarch64 to aarch64), it fails to link:
aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
`json_lex_number':
src/common/jsonapi.c:2379:(.text+0x168): undefined reference to
`appendBinaryPQExpBuffer'
aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
`makeJsonLexContextCstringLen':
src/common/jsonapi.c:418:(.text+0x44c): undefined reference to
`createPQExpBuffer'
aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
`allocate_incremental_state':
src/common/jsonapi.c:465:(.text+0x578): undefined reference to
`initPQExpBuffer'
aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
`makeJsonLexContextIncremental':
src/common/jsonapi.c:535:(.text+0x67c): undefined reference to
`createPQExpBuffer'
aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
`freeJsonLexContext':
src/common/jsonapi.c:699:(.text+0x728): undefined reference to
`destroyPQExpBuffer'
aarch64-poky-linux-ld: src/common/jsonapi.c:696:(.text+0x768): undefined
reference to `destroyPQExpBuffer'
aarch64-poky-linux-ld: src/common/jsonapi.c:703:(.text+0x794): undefined
reference to `termPQExpBuffer'
aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
`json_lex':
src/common/jsonapi.c:1606:(.text+0xaa8): undefined reference to
`resetPQExpBuffer'
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #19727: pg-combinebackup fails to link
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
@ 2026-09-29 14:48 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 14:56 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2026-09-29 14:48 UTC (permalink / raw)
To: ross@burtonini.com; +Cc: pgsql-bugs@lists.postgresql.org
PG Bug reporting form <noreply@postgresql.org> writes:
> When I build pg-combineback from postgress 18.6 (in Yocto, cross-compiled on
> aarch64 to aarch64), it fails to link:
> aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
> `json_lex_number':
> src/common/jsonapi.c:2379:(.text+0x168): undefined reference to
> `appendBinaryPQExpBuffer'
You seem to be using a broken linker :-(
pg_combinebackup doesn't use any of the functions in
src/common/jsonapi.c. Therefore, the jsonapi.o member of
libpgcommon.a shouldn't get pulled into the link, and its
unsatisfied dependency on libpq.so shouldn't matter.
Apparently, whatever build toolchain you're using has
forgotten this decades-old principle of .a library usage.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #19727: pg-combinebackup fails to link
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-09-29 14:56 ` Ross Burton <ross@burtonini.com>
2026-09-29 15:30 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Ross Burton @ 2026-09-29 14:56 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org
That's a good point. This is GCC 16.2 and binutils 2.47...
On Tue, 29 Sept 2026 at 15:48, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> PG Bug reporting form <noreply@postgresql.org> writes:
> > When I build pg-combineback from postgress 18.6 (in Yocto, cross-compiled on
> > aarch64 to aarch64), it fails to link:
>
> > aarch64-poky-linux-ld: src/common/libpgcommon.a(jsonapi.o): in function
> > `json_lex_number':
> > src/common/jsonapi.c:2379:(.text+0x168): undefined reference to
> > `appendBinaryPQExpBuffer'
>
> You seem to be using a broken linker :-(
>
> pg_combinebackup doesn't use any of the functions in
> src/common/jsonapi.c. Therefore, the jsonapi.o member of
> libpgcommon.a shouldn't get pulled into the link, and its
> unsatisfied dependency on libpq.so shouldn't matter.
> Apparently, whatever build toolchain you're using has
> forgotten this decades-old principle of .a library usage.
>
> regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #19727: pg-combinebackup fails to link
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 14:56 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
@ 2026-09-29 15:30 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 16:45 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2026-09-29 15:30 UTC (permalink / raw)
To: Ross Burton <ross@burtonini.com>; +Cc: pgsql-bugs@lists.postgresql.org
Ross Burton <ross@burtonini.com> writes:
> That's a good point. This is GCC 16.2 and binutils 2.47...
Hm. The newest Linux installation I have at hand is Fedora 44
with gcc 16.2 + binutils 2.46, and it doesn't show this behavior.
Can you check on whether gcc is invoking /usr/bin/ld? If it's
calling some other linker maybe that's the cause.
I'm suspicious in any case that there's something odd about either
your installation or your environment. I don't recall ever hearing
a similar complaint, so this linking behavior can't be common.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #19727: pg-combinebackup fails to link
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 14:56 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
2026-09-29 15:30 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-09-29 16:45 ` Ross Burton <ross@burtonini.com>
2026-09-30 11:10 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
0 siblings, 1 reply; 8+ messages in thread
From: Ross Burton @ 2026-09-29 16:45 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org
On Tue, 29 Sept 2026 at 16:30, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Hm. The newest Linux installation I have at hand is Fedora 44
> with gcc 16.2 + binutils 2.46, and it doesn't show this behavior.
>
> Can you check on whether gcc is invoking /usr/bin/ld? If it's
> calling some other linker maybe that's the cause.
>
> I'm suspicious in any case that there's something odd about either
> your installation or your environment. I don't recall ever hearing
> a similar complaint, so this linking behavior can't be common.
It's not calling the host ld. Interestingly, this doesn't happen
reliably, I needed to do a loop of builds with make --shuffle to make
it fail. So that suggests there's a missing Makefile dependency
somewhere that normally doesn't cause problems.
I'll dig a little more later, but now I need to cook dinner. :)
Ross
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #19727: pg-combinebackup fails to link
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 14:56 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
2026-09-29 15:30 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 16:45 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
@ 2026-09-30 11:10 ` Ross Burton <ross@burtonini.com>
2026-09-30 13:36 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Ross Burton @ 2026-09-30 11:10 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org
I now have a build tree that works and a build tree that breaks, from
the same machine and environment, simply by re-running the build with
make --shuffle.
The working jsonapi.o has functions like appendStringInfoChar,
resetStringInfo, etc. The one that fails to link has
appendPQExpBufferChar, resetPQExpBuffer, etc.
This is a makefile dependency bug exposed in parallel builds, and can
be reproduced on demand with a minimal reproducer. From a clean but
configured tree, in src/common:
$ make jsonapi.o
$ readelf -sW jsonapi.o | grep reset
64: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND resetStringInfo
This builds a jsonapi.o using the direct target, and
JSONAPI_USE_PQEXPBUFFER is not defined.
$ make clean
$ make jsonapi_shlib.o
$ readelf -sW jsonapi.o | grep reset
65: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND resetPQExpBuffer
This builds jsonapi_shlib.o, which adds JSONAPI_USE_PQEXPBUFFER to
CPPFLAGS. However jsonapi.o hasn't been built yet so that is also
built, but with the PQEXPBUFFER define set this time.
My builds are with -j32 but also --shuffle, which is very good at
shaking out Makefiles which have dependencies that work via luck: if a
target has dependencies A B C, then normal make would be A then B then
C, but --shuffle might build C then B then A.
Ross
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #19727: pg-combinebackup fails to link
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 14:56 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
2026-09-29 15:30 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 16:45 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
2026-09-30 11:10 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
@ 2026-09-30 13:36 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-09-30 15:24 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2026-09-30 13:36 UTC (permalink / raw)
To: Ross Burton <ross@burtonini.com>; +Cc: pgsql-bugs@lists.postgresql.org
Ross Burton <ross@burtonini.com> writes:
> I now have a build tree that works and a build tree that breaks, from
> the same machine and environment, simply by re-running the build with
> make --shuffle.
> The working jsonapi.o has functions like appendStringInfoChar,
> resetStringInfo, etc. The one that fails to link has
> appendPQExpBufferChar, resetPQExpBuffer, etc.
> This is a makefile dependency bug exposed in parallel builds, and can
> be reproduced on demand with a minimal reproducer. From a clean but
> configured tree, in src/common:
> $ make jsonapi.o
> $ readelf -sW jsonapi.o | grep reset
> 64: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND resetStringInfo
> This builds a jsonapi.o using the direct target, and
> JSONAPI_USE_PQEXPBUFFER is not defined.
> $ make clean
> $ make jsonapi_shlib.o
> $ readelf -sW jsonapi.o | grep reset
> 65: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND resetPQExpBuffer
> This builds jsonapi_shlib.o, which adds JSONAPI_USE_PQEXPBUFFER to
> CPPFLAGS. However jsonapi.o hasn't been built yet so that is also
> built, but with the PQEXPBUFFER define set this time.
Very interesting! So the problem is that the scope of the effects of
this:
jsonapi_shlib.o: override CPPFLAGS += -DJSONAPI_USE_PQEXPBUFFER
isn't terribly well-defined. Looking at the rest of that Makefile,
I'm also suspicious about
$(OBJS_FRONTEND): CPPFLAGS += -DUSE_PRIVATE_ENCODING_FUNCS
Can you break that part? If not, maybe the problem is the
gratuitous-looking "override"?
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: BUG #19727: pg-combinebackup fails to link
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 14:56 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
2026-09-29 15:30 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 16:45 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
2026-09-30 11:10 ` Re: BUG #19727: pg-combinebackup fails to link Ross Burton <ross@burtonini.com>
2026-09-30 13:36 ` Re: BUG #19727: pg-combinebackup fails to link Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-09-30 15:24 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 8+ messages in thread
From: Tom Lane @ 2026-09-30 15:24 UTC (permalink / raw)
To: Ross Burton <ross@burtonini.com>; +Cc: pgsql-bugs@lists.postgresql.org
I wrote:
> Ross Burton <ross@burtonini.com> writes:
>> This is a makefile dependency bug exposed in parallel builds, and can
>> be reproduced on demand with a minimal reproducer.
> Very interesting! So the problem is that the scope of the effects of
> this:
> jsonapi_shlib.o: override CPPFLAGS += -DJSONAPI_USE_PQEXPBUFFER
> isn't terribly well-defined.
I dug in the gmake manual and verified that they document this
behavior and point out that it can produce inconsistent results.
So we need to fix it. The attached seems to do the trick, but
now I wonder where else there might be similar issues. I did
a few rounds of "make -s -j32 --shuffle" and didn't see any
other compilation failures, but that isn't a very thorough test.
regards, tom lane
Attachments:
[text/x-diff] v1-fix-makefile-variable-usage.patch (1.3K, ../../859825.1790781896@sss.pgh.pa.us/2-v1-fix-makefile-variable-usage.patch)
download | inline diff:
diff --git a/src/common/Makefile b/src/common/Makefile
index 3404601b6bf..daeba105af9 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -162,11 +162,6 @@ libpgcommon_shlib.a: $(OBJS_SHLIB)
rm -f $@
$(AR) $(AROPT) $@ $^
-# The JSON API normally exits on out-of-memory; disable that behavior for shared
-# library builds. This requires libpq's pqexpbuffer.h.
-jsonapi_shlib.o: override CPPFLAGS += -DJSONAPI_USE_PQEXPBUFFER
-jsonapi_shlib.o: override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
-
# Because this uses its own compilation rule, it doesn't use the
# dependency tracking logic from Makefile.global. To make sure that
# dependency tracking works anyway for the *_shlib.o files, depend on
@@ -176,6 +171,12 @@ jsonapi_shlib.o: override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
%_shlib.o: %.c %.o
$(CC) $(CFLAGS) $(CFLAGS_SL) $(CPPFLAGS) -c $< -o $@
+# The JSON API normally exits on out-of-memory; disable that behavior for shared
+# library builds. This requires libpq's pqexpbuffer.h.
+# This rule overrides the previous one, so must duplicate its effects.
+jsonapi_shlib.o: jsonapi.c jsonapi.o
+ $(CC) $(CFLAGS) $(CFLAGS_SL) -DJSONAPI_USE_PQEXPBUFFER -I$(libpq_srcdir) $(CPPFLAGS) -c $< -o $@
+
libpgcommon_excluded_shlib.a: $(OBJS_EXCLUDED_SHLIB)
rm -f $@
$(AR) $(AROPT) $@ $^
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2026-09-30 15:24 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-09-29 14:01 BUG #19727: pg-combinebackup fails to link PG Bug reporting form <noreply@postgresql.org>
2026-09-29 14:48 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 14:56 ` Ross Burton <ross@burtonini.com>
2026-09-29 15:30 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-09-29 16:45 ` Ross Burton <ross@burtonini.com>
2026-09-30 11:10 ` Ross Burton <ross@burtonini.com>
2026-09-30 13:36 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-09-30 15:24 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox