public inbox for [email protected]help / color / mirror / Atom feed
Fix race with LLVM and bison. 4+ messages / 3 participants [nested] [flat]
* Fix race with LLVM and bison. @ 2026-03-27 11:31 Maksim.Melnikov <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Maksim.Melnikov @ 2026-03-27 11:31 UTC (permalink / raw) To: [email protected] Hi All. I've found build error in configuration --with-llvm CPPFLAGS="-O2" ./configure --enable-debug --enable-cassert --enable-tap-tests --with-openssl --with-icu --with-llvm .... make world-bin -j3 .... cubescan.c:9:10: fatal error: 'cubeparse.h' file not found 9 | #include "cubeparse.h" /* must be after cubedata.h for YYSTYPE and NDBOX */ .... segscan.c:9:10: fatal error: 'segparse.h' file not found 9 | #include "segparse.h" /* must be after segdata.h for SEG */ The reason is race, that exist between LLVM compilation and bison source code generation and compilation can occur first. Ideally LLVM compilation target should depend on header files targets. The error is difficult to reproduce and I've done simple patch to have stable reproducing. Fix patch is also attached. Thanks. Attachments: [text/x-patch] v1-0002-Patch-simplify-reproducing-of-race-with-LLVM-and-.patch (635B, 2-v1-0002-Patch-simplify-reproducing-of-race-with-LLVM-and-.patch) download | inline diff: From b326ca00b542ca45453efccbd41d97ccdf766e30 Mon Sep 17 00:00:00 2001 From: Maksim Melnikov <[email protected]> Date: Fri, 27 Mar 2026 13:33:46 +0300 Subject: [PATCH v1 2/2] Patch simplify reproducing of race with LLVM and bison code generation. --- src/Makefile.global.in | 1 + 1 file changed, 1 insertion(+) diff --git a/src/Makefile.global.in b/src/Makefile.global.in index a7699b026bb..f186f2e91e1 100644 --- a/src/Makefile.global.in +++ b/src/Makefile.global.in @@ -809,6 +809,7 @@ endif %.c: %.y $(if $(BISON_CHECK_CMD),$(BISON_CHECK_CMD)) + sleep 10s $(BISON) $(BISONFLAGS) -o $@ $< %.i: %.c -- 2.43.0 [text/x-patch] v1-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patch (1.5K, 3-v1-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patch) download | inline diff: From ea157f883d54974f5924d78592813b5f1f082bd5 Mon Sep 17 00:00:00 2001 From: Maksim Melnikov <[email protected]> Date: Fri, 27 Mar 2026 13:33:09 +0300 Subject: [PATCH v1 1/2] Fix race with LLVM compilation and bison code generation. In case when source file is compiled with LLVM and include header file that should be generated by bison, compile error can occur, example: cubescan.c:9:10: fatal error: 'cubeparse.h' file not found 9 | #include "cubeparse.h" The reason is race, that exist between LLVM compilation and bison code generation and compilation can occur first. Ideally LLVM compilation target should depend on header files targets. --- contrib/cube/Makefile | 4 ++++ contrib/seg/Makefile | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/contrib/cube/Makefile b/contrib/cube/Makefile index dfb0d806e4b..b4f3d32450b 100644 --- a/contrib/cube/Makefile +++ b/contrib/cube/Makefile @@ -35,6 +35,10 @@ endif cubeparse.h: cubeparse.c touch $@ +ifeq ($(with_llvm), yes) +cubescan.c: cubeparse.h +endif + cubeparse.c: BISONFLAGS += -d # Force these dependencies to be known even without dependency info built: diff --git a/contrib/seg/Makefile b/contrib/seg/Makefile index b408f4049cb..b98ca73bc64 100644 --- a/contrib/seg/Makefile +++ b/contrib/seg/Makefile @@ -34,6 +34,10 @@ endif segparse.h: segparse.c touch $@ +ifeq ($(with_llvm), yes) +segscan.bc: segparse.h +endif + segparse.c: BISONFLAGS += -d # Force these dependencies to be known even without dependency info built: -- 2.43.0 ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Fix race with LLVM and bison. @ 2026-03-27 18:56 Andres Freund <[email protected]> parent: Maksim.Melnikov <[email protected]> 0 siblings, 2 replies; 4+ messages in thread From: Andres Freund @ 2026-03-27 18:56 UTC (permalink / raw) To: Maksim.Melnikov <[email protected]>; +Cc: [email protected] Hi, On 2026-03-27 14:31:52 +0300, Maksim.Melnikov wrote: > I've found build error in configuration --with-llvm > > CPPFLAGS="-O2" ./configure --enable-debug --enable-cassert > --enable-tap-tests --with-openssl --with-icu --with-llvm > > .... > > make world-bin -j3 > > .... > > cubescan.c:9:10: fatal error: 'cubeparse.h' file not found > 9 | #include "cubeparse.h" /* must be after cubedata.h for YYSTYPE and > NDBOX */ > > .... > segscan.c:9:10: fatal error: 'segparse.h' file not found > 9 | #include "segparse.h" /* must be after segdata.h for SEG */ > > > The reason is race, that exist between LLVM compilation and bison source > code generation and compilation can occur first. > > Ideally LLVM compilation target should depend on header files targets. > > The error is difficult to reproduce and I've done simple patch to have > stable reproducing. Fix patch is also attached. You don't need a sleep to show there's a problem, you can just do make -C contrib/cube cubescan.bc We don't have the same issue in the backend, as for backend code each .bc file depends on the .o file: src/backend/common.mk: ifeq ($(with_llvm), yes) objfiles.txt: $(patsubst %.o,%.bc, $(OBJS)) $(patsubst %.o,%.bc, $(OBJS)): $(OBJS) endif But for some reason I didn't add the same logic to pgxs.mk. I think we need something like the attached to make the dependencies work. I'm a bit worried about breaking some extensions if were to backpatch this. So I'm somewhat inclined to just fix this in master. Greetings, Andres Freund Attachments: [text/x-diff] fix-llvm-contrib-pgxs-deps.diff (1.4K, 2-fix-llvm-contrib-pgxs-deps.diff) download | inline diff: diff --git i/src/makefiles/pgxs.mk w/src/makefiles/pgxs.mk index 039cee3dfe5..d7732024b15 100644 --- i/src/makefiles/pgxs.mk +++ w/src/makefiles/pgxs.mk @@ -219,7 +219,12 @@ endef all: $(PROGRAM) $(DATA_built) $(HEADER_allbuilt) $(SCRIPTS_built) $(addsuffix $(DLSUFFIX), $(MODULES)) $(addsuffix .control, $(EXTENSION)) ifeq ($(with_llvm), yes) +# Ensure that .bc files for MODULES and OBJS get built with all all: $(addsuffix .bc, $(MODULES)) $(patsubst %.o,%.bc, $(OBJS)) +# Ensure that each .bc file depends on the corresponding .o file, to ensure +# the dependencies required for it to be built are present. +$(patsubst %.o,%.bc, $(OBJS)): $(OBJS) +$(addsuffix .bc, $(MODULES)): $(addsuffix .o, $(MODULES)) endif ifdef MODULE_big diff --git i/src/backend/common.mk w/src/backend/common.mk index 61861f5c7eb..85bb4ee2527 100644 --- i/src/backend/common.mk +++ w/src/backend/common.mk @@ -22,7 +22,10 @@ objfiles.txt: Makefile $(SUBDIROBJS) $(OBJS) $(if $(filter-out $(OBJS),$?),( $(if $(SUBDIROBJS),cat $(SUBDIROBJS); )echo $(addprefix $(subdir)/,$(OBJS)) ) >$@,touch $@) ifeq ($(with_llvm), yes) +# Ensure that .bc files get built when building .o files objfiles.txt: $(patsubst %.o,%.bc, $(OBJS)) +# Ensure that each .bc file depends on the corresponding .o file, to ensure +# the dependencies required for it to be built are present. $(patsubst %.o,%.bc, $(OBJS)): $(OBJS) endif ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Fix race with LLVM and bison. @ 2026-03-30 07:56 Maksim.Melnikov <[email protected]> parent: Andres Freund <[email protected]> 1 sibling, 0 replies; 4+ messages in thread From: Maksim.Melnikov @ 2026-03-30 07:56 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: [email protected] Hi On 3/27/26 21:56, Andres Freund wrote: > Hi, > > On 2026-03-27 14:31:52 +0300, Maksim.Melnikov wrote: >> I've found build error in configuration --with-llvm >> >> CPPFLAGS="-O2" ./configure --enable-debug --enable-cassert >> --enable-tap-tests --with-openssl --with-icu --with-llvm >> >> .... >> >> make world-bin -j3 >> >> .... >> >> cubescan.c:9:10: fatal error: 'cubeparse.h' file not found >> 9 | #include "cubeparse.h" /* must be after cubedata.h for YYSTYPE and >> NDBOX */ >> >> .... >> segscan.c:9:10: fatal error: 'segparse.h' file not found >> 9 | #include "segparse.h" /* must be after segdata.h for SEG */ >> >> >> The reason is race, that exist between LLVM compilation and bison source >> code generation and compilation can occur first. >> >> Ideally LLVM compilation target should depend on header files targets. >> >> The error is difficult to reproduce and I've done simple patch to have >> stable reproducing. Fix patch is also attached. > You don't need a sleep to show there's a problem, you can just do > make -C contrib/cube cubescan.bc > > We don't have the same issue in the backend, as for backend code each .bc file > depends on the .o file: > > src/backend/common.mk: > > ifeq ($(with_llvm), yes) > objfiles.txt: $(patsubst %.o,%.bc, $(OBJS)) > $(patsubst %.o,%.bc, $(OBJS)): $(OBJS) > endif > > But for some reason I didn't add the same logic to pgxs.mk. > > I think we need something like the attached to make the dependencies work. > > > I'm a bit worried about breaking some extensions if were to backpatch this. So > I'm somewhat inclined to just fix this in master. > > Greetings, > > Andres Freund Thanks for attentions, I see your patch is better. Few additional thoughts 1. Now I see that we have two different places to configure backend and extensions, maybe we should apply your patch to some common place, for example src/Makefile.global.in, because it seems common build logic. How do you think? 2. If you have some doubts about backpatch, I suggest to apply fix-llvm-contrib-pgxs-deps.diff on master and patch v2-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patch as backpatch, because this one correct it only for problematic contribs. How do you think? Greetings, Maksim Melnikov Attachments: [text/x-patch] v2-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patch (1.5K, 2-v2-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patch) download | inline diff: From 0f722396846277387c88606addbaa8160f3eee9e Mon Sep 17 00:00:00 2001 From: Maksim Melnikov <[email protected]> Date: Fri, 27 Mar 2026 13:33:09 +0300 Subject: [PATCH v2] Fix race with LLVM compilation and bison code generation. In case when source file is compiled with LLVM and include header file that should be generated by bison, compile error can occur, example: cubescan.c:9:10: fatal error: 'cubeparse.h' file not found 9 | #include "cubeparse.h" The reason is race, that exist between LLVM compilation and bison code generation and compilation can occur first. Ideally LLVM compilation target should depend on .o files targets. --- contrib/cube/Makefile | 4 ++++ contrib/seg/Makefile | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/contrib/cube/Makefile b/contrib/cube/Makefile index dfb0d806e4b..d84ad3fc3e9 100644 --- a/contrib/cube/Makefile +++ b/contrib/cube/Makefile @@ -35,6 +35,10 @@ endif cubeparse.h: cubeparse.c touch $@ +ifeq ($(with_llvm), yes) +cubescan.bc: cubescan.o +endif + cubeparse.c: BISONFLAGS += -d # Force these dependencies to be known even without dependency info built: diff --git a/contrib/seg/Makefile b/contrib/seg/Makefile index b408f4049cb..dde996ca5c5 100644 --- a/contrib/seg/Makefile +++ b/contrib/seg/Makefile @@ -34,6 +34,10 @@ endif segparse.h: segparse.c touch $@ +ifeq ($(with_llvm), yes) +segscan.bc: segscan.o +endif + segparse.c: BISONFLAGS += -d # Force these dependencies to be known even without dependency info built: -- 2.43.0 ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Fix race with LLVM and bison. @ 2026-03-30 22:37 Zsolt Parragi <[email protected]> parent: Andres Freund <[email protected]> 1 sibling, 0 replies; 4+ messages in thread From: Zsolt Parragi @ 2026-03-30 22:37 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Maksim.Melnikov <[email protected]>; [email protected] > I think we need something like the attached to make the dependencies work. > > I'm a bit worried about breaking some extensions if were to backpatch this. So > I'm somewhat inclined to just fix this in master. pg_plan_advice also seems to be affected and is missing from the original patch, a generic fix like this handles that too and all future cases. ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-03-30 22:37 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-03-27 11:31 Fix race with LLVM and bison. Maksim.Melnikov <[email protected]> 2026-03-27 18:56 ` Andres Freund <[email protected]> 2026-03-30 07:56 ` Maksim.Melnikov <[email protected]> 2026-03-30 22:37 ` Zsolt Parragi <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox