From: Noah Misch <noah@leadboat.com>
To: pgsql-hackers@postgresql.org
Subject: Race to build pg_isolation_regress in "make -j check-world"
Date: Mon, 6 Nov 2017 00:07:52 -0800
Message-ID: <20171106080752.GA1298146@rfd.leadboat.com> (raw)
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
I've been enjoying the speed of parallel check-world, but I get spurious
failures from makefile race conditions. Commit c66b438 fixed the simple ones.
More tricky is this problem of multiple "make" processes entering
src/test/regress concurrently, which causes failures like these:
gcc: error: pg_regress.o: No such file or directory
make[4]: *** [pg_isolation_regress] Error 1
/bin/sh: ../../../src/test/isolation/pg_isolation_regress: Permission denied
make -C test_extensions check
make[2]: *** [check] Error 126
make[2]: Leaving directory `/home/nm/src/pg/backbranch/10/src/test/isolation'
/bin/sh: ../../../../src/test/isolation/pg_isolation_regress: Text file busy
make[3]: *** [isolationcheck] Error 126
make[3]: Leaving directory `/home/nm/src/pg/backbranch/10/src/test/modules/snapshot_too_old'
This is reproducible since commit 2038bf4 or earlier; "make -j check-world"
had worse problems before that era. A workaround is to issue "make -j; make
-j -C src/test/isolation" before the check-world. This problem doesn't affect
src/test/regress/pg_regress. Every top-level "make" or "make install",
including temp-install, builds pg_regress.
I tried fixing this by building src/test/isolation at the same times we run
install-temp. Naturally, that didn't help installcheck-world. It also caused
multiple "make" processes to enter src/port concurrently. I could fix both
check-world and installcheck-world with the attached hack of building
src/test/isolation during every top-level build or install.
The problem of multiple "make" processes in a directory (especially src/port)
shows up elsewhere. In a cleaned tree, "make -j -C src/bin" or "make -j
installcheck-world" will do it. For more-prominent use cases, src/Makefile
prevents this with ".NOTPARALLEL:" and building first the directories that are
frequent submake targets. Perhaps we could fix the general problem with
directory locking; targets that call "$(MAKE) -C FOO" would first sleep until
FOO's lock is available. That could be tricky to make robust.
For now, I propose back-patching the attached, sad hack. Better ideas?
Thanks,
nm
diff --git a/src/Makefile b/src/Makefileindex 380da92..febbced 100644--- a/src/Makefile+++ b/src/Makefile@@ -28,6 +28,7 @@ SUBDIRS = \
pl \
makefiles \
test/regress \
+ test/isolation \
test/perl
# There are too many interdependencies between the subdirectories, so
diff --git a/src/test/isolation/Makefile b/src/test/isolation/Makefileindex 8eb4969..efbdc40 100644--- a/src/test/isolation/Makefile+++ b/src/test/isolation/Makefile@@ -15,6 +15,13 @@ OBJS = specparse.o isolationtester.o $(WIN32RES)
all: isolationtester$(X) pg_isolation_regress$(X)
+# Though we don't install these binaries, build them during installation+# (including temp-install). Otherwise, "make -j check-world" and "make -j+# installcheck-world" would spawn multiple, concurrent builds in this+# directory. Later builds would overwrite files while earlier builds are+# reading them, causing occasional failures.+install: | all+
submake-regress:
$(MAKE) -C $(top_builddir)/src/test/regress pg_regress.o
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackersAttachments:isolation-build-v1.patch (text/plain, 1012B) downloaddiff --git a/src/Makefile b/src/Makefileindex 380da92..febbced 100644--- a/src/Makefile+++ b/src/Makefile@@ -28,6 +28,7 @@ SUBDIRS = \
pl \
makefiles \
test/regress \
+ test/isolation \
test/perl
# There are too many interdependencies between the subdirectories, so
diff --git a/src/test/isolation/Makefile b/src/test/isolation/Makefileindex 8eb4969..efbdc40 100644--- a/src/test/isolation/Makefile+++ b/src/test/isolation/Makefile@@ -15,6 +15,13 @@ OBJS = specparse.o isolationtester.o $(WIN32RES)
all: isolationtester$(X) pg_isolation_regress$(X)
+# Though we don't install these binaries, build them during installation+# (including temp-install). Otherwise, "make -j check-world" and "make -j+# installcheck-world" would spawn multiple, concurrent builds in this+# directory. Later builds would overwrite files while earlier builds are+# reading them, causing occasional failures.+install: | all+
submake-regress:
$(MAKE) -C $(top_builddir)/src/test/regress pg_regress.o
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com
Subject: Re: Race to build pg_isolation_regress in "make -j check-world"
In-Reply-To: <20171106080752.GA1298146@rfd.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox