From: Jack Kelly <jack@jackkelly.name>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
  PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #15525: Build failures when compiling Postgres with Make
 parallelization
References: <15525-a30da084f17a1faa@postgresql.org>
	<31602.1543363896@sss.pgh.pa.us> <87k1kwlvad.fsf@jackkelly.name>
	<28417.1543458718@sss.pgh.pa.us>
	<CAEepm=3wHjf9ZfekMq5x0qs_r1XBd0YXTrhoZMbmDx-xE8NXwA@mail.gmail.com>
	<31059.1543461919@sss.pgh.pa.us>
	<CAEepm=3WXu0Ke8AB3i9Z8EdKZ_RajcKrkwj1GTkgcdqdaafjqw@mail.gmail.com>
Date: Thu, 29 Nov 2018 22:14:31 +1100
In-Reply-To: 
 <CAEepm=3WXu0Ke8AB3i9Z8EdKZ_RajcKrkwj1GTkgcdqdaafjqw@mail.gmail.com>
	(Thomas Munro's message of "Thu, 29 Nov 2018 19:25:50 +1300")
Message-ID: <87r2f4jfu0.fsf@jackkelly.name>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

Thanks Thomas and Tom for your detective work.

Thomas Munro <thomas.munro@enterprisedb.com> writes:

> As for what could be done about it, it seems like we (or the Nix
> project, in a local patch) could declare individual targets to have
> .LOW_RESOLUTION_TIME:
>
> https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
>
> That doesn't seem any better than using "touch" to make a better mtime
> though.  I'm kinda surprised that the Nix project doesn't have this
> problem on other projects, though, if they're always using a modern
> GNU make.  What are they doing differently?

I think this is slightly better than using "touch", because it's a
Makefile-level fix instead of kludging around with the file system, and
its designed purpose is to deal with broken tools.

IMHO, The Nth-degree "correct" thing for the postgresql build system
would be check if the most recent versions are vulnerable, and if so
update the configure script to detect a high-resolution filesystem and a
truncating ranlib, and if that is true for that build, then set a
variable so the Makefiles can conditionally add static libraries to
`.LOW_RESOLUTION_TIME` targets. This seems like a lot of work for
marginal payoff, particularly if releases newer than 9.x are not brittle
in this way.


On the Nix question: I'm an itinerant nixpkgs contributor, so I can't
speak definitively, but I think there are at least a couple of things
going on:

1. Nix on macOS is still a bit of a second-class citizen. (I recently
   had to fix code that assumed shared objects always ended in ".so",
   instead of substituting a variable that expanded to ".dylib" on
   macOS).

2. If Nix successfully builds postgres and adds it to a binary cache,
   most people will not run the build themselves.

3. This bug seems to be tickled because two different Makefiles use
   attempt to build the same target at the same time, using a tool
   (macOS libtool/ranlib, albeit through a recursive $(MAKE) invocation)
   that doesn't support subsecond timestamps, on a filesystem that does
   (APFS). That's a bit of a corner case. I speculate that building a
   static library with a nonrecursive Makefile would only kick off one
   build of the `.a` file, because make will only invoke the command
   once as it walks the DAG of dependencies.

#1 and #2 mean the actual amount of building done is relatively low, and
#3 means that it is actually somewhat hard to trip over.

I have filed https://github.com/NixOS/nixpkgs/issues/51221 with nixpkgs,
and now @dalroth is talking about patching cctools' ranlib.


Thanks again for all your help. I probably won't have time to dig into
this further but if you need more information I'll see what I can do.

-- Jack