From: Jack Kelly <jack@jackkelly.name>
To: tgl@sss.pgh.pa.us
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15525: Build failures when compiling Postgres with Make
 parallelization
In-Reply-To: <31602.1543363896@sss.pgh.pa.us>
References: <15525-a30da084f17a1faa@postgresql.org>
 <31602.1543363896@sss.pgh.pa.us>
Date: Thu, 29 Nov 2018 08:57:46 +1100
Message-ID: <87k1kwlvad.fsf@jackkelly.name>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

Hi Tom & List,

OP reported the bug I encountered on my behalf, which is reported to
nixpkgs at
https://github.com/NixOS/nixpkgs/issues/51093#issuecomment-442301157
. I'm not a subscriber to this list and have constructed the threading
headers manually (I hope this works). Please CC me in replies where
relevant.

Tom Lane <tgl@sss.pgh.pa.us> writes:

> This isn't too helpful if you don't mention which macOS version nor what
> sort of hardware exactly.  Most PG developers use parallel builds
> routinely, so we know that it's not broken in general.

To complicate matters further, I'm reporting the nixpkgs bug on behalf
of my mac-using colleague. His machine overview:

Model Name:    MacBook Pro
Model Identifier:    MacBookPro14,3
Processor Name:    Intel Core i7
Processor Speed:    2.9 GHz
Number of Processors:    1
Total Number of Cores:    4
L2 Cache (per Core):    256 KB
L3 Cache:    8 MB
Memory:    16 GB
Boot ROM Version:    185.0.0.0.0
SMC Version (system):    2.45f0

The build is happening on an APFS file system, I believe.

> - snip discussion of process-spawn failures at high -j numbers -

> Having said that, we did do a round of patches in the v11 development
> cycle that addressed some parallel-make hazards.  A lot of said hazards
> were new in v11 :-(, but I think that some of them were pre-existing
> problems.  So you might find that PG 11 is more resistant to whatever
> is going on here.

A failing build log is attached to the nixpkgs GH issue at
https://github.com/NixOS/nixpkgs/files/2617687/build.log I note that it
calls `ar rcs libpgtypes.a ...` multiple times during the build, and I
speculate that these `ar` invocations start racing each other.

@delroth on GH notes:

@delroth> Starting in 9.1.0 (postgres/postgres@19e231b) this is in the postgres codebase:
@delroth>
@delroth> ./src/interfaces/ecpg/compatlib/Makefile:	$(MAKE) -C $(top_builddir)/src/interfaces/ecpg/pgtypeslib all
@delroth> ./src/interfaces/ecpg/ecpglib/Makefile:	$(MAKE) -C $(top_builddir)/src/interfaces/ecpg/pgtypeslib all

> BTW, are you using the Apple-supplied make, or some other version?
> In the past we've had to fight with parallelism bugs in old gmake
> versions ...

Whatever nixpkgs asks for, which is probably gnumake, and it looks like
nixpkgs has gnumake 4.2.1.

HTH,

-- Jack