Date: Fri, 8 Apr 2022 19:27:58 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Andres Freund <andres@anarazel.de>
Cc: Andrew Dunstan <andrew@dunslane.net>,
	Melih Mutlu <m.melihmutlu@gmail.com>, pgsql-hackers@postgresql.org
Subject: Re: Mingw task for Cirrus CI
Message-ID: <20220409002758.GM24419@telsasoft.com>
References: 
 <CAGPVpCSKS9E0An4=e7ZDnme+y=WOcQFJYJegKO8kE9=gh8NJKQ@mail.gmail.com>
 <20220322001559.mln5mexnhvijuntn@alap3.anarazel.de>
 <CAGPVpCRTg_zUfDB393EZLkuMkJ0UMu_N06TpgvAjH4Y8NcMzfg@mail.gmail.com>
 <20220331002618.npsfms3sijof7z7n@alap3.anarazel.de>
 <99db8656-9722-65c4-39b7-59fc14670c66@dunslane.net>
 <20220407171021.h5cnwfs355zdug7v@alap3.anarazel.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220407171021.h5cnwfs355zdug7v@alap3.anarazel.de>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20220409002758.GM24419%40telsasoft.com>
Precedence: bulk

On Thu, Apr 07, 2022 at 10:10:21AM -0700, Andres Freund wrote:
> Hi,
> 
> On 2022-04-06 11:03:37 -0400, Andrew Dunstan wrote:
> > On 3/30/22 20:26, Andres Freund wrote:
> > > Could you try using dash to invoke configure here, and whether it makes configure faster?
> > I got weird failures re libxml/parser.h when I tried with dash. See
> > <https://cirrus-ci.com/task/5963254039052288> (It would be nice if we
> > could see config.log on failure.)
> 
> Since dash won't help us to get the build time down sufficiently, and the
> tests don't pass without a separate build tree, I looked at what makes
> config/prep_buildtree so slow.
> 
> It's largely just bad code. The slowest part are spawning one expr and mkdir
> -p for each directory. One 'cmp' for each makefile doesn't help either.
> 
> The expr can be replaced with
>   subdir=${item#$sourcetree}
> that's afaics posix syntax ([1]), not bash.
> 
> Spawning one mkdir for each directory can be replaced by a single mkdir
> invocation with all the directories. On my linux workstation that gets the
> time for the first loop down from 1005ms to 38ms, really.

Even better?

(cd "$sourcetree" && find . -print |grep -E '/Makefile$|/GNUmakefile$' |grep -v "$sourcetree/doc/src/sgml/images/" |xargs tar c) |
        (cd "$buildtree" && tar x)