From: Tom Lane <tgl@sss.pgh.pa.us>
To: Robert Haas <robertmhaas@gmail.com>
cc: Andres Freund <andres@anarazel.de>, Andrew Dunstan <andrew@dunslane.net>,
        Justin Pryzby <pryzby@telsasoft.com>,
        "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>,
        Thomas Munro <thomas.munro@gmail.com>,
        Melanie Plageman <melanieplageman@gmail.com>,
        Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
        Daniel Gustafsson <daniel@yesql.se>
Subject: Re: Adding CI to our tree
In-reply-to: 
 <CA+Tgmoap6WXzta=zw97SF73hCfzEWVMtfDFB01svYXMtTzKVVA@mail.gmail.com>
References: <20211001222752.wrz7erzh4cajvgp6@alap3.anarazel.de>
 <20211231014652.kgrdk2wytiallix3@alap3.anarazel.de>
 <20220109191649.GL14051@telsasoft.com>
 <20220109195744.mjoue2pr6xtnsquw@alap3.anarazel.de>
 <20220110220748.GS14051@telsasoft.com>
 <20220113185527.kgzmxutkydkyktuq@alap3.anarazel.de>
 <63f3be31-d7c4-0ff4-e5f4-7368863da1bc@dunslane.net>
 <20220114233411.2byuid4umwjqbhug@alap3.anarazel.de>
 <20220114235457.GQ14051@telsasoft.com>
 <87a81b91-87bf-c0bc-7e4f-06dffadcf737@dunslane.net>
 <20220117181946.bmvubqpzqxlvmgeh@alap3.anarazel.de>
 <CA+Tgmoap6WXzta=zw97SF73hCfzEWVMtfDFB01svYXMtTzKVVA@mail.gmail.com>
Comments: In-reply-to Robert Haas <robertmhaas@gmail.com>
	message dated "Mon, 17 Jan 2022 13:50:04 -0500"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <47039.1642447089.1@sss.pgh.pa.us>
Date: Mon, 17 Jan 2022 14:18:09 -0500
Message-ID: <47040.1642447089@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/47040.1642447089%40sss.pgh.pa.us>
Precedence: bulk

Robert Haas <robertmhaas@gmail.com> writes:
> I have a lot of sympathy with Andrew here, actually. If you just do
> 'make check-world' and assume that will cover everything, you get one
> giant output file. That is not great at all.

Yeah.  I agree with Andrew that we want output that is more modular,
not less so.  But we do need to find a way to have less knowledge
hard-wired in the buildfarm client script.

> But having said that, I also agree that it sucks to have to keep
> updating the BF client every time we want to do any kind of
> test-related changes in the main source tree. One way around that
> would be to put a file in the main source tree that the build farm
> client can read to know what to do. Another would be to have the BF
> client download the latest list of steps from somewhere instead of
> having it in the source code, so that it can be updated without
> everyone needing to update their machine.

The obvious place for "somewhere" is "the main source tree", so I
doubt your second suggestion is better than your first.  But your
first does seem like a plausible way to proceed.

Another way to think of it, maybe, is to migrate chunks of the
buildfarm client script itself into the source tree.  I'd rather
that developers not need to become experts on the buildfarm client
to make adjustments to the test process --- but I suspect that
a simple script like "run make check in these directories" is
not going to be flexible enough for everything.

			regards, tom lane