From: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Subject: Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL
To: Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>
Cc: Thomas Munro <thomas.munro@gmail.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
References: <20190521015840.cztw2qxthh7xuutk@alap3.anarazel.de>
 <aaee2cc8-d722-990c-b848-413ba9913703@2ndQuadrant.com>
 <20190521191918.z7kwnrlj45mk2k67@alap3.anarazel.de>
 <20190521194129.2kl2mldesrr65gmb@alap3.anarazel.de>
 <CA+hUKGKY=p7ZeZmAiC0zK72moLyYC=HaF-Fg36s+tNEAJD6R+g@mail.gmail.com>
 <7333.1558537134@sss.pgh.pa.us>
 <20190522175150.c26f4jkqytahajdg@alap3.anarazel.de>
 <7200.1558548407@sss.pgh.pa.us>
 <20190522182027.fzytm3tciqefgz43@alap3.anarazel.de>
 <7965.1558549671@sss.pgh.pa.us>
 <20190522184254.vi5w4icue7vu4km5@alap3.anarazel.de>
Openpgp: preference=signencrypt
Autocrypt: addr=andrew.dunstan@2ndQuadrant.com; prefer-encrypt=mutual;
 keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNL0FuZHJldyBEdW5zdGFuIDxhbmRyZXcuZHVuc3RhbkAybmRRdWFkcmFudC5jb20+wsCY
 BBMBAgBCAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAUJDyV6hBYhBOQ+WEYd/Hy/RGkV
 pZn6f8tZ/DuBBQJcH/EVAhkBAAoJEJn6f8tZ/DuB7BEH/iDWZEbVft2m2MT6SPjARzTupt/l
 xLLZ/Cl6n2US2hov5+fBbnR5xvECk/YCOfG9ICvKNlpFjceqNLRKStd8cmndnI+kDw91uNM2
 hpL4aJMeW65XCHvqXlOLSwv5chBpXz9NG7FMaJ+q+59sWEWjcTNmjXgkEHn53y+1pc7nSLDe
 cr5ZI1aB3nGXRm1SoDY5PYZkUKMykW+YBZwgcs/Q5tRaE+siN7DKbz8E628+PerwAkmHPVXN
 MdJFD6gs4S0gDFA8IxcyIwZU2v4By+brzcJ9Se3FK5BYxBDHtaYZxgSrJAvOUOrLIZPQTQfL
 TtnVs3C3JTfrxCH1ceHtnA6dRuTOwE0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oK
 Waqf+wmW4tpDe/VVcxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7
 llV0xSSoFiieQBsrGvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8
 BoVrRFcdzlIY/NFNhFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C
 4R+M6w7o2PLGOjdZ+kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaY
 L++LAEOqrlvm2TvqQqZRtnEAEQEAAcLAZQQYAQIADwIbDAUCXA6fpAUJDyV6xAAKCRCZ+n/L
 Wfw7geU4B/9vFFsVmWxerDWgNjyiK80zIPmlw/iwdUlU6FIu0JWLMg4093kvxBs7RMZV9hBq
 n3ekBvD4IvFsVLJrw9qzSI2FH3UeYy5WXpEs01Ppia60NOlvRDqXbryzuvfe+iYo6bVP3Ikr
 TPTNYL/BePUTD2DPvSI82SbGTyzuA7SJv6NRqqQDMIy1fU6KyQFLmQJp/QJyhKWzm9jWh8ee
 w8jVaoMY0Kbq5N7RqisEx1Lu/Iw646UphdYzQNymf+kbiU5Kf4I6DhzV8w9sie1Bo1juNj3I
 c5/fFJsmxljbnaDbXPcnFsur1Oo9mIHFakgWgOBMuxoWfjONiLjjLh/9+ZIqSSLX
Message-ID: <2b42bb13-7a7e-e295-2054-9a209749c3a7@2ndQuadrant.com>
Date: Wed, 22 May 2019 16:04:34 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <20190522184254.vi5w4icue7vu4km5@alap3.anarazel.de>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-MW
Precedence: bulk


On 5/22/19 2:42 PM, Andres Freund wrote:
> Hi,
>
> On 2019-05-22 14:27:51 -0400, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> On 2019-05-22 14:06:47 -0400, Tom Lane wrote:
>>>> Not sure about that last bit.  pg_upgrade has the issue of possibly
>>>> wanting to deal with 2 installations, unlike the rest of the tree,
>>>> so I'm not sure that fixing its problem means there's something we
>>>> need to change everywhere else.
>>> I'm not quite following? We need to move it into global scope to fix the
>>> issue at hand (namely that we currently need to make install first, just
>>> to get psql).  And at which scope could it be in master, other than
>>> global?
>> Maybe I misunderstood you --- I thought you were talking about something
>> like defining EXTRA_REGRESS_OPTS in Makefile.global.  If you mean
>> running this unconditionally within test.sh, I've got no objection
>> to that.
> Oh, yes, that's what I meant.
>
>
>>> I do think we will have to move it to the global scope in the back
>>> branches too, because NO_TEMP_INSTALL does indeed fail without a global
>>> install first (rather than using the temp install, as intended):
>> Agreed, we should fix it in all branches, because it seems like it's
>> probably testing the wrong thing, ie using the later branch's psql
>> to run the earlier branch's regression tests.


If I disable install, the buildfarm fails the upgrade check even when
not using NO_TEMP_INSTALL.


excerpts from the log:


rm -rf '/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build'/tmp_install
/bin/mkdir -p
'/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build'/tmp_install/log
make -C '../../..'
DESTDIR='/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build'/tmp_install
install
>'/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build'/tmp_install/log/install.log
2>&1
make -j1  checkprep
>>'/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build'/tmp_install/log/install.log
2>&1
MAKE=make
PATH="/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build/tmp_install/home/pgl/npgl/pg_head/bfroot/HEAD/inst/bin:$PATH"
LD_LIBRARY_PATH="/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build/tmp_install/home/pgl/npgl/pg_head/bfroot/HEAD/inst/lib" 
bindir=/home/pgl/npgl/pg_h
ead/bfroot/HEAD/pgsql.build/tmp_install//home/pgl/npgl/pg_head/bfroot/HEAD/inst/bin
EXTRA_REGRESS_OPTS="--port=5678" /bin/sh
/home/pgl/npgl/pg_head/src/bin/pg_upgrade/test.sh


rm -rf ./testtablespace
mkdir ./testtablespace
../../../src/test/regress/pg_regress
--inputdir=/home/pgl/npgl/pg_head/src/test/regress
--bindir='/home/pgl/npgl/pg_head/bfroot/HEAD/inst/bin'   --port=5678
--port=54464 --dlpath=. --max-concurrent-tests=20 --port=5678
--port=54464
--schedule=/home/pgl/npgl/pg_head/src/test/regress/parallel_schedule 
(using postmaster on /tmp/pg_upgrade_check-GCUkGu, port 54464)
============== dropping database "regression"         ==============
sh: /home/pgl/npgl/pg_head/bfroot/HEAD/inst/bin/psql: No such file or
directory
command failed: "/home/pgl/npgl/pg_head/bfroot/HEAD/inst/bin/psql" -X -c
"DROP DATABASE IF EXISTS \"regression\"" "postgres"
make[1]: *** [GNUmakefile:141: installcheck-parallel] Error 2
make[1]: Leaving directory
'/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build/src/test/regress'
make: *** [GNUmakefile:68: installcheck-parallel] Error 2
make: Leaving directory '/home/pgl/npgl/pg_head/bfroot/HEAD/pgsql.build'
waiting for server to shut down.... done
server stopped
make: *** [Makefile:48: check] Error 1


It looks to me like the bindir needs to be passed to the make called by
test.sh (maybe LD_LIBRARY_PATH too?)


cheers


andrew


-- 

Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services