From: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Subject: Re: backup manifests and contemporaneous buildfarm failures
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>, Fabien COELHO <coelho@cri.ensmp.fr>,
 Alvaro Herrera <alvherre@2ndquadrant.com>, David Steele
 <david@pgmasters.net>, Andres Freund <andres@anarazel.de>,
 Noah Misch <noah@leadboat.com>, Stephen Frost <sfrost@snowman.net>,
 Amit Kapila <amit.kapila16@gmail.com>,
 Suraj Kharage <suraj.kharage@enterprisedb.com>,
 tushar <tushar.ahuja@enterprisedb.com>,
 Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>,
 Rushabh Lathia <rushabh.lathia@gmail.com>,
 Tels <nospam-pg-abuse@bloodgate.com>,
 PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
 Jeevan Chalke <jeevan.chalke@enterprisedb.com>,
 vignesh C <vignesh21@gmail.com>, Thomas Munro <thomas.munro@gmail.com>
References: 
 <CA+TgmoazJ5wh+H77kjVfMnDh6WLh=oJodrwJ21SACSq5fojwPg@mail.gmail.com>
 <20200403205412.GA8279@alvherre.pgsql>
 <CA+TgmoaRbm-L1C2YkDVPx1TGN9ufxWjnaweTwKE=TQC-YOD=rg@mail.gmail.com>
 <CA+TgmoZ4bAbjd_Am=UiMpBrgPEO=7YarqvaAMrQYKP1GkkOaRg@mail.gmail.com>
 <alpine.DEB.2.21.2004032346550.16227@pseudo> <22394.1585951968@sss.pgh.pa.us>
 <CA+Tgmoah0khUqHG5vn8k=LJFO0mmNLyM3HtUmh4RBkq7W2GFgQ@mail.gmail.com>
 <3419.1585965125@sss.pgh.pa.us>
 <CA+TgmoYR6GKr1kc4KTBwXbNChDaJX8qnx7k+KKgtNnSiFgCumA@mail.gmail.com>
 <CA+TgmoaUOS5X64nKgFxNV7JHN4sRkNAJYW2gHz-LMb0Ej4xHig@mail.gmail.com>
 <24829.1586234244@sss.pgh.pa.us>
 <CAA8=A7-FFbnpVGg5-T-tvNLhLN_XkD6yGpJ9VHKqsROiHnsFFw@mail.gmail.com>
Autocrypt: addr=andrew.dunstan@2ndQuadrant.com; prefer-encrypt=mutual;
 keydata=
 mQENBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAG0L0FuZHJldyBEdW5zdGFuIDxhbmRyZXcuZHVuc3RhbkAybmRRdWFkcmFudC5jb20+iQFY
 BBMBAgBCAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgBYhBOQ+WEYd/Hy/RGkVpZn6f8tZ
 /DuBAhkBBQJd1pSYBQkSzqMsAAoJEJn6f8tZ/DuBJmEIAIVFBIC3qGqh+YTPVTkfowwxpHz0
 tZSPjZ6GS/5y62kgoZrZOVXw+Rg2ceuMM3a+In/RRjTx4bembgYqRKgzXlDfiNBJeq5ioVuN
 H1faDKE2lm5a6QkRQ2JU6DKf60fqSJGP5LisCUr1V0wz6EnlO31ArrKtXCNB+yk0KDAS6o6I
 Ql4pkXpr+vq4iilwJmiJOqTBam0LmQE7ulYCCYU8Z6MkpiSl+dO7QSramcGyxauQRI3a7gcq
 dpo8Vrv3gFTQDeMYxk7cqZp08pIVH5XR0+LgBz4X84f59rA/VAKlC/r6CyEQk59rvDLxCMyP
 2buoj51Z25N3MVghRDGcsU6mT7e5AQ0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oK
 Waqf+wmW4tpDe/VVcxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7
 llV0xSSoFiieQBsrGvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8
 BoVrRFcdzlIY/NFNhFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C
 4R+M6w7o2PLGOjdZ+kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaY
 L++LAEOqrlvm2TvqQqZRtnEAEQEAAYkBJQQYAQIADwIbDAUCXdaUtAUJEs6jVQAKCRCZ+n/L
 Wfw7gQi5B/9sHbmRloZNCWWwZGm+O9iG/JGQxS0WWa4GaZqzqSTWOft2/gO0Bgag89r1djq4
 PFgSZtzHvKQpqYz/7HkPrWh1jpcS83i/uYY1uyHXUUtpdEmn4buNObEtD0YNaJkGdhASsECq
 6akw+u7WJYmAHs/SpR6ok6HqLw7JHDVc2h5dViLfTk0qrgDa7sq/LVMoJje5Ct/Ku3YaWAV6
 YSmzoPP5BmPVFqP11OMQkCqurIu9vTLXKFWX623vCmqRDHJ6e5GCDHbH8USU+4wJBfPymZkd
 cA7NGm/mmWeJiWBFIDyjzYxrjku8ty9TZJVsaRXnS+OFQzXz9Eu2wTpKdHij46uY
Message-ID: <7330a7c7-ce5f-9769-39a1-bdb0b32bb4a6@2ndQuadrant.com>
Date: Wed, 8 Apr 2020 13:45:39 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.6.0
MIME-Version: 1.0
In-Reply-To: 
 <CAA8=A7-FFbnpVGg5-T-tvNLhLN_XkD6yGpJ9VHKqsROiHnsFFw@mail.gmail.com>
Content-Type: multipart/mixed;
 boundary="------------524D116BD8080C28075D26CC"
Content-Language: en-US
Precedence: bulk

This is a multi-part message in MIME format.
--------------524D116BD8080C28075D26CC
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


On 4/7/20 9:42 AM, Andrew Dunstan wrote:
> On Tue, Apr 7, 2020 at 12:37 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Robert Haas <robertmhaas@gmail.com> writes:
>>> Taking stock of the situation this morning, most of the buildfarm is
>>> now green. There are three failures, on eelpout (6 hours ago),
>>> fairywren (17 hours ago), and hyrax (3 days, 7 hours ago).
>> fairywren has now done this twice in the pg_validatebackupCheck step:
>>
>> exec failed: Bad address at /home/pgrunner/bf/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm line 340.
>>  at /home/pgrunner/bf/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm line 340.
>>
>> I'm a tad suspicious that it needs another perl2host()
>> somewhere, but the log isn't very clear as to where.
>>
>> More generally, I wonder if we ought to be trying to
>> centralize those perl2host() calls instead of sticking
>> them into individual test cases.
>>
>>
>
> Not sure about that. I'll see if I can run it by hand and get some
> more info. What's quite odd is that jacana (a very similar setup) is
> passing this happily.
>


OK, tricky, but here's what I did to get this working on fairywren.


First, on Msys2 there is a problem with name mangling. We've had to fix
this before by telling it to ignore certain argument prefixes.


Second, once that was fixed rmdir was failing on the tablespace. On
Windows this is a junction, so unlink is the correct thing to do, I
believe, just as it is on Unix where it's a symlink.


cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--------------524D116BD8080C28075D26CC
Content-Type: text/x-patch; charset=UTF-8;
 name="fix_validate_regression.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="fix_validate_regression.patch"

diff --git a/src/bin/pg_validatebackup/t/003_corruption.pl b/src/bin/pg_validatebackup/t/003_corruption.pl
index 7a09d02e6c..fe717dfc73 100644
--- a/src/bin/pg_validatebackup/t/003_corruption.pl
+++ b/src/bin/pg_validatebackup/t/003_corruption.pl
@@ -16,6 +16,8 @@ $master->start;
 # Include a user-defined tablespace in the hopes of detecting problems in that
 # area.
 my $source_ts_path = TestLib::perl2host(TestLib::tempdir_short());
+my $source_ts_prefix = $source_ts_path;
+$source_ts_prefix =~ s!([^A-Z]:/[^/]*)/.*!$1!;
 $master->safe_psql('postgres', <<EOM);
 CREATE TABLE x1 (a int);
 INSERT INTO x1 VALUES (111);
@@ -105,6 +107,7 @@ for my $scenario (@scenario)
 		# Take a backup and check that it validates OK.
 		my $backup_path = $master->backup_dir . '/' . $name;
 		my $backup_ts_path = TestLib::perl2host(TestLib::tempdir_short());
+                local $ENV{MSYS2_ARG_CONV_EXCL} = $source_ts_prefix;
 		$master->command_ok(['pg_basebackup', '-D', $backup_path, '--no-sync',
 							'-T', "${source_ts_path}=${backup_ts_path}"],
 							"base backup ok");
@@ -177,14 +180,7 @@ sub mutilate_missing_tablespace
 	my ($tsoid) = grep { $_ ne '.' && $_ ne '..' }
 		 slurp_dir("$backup_path/pg_tblspc");
 	my $pathname = "$backup_path/pg_tblspc/$tsoid";
-	if ($windows_os)
-	{
-		rmdir($pathname) || die "$pathname: $!";
-	}
-	else
-	{
-		unlink($pathname) || die "$pathname: $!";
-	}
+	unlink($pathname) || die "$pathname: $!";
 	return;
 }
 

--------------524D116BD8080C28075D26CC--