From: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Subject: Re: [PATCH] pgbench tap tests fail if the path contains a perl
 special character
To: Fabien COELHO <coelho@cri.ensmp.fr>, Tom Lane <tgl@sss.pgh.pa.us>
Cc: =?UTF-8?B?UmHDumwgTWFyw61uIFJvZHLDrWd1ZXo=?= <rmrodriguez@carto.com>,
 Alvaro Herrera <alvherre@2ndquadrant.com>,
 PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
References: <201901181927.sbv5o72uu4cc@alvherre.pgsql>
 <29026.1547841766@sss.pgh.pa.us> <32650.1547843169@sss.pgh.pa.us>
 <CAM6_UM7fi-v5e90ghaZCLEfU4_Tk0n9=M6no5f1eXnMeLSeN0g@mail.gmail.com>
 <20202.1548035461@sss.pgh.pa.us> <alpine.DEB.2.21.1901210834280.2246@lancre>
Openpgp: preference=signencrypt
Autocrypt: addr=andrew.dunstan@2ndQuadrant.com; prefer-encrypt=mutual;
 keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNL0FuZHJldyBEdW5zdGFuIDxhbmRyZXcuZHVuc3RhbkAybmRRdWFkcmFudC5jb20+wsCY
 BBMBAgBCAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAUJDyV6hBYhBOQ+WEYd/Hy/RGkV
 pZn6f8tZ/DuBBQJcH/EVAhkBAAoJEJn6f8tZ/DuB7BEH/iDWZEbVft2m2MT6SPjARzTupt/l
 xLLZ/Cl6n2US2hov5+fBbnR5xvECk/YCOfG9ICvKNlpFjceqNLRKStd8cmndnI+kDw91uNM2
 hpL4aJMeW65XCHvqXlOLSwv5chBpXz9NG7FMaJ+q+59sWEWjcTNmjXgkEHn53y+1pc7nSLDe
 cr5ZI1aB3nGXRm1SoDY5PYZkUKMykW+YBZwgcs/Q5tRaE+siN7DKbz8E628+PerwAkmHPVXN
 MdJFD6gs4S0gDFA8IxcyIwZU2v4By+brzcJ9Se3FK5BYxBDHtaYZxgSrJAvOUOrLIZPQTQfL
 TtnVs3C3JTfrxCH1ceHtnA6dRuTOwE0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oK
 Waqf+wmW4tpDe/VVcxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7
 llV0xSSoFiieQBsrGvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8
 BoVrRFcdzlIY/NFNhFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C
 4R+M6w7o2PLGOjdZ+kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaY
 L++LAEOqrlvm2TvqQqZRtnEAEQEAAcLAZQQYAQIADwIbDAUCXA6fpAUJDyV6xAAKCRCZ+n/L
 Wfw7geU4B/9vFFsVmWxerDWgNjyiK80zIPmlw/iwdUlU6FIu0JWLMg4093kvxBs7RMZV9hBq
 n3ekBvD4IvFsVLJrw9qzSI2FH3UeYy5WXpEs01Ppia60NOlvRDqXbryzuvfe+iYo6bVP3Ikr
 TPTNYL/BePUTD2DPvSI82SbGTyzuA7SJv6NRqqQDMIy1fU6KyQFLmQJp/QJyhKWzm9jWh8ee
 w8jVaoMY0Kbq5N7RqisEx1Lu/Iw646UphdYzQNymf+kbiU5Kf4I6DhzV8w9sie1Bo1juNj3I
 c5/fFJsmxljbnaDbXPcnFsur1Oo9mIHFakgWgOBMuxoWfjONiLjjLh/9+ZIqSSLX
Message-ID: <fde787ee-9c2d-46da-3ece-f7ae64a70b33@2ndQuadrant.com>
Date: Mon, 21 Jan 2019 10:40:47 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.3.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.21.1901210834280.2246@lancre>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Language: en-MW
Precedence: bulk


On 1/21/19 4:50 AM, Fabien COELHO wrote:
>
> Hello Tom,
>
>> Hm, so bowerbird (a Windows machine) has been failing the pgbench test=
s
>> since this went in, cf
>>
>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dbowerbird&dt=
=3D2019-01-20%2004%3A57%3A01
>>
>>
>> I'm just guessing, but I suspect that bowerbird is using a path spec
>> that
>> includes a backslash directory separator and that's somehow bollixing
>> things.
>
> This point is unclear from the log, where plain slashes are used in
> the log prefix path, and furthermore no strange characters appear in
> the log prefix path:
>
> =C2=A0# Running: pgbench -n -S -t 50 -c 2 --log
> --log-prefix=3DG:/prog/bf/root/HEAD/pgsql.build/src/bin/pgbench/tmp_che=
ck/t_001_pgbench_with_server_main_data/001_pgbench_log_2
> --sampling-rate=3D0.5
> =C2=A0ok 345 - pgbench logs status (got 0 vs expected 0)
> =C2=A0ok 346 - pgbench logs stdout /(?^:select only)/
> =C2=A0ok 347 - pgbench logs stdout /(?^:processed: 100/100)/
> =C2=A0ok 348 - pgbench logs stderr /(?^:^$)/
> =C2=A0not ok 349 - number of log files
>
>> If so, we might be able to fix it by converting backslashes to
>> forward slashes before applying quotemeta().
>>
>> It's also possible that on Windows, "glob" handles backslashes
>> differently than it does elsewhere, which would be harder to fix
>> --- that would bring back my original fear that the appropriate
>> quoting rules are different for "glob" than for regexes.
>
> I'm afraid it could indeed be due to platform-specific behavior, so
> testing on the target machine to understand the actual behavior would
> help.
>
> I just tested the current version on my laptop within a directory
> containing spaces and other misc chars: Pgbench TAP tests are
> currently broken in this context on master, and this may be used to a
> collection of issues, not just one, eg pgbench function splits
> parameters on spaces which breaks if there are spaces in bdir.
>
> I'm going to investigate, possibly over next week-end, so please be
> patient.
>
> About windows-specific issues, from File::Glob man page:
>
> """ On DOSISH systems, backslash is a valid directory separator
> character. In this case, use of backslash as a quoting character (via
> GLOB_QUOTE) interferes with the use of backslash as a directory
> separator. ... """
>
> It seems to suggest that quotemeta backslash-on-nearly-everything
> approach is not appropriate.
>


File globbing is not the same thing as regex matching. For example, '.'
is a literal character in a glob pattern but a metacharacter in a regex,
while ' ' is a literal character in a regex but a globbing metacharacter
(but see below), and '*' is a metacharacter in both but has different
meanings. quotemeta is intended for regexes but being used here on an
expression used for globbing.

Perhaps it would be OK we changed back the glob line to use $prefix
instead of $qprefix, but kept the use of $qprefix in the later regex.

To deal with the issue of spaces in file names (not an issue eher ob
bowerbird), we should consider adding this:

=C2=A0=C2=A0=C2=A0 use File::Glob ':bsd_glob';

This removes the globbing metcharacter nature of the space character,
although it might have other effects that cause pain. See `perldoc
File::Glob`


cheers


andrew


--=20
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services