Message-ID: <2d7de342-2045-42b4-8dbe-86d18e7c9302@iki.fi>
Date: Fri, 12 Apr 2024 20:03:03 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: PG_TEST_EXTRAs by theme rather than test name (Re: pgsql: Add tests
 for libpq gssencmode and sslmode options)
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Peter Eisentraut <peter@eisentraut.org>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
References: <E1rtcH1-0012Sg-HH@gemulon.postgresql.org>
 <09d4bf5d-d0fa-4c66-a1d7-5ec757609646@eisentraut.org>
 <b396d28a-579f-4b7c-9d9f-8a0a52b5c9b2@iki.fi>
Content-Language: en-US
In-Reply-To: <b396d28a-579f-4b7c-9d9f-8a0a52b5c9b2@iki.fi>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/2d7de342-2045-42b4-8dbe-86d18e7c9302%40iki.fi>
Precedence: bulk

(moved to pgsql-hackers, change subject)

On 10/04/2024 18:54, Heikki Linnakangas wrote:
> On 10/04/2024 17:48, Peter Eisentraut wrote:
>> On 08.04.24 01:50, Heikki Linnakangas wrote:
>>> Add tests for libpq gssencmode and sslmode options
>>
>> Why aren't these tests at
>> src/interfaces/libpq/t/nnn_negotiate_encryption.pl ?
> 
> To be honest, it never occurred to me. It started out as extra tests
> under src/test/ssl/, and when I decided to move them out to its own
> module, I didn't think of moving them to src/interfaces/libpq/t/.
> 
> I will move it, barring any objections or better ideas.

Moved.

I also added an extra check for PG_TEST_EXTRA=kerberos, so that the 
tests that require a MIT Kerberos installation are only run if 
PG_TEST_EXTRA=kerberos is specified. That seems prudent; it seems 
unlikely that you would want to run libpq_encryption tests with Kerberos 
tests included, but not the main kerberos tests. If you specify 
PG_TEST_EXTRA=libpq_encryption, but not 'kerberos', it's probably 
because you don't have an MIT Kerberos installation on your system.

I added documentation for the new PG_TEST_EXTRA=libpq_encryption option, 
I missed that earlier, with a note on the above interaction with 'kerberos'.


As we accumulate more PG_TEST_EXTRA options, I think we should 
categorize the tests by the capabilities they need or the risk 
associated, rather than by test names. Currently we have:

- kerberos: Requires MIT Kerberos installation and opens TCP/IP listen 
sockets
- ldap: Requires OpenLDAP installation and opens TCP/IP listen sockets
- ssl: Opens TCP/IP listen sockets.
- load_balance: Requires editing the system 'hosts' file and opens 
TCP/IP listen sockets.
- libpq_encryption: Opens TCP/IP listen sockets. For the GSSAPI tests, 
requires MIT Kerberos installation
- wal_consistency_checking: is resource intensive
- xid_wraparound: is resource intensive

There are a few clear themes here:

- tests that open TCP/IP listen sockets
- tests that require OpenLDAP installation
- tests that require MIT Kerberos installation
- tests that require editing 'hosts' file
- tests that are resource intensive

We could have PG_TEST_EXTRA options that match those themes, and 
enable/disable the individual tests based on those requirements. For 
example, if you're on a single-user system and have no issue with 
opening TCP/IP listen sockets, you would specify 
"PG_TEST_EXTRA=tcp-listen", and all the tests that need to open TCP/IP 
listen sockets would run. Also it would be nice to have autoconf/meson 
tests for the presence of OpenLDAP / MIT Kerberos installations, instead 
of having to enable/disable them with PG_TEST_EXTRA.

-- 
Heikki Linnakangas
Neon (https://neon.tech)