Subject: Re: Support for NSS as a libpq TLS backend
To: Daniel Gustafsson <daniel@yesql.se>
Cc: Andres Freund <andres@anarazel.de>,
 Postgres hackers <pgsql-hackers@lists.postgresql.org>,
 Michael Paquier <michael@paquier.xyz>,
 Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
 Stephen Frost <sfrost@snowman.net>, Thomas Munro <thomas.munro@gmail.com>
References: 
 <CAA8=A7-suZg90M7Fgyo1Rfymkrpdh_Z547RKXrbHirsAmojQSw@mail.gmail.com>
 <20200904012334.GF19499@paquier.xyz> <20200917074134.GX2873@paquier.xyz>
 <BAD11CCA-E712-4906-8A27-94C2AA0E696E@yesql.se>
 <20200929055939.GF7117@paquier.xyz>
 <DE377D9A-BFFC-492C-A2B8-2AE6EBBBE740@yesql.se>
 <411593A7-E037-474D-BFD7-D3D6683C1D46@yesql.se>
 <B1ADEE76-3E33-4E9D-B830-3AA89B1C42D2@yesql.se>
 <20201020191529.5yverw3ybaube3pg@alap3.anarazel.de>
 <907FA7D1-9BD9-464B-A6EC-DEEB53438D36@yesql.se>
 <20201028063957.cvln377jgao33ssu@alap3.anarazel.de>
 <C86C2DFE-CB21-431B-AA25-CF5BD0779717@yesql.se>
 <994d5fdb-16ff-bb8b-c29f-8bc9f809208c@dunslane.net>
 <977041A2-0103-4150-8174-FB77F6C238A2@yesql.se>
From: Andrew Dunstan <andrew@dunslane.net>
Autocrypt: addr=andrew@dunslane.net; prefer-encrypt=mutual; keydata=
 mQENBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAG0JEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PokBPgQTAQIAKAIbAwYL
 CQgHAwIGFQgCCQoLBBYCAwECHgECF4AFAl3WlJ0FCRLOoywACgkQmfp/y1n8O4GxaQgAmtHC
 VYmtz4XxLLOu0p3aTcYrRNeTiFA5NahXljvOTODLO923Jh2HT4/PNsAOJ3L4Q5SEYoCn9Qh9
 BQUScpUGZmzInrcgE6iAYMqDX+T2CwJciuPt8SkPlYuupwkBmtcKKGH70+dyH4VL3zrK24as
 QUwncv+/632AqNCiIKa0nRCrlq/2bJaeD3ULgxrcuRVNyAwnfHGXqDo+7q865XE8JhQXjkml
 gjB06xaztVrZ8GMCDskSentmgC7a0q5w7MFFWztb1u0rDHfb0jlDaD9yOYGobkAY/E/xuHho
 7sJLotAWWAaN3t6j5Px7j2OW4sCIYM+sxlHAhrE8yiKlyyBmBbkBDQROylhZAQgA0ZzjNBQs
 7GFuo7Z3IPnViGoI1Gab2gpZqp/7CZbi2kN79VVzECc2LN3gtYId/OlvlwcC0aQ4948x/NZV
 /qe90Xt3d6CVu1/n37Mbs7uWVXTFJKgWKJ5AGysa+oMiddhf/iZFFcy3KEzfnf+2LEL1FYuz
 Wih/JiYI89ddjuH1Ht6fcHwGhWtEVx3OUhj80U2EUZb/12QSIaOCdrZGTp7XmSWfhm8UFZSD
 ben3fD+mpdzFjr6T/xMST4LhH4zrDujY8sY6N1n6Q+JRzDkfznN//JknBBfOKxdo127BYHpV
 p2z1gL12PqQ9HVzhUZGoBpgv74sAQ6quW+bZO+pCplG2cQARAQABiQElBBgBAgAPAhsMBQJd
 1pS0BQkSzqNVAAoJEJn6f8tZ/DuBCLkH/2wduZGWhk0JZbBkab472Ib8kZDFLRZZrgZpmrOp
 JNY5+3b+A7QGBqDz2vV2Org8WBJm3Me8pCmpjP/seQ+taHWOlxLzeL+5hjW7IddRS2l0Safh
 u405sS0PRg1omQZ2EBKwQKrpqTD67tYliYAez9KlHqiToeovDskcNVzaHl1WIt9OTSquANru
 yr8tUygmN7kK38q7dhpYBXphKbOg8/kGY9UWo/XU4xCQKq6si729MtcoVZfrbe8KapEMcnp7
 kYIMdsfxRJT7jAkF8/KZmR1wDs0ab+aZZ4mJYEUgPKPNjGuOS7y3L1NklWxpFedL44VDNfP0
 S7bBOkp0eKPjq5g=
Message-ID: <89a6fa41-10fa-06c7-19c6-4f1dc1a9b123@dunslane.net>
Date: Mon, 2 Nov 2020 09:17:00 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.9.0
MIME-Version: 1.0
In-Reply-To: <977041A2-0103-4150-8174-FB77F6C238A2@yesql.se>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Content-Language: en-US
Precedence: bulk


On 11/1/20 5:04 PM, Daniel Gustafsson wrote:
>> On 1 Nov 2020, at 14:13, Andrew Dunstan <andrew@dunslane.net> wrote:
>> I've been looking through the new patch set, in particular the testing=

>> setup.
> Thanks!
>
>> The way it seems to proceed is to use the existing openssl generated
>> certificates and imports them into NSS certificate databases. That see=
ms
>> fine to bootstrap testing,
> That's pretty much why I opted for using the existing certs: to bootstr=
ap the
> patch and ensure OpenSSL-backend compatibility.
>
>> but it seems to me it would be more sound not
>> to rely on openssl at all. I'd rather see the Makefile containing
>> commands to create these from scratch, which mirror the openssl
>> variants. IOW you should be able to build and test this from scratch,
>> including certificate generation, without having openssl installed at =
all.
> I don't disagree with this, but I do also believe there is value in tes=
ting all
> TLS backends with exactly the same certificates to act as a baseline.  =
The
> nssfiles target should definitely be able to generate from scratch, but=
 maybe a
> combination is the best option?


Yeah. I certainly think we need something that should how we would
generate them from scratch using nss. That said, the importation code is
also useful.


>
> Being well versed in the buildfarm code, do you have an off-the-cuff id=
ea onIU
> how to do cross library testing such that OpenSSL/NSS compatibility can=
 be
> ensured?  Andres was floating the idea of making a single sourcetree be=
 able to
> have both for testing but more discussion is needed to settle on a way =
forward.


Well, I'd probably try to leverage the knowledge we have in doing
cross-version upgrade testing. It works like this: After the
install-check-C stage each branch saves its binaries and data files in a
special location, adjusting things like library locations to match. then
to test that version it uses that against all the older versions
similarly saved.


We could generalize that saving mechanism and do it if any module
required it. But instead of testing against a different branch, we'd
test against a different animal. So we'd have two animals, one building
with openssl and one with nss, and they would test against each other
(i.e. one as the client and one as the sever, and vice versa).


This would involve a deal of work on my part, but it's very doable, I
believe.


We'd need a way to run tests where we could specify the client and
server binary locations.


Anyway, those are my thoughts. Comments welcome.


cheers


andrew


--=20
Andrew Dunstan
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company