Date: Mon, 25 Oct 2021 11:58:14 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Bruce Momjian <bruce@momjian.us>
Cc: Sasasu <i@sasa.su>, Robert Haas <robertmhaas@gmail.com>,
	Andres Freund <andres@anarazel.de>,
	PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: XTS cipher mode for cluster file encryption
Message-ID: <20211025155814.GD20998@tamriel.snowman.net>
References: <20211013222648.GA373@momjian.us>
 <20211015192248.GP20998@tamriel.snowman.net>
 <20211015212109.ugwjuhe4lzymnorg@alap3.anarazel.de>
 <CA+TgmoaMHLXrxOj1HA+K=Z_qqgZUOpfdto8+QiVXsfP0ds+ojw@mail.gmail.com>
 <4b73c57e-0941-9e66-ea7e-087793c4c927@sasa.su>
 <20211019185456.GV20998@tamriel.snowman.net>
 <20211023160336.GB22621@momjian.us>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="3L1hu/R33NquA63z"
Content-Disposition: inline
In-Reply-To: <20211023160336.GB22621@momjian.us>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20211025155814.GD20998%40tamriel.snowman.net>
Precedence: bulk


--3L1hu/R33NquA63z
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Bruce Momjian (bruce@momjian.us) wrote:
> On Tue, Oct 19, 2021 at 02:54:56PM -0400, Stephen Frost wrote:
> > * Sasasu (i@sasa.su) wrote:
> > > A unified block-based I/O API sounds great. Has anyone tried to do th=
is
> > > before? It would be nice if the front-end tools could also use these =
API.
> >=20
> > The TDE patch from Cybertec did go down this route, but the API ended up
> > being rather different which menat a lot of changes in other parts of
> > the system.  If we can get a block-based temporary file method that
> > maintains more-or-less the same API, that'd be great, but I'm not sure
> > that we can really do so and I am not entirely convinced that we should
> > make the TDE effort depend on an otherwise quite independent effort of
> > making all temp files usage be block based.
>=20
> Uh, I thought people felt the Cybertec patch was too large and that a
> unified API for temporary file I/O-encryption was a requirement.  Would
> a CTR-steaming-encryption API for temporary tables be easier to
> implement?

Having a unified API for temporary file I/O (that could then be extended
to provide encryption) would definitely help with reducing the size of a
TDE patch.  The approach used in the Cybertec patch was to make
temporary file access block based, but the way that was implemented was
with an API different from pread/pwrite and that meant changing pretty
much all of the call sites for temporary file access, which naturally
resulted in changes in a lot of otherwise unrelated code.

There was an argument put forth that a block-based API for temporary
file access would generally be good as it would mean fewer syscalls.  If
we can get behind that and make it happen in (relatively) short order
then we'd certainly be better off when it comes to implementing TDE
which also deals with temporary files.  I'm a bit concerned about that
approach due to the changes needed but I'm also not against it.  I do
think that an API which was more-or-less the same as what's used today
would be a smaller change and therefore might be easier to get in and
that it'd also make a TDE patch smaller.  Perhaps both could be
accomplished (an API that's similar to today, but the actual file access
being block-based).

Either way, we should get that unification done in the core code first
as an independent effort.  That's what I hope the Cybertec folks have
had a chance to work on.

As for the specific encryption method to use, using CTR would be simpler
as it doesn't require access to be block-based, though we would need to
make sure to not re-use the IV across any of the temporary files being
created (potentially concurrently).  Probably not that hard to do but
just something to make sure we do.  Of course, if we arrange for
block-based access then we could use XTS or perhaps GCM/GCM-SIV if we
wanted to.

Thanks,

Stephen

--3L1hu/R33NquA63z
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJhdtQWAAoJEO1sijiDR2RVUWcP/RGoqAndca52Wp6Unhfqy1gC
SjsjBext2GG2g+gq5wWYyQy6gB9T/UnGPjxxCEZgSuQX7pAjfUmc5soNzjykxV5/
t6TA+yPflOldxMcqmgt16tvomD39Znl6s/6svN8656BZBj7+FvuNGt3YWMKqPI0E
hKGrvXLpd1BlP1fjqRI1rayhIUAQ4U+eUgqYah4GGzbkUgS7Np9xJdll6aABqUr7
gUsLOKk4jugXSrLd8JwMkrq3QaXJyU03XZKYmtmTmTmvTD/78Vb+O8MD1XXNZ0T5
oVETXXjqe3/SmpgDkvlgaCUuqbhSeb5/H0SF7iREXJyh4+9eaI+/mEn5qsbE1LEt
RofCVkuS/huYmIdm+Chr6gzrLWrinf3/0k0d5aUulZW5ORrYf+lkQKbBITqjo9fn
qbXWTnFnYS5BE0xromc25h4OlUabz0UDR8V2cYOJHUTvnMbJWlra/JVcg8vj0Yx6
4Qk38lZ2sm1/BdG/GF9D6llenrmY9MGX1OR48911XkMuXEUyhB8k6i5WYFtMZfg8
1o6aPwh0LaXJX09X0R5cv5LrN0MFSFYdxOD5lgf7+m879fRWwzIDlTG+LiBCXFZg
AbTPaNokdP8LTfjoLV9qLe9RLw8MNoxBp2PDc8+yWAImCtbJUQB5RhIjdIusWXsb
YkHTmFlhe0k6NAYur+uc
=m2y/
-----END PGP SIGNATURE-----

--3L1hu/R33NquA63z--