Date: Thu, 18 Mar 2021 13:46:28 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Bruce Momjian <bruce@momjian.us>,
	Masahiko Sawada <sawada.mshk@gmail.com>,
	Tom Kincaid <tomjohnkincaid@gmail.com>,
	Robert Haas <robertmhaas@gmail.com>,
	Andres Freund <andres@anarazel.de>,
	Amit Kapila <amit.kapila16@gmail.com>,
	Thomas Munro <thomas.munro@gmail.com>,
	PostgreSQL-development <pgsql-hackers@postgresql.org>,
	Masahiko Sawada <masahiko.sawada@2ndquadrant.com>
Subject: Re: Key management with tests
Message-ID: <20210318174628.GK20766@tamriel.snowman.net>
References: <20210318172152.GJ20766@tamriel.snowman.net>
 <20210318173743.GA13514@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="yFHZhmpl2995/sJc"
Content-Disposition: inline
In-Reply-To: <20210318173743.GA13514@alvherre.pgsql>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20210318174628.GK20766%40tamriel.snowman.net>
Precedence: bulk


--yFHZhmpl2995/sJc
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Alvaro Herrera (alvherre@alvh.no-ip.org) wrote:
> On 2021-Mar-18, Stephen Frost wrote:
>=20
> > * Alvaro Herrera (alvherre@alvh.no-ip.org) wrote:
> > > Patch 10 uses the term "WAL-skip relations".  What does that mean?  Is
> > > it "relations that are not WAL-logged"?  I suppose we already have a
> > > term for this; I'm not sure it's a good idea to invent a different te=
rm
> > > that is only used in this new place.
> >=20
> > This is discussed in src/backend/access/transam/README, specifically the
> > section that talks about Skipping WAL for New RelFileNode.  Basically,
> > it's the 'wal_level=3Dminimal' optimization which allows WAL to be
> > skipped.
>=20
> Hmm ... that talks about WAL-skipping *changes*, not WAL-skipping
> *relations*.  I thought WAL-skipping meant unlogged relations, but
> I understand now that that's unrelated.  In the transam/README, WAL-skip
> means a change in a transaction in a relfilenode that, if rolled back,
> would disappear; and I'm not sure I understand how the code is handling
> the case that a relation is under that condition.
>=20
> This caught my attention because a comment says "encryption does not
> support WAL-skipped relations", but there's no direct change to the
> definition of RelFileNodeSkippingWAL() to account for that.  Perhaps I
> am just overlooking something, since I'm just skimming anyway.

This is relatively current activity and so it's entirely possible
comments and perhaps code need further updating in this area, but to
explain what's going on in a bit more detail-=20

Ultimately, we need to make sure that LSNs aren't re-used.  There's two
sources of LSNs today: those for relations which are being written into
the WAL and those for relations which are not (UNLOGGED relations,
specifically).  The 'minimal' WAL level introduces complications with
this requirement because tables created (or truncated) inside a
transaction are considered permanent once they're committed, but the
data pages in those relations don't go into the WAL and the LSNs on the
pages of those relations isn't guaranteed to be either unique or even
necessarily set, and if we were to generate LSNs for those it would be
required to be done by actually advancing the WAL LSN, which would
require writing into the WAL and therefore wouldn't be quite the
optimization that's expected.

I'm not sure if it's been explicitly done yet but I believe the idea is,
based on my last discussion with Bruce, at least initially, simply
disallow encrypted clusters from running with wal_level=3Dminimal to avoid
this issue.

Thanks,

Stephen

--yFHZhmpl2995/sJc
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJgU5HzAAoJEO1sijiDR2RVBh0QAJZWL7FToayHbyAKjeRw8fuc
Er1Ug9RNxHqfZY4jcZC4C4PJjzjotf3rSIkXq14ZEaGEYdo6obrppl9K6mrqXpDZ
kCsVqEyUnc/liMGj6uCWU/V47O318kIiM+CDsbfUd4dclpE4M/5j/8D3U1yTb6x1
28fDnMZRtW9tWVhydOTQKVg6IpqdCfOd9jN5O1gAdqkj4CzzYm+iZThVRTG6f9ik
U+FXmZCzWF1nonYajyDmGDWu6lsWVu/3Exw0rgu+pZp6updLdkLWuZ33+LUargb+
NjDiF2gaXcoANiucYIDAM6YNEgUNaBZ0tfFSqYOKuPBPK3SbvgUKsNLI/8CuEEqo
QPyKOsOzF0LC4yfWzMLgM8AXcuFtCxj5tMG6lCJVCnMJyrF3xFKczhFfDdEfJ3FW
OBQTiRbdzNzF5T2J1x/gItV1KFT3EuxKYDVAnvK/6SfTJSmVDFUTOsh0GDilzqtK
S5o3q4P8ZPDqze84Uew8+ytPEbFzQmbTC2/vTWUfPsGGwHF+DOTq9crlTAhzYtwj
jugQNmcertaeG+5Y8n0QT4XncL49yQRrRuMULfPR5NYr4aO4ScM/QlVAY9W+G04P
S/PIOgqouQB1ZUZ28T/RR2V3xrt0jJHD7xV38+5Hr/LyV44HaxtNYFMovb9BzZ1x
1FvXX53oydzDHmASgST/
=ahwv
-----END PGP SIGNATURE-----

--yFHZhmpl2995/sJc--