Message-ID: <f0f8ba63-40b4-44cd-a8f3-625abdae0862@bluegap.ch>
Date: Sat, 1 Nov 2025 00:39:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Enquiry about TDE with PgSQL
To: Bruce Momjian <bruce@momjian.us>,
  Kai Wagner <kai.wagner@percona.com>
Cc: Laurenz Albe <laurenz.albe@cybertec.at>,
  Ron Johnson <ronljohnsonjr@gmail.com>,
  pgsql-general <pgsql-general@postgresql.org>
References: 
 <CACgMzfwSDRF+kQr59h0-xGUobCeFZxwVzE_tUxF18DkVb+vuDQ@mail.gmail.com>
 <CAKAnmmKDCOdUT5JtJZz5papMO0zW1cnG4934d6aQVCQ_KdbUeg@mail.gmail.com>
 <CANzqJaA41CzNjkiQex+A0u9z11i6R3WQZJ+fkXfJO7VJwOMWzg@mail.gmail.com>
 <a1dae4a583d560a90c67872b766ead02ddb13e51.camel@cybertec.at>
 <aQPDvqUzZVKkaxsS@momjian.us>
 <CAG0qCNhL=SEB4vc4v48PxN1F-t8htC463TpX7KDNWQ-s3s8dtA@mail.gmail.com>
 <aQTNmM-3VeTRZdx0@momjian.us>
From: Markus Wanner <markus@bluegap.ch>
Content-Language: en-US
Autocrypt: addr=markus@bluegap.ch; keydata=
 xsNNBEyA6xoBIAD2b/0sj830m1pqjX0LXY6ICB/oE4/UMxij07neWCHdGwCNGdXroz9BRucA
 cjRnVKnt/Xe10PjfgBMo41d20g0vBtGQnRcZzI5r98cfbwkCoND3M2Svh+/NZV9rPjfa5dBo
 Ihd73e/qBpmAc+YM9qGfhzgSXXg8N5lHCLo0CQxN7WrYhhB+G/mPvLjyQEenGi793R9uT5GA
 MLB2ZjbJXIRtdB5Nn+OILwELp08mrQrc6UwN60KmsrH1+1OxJUviMmCSjF573egAmhxdYOSQ
 /3JAgg5rPEAflwLodSUGFd6VQ5AaaOElqJjXTWT23YLN7r6Qq5EbF4VvsMB7ClVvXRr3vzXU
 dwzdukOsnXSv5Do0KL4gQvgNuBj5z//VMpHXoAQJvZpNgtdd4UFe4hJx4esJX0iGtqytl2xc
 QV8aZxdfksuTgzkMT+QcdcP5lB3VngxwObqyKl8NwJcDS46clNDc8qfBz5x0cCU0x4MIb179
 rf9CZ84bVrTbe1zOqeu5rnb+sqxfzSX2IDaKqZEXEUz5s1+pHo3SuxkXVm/xbqJPUIFtrAgQ
 /lt/0UEoNyXm3wxokPTrb9C3QJNyIb3IpQhGWcmBFMNVd+Kb74gjS3IQAdjETGdVYwn1BBEk
 UhMQ07RnFoojFX9hfMyVALCcmuU8mdfJqdO0DfoO9Wo5VZ5CTa8094Kzg7X0oLF1/AMZoB61
 uM2h3eBkzoHsSvgJVOGOZzPUD56Q+wjgJt9QMx7CyMRg2JyrLxmxDxB6e1GUyDK1QDMjq7iB
 nrBI4w4ivMPNC+UH15m0Kb8UmeBpKTcPnLat/2WHVeYcnm9oad9Jp6iEmCPcNsZIOkJcOxLu
 1RJ+Xfeb+1owUS7oCDSwzlM6bzG1TIVcEmXFIdwC3r0Rp9U822JDDHUTapKmRSdLpO59cuZT
 vX9E9k0TU81WtWNsy++MfNC1ed9SrcyTKijnNGz90lb9ewB0IATzyUJpKe2VGAsvj67t9dJB
 pXjg5iiKlQe0ULneg+63YovSVqQpk9E5CCLzchJ3mKCHQwc1qkXOkPC715sE/2uGtdKD3ne8
 rrJvSbz1bMq5G69MJ+QWenDtoFDDiz4LFwEH4uTM3tQtwtEjVnp4UeFErEshM1ZzJG2GP1E4
 CWmGRBcKvpQKy5yg4i4LDApWwa6hhLzkwzRpSl+wCGDSJpo4cuDDnVzuT1LRhX0t80mrRZVi
 a1DY+70tFAGVHweVLWC+LHcQAIxBx3uRdOXPQzzJVM5lIxgd+Slp1aRGqmcxNCIOTofYp9rS
 SIggvq/JgvibJx9WHTPk46X5SZop2Fnm+lzube/fPF8MK7z830wPl+7atB+WarLayo7pMSMU
 OaPqmQPMoY4XABEBAAHNIU1hcmt1cyBXYW5uZXIgPG1hcmt1c0BibHVlZ2FwLmNoPsLDnAQT
 AQgARgIbAwcLCQgKBwQDCBUKCQgLAwIBBRYCAwEAAh4BAheAAhkBFiEE7WdiNgeE4zHiUwPW
 Alr+layd8xsFAlrhndcFCSEstb0ACgkQAlr+layd8xtjXx/9GhTKwxa1Los78B+3y9kmD5kI
 N97tpCd/vJG7oFT13EIe4OXXJiYYBClpaWw6nj/bLkmIZHo4Otc8wLGisRWE+s3L5eXHiBt+
 RKlXmxamVFKGCQDhxPbUBrAsTlI5WWMkbj8J0pOYf2zfWJVRahSE+5840Hc/TzEW3gYF0oFK
 ZqblJO3wlgKs5MJzzsSnite9a1B40TJ7YjknHDDv9fUlr9WIoR/XIwaaphibI0tSGMYOSqSB
 Nu9haQv57eC2RyDz/DP8J3y5RwZJ/lfD1l9AxGCGkwFAX/VyzWjSmDvNz1kSqbs7nn57su4N
 aRl9TtSN/51B7j9WxIsbuyNPtmd26+KDgSteZT9DDNg3e455CZRRv2zo0LaOwpvLfsRh6F5c
 jnOaHYR6fBdjk8Zdw8yN1aLm2Ls0HGWzHSttBLLy7Wa75hIwrHYqz7NykMmy5bDJUvKCjmug
 aE5dJZ5mFsawYTV8PBxhOp+m8584pacBa0wcBrHpQr9+tNZJe8lQGLvxVVsYV+WEDUwc1MCJ
 tFpjS/tq420F5hLVQRCzypi8iXlohGwnlN+qRWE6NuIXBdYYuLQzed9THeNZgMd8zptGByrl
 FwJHPBlDMPLcyQPhJMq/4NksL+6F+iKpbclNpq3gq7soHjlqMQwUTK14LUAQcfzRkExuO7go
 vwtJBf286WY+LU3qG5ErdeNP1OciTKVsC4lH9GevZ0q94OJfFppPIenUK6Co7DLsM4/NKaEI
 JCn9LV2S1sktVLCtNsIiOjAsRS3PRcHVmSZcWDI6QIfINBj5omBiigtR9r38dCPdce+0aYXB
 T4RCleeyqlN/nYFQcMzRdZ/q3LbfGmhDrHUE795q4RwU8qvLIR41NwEi4dYyTKWQphA1iVaa
 D4clk70NrqNZtMacE99iG6k+0PQoTEw2O8BnXTcAXlu3wKaq1QsLTrEhrReqhNRiy4kAMORT
 dlerMgTmh4nbjBUdrhARARxruPp11Dkq3CiytSWwIbXvmIhffGBSV3mQq4iw/d5CWpzBuOzq
 1b87Tnqh36yyUc9l4PkXw6vWgRdwf+0PQRl1VGzuO0j3PY4J/H6RfLMaw8y1VFGqY76c6whw
 XnV5QxzdBw0gl7ekvTYOz/PzDistB6yzcAqdfUZfwIOmD+ewmDzpkzZgClHzrW9tM6L3dO9o
 u8DkPdMDt2CgFYz6gdSPpN3KTI4ASY8A+X5u4SHhMksqm/1M7XpwGAULa77hDidI74MtVd0H
 WVVJjfMcg8SHyOddPDKmwHWCMVLYB5IzdCZ8ZUNplMcbVlEH0K97p/e7IVH4Bwp/aaW0+n2o
 ZoJZ0vjmvjdUQoCpxtOSWIYDYtDymsgSXMnK4/Xc3joZQ844BGN9KYUSCisGAQQBl1UBBQEB
 B0BQ7XndauIOu0Owz1dPCUbA3va75F2YNBody5GaoZI5DgMBCAfCw3wEGAEIACYWIQTtZ2I2
 B4TjMeJTA9YCWv6VrJ3zGwUCY30phQIbDAUJBaOagAAKCRACWv6VrJ3zG+GxH/9PXFilXAQY
 CMiplBKCzAZUkHqmWvkFiC+rwQiUrr2GVLcYY3QC0pTb9Lq6sSJjgqrfrvmBgcG8Gy9PdfM6
 SUb1gi6T2D6wZnWSc2lwiCIJ1sJYl9MN+c0JbkDTybFJ+17IgjUVocousBh+pN4637f9qVVB
 b6FIV77sw4HfH9OVGR0iAUHHqLfbA/3thlz7BjB5mbSUfu/G87kUq9IVX5Nk9MNsT4+x+Gmg
 WqAfcyPwhok2RlSm7UnZjjwiK4so5w1tiKqV6fwNsAntVX0ikT538gSIkB4oQINeQJ6OACVP
 CaB9YqysAZyEiZZazNaU5a1lXa0AUBL+kwmNOMxB/kArDvwcxZNtzKNpW6B9Xug8sawLgD8n
 v/FTI4j6CDZnNw9HrZi1eQ6piEI0qqvGGGQ/aXtjyGKlzkE3xZa7FzjkJQIMsitZ7ZLAgFSP
 +Oj9wLD0mtBEMVWA8Cx/B/DiiEjMUOfg7yRmd3NKVYLfUkQVdQf6fDLXKgiQ0+Fb+c8gWWGF
 HZq0aBOJZ6QcG0CXZEjDtUbxQZALrinKPrjfZWPsOM2GcYxgeLbp51f8sdMg0nzwB9UfoW1o
 5HXhvcqtwehm5Ld5oHLCcJ7aUV04zvSCe0SgRpEfLtOKNVRhG+8skymV4ztjHncV0p+RtJxo
 J2gJiSWk3VTikdEC0V/oM65wt9VEY7LBi3CLu+jczjYyZrExZXKMfTPjyhmK2pPsW/rQS8Le
 e+BEOhEqlt200FhBySPzNX0/DMx0KKg6mKK6/G6DNEcSptqbGB4MsgBlKG9mzSsNd1E/nzqM
 MME/+UM+efSxQdTt1tTFF5cTofbCUmOcIoH/BXS3xl/K2tTUoa23pdpWLvJfM8Y4CzgmwIEi
 uLyWCtzqVjpqhm9f6jJr6qnSZ6hrscoV1c+rK2ok8e/tda1kPTjd3OCzP7akO+kkS7sEzvp3
 XLpJm/FoWyb1j9OGDAvNyBMjZz/WJ2XbFsnt2PdvUW9Kp0oqjatwYh8kaeLZg5PTgCEYtzeo
 qefA3eEB5RxZgyJgjDa3wXkX9n/CczHpXxUc/qFb4FyC5CrzV+EI6eO46PYHfx478C0XurSW
 gmPaSsZ/uwJtEaVHjl+tWIUbpzX0gcUAG9o3Tap6bjecgJFTm4SnJaXfp5JcYJNTPPsuVWCB
 nI5X12lr6Cbc1wKW/uKXe6lHxEHNkflHoYvxwlkWsi3rkzyIRy1F/fwDmD8ZgySD0/AeyFoM
 ISHwIoDr7mqBMgvuWBrsoibaH+vYmo7aYyGsEmJ10EHFckg1MFnmmul0UN35GjyWzICKQVt5
 rXX2QBDETwRSb3B+cb7tmevoHmYn61xM7Z4Aglt2l39NG4mLdrGQzHBRelRo
In-Reply-To: <aQTNmM-3VeTRZdx0@momjian.us>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/f0f8ba63-40b4-44cd-a8f3-625abdae0862%40bluegap.ch>
Precedence: bulk

It's always entertaining to read PCI DSS...

In the "guidance and purpose" column of page 95, the standard reads:

     Disk-level encryption typically encrypts the entire disk
     or partition using the same key, with all data
     automatically decrypted when the system runs or when
     an authorized user requests it.

Granted, there would be benefits from a more fine grained encryption, 
e.g. where you'd encrypt PAN data with a different key. I guess that 
would be called column encryption. But I don't think any version of TDE 
implements it that way. The best I've seen is per tablespace.

Further:

     Many disk-encryption solutions [..] perform the appropriate
     cryptographic transformations without any special action by the user
     other than supplying a password or passphrase at system start-up or
     at the beginning of a session.

Well, isn't that exactly how TDE also works? It loads the key at 
start-up (of the database system, not the operating system, but that's 
equivalent for dedicated database servers) and decrypts "without any 
special action by the user". Otherwise, it would not be very transparent.

     This provides no protection from a malicious individual that
     has already managed to gain access to a valid user account

I see a point here in possible separation of concerns. With a sysadmin 
being able to manage the system, but not having (direct) access to the 
data. That's more of an obscurity rather than hard security, though (as 
the key likely is somewhere in memory fully accessible for a malicious 
sysadmin). Still potentially beneficial in combination with all the 
auditing etc...

So, my conclusions:


  * some potential process level security, no scientific security gains
  * most (if not all) existing TDE products on the market don't actually
    satisfy the stated purpose (or differ enough from disk-level
    encryption)
  * these look like unnecessary hurdles pushed into the standard by
    companies trying to shake off competitors who miss these (somewhat
    dubious) features
  * as an OSS project, it's still pointless to fight the standard and
    much more feasible to just implement that darn feature.

Best Regards,
Markus


On 10/31/25 15:54, Bruce Momjian wrote:
> On Fri, Oct 31, 2025 at 03:01:48PM +0100, Kai Wagner wrote:
>> As I personally believe, there is no real way around TDE in the future, either
>> by extensibility of the core (start with the storage manager and move your way
>> on from there), to make an extension possible, or by directly adding it to the
>> core, there are more reasons coming or are already on their way.
>>
>> With the PCI DSS v4.1 standard, one key rule to comply with is, that "If PAN is
> 
> Uh, I think you mean the 4.0.1 standard, which became active on January
> 1, 2025.  I am surprised this is only being mentioned now:
> 
> 	https://blog.pcisecuritystandards.org/just-published-pci-dss-v4-0-1
> 	
> 	When will PCI DSS v4.0 be retired?
> 	
> 	As with all new versions of PCI DSS, there will be a period where both
> 	the current and updated version will be active at the same time. PCI DSS
> -->	v4.0 will be retired on 31 December 2024. After that point, PCI DSS
> -->	v4.0.1 will be the only active version of the standard supported by PCI
> 	SSC.
> 
> While it was active on Jan 1, it became effective on March 31, 2025:
> 
> 	Does PCI DSS v4.0.1 change the 31 March 2025 effective date for the new
> 	requirements?
> 	
> 	No. This limited revision does not impact the effective date of these
> 	new requirements.
> 
>> stored, it must be rendered unreadable". Of course there are other ways, like
>> tokenization, hashing etc. but this regulation is pushing towards at rest
>> encryption in the long run, and not only disk encryption. We can dislike it,
>> but we are already seeing the need coming from large industries and companies
>> that they cannot work around this anymore, as the auditors doing the checkboxes
>> do not really care about "good alternatives", as they do not even technically
>> understand what this is about. They do compare postgres simply against other
>> already in use databases at these orgs (MySQL or MongoDB), and as such, we are
>> currently the only one that cannot be used in such a use case, at least not
>> without the willingness of the auditor to make it happen.
> 
> I see the new specification that disk-level encryption is insufficient,
> starting on page 93 (page 97 in the PDF URL):
> 
> 	https://www.middlebury.edu/sites/default/files/2025-01/PCI-DSS-v4_0_1.pdf#page=97
> 
> -->	3.5.1.2 If disk-level or partition-level encryption (rather than
> 	file-, column-, or field-level database encryption) is used to
> 	render PAN unreadable, it is implemented only as follows:
> 	
> 	• On removable electronic media
> 	
> 	OR
> 	
> 	• If used for non-removable electronic media, PAN is also
> -->	rendered unreadable via another mechanism that meets Requirement
> 	3.5.1.
> 
> 	...
> 
> 	While disk or partition encryption may still be present on these
> 	types of devices, it cannot be the only mechanism used to protect
> 	PAN stored on those systems. Any stored PAN must also be rendered
> 	unreadable per Requirement 3.5.1—for example, through truncation
> 	or a data-level encryption mechanism. Full disk encryption helps
> 	to protect data in the event of physical loss of a disk and
> 	therefore its use is appropriate only for removable electronic
> 	media storage devices.
> 
> 	Purpose
> 
> 	Disk-level and partition-level encryption typically encrypts
> 	the entire disk or partition using the same key, with all data
> 	automatically decrypted when the system runs or when an authorized
> -->	user requests it. For this reason, disk-level encryption is not
> -->	appropriate to protect stored PAN on computers, laptops, servers,
> 	storage arrays, or any other system that provides transparent
> 	decryption upon user authentication.
> 
> PAN is:
> 
> 	https://www.middlebury.edu/sites/default/files/2025-01/PCI-DSS-v4_0_1.pdf#page=391	
> 	
> 	PAN Acronym for “primary account number.” Unique payment card
> 	number (credit, debit, or prepaid cards, etc.) that identifies
> 	the issuer and the cardholder account.
> 
> So it seems we have somewhat of a stand-off, with the Postgres project
> questioning the value of TDE and the PCI writers doubling-down on
> specifying disk-level encryption as insufficient.
> 
> The biggest possible downside of this standoff is that enterprises that
> need to meet PCI compliance specifications are forced to use specialized
> versions of Postgres or Postgres extensions that support TDE.
> 
> The fact that it has been seven months since PCI 4.0.1 was effective,
> with little to no discussion or movement on adding TDE to community
> Postgres means to me that we are unlikely to see TDE added to community
> Postgres anytime soon.  I have a small hope that adding compression to
> the writing of temporary files will reduce the code changes needed to
> encrypt temporary files, thus reducing the amount of TDE code changes
> needed.
>