Subject: Re: Enquiry about TDE with PgSQL

From: Greg Sabino Mullane <[email-masked]>
Date: Thu, 16 Oct 2025 18:04:49 -0400

>
> I would like to enquire that based on the anecdotal experience of group
> members, which TDE solution works best for PgSQL 17 databases.

Generally speaking, there is no "best". People use whatever vendor they
happen to already use. Your best solution is to avoid TDE altogether. If
you really need encryption at rest, have the OS do it. That works well
(transparently, even), is very battle-tested, and has minimal performance
impact. TDE, on the other hand, is a very complex and difficult thing to
add into Postgres. Currently it means you are using a forked version of
Postgres and are incurring overhead every time you read or write to disk.

 The scenario I have is of a large number of tables (15-20K) and some
> tables with 100M tuples each. The total database size is 4TB.

The size and number of tables does not really matter. How often you write
WAL, and how often things move in and out of shared buffers is what matters.

Cheers,
Greg

---

From: Ron Johnson <[email-masked]>
Date: Fri, 17 Oct 2025 00:49:06 -0400

On Thu, Oct 16, 2025 at 6:05 PM Greg Sabino Mullane <[email-masked]>
wrote:

> impact.

But filesystem encryption still means that validly logged-in users see the
unencrypted data.  That's great for a laptop that might get stolen, or for
drives that are discarded without being wiped, but are no protection
against hackers who want to exfiltrate your data.

(Neither protect against ransomware, but that's a different problem.)

> TDE, on the other hand, is a very complex and difficult thing to add
> into Postgres.
>

TDE was added to SQL Server, with (to us, at least) minimally-noticed
overhead.  Oracle has it, too, but I don't know the details.

The bottom line is that requirements for TDE are escalating, whether you
like it or not, as Yet Another Layer Of Defense against hackers
exfiltrating data, and then threatening to leak it to the public.

-- 
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

---

From: Laurenz Albe <[email-masked]>
Date: Fri, 17 Oct 2025 09:01:52 +0200

On Fri, 2025-10-17 at 00:49 -0400, Ron Johnson wrote:
> On Thu, Oct 16, 2025 at 6:05 PM Greg Sabino Mullane <[email-masked]> wrote:
> > 
> > TDE, on the other hand, is a very complex and difficult thing to add into Postgres.
> 
> TDE was added to SQL Server, with (to us, at least) minimally-noticed overhead.
> Oracle has it, too, but I don't know the details.
> 
> The bottom line is that requirements for TDE are escalating, whether you like it or
> not, as Yet Another Layer Of Defense against hackers exfiltrating data, and then
> threatening to leak it to the public.

Bruce Momjian has interesting things to say about that in
https://compiledconversations.com/6/ (unfortunately I don't remember where
exactly in this 84 minute piece).

It is a feature that users want (or need to comply with whatever they feel
they have to comply with).  On the other hand, it has very limited technical
or security value, which hampers its acceptance into core.

Yours,
Laurenz Albe

---

From: Ron Johnson <[email-masked]>
Date: Fri, 17 Oct 2025 08:47:40 -0400

On Fri, Oct 17, 2025 at 3:01 AM Laurenz Albe <[email-masked]>
wrote:

> overhead.
> like it or

I gave you a reason: "Yet Another Layer Of Defense against hackers
exfiltrating data".  It's the same reason PgBackRest encrypts backups.

---

From: Bruce Momjian <[email-masked]>
Date: Thu, 30 Oct 2025 15:59:58 -0400

> exactly in this 84 minute piece).

Here is my most recent blog about TDE:

	https://momjian.us/main/blogs/pgblog/2025.html#February_22_2025

-- 
  Bruce Momjian  <[email-masked]>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

---

From: Kai Wagner <[email-masked]>
Date: Fri, 31 Oct 2025 15:01:48 +0100

As I personally believe, there is no real way around TDE in the future,
either by extensibility of the core (start with the storage manager and
move your way on from there), to make an extension possible, or by directly
adding it to the core, there are more reasons coming or are already on
their way.

With the PCI DSS v4.1 standard, one key rule to comply with is, that "If
PAN is stored, it must be rendered unreadable". Of course there are other
ways, like tokenization, hashing etc. but this regulation is pushing
towards at rest encryption in the long run, and not only disk encryption.
We can dislike it, but we are already seeing the need coming from large
industries and companies that they cannot work around this anymore, as the
auditors doing the checkboxes do not really care about "good alternatives",
as they do not even technically understand what this is about. They do
compare postgres simply against other already in use databases at these
orgs (MySQL or MongoDB), and as such, we are currently the only one that
cannot be used in such a use case, at least not without the willingness of
the auditor to make it happen.

On Thu, Oct 30, 2025 at 9:00 PM Bruce Momjian <[email-masked]> wrote:

> https://url.avanan.click/v2/r01/___https://compiledconversations.com/6/___.YXAzOnBlcmNvbmE6YTpnOjMxNTMyOGQ0MzIyODAzOTU2M2E4OWUxZjNjZmMyYTRlOjc6MDJjZDplMmZkOGI3NTExNjAzNDI4YzZlZTZjMDQwNjU1YWQyZTVlYzU4NmQ4NjMzYzQxZGVjNzUxMGM5MmM0YThkM2M5OnA6VDpO
> (unfortunately I don't remember where
> https://url.avanan.click/v2/r01/___https://momjian.us/main/blogs/pgblog/2025.html%23February_22_2025___.YXAzOnBlcmNvbmE6YTpnOjMxNTMyOGQ0MzIyODAzOTU2M2E4OWUxZjNjZmMyYTRlOjc6ODI4Nzo4OTUwMTcwNDljNjA0OGYxNzU3MDhlMjhiNDIwZjNiNzNjYTZmNWJjZmM2MmNmNWJkMGFhNTllOTMzNjA2Y2EyOnA6VDpO
>
> --
>   Bruce Momjian  <[email-masked]>
> https://url.avanan.click/v2/r01/___https://momjian.us___.YXAzOnBlcmNvbmE6YTpnOjMxNTMyOGQ0MzIyODAzOTU2M2E4OWUxZjNjZmMyYTRlOjc6ZTBkOTozZDU5MmRlNGI0YTU5ZmIxM2UzNmE1NTgzY2U1YjBjNmZlZWMwNmEyNzBhYjdlYTlhNDhlZTU4MGVjMDQ4MTk5OnA6VDpO
>   EDB
> https://url.avanan.click/v2/r01/___https://enterprisedb.com___.YXAzOnBlcmNvbmE6YTpnOjMxNTMyOGQ0MzIyODAzOTU2M2E4OWUxZjNjZmMyYTRlOjc6ZWFlNjoyYWE0NWVmY2EwZTBhNGM3Y2Q2NzQwNDQ5NmM5OGMwODkxNDUxYzY2YmI4NWZhNzM0NmUwZjI1Mzg4NzE4ZDhhOnA6VDpO
>

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 31 Oct 2025 10:54:16 -0400

On Fri, Oct 31, 2025 at 03:01:48PM +0100, Kai Wagner wrote:
> As I personally believe, there is no real way around TDE in the future, either
> by extensibility of the core (start with the storage manager and move your way
> on from there), to make an extension possible, or by directly adding it to the
> core, there are more reasons coming or are already on their way.
> 
> With the PCI DSS v4.1 standard, one key rule to comply with is, that "If PAN is

Uh, I think you mean the 4.0.1 standard, which became active on January
1, 2025.  I am surprised this is only being mentioned now:

	https://blog.pcisecuritystandards.org/just-published-pci-dss-v4-0-1
	
	When will PCI DSS v4.0 be retired?
	
	As with all new versions of PCI DSS, there will be a period where both
	the current and updated version will be active at the same time. PCI DSS
-->	v4.0 will be retired on 31 December 2024. After that point, PCI DSS
-->	v4.0.1 will be the only active version of the standard supported by PCI
	SSC.

While it was active on Jan 1, it became effective on March 31, 2025:

	Does PCI DSS v4.0.1 change the 31 March 2025 effective date for the new
	requirements?
	
	No. This limited revision does not impact the effective date of these
	new requirements. 

> stored, it must be rendered unreadable". Of course there are other ways, like
> tokenization, hashing etc. but this regulation is pushing towards at rest
> encryption in the long run, and not only disk encryption. We can dislike it,
> but we are already seeing the need coming from large industries and companies
> that they cannot work around this anymore, as the auditors doing the checkboxes
> do not really care about "good alternatives", as they do not even technically
> understand what this is about. They do compare postgres simply against other
> already in use databases at these orgs (MySQL or MongoDB), and as such, we are
> currently the only one that cannot be used in such a use case, at least not
> without the willingness of the auditor to make it happen.

I see the new specification that disk-level encryption is insufficient,
starting on page 93 (page 97 in the PDF URL):

	https://www.middlebury.edu/sites/default/files/2025-01/PCI-DSS-v4_0_1.pdf#page=97

-->	3.5.1.2 If disk-level or partition-level encryption (rather than
	file-, column-, or field-level database encryption) is used to
	render PAN unreadable, it is implemented only as follows:
	
	• On removable electronic media
	
	OR
	
	• If used for non-removable electronic media, PAN is also
-->	rendered unreadable via another mechanism that meets Requirement
	3.5.1.

	...

	While disk or partition encryption may still be present on these
	types of devices, it cannot be the only mechanism used to protect
	PAN stored on those systems. Any stored PAN must also be rendered
	unreadable per Requirement 3.5.1—for example, through truncation
	or a data-level encryption mechanism. Full disk encryption helps
	to protect data in the event of physical loss of a disk and
	therefore its use is appropriate only for removable electronic
	media storage devices.

	Purpose

	Disk-level and partition-level encryption typically encrypts
	the entire disk or partition using the same key, with all data
	automatically decrypted when the system runs or when an authorized
-->	user requests it. For this reason, disk-level encryption is not
-->	appropriate to protect stored PAN on computers, laptops, servers,
	storage arrays, or any other system that provides transparent
	decryption upon user authentication.

PAN is:

	https://www.middlebury.edu/sites/default/files/2025-01/PCI-DSS-v4_0_1.pdf#page=391	
	
	PAN Acronym for “primary account number.” Unique payment card
	number (credit, debit, or prepaid cards, etc.) that identifies
	the issuer and the cardholder account.

So it seems we have somewhat of a stand-off, with the Postgres project
questioning the value of TDE and the PCI writers doubling-down on
specifying disk-level encryption as insufficient.

The biggest possible downside of this standoff is that enterprises that
need to meet PCI compliance specifications are forced to use specialized
versions of Postgres or Postgres extensions that support TDE.

The fact that it has been seven months since PCI 4.0.1 was effective,
with little to no discussion or movement on adding TDE to community
Postgres means to me that we are unlikely to see TDE added to community
Postgres anytime soon.  I have a small hope that adding compression to
the writing of temporary files will reduce the code changes needed to
encrypt temporary files, thus reducing the amount of TDE code changes
needed.

---

From: Adrian Klaver <[email-masked]>
Date: Fri, 31 Oct 2025 08:21:18 -0700

> specifying disk-level encryption as insufficient.

Yeah, what I would like to know is how many of the data breaches 
actually grab directly from the storage versus getting it through the 
database or other software above the storage? It seems to me social 
engineering plays a bigger role in this.

-- 
Adrian Klaver
[email-masked]

---

From: Greg Sabino Mullane <[email-masked]>
Date: Fri, 31 Oct 2025 11:25:04 -0400

>         decryption upon user authentication.

Hmm, I read this a few times but still not sure what the technical
objection is. Yes, the entire disk is encrypted with the same key, but why
is that insufficient to protect things? Anyone care to guess what they are
thinking here?

> versions of Postgres or Postgres extensions that support TDE.

Not always a downside for the companies selling those specialized versions
though.

Cheers,
Greg

--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 31 Oct 2025 11:34:30 -0400

On Fri, Oct 31, 2025 at 11:25:04AM -0400, Greg Sabino Mullane wrote:
> Hmm, I read this a few times but still not sure what the technical objection
> is. Yes, the entire disk is encrypted with the same key, but why is that
> insufficient to protect things? Anyone care to guess what they are thinking
> here?

This is more an argument for security using layers.  With storage
encryption, the file system as visible is unencrypted, and backups of
that file system can be unencrypted.

Community Postgres relies on file system permissions to secure the data
directory.  You can argue that once file system permissions are
bypassed, security is impossible, and agree with that, but some feel the
extra step needed to pull the Postgres encryption key from memory is a
security feature.  And for backups, the Postgres encryption key might
not be in memory.  Of course, forcing encrypted backups is a solution,
but an extra step.

> Not always a downside for the companies selling those specialized versions

Yes, no question they are happy.

---

From: Ron Johnson <[email-masked]>
Date: Fri, 31 Oct 2025 11:34:45 -0400

> is that insufficient to protect things? Anyone care to guess what they are

Networking.

Who breaks into a DC and steals a rack of disks or SSDs?  Very, very few
evil-doers.
Who hacks into networks and exfiltrates data over the wire?  Many hackers.

---

From: Adrian Klaver <[email-masked]>
Date: Fri, 31 Oct 2025 08:37:44 -0700

On 10/31/25 08:25, Greg Sabino Mullane wrote:
> On Fri, Oct 31, 2025 at 10:54 AM Bruce Momjian <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
> objection is. Yes, the entire disk is encrypted with the same key, but 
> why is that insufficient to protect things? Anyone care to guess what 
> they are thinking here?

My best guess, is that the more the storage encryption is fragmented by 
different keys the less likely all the disk could be decrypted by a 
single action. The weak link is '... other system that provides 
transparent decryption upon user authentication.'. At some point the 
data needs to be decrypted for end user use. That means the point of 
attack is moved to the user and storage encryption does not really matter.

> 
>     The biggest possible downside of this standoff is that enterprises
>     that need to meet PCI compliance specifications are forced to use
>     specialized versions of Postgres or Postgres extensions that support
>     TDE.
> 
> 
> Not always a downside for the companies selling those specialized 
> versions though.
> 
> Cheers,
> Greg
> 
> --
> Crunchy Data - https://www.crunchydata.com <https://www.crunchydata.com>

---

From: Laurenz Albe <[email-masked]>
Date: Fri, 31 Oct 2025 17:40:31 +0100

> engineering plays a bigger role in this.

This is not about actual security considerations, it is about checkboxes.
Consequently, rational arguments are missing the point.

Yours,
Laurenz Albe

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 31 Oct 2025 12:49:25 -0400

> Consequently, rational arguments are missing the point.

I think the big question is that, now with the effective PCI spec
disallowing only storage-level encryption, can we, as a project,
continue to reject in-core TDE because it is a check-box item.

---

From: Adrian Klaver <[email-masked]>
Date: Fri, 31 Oct 2025 10:01:07 -0700

> Consequently, rational arguments are missing the point.

Alright, been there.

Years ago I used to drive a delivery truck for wholesale greenhouse and 
one of my chores was to go to a remote greenhouse we operated and 
pickup/deliver plants. There was a whole process for securing the key 
that you used to open the entry door. I pointed out that the greenhouse 
walls where two layers of plastic inflated by an air blower and then I 
proceeded to pull out my pocket knife as an example of a 'universal' 
key. The door key process stayed because it made people feel the 
greenhouse contents where safe. FYI, things did get stolen though that 
was because folks left them outside and the thieves did not have to 
bother with a knife.

---

From: Christophe Pettus <[email-masked]>
Date: Fri, 31 Oct 2025 10:04:35 -0700

> On Oct 31, 2025, at 08:21, Adrian Klaver <[email-masked]> wrote:
> Yeah, what I would like to know is how many of the data breaches actually grab directly from the storage versus getting it through the database or other software above the storage?

Essentially zero.

PCI, like a lot of data security standards, are a magpie's assemblage of things that the authors have heard about all of which sound "secure" to them.  However, since these particular magpies have machine guns (metaphorically) and can do serious damage to businesses, we must play along with the masquerade.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 31 Oct 2025 13:06:21 -0400

On Fri, Oct 31, 2025 at 10:04:35AM -0700, Christophe Pettus wrote:
>
>
> > On Oct 31, 2025, at 08:21, Adrian Klaver <[email-masked]>
> > wrote:  Yeah, what I would like to know is how many of the data
> > breaches actually grab directly from the storage versus getting it
> > through the database or other software above the storage?
>
> Essentially zero.
>
> PCI, like a lot of data security standards, are a magpie's assemblage
> of things that the authors have heard about all of which sound
> "secure" to them.  However, since these particular magpies have
> machine guns (metaphorically) and can do serious damage to businesses,
> we must play along with the masquerade.

Yes, we have been avoiding the masquerade for years.  The question is
can we continue.  From the lack of discussion since April 1, 2025, it
seems the answer is yes.

---

From: "Clay Jackson (cjackson)" <[email-masked]>
Date: Fri, 31 Oct 2025 17:32:23 +0000

Pardo me for jumping in here - but would filesystem level encryption possibly meet your requirements?

Clay Jackson
Database Solutions Sales Engineer
[email-masked]
office  949-754-1203  mobile 425-802-9603

-----Original Message-----
From: Bruce Momjian <[email-masked]>
Sent: Friday, October 31, 2025 10:06 AM
To: Christophe Pettus <[email-masked]>
Cc: Adrian Klaver <[email-masked]>; Kai Wagner <[email-masked]>; Laurenz Albe <[email-masked]>; Ron Johnson <[email-masked]>; pgsql-general <[email-masked]>
Subject: Re: Enquiry about TDE with PgSQL

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

> we must play along with the masquerade.

Yes, we have been avoiding the masquerade for years.  The question is can we continue.  From the lack of discussion since April 1, 2025, it seems the answer is yes.

--
  Bruce Momjian  <[email-masked]>        https://momjian.us/
  EDB                                      https://enterprisedb.com/

---

From: Christophe Pettus <[email-masked]>
Date: Fri, 31 Oct 2025 17:16:09 -0700

> specifying disk-level encryption as insufficient.

PCI definitely exhibits a preference away from disk-level encryption, although it doesn't prohibit it: you have to make sure that simply mounting the disk doesn't decrypt it.  Their concern is that if user credentials are compromised, and an attacker then has to do something else in order to see the plaintext.  This kind of implies TDE, although they don't use that term.

Now, the road forks here:

1. If a customer wants TDE and isn't interested in hearing about other solutions, then TDE is only thing that will meet that goal.

2. The PCI spec doesn't specifically offer up TDE as an alternative to disk-level encryption, though.  It exhibits a strong preference for column-level encryption of sensitive data, which doesn't require TDE.

In some ways, there's no real point of discussion.  You can comply with PCI without TDE (I would argue that, in fact, you are in a better position with column-level encryption), but if the organization wants TDE, then the technical arguments rarely matter.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 31 Oct 2025 20:21:04 -0400

On Fri, Oct 31, 2025 at 05:16:09PM -0700, Christophe Pettus wrote:
> > So it seems we have somewhat of a stand-off, with the Postgres
> > project questioning the value of TDE and the PCI writers
> > doubling-down on specifying disk-level encryption as insufficient.
>
> PCI definitely exhibits a preference away from disk-level encryption,
> although it doesn't prohibit it: you have to make sure that simply
> mounting the disk doesn't decrypt it.  Their concern is that if
> user credentials are compromised, and an attacker then has to do
> something else in order to see the plaintext.  This kind of implies
> TDE, although they don't use that term.
>
> 1. If a customer wants TDE and isn't interested in hearing about other
> solutions, then TDE is only thing that will meet that goal.
>
> 2. The PCI spec doesn't specifically offer up TDE as an alternative to
> disk-level encryption, though.  It exhibits a strong preference for
> column-level encryption of sensitive data, which doesn't require TDE.
>
> In some ways, there's no real point of discussion.  You can comply
> with PCI without TDE (I would argue that, in fact, you are in a better
> position with column-level encryption), but if the organization wants
> TDE, then the technical arguments rarely matter.

I think column-level encryption, on the client side, actually does
improve security and is preferable to file system level TDE, and I think
many here feel the same way.

---

From: "Clay Jackson (cjackson)" <[email-masked]>
Date: Sat, 1 Nov 2025 00:24:25 +0000

I can't disagree - but the question them becomes, as Markus and other have pointed out; would that allow a customer/user to check the "Encryption" box for PCI or any other "compliance review"

Clay Jackson

[email-masked]

-----Original Message-----
From: Bruce Momjian <[email-masked]>
Sent: Friday, October 31, 2025 5:21 PM
To: Christophe Pettus <[email-masked]>
Cc: pgsql-general <[email-masked]>; Kai Wagner <[email-masked]>; Laurenz Albe <[email-masked]>; Ron Johnson <[email-masked]>
Subject: Re: Enquiry about TDE with PgSQL

On Fri, Oct 31, 2025 at 05:16:09PM -0700, Christophe Pettus wrote:
> mounting the disk doesn't decrypt it.  Their concern is that if user
> credentials are compromised, and an attacker then has to do something
> else in order to see the plaintext.  This kind of implies TDE,
> although they don't use that term.
>

I think column-level encryption, on the client side, actually does improve security and is preferable to file system level TDE, and I think many here feel the same way.

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 31 Oct 2025 20:33:49 -0400

On Sat, Nov  1, 2025 at 12:24:25AM +0000, Clay Jackson (cjackson) wrote:
> I can't disagree - but the question them becomes, as Markus and
> other have pointed out; would that allow a customer/user to check the
> "Encryption" box for PCI or any other "compliance review"

I think so.  It says storage encryption is insufficient, but it doesn't
say client-side column-level encryption is insufficient.

---

From: Christophe Pettus <[email-masked]>
Date: Fri, 31 Oct 2025 18:32:17 -0700

> On Oct 31, 2025, at 17:21, Bruce Momjian <[email-masked]> wrote:
> 

Absolutely.  Unfortunately, too many IT security policies are basically a grab-bag of things that someone has read that all claimed to be "best practice," and the degree to which they can be educated on the topic is variable.

---

From: Christophe Pettus <[email-masked]>
Date: Fri, 31 Oct 2025 18:35:52 -0700

> I can't disagree - but the question them becomes, as Markus and other have pointed out; would that allow a customer/user to check the "Encryption" box for PCI or any other "compliance review"

The answer is: it depends (doesn't it always?).  Doing secure column-level encryption meets the PCI standard, and a competent PCI auditor will know that.  However, TDE has this cache as being "the way one does it," and if the organization is that way, it's hard to move them off of it.

As a sign of how the PCI world views TDE, at least one of the major credit card associations does not use it, and they have literally everyone's credit card number, with expiration date and CVV, sitting on their disks.

---

From: Chris Travers <[email-masked]>
Date: Sat, 1 Nov 2025 11:18:43 +0700

I maintain that the way forward is to get TDE in core.  Perhaps someone
could pick up the previous patches and try to push them again

Best Wishes,
Chris Travers

On Sat, Nov 1, 2025, 8:36 AM Christophe Pettus <[email-masked]> wrote:

> > I can't disagree - but the question them becomes, as Markus and other
> have pointed out; would that allow a customer/user to check the
> The answer is: it depends (doesn't it always?).  Doing secure column-level
> encryption meets the PCI standard, and a competent PCI auditor will know
> that.  However, TDE has this cache as being "the way one does it," and if
> the organization is that way, it's hard to move them off of it.
>
> As a sign of how the PCI world views TDE, at least one of the major credit
> card associations does not use it, and they have literally everyone's
> credit card number, with expiration date and CVV, sitting on their disks.
>
>

---

From: Kai Wagner <[email-masked]>
Date: Sat, 1 Nov 2025 08:34:57 +0100

> could pick up the previous patches and try to push them again
I wholeheartedly agree, as in this thread we are trying to do the same
thing again, that has already happened all they years before. We lose
ourselves in technical reasons, wondering why this makes no sense and how
it could be achieved differently, but we forget that we live in a vacuum
and bubble here.  The auditor, most of the time (as I've seen many times),
has no knowledge of these technical aspects. It's a box to check, with a
simple 'yes' or 'no'. They don't even wanna hear any "but this also
satisfies it, as this isn't clearly stated and worded in the standard".
This doesn't get us anywhere anymore; they will not put their checkbox
there if there is no simple answer to it.

@Bruce Momjian <[email-masked]> I totally understand your frustration
from previous times and also your point of view, that's absolutely valid,
no doubt about that. The time has changed over the course of the last 5+
years, and maybe it is time to reconsider. Just because it didn't succeed
last time doesn't mean we have to end up in the same spot this time. We
discussed it at length, and I am committed to supporting and making happen
what's necessary to get TDE fully functional with postgres directly. The
way of the implementation is a different question. Who from the former
times, or maybe even now, being interested in the topic, would be open for
a TDE group, to technically discuss options, possibilities etc. that we can
POC on and share for further feedback?!

>> The answer is: it depends (doesn't it always?).  Doing secure
>> column-level encryption meets the PCI standard, and a competent PCI auditor
>> will know that.  However, TDE has this cache as being "the way one does
>> it," and if the organization is that way, it's hard to move them off of it.
>>
>> As a sign of how the PCI world views TDE, at least one of the major
>> credit card associations does not use it, and they have literally
>> everyone's credit card number, with expiration date and CVV, sitting on
>> their disks.
>>
>>

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 1 Nov 2025 09:58:24 -0400

On Sat, Nov  1, 2025 at 08:34:57AM +0100, Kai Wagner wrote:
> I wholeheartedly agree, as in this thread we are trying to do the same thing
> again, that has already happened all they years before. We lose ourselves in
> technical reasons, wondering why this makes no sense and how it could be
> achieved differently, but we forget that we live in a vacuum and bubble here. 
> The auditor, most of the time (as I've seen many times), has no knowledge of
> these technical aspects. It's a box to check, with a simple 'yes' or 'no'. They
> don't even wanna hear any "but this also satisfies it, as this isn't clearly
> stated and worded in the standard". This doesn't get us anywhere anymore; they
> will not put their checkbox there if there is no simple answer to it.
> 
> @Bruce Momjian I totally understand your frustration from previous times and
> also your point of view, that's absolutely valid, no doubt about that. The time
> has changed over the course of the last 5+ years, and maybe it is time to
> reconsider. Just because it didn't succeed last time doesn't mean we have to
> end up in the same spot this time. We discussed it at length, and I am
> committed to supporting and making happen what's necessary to get TDE fully
> functional with postgres directly. The way of the implementation is a different
> question. Who from the former times, or maybe even now, being interested in the
> topic, would be open for a TDE group, to technically discuss options,
> possibilities etc. that we can POC on and share for further feedback?!

Without the calculus of feature value vs code overhead changing, talking
isn't going to accomplish anything.  My patch was already very small for
shared buffers, and the WAL code would have been small, but the code
weight of encrypting temporary files wasn't justified by the community,
and I couldn't argue against that conclusion.

Crunchy had someone working on restructuring temporary file writes to
make it easier, but they were unsuccessful.  Maybe the posted temporary
file compression patch will help reduce the code weight.

If you want to move forward with TDE without waiting to see if the
temporary file compression patch will reduce the TDE code impact, you
need to dig into how the community does feature calculus and how that
calculus can be changed --- this is not something technology can fix.

Companies are willing to add the code weight because it is a sale for
them, and customers are willing to pay to meet the check box --- that
calculus just doesn't work in the community.

---

From: Adrian Klaver <[email-masked]>
Date: Sat, 1 Nov 2025 08:32:17 -0700

> calculus just doesn't work in the community.

That is not true:

https://docs.percona.com/pg-tde/index.html

and

https://www.cybertec-postgresql.com/en/products/postgresql-transparent-data-encryption/

The community does provide it, just not the version of Postgres released 
here:

https://www.postgresql.org/download/

---

From: Greg Sabino Mullane <[email-masked]>
Date: Sat, 1 Nov 2025 14:34:36 -0400

> The community does provide it, just not the version of Postgres released

I'm not sure of your point here. Companies have forked versions / patches,
yes, but not sure how that equates to "community-provided"

Cheers,
Greg

---

From: Ken Marshall <[email-masked]>
Date: Sat, 1 Nov 2025 13:54:56 -0500

+1 from me for having TDE in-core or available as an extension

The security auditors that I have worked with have been increasingly
unwilling to actual evaluate the merits of an implementation or perhaps
no longer have the knowledge or skills needed. This is a needed
checkbox to allow PostgreSQL to be deployed in those environments.

Regards,
Ken

---

From: Rainer Duffner <[email-masked]>
Date: Sat, 1 Nov 2025 21:07:01 +0100

> Am 01.11.2025 um 19:54 schrieb Ken Marshall <[email-masked]>:
> 

Do you actually have HSMs with your TDE (assuming you use it elsewhere?
We run, for a customer, an Oracle DataGuard configuration with TDE with a HSM.

We have a support-contract with a 3rd party company that helps us with the more obscure problems on Oracle that we don’t encounter every day and they told us of all their clients (banks, insurance companies), we are the only ones with TDE. They loath working with it ;-)

There’s apparently another non-disclosed customer that uses it.

It may be that a lot of people now use „cloud HSMs“ - but I’m a bit of a purist for these kinds of things in that I believe that unless you own the hardware (HSMs are usually tamper-proof enough so you can deploy them in 3rd-party datacenters that aren’t your own), you don’t really control the keys.

In our case, the databases are backed up with rman to an NFS share that is provided by a virtualized linux server - the severs itself are hardware.

If you don’t have TDE, your backups aren’t encrypted and they end up on the veeam server like everything else, where an admin could copy them somewhere else and potentially take them elsewhere.

With the HSM, we don’t actually know the secret to decrypt the data (there may be a way to get it, I don’t know). We know the secret to unseal the wallet (that sits on the HSM, I believe) so that the database actually mounts and starts.

It’s pretty bullet-proof (I believe there’s techniques to prevent sniffing out the secret from RAM and HSMs usually implement those in their client software).
In fact, it’s so bullet-proof that should you lose the keys on the HSM, your data is gone if you have no other backups or backups of the HSM. 

If the amount of data is small enough, you can GPG encrypt a „normal“ full dump - but that become unfeasible as database size grows.

---

From: Adrian Klaver <[email-masked]>
Date: Sat, 1 Nov 2025 14:15:43 -0700

On 11/1/25 11:34, Greg Sabino Mullane wrote:
> On Sat, Nov 1, 2025 at 11:32 AM Adrian Klaver <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     The community does provide it, just not the version of Postgres
>     released here:
> 
> 
> I'm not sure of your point here. Companies have forked versions / 
> patches, yes, but not sure how that equates to "community-provided"

Well this gets in to a pet peeve of my with regard to OSS and the 
definition of 'community', it seems to adjust in size of coverage 
depending on the argument.

For a recent thread on -www that gets into this, as concerns 
contributors, see:

https://www.postgresql.org/message-id/aJ9ur1eSnk5pp9aD%40msg.df7cb.de

To get to said contributors list you go:

https://www.postgresql.org/community/contributors/

where you see folks recognized for their work outside the core code.

Examples of the expanded 'commmunity':

1) From the 'community' site you can go to:

where that packaging is done outside of core.

2) Again from postgresql.org:

https://www.postgresql.org/docs/18/external-extensions.html

"PostgreSQL is designed to be easily extensible. For this reason, 
extensions loaded into the database can function just like features that 
are built in. The contrib/ directory shipped with the source code 
contains several extensions, which are described in Appendix F. Other 
extensions are developed independently, like PostGIS. Even PostgreSQL 
replication solutions can be developed externally. For example, Slony-I 
is a popular primary/standby replication solution that is developed 
independently from the core project."

3) Of course there is:

https://www.postgresql.org/community/

"PostgreSQL is well-supported by its active community."

To me that means extensions provided by members of the 'community' are 
'community-provided'?

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 1 Nov 2025 19:42:39 -0400

> To me that means extensions provided by members of the 'community' are

Yes, this point was so prone to confusion I decided not to reply to the
original email.  ;-)

---

From: Kai Wagner <[email-masked]>
Date: Sun, 2 Nov 2025 11:14:58 +0100

> calculus can be changed --- this is not something technology can fix.
I fully agree here, and as I stated above, I don't question this at all, as
this is the strength of the diverse and spread community. Looking at this
thread alone, and the multiple different "users" popping up, that see the
need and would like to see TDE in core or in an extension drives my
thinking as it makes sense to start thinking and looking into the temp file
compression already right now, if and how this could make our code changes
easier, or if additional extensibility should be directly part of the first
patch, so you can enable it through its extension. Either way, it should be
considered and kept in mind now, not after the fact, or we will continue to
spin this wheel. And this thinking and willingness alone, to actually be
open to that idea, requires some upfront discussion, so you at least know
you are welcome to work on it without wasting effort, because no one will
ever want to merge. (and yes, of course everyone can work on whatever they
want at any point in time, but people might prefer working on something
that actually benefits the project and is welcome, rather than ending in
>/dev/null)

Given your experience Bruce, can you offer us some advice on how you would
approach it currently? What do you think makes the most sense, and how
should we proceed with collaboration to at least see a small change in
making this happen in the future?

> calculus just doesn't work in the community.
And this should always be at the forefront, as this makes the project
exactly as strong as it is today.

> https://url.avanan.click/v2/r01/___https://momjian.us___.YXAzOnBlcmNvbmE6YTpnOmZhM2VjN2JlNjcyOTZmMGQwMjEzOGM1NWU5OTA5NzM0Ojc6ZDgxOTo3NzQ0MzQwZjhmODA4YmY1ZjFiZmFlMzdlNmE4MzUxYzc0NGU0NTIwMjA5YWU1Mjk2NjQ5YjFiZjk0NDI3MDNjOnA6VDpO
>   EDB
> https://url.avanan.click/v2/r01/___https://enterprisedb.com___.YXAzOnBlcmNvbmE6YTpnOmZhM2VjN2JlNjcyOTZmMGQwMjEzOGM1NWU5OTA5NzM0Ojc6MjZhZTplMDhhZWQ5Nzg3YmVjODI4MDQ5YmU2ODc1MGExMTVmMmRmOTVlOTk2N2FmM2Y1N2VkN2VmYTZhYWMyZDQ1MzE3OnA6VDpO
>

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 3 Nov 2025 11:56:25 -0500

On Sun, Nov  2, 2025 at 11:14:58AM +0100, Kai Wagner wrote:
> thread alone, and the multiple different "users" popping up, that see the need
> and would like to see TDE in core or in an extension drives my thinking as it
> makes sense to start thinking and looking into the temp file compression
> already right now, if and how this could make our code changes easier, or if
> additional extensibility should be directly part of the first patch, so you can
> enable it through its extension. Either way, it should be considered and kept
> in mind now, not after the fact, or we will continue to spin this wheel. And
> this thinking and willingness alone, to actually be open to that idea, requires
> some upfront discussion, so you at least know you are welcome to work on it
> without wasting effort, because no one will ever want to merge. (and yes, of
> course everyone can work on whatever they want at any point in time, but people
> might prefer working on something that actually benefits the project and is
> welcome, rather than ending in >/dev/null)
> 
> approach it currently? What do you think makes the most sense, and how should
> we proceed with collaboration to at least see a small change in making this
> happen in the future? 

Good questions.  I think we have to decide if the community wants TDE,
and if so how much code change will the community accept to get it.  And
then we have to decide if this happens in an extension, like Percona's,
or in the core code.

The problem with the Percona extension is it seems like it was developed
mostly/all by Percona employees, meaning development was driven/steered
by Percona, and there was insufficient feedback from the community for
it to be polished enough to be a general community solution.  (I
fortunately attended all the Percona TDE talks at the Riga conference.)
The community needs to get involved with that extension if it is to
become a community-polished solution.

An extension does allow for fewer core code changes, but as I stated in
a previous email, the changes needed for command-line tools, external
tools, and replication to work in a TDE environment could make an
extension-based-TDE API too complex to be useful.

As far as how to move forward, I think we need to look at the temporary
file compression patch, see if that will help TDE code impact, and then
decide, with that patch, if the TDE core code impact is acceptable.  If
not, we then need to look at an extension, what hooks are needed, and
then decide if the final result have a simple enough API to be useful. 
(Do we start from scratch or use Percona's?)  If the API is complex, and
could lead to corruption/data loss due to encryption, that solution is
unacceptable.

---

From: Laurenz Albe <[email-masked]>
Date: Mon, 03 Nov 2025 19:42:06 +0100

On Mon, 2025-11-03 at 11:56 -0500, Bruce Momjian wrote:
> it to be polished enough to be a general community solution.

Reading a Percona blog, it looks like you need a modified server to get
to encrypt WAL, and they probably have no support for encrypting
temporary files.  So I'd say that TDE can probably not be a pure extension.
Perhaps somebody from Percona can confirm.

But I don't think it's a shortage of implementations for TDE that is the
problem.

Since you say that encrypting the temp files is the biggest hurdle for
community acceptance, what about a first version that does not encrypt
temp files?  For one, that will be good for encrypted backups (which is
one of the good use cases for TDE), and then you could argue that temp
files are not data *at rest*, so data-at-rest-encryption does not apply
to them.  Rome wasn't built in a day, and neither were parallel query
or declarative partitioning.

Yours,
Laurenz Albe