public inbox for [email protected]
help / color / mirror / Atom feedRe: Enquiry about TDE with PgSQL
7+ messages / 6 participants
[nested] [flat]
* Re: Enquiry about TDE with PgSQL
@ 2025-11-04 02:05 Bruce Momjian <[email protected]>
2025-11-04 05:23 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
2025-11-04 05:40 ` Re: Enquiry about TDE with PgSQL Laurenz Albe <[email protected]>
2025-11-04 15:15 ` RE: Enquiry about TDE with PgSQL Clay Jackson (cjackson) <[email protected]>
2025-11-06 09:29 ` Re: Enquiry about TDE with PgSQL Jan Wieremjewicz <[email protected]>
0 siblings, 4 replies; 7+ messages in thread
From: Bruce Momjian @ 2025-11-04 02:05 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; +Cc: Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; Clay Jackson (cjackson) <[email protected]>; pgsql-general; Ron Johnson <[email protected]>
On Mon, Nov 3, 2025 at 07:42:06PM +0100, Laurenz Albe wrote:
> On Mon, 2025-11-03 at 11:56 -0500, Bruce Momjian wrote:
> > The problem with the Percona extension is it seems like it was developed
> > mostly/all by Percona employees, meaning development was driven/steered
> > by Percona, and there was insufficient feedback from the community for
> > it to be polished enough to be a general community solution.
>
> Reading a Percona blog, it looks like you need a modified server to get
> to encrypt WAL, and they probably have no support for encrypting
> temporary files. So I'd say that TDE can probably not be a pure extension.
> Perhaps somebody from Percona can confirm.
Yes, the server has to be modified because the hooks they need don't
exist in the community source code. They also have encryption control
on the table level, which I frankly think will never work long-term
because the storage API doesn't have enough table-level detail, so I
think they are considering tablespace-level or cluster-level encryption.
> But I don't think it's a shortage of implementations for TDE that is the
> problem.
>
> Since you say that encrypting the temp files is the biggest hurdle for
> community acceptance, what about a first version that does not encrypt
> temp files? For one, that will be good for encrypted backups (which is
> one of the good use cases for TDE), and then you could argue that temp
> files are not data *at rest*, so data-at-rest-encryption does not apply
> to them. Rome wasn't built in a day, and neither were parallel query
> or declarative partitioning.
Uh, people will say that if the solution is not 100% secure in its
coverage, it is much less useful and therefore not worth it.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Do not let urgent matters crowd out time for investment in the future.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-04 02:05 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-04 05:23 ` Adrian Klaver <[email protected]>
2025-11-08 03:25 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
3 siblings, 1 reply; 7+ messages in thread
From: Adrian Klaver @ 2025-11-04 05:23 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; Laurenz Albe <[email protected]>; +Cc: Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; Clay Jackson (cjackson) <[email protected]>; pgsql-general; Ron Johnson <[email protected]>
On 11/3/25 18:05, Bruce Momjian wrote:
> On Mon, Nov 3, 2025 at 07:42:06PM +0100, Laurenz Albe wrote:
>> On Mon, 2025-11-03 at 11:56 -0500, Bruce Momjian wrote:
>>> The problem with the Percona extension is it seems like it was developed
>>> mostly/all by Percona employees, meaning development was driven/steered
>>> by Percona, and there was insufficient feedback from the community for
>>> it to be polished enough to be a general community solution.
>
> Uh, people will say that if the solution is not 100% secure in its
> coverage, it is much less useful and therefore not worth it.
>
In a different realm you have been there before and it worked out:
https://momjian.us/main/blogs/pgblog/2010.html#May_12_2010
--
Adrian Klaver
[email protected]
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-04 02:05 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-04 05:23 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
@ 2025-11-08 03:25 ` Bruce Momjian <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Bruce Momjian @ 2025-11-08 03:25 UTC (permalink / raw)
To: Adrian Klaver <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; Clay Jackson (cjackson) <[email protected]>; pgsql-general; Ron Johnson <[email protected]>
On Mon, Nov 3, 2025 at 09:23:33PM -0800, Adrian Klaver wrote:
> On 11/3/25 18:05, Bruce Momjian wrote:
> > On Mon, Nov 3, 2025 at 07:42:06PM +0100, Laurenz Albe wrote:
> > > On Mon, 2025-11-03 at 11:56 -0500, Bruce Momjian wrote:
> > > > The problem with the Percona extension is it seems like it was developed
> > > > mostly/all by Percona employees, meaning development was driven/steered
> > > > by Percona, and there was insufficient feedback from the community for
> > > > it to be polished enough to be a general community solution.
>
> >
> > Uh, people will say that if the solution is not 100% secure in its
> > coverage, it is much less useful and therefore not worth it.
> >
>
> In a different realm you have been there before and it worked out:
>
> https://momjian.us/main/blogs/pgblog/2010.html#May_12_2010
Yes, pg_upgrade was a hard one too, and it took a long time be accepted,
so there is hope.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Do not let urgent matters crowd out time for investment in the future.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-04 02:05 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-04 05:40 ` Laurenz Albe <[email protected]>
2025-11-04 12:12 ` Re: Enquiry about TDE with PgSQL Álvaro Herrera <[email protected]>
3 siblings, 1 reply; 7+ messages in thread
From: Laurenz Albe @ 2025-11-04 05:40 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; Clay Jackson (cjackson) <[email protected]>; pgsql-general; Ron Johnson <[email protected]>
On Mon, 2025-11-03 at 21:05 -0500, Bruce Momjian wrote:
> On Mon, Nov 3, 2025 at 07:42:06PM +0100, Laurenz Albe wrote:
>
> > Since you say that encrypting the temp files is the biggest hurdle for
> > community acceptance, what about a first version that does not encrypt
> > temp files? For one, that will be good for encrypted backups (which is
> > one of the good use cases for TDE), and then you could argue that temp
> > files are not data *at rest*, so data-at-rest-encryption does not apply
> > to them. Rome wasn't built in a day, and neither were parallel query
> > or declarative partitioning.
>
> Uh, people will say that if the solution is not 100% secure in its
> coverage, it is much less useful and therefore not worth it.
Some people will doubtless say that. Others will consider the checkbox
requirement satisfied and use it. Yet others will consider a mislaid
backup their biggest problem and will consider TDE a technically useful
solution.
9.6, which introduced parallel query, only supported it for sequential
scans, which was much less useful than what we have today. I for one
wouldn't consider an implementation of TDE with some features missing
"not worth it". If anything, I consider the marginal security improvement
that TDE as a whole provides not worth it. But I am sold on the claim
that having TDE would promote the adoption of PostgreSQL.
I am curious what others think.
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-04 02:05 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-04 05:40 ` Re: Enquiry about TDE with PgSQL Laurenz Albe <[email protected]>
@ 2025-11-04 12:12 ` Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Álvaro Herrera @ 2025-11-04 12:12 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; +Cc: Bruce Momjian <[email protected]>; Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; Clay Jackson (cjackson) <[email protected]>; pgsql-general; Ron Johnson <[email protected]>
On 2025-Nov-04, Laurenz Albe wrote:
> 9.6, which introduced parallel query, only supported it for sequential
> scans, which was much less useful than what we have today. I for one
> wouldn't consider an implementation of TDE with some features missing
> "not worth it".
We call this incremental development, and we've accepted that it might
be the only way to get complex features developed. We did it for
partitioning, parallel query, AIO, and lots of other things. IMO it's a
good strategy, even if we sometimes have to backtrack.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"No tengo por qué estar de acuerdo con lo que pienso"
(Carlos Caszeli)
^ permalink raw reply [nested|flat] 7+ messages in thread
* RE: Enquiry about TDE with PgSQL
2025-11-04 02:05 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-04 15:15 ` Clay Jackson (cjackson) <[email protected]>
3 siblings, 0 replies; 7+ messages in thread
From: Clay Jackson (cjackson) @ 2025-11-04 15:15 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; Laurenz Albe <[email protected]>; +Cc: Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; pgsql-general; Ron Johnson <[email protected]>
Again, speaking for myself only and not officially for Quest.
" Uh, people will say that if the solution is not 100% secure in its coverage, it is much less useful and therefore not worth it."
I would assert that NO system is "100% secure" given enough money and resources. I think the more important point of this discussion is how users of PostgreSQL can "check the box" for compliance with whatever security and encryption "standards" are "required" in their environment, and/or mitigate the risks of not being "compliant".
Clay Jackson
-----Original Message-----
From: Bruce Momjian <[email protected]>
Sent: Monday, November 3, 2025 6:06 PM
To: Laurenz Albe <[email protected]>
Cc: Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; Clay Jackson (cjackson) <[email protected]>; pgsql-general <[email protected]>; Ron Johnson <[email protected]>
Subject: Re: Enquiry about TDE with PgSQL
CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
On Mon, Nov 3, 2025 at 07:42:06PM +0100, Laurenz Albe wrote:
> On Mon, 2025-11-03 at 11:56 -0500, Bruce Momjian wrote:
> > The problem with the Percona extension is it seems like it was
> > developed mostly/all by Percona employees, meaning development was
> > driven/steered by Percona, and there was insufficient feedback from
> > the community for it to be polished enough to be a general community solution.
>
> Reading a Percona blog, it looks like you need a modified server to
> get to encrypt WAL, and they probably have no support for encrypting
> temporary files. So I'd say that TDE can probably not be a pure extension.
> Perhaps somebody from Percona can confirm.
Yes, the server has to be modified because the hooks they need don't exist in the community source code. They also have encryption control on the table level, which I frankly think will never work long-term because the storage API doesn't have enough table-level detail, so I think they are considering tablespace-level or cluster-level encryption.
> But I don't think it's a shortage of implementations for TDE that is
> the problem.
>
> Since you say that encrypting the temp files is the biggest hurdle for
> community acceptance, what about a first version that does not encrypt
> temp files? For one, that will be good for encrypted backups (which
> is one of the good use cases for TDE), and then you could argue that
> temp files are not data *at rest*, so data-at-rest-encryption does not
> apply to them. Rome wasn't built in a day, and neither were parallel
> query or declarative partitioning.
Uh, people will say that if the solution is not 100% secure in its coverage, it is much less useful and therefore not worth it.
--
Bruce Momjian <[email protected]> https://momjian.us/
EDB https://enterprisedb.com/
Do not let urgent matters crowd out time for investment in the future.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-04 02:05 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-06 09:29 ` Jan Wieremjewicz <[email protected]>
3 siblings, 0 replies; 7+ messages in thread
From: Jan Wieremjewicz @ 2025-11-06 09:29 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Kai Wagner <[email protected]>; Chris Travers <[email protected]>; Christophe Pettus <[email protected]>; Clay Jackson (cjackson) <[email protected]>; pgsql-general; Ron Johnson <[email protected]>
Just wanted to throw in my two cents, or actually, two points, on TDE:
1.
Extensions seem like the right place for most of the TDE logic.
Encryption usually depends on organization specific or even
proprietary security systems. Think of features like KMS integrations that
need custom handling and are often owned by Security, Compliance, or IT
rather than DBAs, so it is hard to convince DBAs to stick to a "supported"
one.
That kind of setup is difficult to generalize into PostgreSQL core, and
extensions give the flexibility to make it work. The same goes for cloud
KMS or HSM certification integrations, which add another layer of
complexity that is easier to manage outside core.
Also, by relying mainly on new hooks, we can keep changes to the core
minimal and make TDE as non-intrusive as possible for those who do not need
it.
2.
About the "single-vendor open source" comment...
That was never the goal when we kicked off pg_tde. Sure, Percona funded
the initial work, but we would really like to see this become a broader
community effort.
We saw a gap that enterprise users were facing and tried to fill
it based on what we had learned from other databases, but the more people
get involved, the better this can become.
On Tue, Nov 4, 2025 at 3:06 AM Bruce Momjian <[email protected]> wrote:
> On Mon, Nov 3, 2025 at 07:42:06PM +0100, Laurenz Albe wrote:
> > On Mon, 2025-11-03 at 11:56 -0500, Bruce Momjian wrote:
> > > The problem with the Percona extension is it seems like it was
> developed
> > > mostly/all by Percona employees, meaning development was driven/steered
> > > by Percona, and there was insufficient feedback from the community for
> > > it to be polished enough to be a general community solution.
> >
> > Reading a Percona blog, it looks like you need a modified server to get
> > to encrypt WAL, and they probably have no support for encrypting
> > temporary files. So I'd say that TDE can probably not be a pure
> extension.
> > Perhaps somebody from Percona can confirm.
>
> Yes, the server has to be modified because the hooks they need don't
> exist in the community source code. They also have encryption control
> on the table level, which I frankly think will never work long-term
> because the storage API doesn't have enough table-level detail, so I
> think they are considering tablespace-level or cluster-level encryption.
>
> > But I don't think it's a shortage of implementations for TDE that is the
> > problem.
> >
> > Since you say that encrypting the temp files is the biggest hurdle for
> > community acceptance, what about a first version that does not encrypt
> > temp files? For one, that will be good for encrypted backups (which is
> > one of the good use cases for TDE), and then you could argue that temp
> > files are not data *at rest*, so data-at-rest-encryption does not apply
> > to them. Rome wasn't built in a day, and neither were parallel query
> > or declarative partitioning.
>
> Uh, people will say that if the solution is not 100% secure in its
> coverage, it is much less useful and therefore not worth it.
>
> --
> Bruce Momjian <[email protected]>
> https://url.avanan.click/v2/r01/___https://momjian.us___.YXAzOnBlcmNvbmE6YTpnOjI1Nzk5MzJmMmQ0NDFlMGE...
> EDB
> https://url.avanan.click/v2/r01/___https://enterprisedb.com___.YXAzOnBlcmNvbmE6YTpnOjI1Nzk5MzJmMmQ0N...
>
> Do not let urgent matters crowd out time for investment in the future.
>
>
>
--
<https://www.percona.com/;
Jan Wieremjewicz
Sr. Product Manager, Percona
[email protected] <[email protected]>
CET (GMT+1)
Databases Run Better With Percona
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2025-11-08 03:25 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-11-04 02:05 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-04 05:23 ` Adrian Klaver <[email protected]>
2025-11-08 03:25 ` Bruce Momjian <[email protected]>
2025-11-04 05:40 ` Laurenz Albe <[email protected]>
2025-11-04 12:12 ` Álvaro Herrera <[email protected]>
2025-11-04 15:15 ` Clay Jackson (cjackson) <[email protected]>
2025-11-06 09:29 ` Jan Wieremjewicz <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox