public inbox for [email protected]
help / color / mirror / Atom feedRe: Enquiry about TDE with PgSQL
9+ messages / 4 participants
[nested] [flat]
* Re: Enquiry about TDE with PgSQL
@ 2025-11-03 15:08 Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
0 siblings, 1 reply; 9+ messages in thread
From: Bruce Momjian @ 2025-11-03 15:08 UTC (permalink / raw)
To: Rainer Duffner <[email protected]>; +Cc: Ken Marshall <[email protected]>; pgsql-general
On Sat, Nov 1, 2025 at 09:07:01PM +0100, Rainer Duffner wrote:
> Do you actually have HSMs with your TDE (assuming you use it
> elsewhere? We run, for a customer, an Oracle DataGuard configuration
> with TDE with a HSM.
There were some interesting ideas in this email I want to reply to.
> We have a support-contract with a 3rd party company that helps us
> with the more obscure problems on Oracle that we don’t encounter
> every day and they told us of all their clients (banks, insurance
> companies), we are the only ones with TDE. They loath working with it
> ;-)
Is it the Oracle API they don't like, that Postgres can improve upon, or
something fundamental they don't like, or don't see the value in?
> There’s apparently another non-disclosed customer that uses it.
>
> It may be that a lot of people now use „cloud HSMs“ - but I’m
> a bit of a purist for these kinds of things in that I believe that
> unless you own the hardware (HSMs are usually tamper-proof enough so
> you can deploy them in 3rd-party datacenters that aren’t your own),
> you don’t really control the keys.
>
> In our case, the databases are backed up with rman to an NFS share
> that is provided by a virtualized linux server - the severs itself are
> hardware.
>
> If you don’t have TDE, your backups aren’t encrypted and they end
> up on the veeam server like everything else, where an admin could copy
> them somewhere else and potentially take them elsewhere.
>
> With the HSM, we don’t actually know the secret to decrypt the data
> (there may be a way to get it, I don’t know). We know the secret
> to unseal the wallet (that sits on the HSM, I believe) so that the
> database actually mounts and starts.
>
> It’s pretty bullet-proof (I believe there’s techniques to prevent
> sniffing out the secret from RAM and HSMs usually implement those in
> their client software). In fact, it’s so bullet-proof that should
> you lose the keys on the HSM, your data is gone if you have no other
> backups or backups of the HSM.
As far as I know, there are two ways to generate the data encryption
key. One is for the HSM to generate it, and then only the HSM knows it.
The other method is to create the encryption key on a USB memory stick,
copy the key into the HSM, and then remove the USB memory stick and
store it in a secure location like a safe. The second method seems like
a better option to me. Oh, and make a second copy of the USB memory
stick.
> If the amount of data is small enough, you can GPG encrypt a
> „normal“ full dump - but that become unfeasible as database size
> grows.
This is a good point. For TDE storage, the data is encrypted once on
write, and then can be backed up as many times as needed without
re-encryption. With storage-level encryption, the data has to be
encrypted for every backup. However, considering how much TLS is used,
I assumed the encryption overhead would be minimal compared to the
transfer overhead.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Do not let urgent matters crowd out time for investment in the future.
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-03 15:39 ` [email protected]
2025-11-03 16:01 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 18:30 ` Re: Enquiry about TDE with PgSQL Laurenz Albe <[email protected]>
0 siblings, 2 replies; 9+ messages in thread
From: [email protected] @ 2025-11-03 15:39 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Ken Marshall <[email protected]>; pgsql-general
Am 2025-11-03 16:08, schrieb Bruce Momjian:
> Is it the Oracle API they don't like, that Postgres can improve upon,
> or
> something fundamental they don't like, or don't see the value in?
I am not sure.
It just complicates everything.
Documentation isn't thin, it's skeletal.
And of course, actual support from the HSM-vendor for this use-case is
non-existent.
Same for Oracle.
They'll both point at each other.
Who'd have thought that.
> As far as I know, there are two ways to generate the data encryption
> key. One is for the HSM to generate it, and then only the HSM knows
> it.
> The other method is to create the encryption key on a USB memory stick,
> copy the key into the HSM, and then remove the USB memory stick and
> store it in a secure location like a safe. The second method seems
> like
> a better option to me. Oh, and make a second copy of the USB memory
> stick.
The keys are generated on the HSM.
There's HSM client you've got to install that manages the communication
to the HSM.
The HSM should be backed up, too. Which is only possible by connecting
physically to it with a notebook and inserting an USB stick.
Which begs the question: where do you source an USB stick with the same
trust-level as the 20k-a-pop HSM?
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
@ 2025-11-03 16:01 ` Bruce Momjian <[email protected]>
2025-11-03 16:20 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
1 sibling, 1 reply; 9+ messages in thread
From: Bruce Momjian @ 2025-11-03 16:01 UTC (permalink / raw)
To: [email protected]; +Cc: Ken Marshall <[email protected]>; pgsql-general
On Mon, Nov 3, 2025 at 04:39:45PM +0100, [email protected] wrote:
> Am 2025-11-03 16:08, schrieb Bruce Momjian:
>
> > Is it the Oracle API they don't like, that Postgres can improve upon, or
> > something fundamental they don't like, or don't see the value in?
>
>
> I am not sure.
>
> It just complicates everything.
> Documentation isn't thin, it's skeletal.
Okay, these are things we can improve on. It think the API of my final
posted patch had a pretty simple API, but pushing that API out to
external tools will add complexity I didn't implement, and that
complexity could be a reason to reject TDE.
> And of course, actual support from the HSM-vendor for this use-case is
> non-existent.
> Same for Oracle.
Yes, my patch used shell scripts --- not sure if that is good or bad.
I will admit that companies are better at integrating with external
vendors, particulary hardware vendors. There is an organization
mismatch betwween the community and companies, and the community
basically forces companies to intract on community terms --- companies
are a more natural iteraction for other companies.
> > As far as I know, there are two ways to generate the data encryption
> > key. One is for the HSM to generate it, and then only the HSM knows it.
> > The other method is to create the encryption key on a USB memory stick,
> > copy the key into the HSM, and then remove the USB memory stick and
> > store it in a secure location like a safe. The second method seems like
> > a better option to me. Oh, and make a second copy of the USB memory
> > stick.
>
>
> The keys are generated on the HSM.
> There's HSM client you've got to install that manages the communication to
> the HSM.
>
> The HSM should be backed up, too. Which is only possible by connecting
> physically to it with a notebook and inserting an USB stick.
The problem is that if anything happens with the HSM, you are stuck.
The HSM adds an additional risk.
> Which begs the question: where do you source an USB stick with the same
> trust-level as the 20k-a-pop HSM?
I don't know.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Do not let urgent matters crowd out time for investment in the future.
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
2025-11-03 16:01 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-03 16:20 ` Adrian Klaver <[email protected]>
2025-11-03 16:59 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Adrian Klaver @ 2025-11-03 16:20 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; [email protected]; +Cc: Ken Marshall <[email protected]>; pgsql-general
On 11/3/25 08:01, Bruce Momjian wrote:
> On Mon, Nov 3, 2025 at 04:39:45PM +0100, [email protected] wrote:
>> Am 2025-11-03 16:08, schrieb Bruce Momjian:
> I will admit that companies are better at integrating with external
> vendors, particulary hardware vendors. There is an organization
> mismatch betwween the community and companies, and the community
> basically forces companies to intract on community terms --- companies
> are a more natural iteraction for other companies.
Again a distinction without a difference.
If you go here:
https://www.postgresql.org/community/contributors/
and look at the names under Core Team and Major Contributors, out of 59
names only 6 do not list a company name. Companies live at the heart of
the community. Hence the the policy of no company having no more then
50%(?) of the Core team.
--
Adrian Klaver
[email protected]
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
2025-11-03 16:01 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 16:20 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
@ 2025-11-03 16:59 ` Bruce Momjian <[email protected]>
2025-11-03 18:07 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Bruce Momjian @ 2025-11-03 16:59 UTC (permalink / raw)
To: Adrian Klaver <[email protected]>; +Cc: [email protected]; Ken Marshall <[email protected]>; pgsql-general
On Mon, Nov 3, 2025 at 08:20:21AM -0800, Adrian Klaver wrote:
> On 11/3/25 08:01, Bruce Momjian wrote:
> > On Mon, Nov 3, 2025 at 04:39:45PM +0100, [email protected] wrote:
> > > Am 2025-11-03 16:08, schrieb Bruce Momjian:
>
> > I will admit that companies are better at integrating with external
> > vendors, particulary hardware vendors. There is an organization
> > mismatch between the community and companies, and the community
> > basically forces companies to intract on community terms --- companies
> > are a more natural iteraction for other companies.
>
> Again a distinction without a difference.
>
> If you go here:
>
> https://www.postgresql.org/community/contributors/
>
> and look at the names under Core Team and Major Contributors, out of 59
> names only 6 do not list a company name. Companies live at the heart of the
> community. Hence the the policy of no company having no more then 50%(?) of
> the Core team.
The issue is that interacting with external companies is rarely done by
the community, so if you need to get community software to interact with
external software, the external software vendor has to get involved with
the community, and for non-Postgres-focused external companies, that is
a big hurdle. Companies have many employees who can make it their job
to get the integration working, as has happened with the many
company-controlled Postgres TDE solutions.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Do not let urgent matters crowd out time for investment in the future.
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
2025-11-03 16:01 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 16:20 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
2025-11-03 16:59 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-03 18:07 ` Adrian Klaver <[email protected]>
2025-11-03 18:14 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Adrian Klaver @ 2025-11-03 18:07 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: [email protected]; Ken Marshall <[email protected]>; pgsql-general
On 11/3/25 8:59 AM, Bruce Momjian wrote:
> On Mon, Nov 3, 2025 at 08:20:21AM -0800, Adrian Klaver wrote:
>> On 11/3/25 08:01, Bruce Momjian wrote:
>>> On Mon, Nov 3, 2025 at 04:39:45PM +0100, [email protected] wrote:
>>>> Am 2025-11-03 16:08, schrieb Bruce Momjian:
>>
>> Again a distinction without a difference.
>>
>> If you go here:
>>
>> https://www.postgresql.org/community/contributors/
>>
>> and look at the names under Core Team and Major Contributors, out of 59
>> names only 6 do not list a company name. Companies live at the heart of the
>> community. Hence the the policy of no company having no more then 50%(?) of
>> the Core team.
>
> The issue is that interacting with external companies is rarely done by
> the community, so if you need to get community software to interact with
> external software, the external software vendor has to get involved with
> the community, and for non-Postgres-focused external companies, that is
> a big hurdle. Companies have many employees who can make it their job
> to get the integration working, as has happened with the many
> company-controlled Postgres TDE solutions.
To be specific it is a customer service issue.
So why is this page:
https://www.postgresql.org/support/professional_support/
"Commercial support is available from many different companies providing
professional services to the PostgreSQL community."
not a solution done by the community?
--
Adrian Klaver
[email protected]
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
2025-11-03 16:01 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 16:20 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
2025-11-03 16:59 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 18:07 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
@ 2025-11-03 18:14 ` Bruce Momjian <[email protected]>
2025-11-03 18:46 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Bruce Momjian @ 2025-11-03 18:14 UTC (permalink / raw)
To: Adrian Klaver <[email protected]>; +Cc: [email protected]; Ken Marshall <[email protected]>; pgsql-general
On Mon, Nov 3, 2025 at 10:07:50AM -0800, Adrian Klaver wrote:
> On 11/3/25 8:59 AM, Bruce Momjian wrote:
> > On Mon, Nov 3, 2025 at 08:20:21AM -0800, Adrian Klaver wrote:
> > > On 11/3/25 08:01, Bruce Momjian wrote:
> > > > On Mon, Nov 3, 2025 at 04:39:45PM +0100, [email protected] wrote:
> > > > > Am 2025-11-03 16:08, schrieb Bruce Momjian:
> > >
>
> > > Again a distinction without a difference.
> > >
> > > If you go here:
> > >
> > > https://www.postgresql.org/community/contributors/
> > >
> > > and look at the names under Core Team and Major Contributors, out of 59
> > > names only 6 do not list a company name. Companies live at the heart of the
> > > community. Hence the the policy of no company having no more then 50%(?) of
> > > the Core team.
> >
> > The issue is that interacting with external companies is rarely done by
> > the community, so if you need to get community software to interact with
> > external software, the external software vendor has to get involved with
> > the community, and for non-Postgres-focused external companies, that is
> > a big hurdle. Companies have many employees who can make it their job
> > to get the integration working, as has happened with the many
> > company-controlled Postgres TDE solutions.
>
> To be specific it is a customer service issue.
>
> So why is this page:
>
> https://www.postgresql.org/support/professional_support/
>
> "Commercial support is available from many different companies providing
> professional services to the PostgreSQL community."
>
> not a solution done by the community?
I am not sure why you are not understanding. To get Postgres to
interoperate with company product X, you need communication between the
Postgres community and the company producing product X. The community
rarely interacts directly with companies, though it does interact with
employees of Postgres-focused companies. The smaller the company and
the less the company relies on Postgres, the less likely the company
will interact with the community on the community's terms.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Do not let urgent matters crowd out time for investment in the future.
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
2025-11-03 16:01 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 16:20 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
2025-11-03 16:59 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 18:07 ` Re: Enquiry about TDE with PgSQL Adrian Klaver <[email protected]>
2025-11-03 18:14 ` Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
@ 2025-11-03 18:46 ` Adrian Klaver <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Adrian Klaver @ 2025-11-03 18:46 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: [email protected]; Ken Marshall <[email protected]>; pgsql-general
On 11/3/25 10:14, Bruce Momjian wrote:
> On Mon, Nov 3, 2025 at 10:07:50AM -0800, Adrian Klaver wrote:
>>
>> To be specific it is a customer service issue.
>>
>> So why is this page:
>>
>> https://www.postgresql.org/support/professional_support/
>>
>> "Commercial support is available from many different companies providing
>> professional services to the PostgreSQL community."
>>
>> not a solution done by the community?
>
> I am not sure why you are not understanding. To get Postgres to
> interoperate with company product X, you need communication between the
> Postgres community and the company producing product X. The community
> rarely interacts directly with companies, though it does interact with
> employees of Postgres-focused companies. The smaller the company and
> the less the company relies on Postgres, the less likely the company
> will interact with the community on the community's terms.
>
I do understand it comes down to two possibilities:
1) Community is what you download here:
https://www.postgresql.org/ftp/source/
and anything above that is third party involvement and nothing to do
with the community.
Or
2) As the postgresql.org mentions in many places that I have previously
cited community includes layers above the source, extending to
commercial support.
If 1) is correct then yes there is a community communication issue.
If 2) is correct there is not, as you can go here:
https://www.postgresql.org/support/professional_support/
and find support.
--
Adrian Klaver
[email protected]
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Enquiry about TDE with PgSQL
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` Re: Enquiry about TDE with PgSQL [email protected]
@ 2025-11-03 18:30 ` Laurenz Albe <[email protected]>
1 sibling, 0 replies; 9+ messages in thread
From: Laurenz Albe @ 2025-11-03 18:30 UTC (permalink / raw)
To: [email protected]; Bruce Momjian <[email protected]>; +Cc: Ken Marshall <[email protected]>; pgsql-general
On Mon, 2025-11-03 at 16:39 +0100, [email protected] wrote:
> The HSM should be backed up, too. Which is only possible by connecting
> physically to it with a notebook and inserting an USB stick.
>
> Which begs the question: where do you source an USB stick with the same
> trust-level as the 20k-a-pop HSM?
I'd say that you don't need a very secure USB stick. You just put the
USB stick in a very secure safe that only two very trustworthy people
can open together.
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2025-11-03 18:46 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-11-03 15:08 Re: Enquiry about TDE with PgSQL Bruce Momjian <[email protected]>
2025-11-03 15:39 ` [email protected]
2025-11-03 16:01 ` Bruce Momjian <[email protected]>
2025-11-03 16:20 ` Adrian Klaver <[email protected]>
2025-11-03 16:59 ` Bruce Momjian <[email protected]>
2025-11-03 18:07 ` Adrian Klaver <[email protected]>
2025-11-03 18:14 ` Bruce Momjian <[email protected]>
2025-11-03 18:46 ` Adrian Klaver <[email protected]>
2025-11-03 18:30 ` Laurenz Albe <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox