From: Bruce Momjian <bruce@momjian.us>
To: Ants Aasma <ants.aasma@eesti.ee>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: WIP: Data at rest encryption
Date: Wed, 14 Jun 2017 11:26:03 -0400
Message-ID: <20170614152603.GE4750@momjian.us> (raw)
In-Reply-To: <CA+CSw_sErZyFn=xAY-2i0QJi8Mtqx334350auBFchgObUQtGSg@mail.gmail.com>
References: <CA+CSw_tb3bk5i7if6inZFc3yyf+9HEVNTy51QFBoeUk7UE_V=w@mail.gmail.com>
<06b6db11-3a3f-4d24-6f57-450f96ecda84@2ndquadrant.com>
<CA+TgmoaXGx=7VEVz7eq9T=raZRHm3iNc244Sx=T1UxX-+szOUQ@mail.gmail.com>
<CA+CSw_uO6wGv2YF-RQvVOGeezJXWY+SGTKYomOY=fDNp6OmHqA@mail.gmail.com>
<CA+TgmoZDd+dNRhfBmJf7rO8PLwSPaP+7DkdjmpPpnjwQu4-osw@mail.gmail.com>
<CA+CSw_sErZyFn=xAY-2i0QJi8Mtqx334350auBFchgObUQtGSg@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On Wed, Jun 14, 2017 at 06:10:32PM +0300, Ants Aasma wrote:
> On Tue, Jun 13, 2017 at 6:35 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> > Performance is likely to be poor on large databases,
> > because every time a page transits between shared_buffers and the
> > buffer cache we've got to en/decrypt, but as long as it's only poor
> > for the people who opt into the feature I don't see a big problem with
> > that.
>
> It would make sense to tune the database with large shared buffers if
> encryption is enabled. That should make sure that most shared buffers
> traffic is going to disk anyway. As for performance, I have a
> prototype assembly implementation of AES that does 3GB/s/core on my
> laptop. If we add that behind a CPUID check the overhead should be
> quite reasonable.
Are you checking the CPU type or if AES instructions are enabled on the
CPU? I ask this because I just realized in researching my new TLS talk
that my BIOS defaults to AES instructions disabled, and I had to
manually enable it.
> > I anticipate that one of the trickier problems here will be handling
> > encryption of the write-ahead log. Suppose you encrypt WAL a block at
> > a time. In the current system, once you've written and flushed a
> > block, you can consider it durably committed, but if that block is
> > encrypted, this is no longer true. A crash might tear the block,
> > making it impossible to decrypt. Replay will therefore stop at the
> > end of the previous block, not at the last record actually flushed as
> > would happen today.
>
> My patch is currenly doing a block at a time for WAL. The XTS mode
Uh, how are you writing partial writes to the WAL. I assume you are
doing a streaming cipher so you can write in increments, right?
> used to encrypt has the useful property that blocks that share
> identical prefix unencrypted also have identical prefix when
> encrypted. It requires that the tearing is 16B aligned, but I think
> that is true for pretty much all storage systems. That property of
> course has security downsides, but for table/index storage we have a
> nonce in the form of LSN in the page header eliminating the issue.
>
> > So, your synchronous_commit suddenly isn't. A
> > similar problem will occur any other page where we choose not to
> > protect against torn pages using full page writes. For instance,
> > unless checksums are enabled or wal_log_hints=on, we'll write a data
> > page where a single bit has been flipped and assume that the bit will
> > either make it to disk or not; the page can't really be torn in any
> > way that hurts us. But with encryption that's no longer true, because
> > the hint bit will turn into much more than a single bit flip, and
> > rereading that page with half old and half new contents will be the
> > end of the world (TM). I don't know off-hand whether we're
> > protecting, say, CLOG page writes with FPWs.: because setting a couple
> > of bits is idempotent and doesn't depend on the existing page
> > contents, we might not need it currently, but with encryption, every
> > bit in the page depends on every other bit in the page, so we
> > certainly would. I don't know how many places we've got assumptions
> > like this baked into the system, but I'm guessing there are a bunch.
>
> I think we need to require wal_log_hints=on when encryption is
> enabled. Currently I have not considered tearing on CLOG bits. Other
> SLRUs probably have similar issues. I need to think a bit about how to
> solve that.
I am not sure if clog even needs to be encrypted.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: bruce@momjian.us, ants.aasma@eesti.ee, robertmhaas@gmail.com, peter.eisentraut@2ndquadrant.com
Subject: Re: WIP: Data at rest encryption
In-Reply-To: <20170614152603.GE4750@momjian.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox