Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256)
	(Exim 4.89)
	(envelope-from <pgsql-hackers-owner+archive@lists.postgresql.org>)
	id 1hC2s1-0001fH-L0
	for pgsql-hackers@arkaria.postgresql.org; Thu, 04 Apr 2019 13:57:33 +0000
Received: from localhost ([127.0.0.1] helo=malur.postgresql.org)
	by malur.postgresql.org with esmtp (Exim 4.89)
	(envelope-from <pgsql-hackers-owner+archive@lists.postgresql.org>)
	id 1hC2s0-0003j6-51
	for pgsql-hackers@arkaria.postgresql.org; Thu, 04 Apr 2019 13:57:32 +0000
Received: from makus.postgresql.org ([2001:4800:3e1:1::229])
	by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256)
	(Exim 4.89)
	(envelope-from <ah@cybertec.at>)
	id 1hC2rz-0003iz-PM
	for pgsql-hackers@lists.postgresql.org; Thu, 04 Apr 2019 13:57:31 +0000
Received: from mx1.mailbox.org ([80.241.60.212])
	by makus.postgresql.org with esmtps (TLS1.2:RSA_AES_256_CBC_SHA1:256)
	(Exim 4.89)
	(envelope-from <ah@cybertec.at>)
	id 1hC2rw-0001QL-Gn
	for pgsql-hackers@postgresql.org; Thu, 04 Apr 2019 13:57:30 +0000
Received: from smtp2.mailbox.org (smtp2.mailbox.org
 [IPv6:2001:67c:2050:105:465:1:2:0])
	(using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits))
	(No client certificate requested)
	by mx1.mailbox.org (Postfix) with ESMTPS id 8D8084A16D;
	Thu,  4 Apr 2019 15:57:24 +0200 (CEST)
X-Virus-Scanned: amavisd-new at heinlein-support.de
Received: from smtp2.mailbox.org ([80.241.60.241])
	by spamfilter06.heinlein-hosting.de (spamfilter06.heinlein-hosting.de
 [80.241.56.125]) (amavisd-new, port 10030)
	with ESMTP id bpgtAlRVPxEy; Thu,  4 Apr 2019 15:57:15 +0200 (CEST)
From: Antonin Houska <ah@cybertec.at>
To: Robert Haas <robertmhaas@gmail.com>
cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>,
    pgsql-hackers@postgresql.org
Subject: Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
In-reply-to: 
 <CA+Tgmoawc7hQW2SrcvVSfw8JyYV8UU4fyQ56nxKn_BtPhXfhdQ@mail.gmail.com>
References: <5907.1534140361@localhost>
 <201808150211.w7F2ApMj023097@ccmail04.silk.ntt-tx.co.jp>
 <19039.1536933944@localhost>
 <CA+TgmoawhAmdVaSYt3YLA5dsgoc-oXf72O3ybFAqiARfyTL4iA@mail.gmail.com>
 <14646.1552570130@localhost>
 <CA+Tgmoawc7hQW2SrcvVSfw8JyYV8UU4fyQ56nxKn_BtPhXfhdQ@mail.gmail.com>
Comments: In-reply-to Robert Haas <robertmhaas@gmail.com>
   message dated "Fri, 15 Mar 2019 11:34:17 -0400."
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <11811.1554386232.1@localhost>
Content-Transfer-Encoding: quoted-printable
Date: Thu, 04 Apr 2019 15:57:12 +0200
Message-ID: <11812.1554386232@localhost>
List-Id: <pgsql-hackers.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-hackers@lists.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-hackers>
Precedence: bulk

Robert Haas <robertmhaas@gmail.com> wrote:

> On Thu, Mar 14, 2019 at 9:26 AM Antonin Houska <ah@cybertec.at> wrote:
> > > Hint bits also rely on this principle.  I thought there might be som=
e
> > > interaction between this work and wal_log_hints for this reason, but=
 I see
> > > nothing of that sort in the patch.
> >
> > I'm not sure if the hint bit is still a problem if we first copy the s=
hared
> > buffer to backend-local memory and encrypt it there. That's what the p=
atch
> > does.
> =

> That has the same problem that I described in the previous paragraph,
> except for the relation data files rather than the WAL itself.  See
> the manual's description of wal_log_hints:
> =

>        <para>
>         When this parameter is <literal>on</literal>, the
> <productname>PostgreSQL</productname>
>         server writes the entire content of each disk page to WAL during=
 the
>         first modification of that page after a checkpoint, even for
>         non-critical modifications of so-called hint bits.
>        </para>
> =

> For encryption to work, you need that.  A hint bit write might convert
> the page to garbage, just as in the previous example, and this option
> make sure that if that happens, there will be an FPW, allowing you to
> recover...

I think I finally understand. Originally I thought the question is how to
compute correct page checksum while the hint bits can be changed w/o exclu=
sive
lock on the buffer. Now I realize that it's more about *recovery*: if the =
hint
bit change is followed by a torn page write, the hint bit can get changed =
on
disk but the checksum might not get updated. The wrong checksum is detecte=
d
during recovery, but if XLOG does not contain the corresponding full page
image, we're not able to recover.

And with encryption, the consequence is even worse because torn page write
causes not only wrong checksum of otherwise useful page, but really damage=
d
page.

I'll enforce the FPW in the next version of the patch.

-- =

Antonin Houska
Web: https://www.cybertec-postgresql.com