public inbox for [email protected]
help / color / mirror / Atom feedFrom: David Christensen <[email protected]>
To: Aleksander Alekseev <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Cc: Stephen Frost <[email protected]>
Subject: Re: Moving forward with TDE
Date: Thu, 3 Nov 2022 17:06:23 -0500
Message-ID: <CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com> (raw)
In-Reply-To: <CAJ7c6TMJGYGkgb7RKvwj7UkfLQ-TEVJ98_bz=rZpnQnjcD5CRA@mail.gmail.com>
References: <CAOxo6XJh95xPOpvTxuP_kiGRs8eHcaNrmy3kkzWrzWxvyVkKkQ@mail.gmail.com>
<CAJ7c6TMJGYGkgb7RKvwj7UkfLQ-TEVJ98_bz=rZpnQnjcD5CRA@mail.gmail.com>
> Unless somebody in the community remembers open questions/issues with
> TDE that were never addressed I suggest simply iterating with our
> usual testing/reviewing process. For now I'm going to change the
> status of the CF entry [1] to "Waiting for Author" since the patchset
> doesn't pass the CI [2].
Thanks, enclosed is a new version that is rebased on HEAD and fixes a
bug that the new pg_control_init() test picked up.
Known issues (just discovered by me in testing the latest revision) is
that databases created from `template0` are not decrypting properly,
but `template1` works fine, so going to dig in on that soon.
> One limitation of the design described on the wiki I see is that it
> seems to heavily rely on AES:
>
> > We will use Advanced Encryption Standard (AES) [4]. We will offer three key length options (128, 192, and 256-bits) selected at initdb time with --file-encryption-method
>
> (there doesn't seem to be any mention of the hash/MAC algorithms,
> that's odd). In the future we should be able to add the support of
> alternative algorithms. The reason is that the algorithms can become
> weak every 20 years or so, and the preferred algorithms may also
> depend on the region. This should NOT be implemented in this
> particular patchset, but the design shouldn't prevent from
> implementing this in the future.
Yes, we definitely are considering multiple algorithms support as part
of this effort.
Best,
David
Attachments:
[application/octet-stream] v2-0001-cfe-01-doc_over_master-squash-commit.patch (10.1K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/2-v2-0001-cfe-01-doc_over_master-squash-commit.patch)
download | inline diff:
From 6f4cb27162b8b719e1f00d9d4bec77c72b629d3e Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:43 -0400
Subject: [PATCH v2 01/12] cfe-01-doc_over_master squash commit
---
doc/src/sgml/database-encryption.sgml | 149 ++++++++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/installation.sgml | 2 +-
doc/src/sgml/monitoring.sgml | 13 +++
doc/src/sgml/postgres.sgml | 1 +
5 files changed, 165 insertions(+), 1 deletion(-)
create mode 100644 doc/src/sgml/database-encryption.sgml
diff --git a/doc/src/sgml/database-encryption.sgml b/doc/src/sgml/database-encryption.sgml
new file mode 100644
index 0000000000..c4377ada64
--- /dev/null
+++ b/doc/src/sgml/database-encryption.sgml
@@ -0,0 +1,149 @@
+<!-- doc/src/sgml/database-encryption.sgml -->
+
+<chapter id="cluster-file-encryption">
+ <title>Cluster File Encryption</title>
+
+ <indexterm zone="cluster-file-encryption">
+ <primary>Cluster File Encryption</primary>
+ </indexterm>
+
+ <para>
+ The purpose of cluster file encryption is to prevent users with read
+ access on the directories used to store database files and write-ahead
+ log files from being able to access the data stored in those files.
+ For example, when using cluster file encryption, users who have read
+ access to the cluster directories for backup purposes will not be able
+ to decrypt the data stored in these files. Read-only access for a group
+ of users can be enabled using the <application>initdb</application>
+ <option>--allow-group-access</option> option. Cluster file encryption
+ also provides data-at-rest security, protecting users from data loss
+ should the physical storage media be stolen or improperly erased before
+ disposal.
+ </para>
+
+ <para>
+ Cluster file encryption does not protect against unauthorized file
+ system writes. Such writes can allow data decryption if used to weaken
+ the system's security and the weakened system is later supplied with
+ the externally-stored cluster encryption key. This also does not always
+ detect if users with write access remove or modify database files.
+ </para>
+
+ <para>
+ This also does not protect against users who have read access to database
+ process memory because all in-memory data pages and data encryption keys
+ are stored unencrypted in memory. Therefore, an attacker who is able
+ to read memory can read the data encryption keys and decrypt the entire
+ cluster. The Postgres operating system user and the operating system
+ administrator, e.g., the <literal>root</literal> user, have such access.
+ </para>
+
+ <sect1 id="cluster-encryption-keys">
+ <title>Keys</title>
+
+ <para>
+ Cluster file encryption uses two levels of encryption — an upper
+ key which encrypts lower-level keys. The upper-level key is often
+ referred to as a Key Encryption Key (<acronym>KEK</acronym>). This key
+ is <emphasis>not</emphasis> stored in the file system, but provided at
+ <command>initdb</command> time and each time the server is started. This
+ key can be easily changed via <command>pg_alterckey</command> without
+ requiring any changes to the the data files or <command>WAL</command>
+ files.
+ </para>
+
+ <para>
+ The lower level keys are data encryption keys, specifically for relations
+ and <acronym>WAL</acronym>. The relation key is used to encrypt database
+ heap and index files. The WAL key is used to encrypt write-ahead log
+ (WAL) files. Two different keys are used so that primary and standby
+ servers can use different relation keys, but the same WAL key, so that
+ these keys can (in a future release) be rotated by switching the
+ primary to the standby and then changing the WAL key. Eventually,
+ encryption will be able to added to non-encrypted clusters by creating
+ encrypted replicas and switching over to them.
+ </para>
+
+ <para>
+ Postgres stores the data encryption (lower-level) keys in the data
+ directory encrypted (wrapped) by key encryption (upper-level) key.
+ Though the data encryption keys technically exist in the file system,
+ the key encryption key does not, so the data encryption keys are
+ securely stored. Data encryption keys are used to security encrypt
+ other database files.
+ </para>
+ </sect1>
+
+ <sect1 id="cluster-encryption-initialization">
+ <title>Initialization</title>
+
+ <para>
+ Cluster file encryption is enabled when
+ <productname>PostgreSQL</productname> is built
+ with <literal>--with-openssl</literal> and <xref
+ linkend="app-initdb-cluster-key-command"/> is specified
+ during <command>initdb</command>. The cluster key
+ provided by the <option>--cluster-key-command</option>
+ option during <command>initdb</command> and the one generated
+ by <xref linkend="guc-cluster-key-command"/> in the
+ <filename>postgresql.conf</filename> must match for the database
+ cluster to start. Note that the cluster key command
+ passed to <command>initdb</command> must return a key of
+ 64 hexadecimal characters. For example:
+<programlisting>
+initdb -D dbname --cluster-key-command='ckey_passphrase.sh'
+</programlisting>
+ Cluster file encryption does not support a <varname>wal_level</varname>
+ of <literal>minimal</literal>.
+ </para>
+ </sect1>
+
+ <sect1 id="cluster-encryption-operation">
+ <title>Operation</title>
+
+ <para>
+ During the <command>initdb</command> process, if
+ <option>--cluster-key-command</option> is specified, two data-level
+ encryption keys are created. These two keys are then encrypted with
+ the key encryption key (KEK) supplied by the cluster key command before
+ being stored in the database directory. The key or passphrase that
+ derives the key must be supplied from the terminal or stored in a
+ trusted key store, such as key vault software or a hardware security
+ module.
+ </para>
+
+ <para>
+ If the <productname>PostgreSQL</productname> server has
+ been initialized to require a cluster key, each time the
+ server starts the <filename>postgresql.conf</filename>
+ <varname>cluster_key_command</varname> command will be executed
+ and the cluster key retrieved. The data encryption keys in the
+ <filename>pg_cryptokeys</filename> directory will then be decrypted
+ using the supplied key and integrity-checked to ensure it matches the
+ initdb-supplied key. (If this check fails, the server will refuse
+ to start.) The cluster encryption key will then be removed from
+ system memory. The decrypted data encryption keys will remain in
+ shared memory until the server is stopped.
+ </para>
+
+ <para>
+ The data encryption keys are randomly generated and can be 128, 192,
+ or 256-bits in length, depending on whether <literal>AES128</literal>,
+ <literal>AES192</literal>, or <literal>AES256</literal> is specified.
+ They are encrypted by the key encryption key (KEK) using Advanced
+ Encryption Standard (<acronym>AES256</acronym>) encryption in Key
+ Wrap Padded Mode, which also provides KEK authentication; see <ulink
+ url="https://tools.ietf.org/html/rfc5649">RFC 5649</ulink>. While
+ 128-bit encryption is sufficient for most sites, 256-bit encryption
+ is thought to be more immune to future quantum cryptographic attacks.
+ </para>
+
+ <para>.
+ If you prefer to create the random keys on your own, you can create
+ a empty directory with a <filename>pg_cryptokeys/live</filename>
+ subdirectory, generate the keys there using your tools. and use the
+ <command>initdb</command> <option>--copy-encryption-keys</option>
+ to copy those keys into the newly-created cluster.
+ </para>
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index de450cd661..efeb542cf7 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -49,6 +49,7 @@
<!ENTITY wal SYSTEM "wal.sgml">
<!ENTITY logical-replication SYSTEM "logical-replication.sgml">
<!ENTITY jit SYSTEM "jit.sgml">
+<!ENTITY database-encryption SYSTEM "database-encryption.sgml">
<!-- programmer's guide -->
<!ENTITY bgworker SYSTEM "bgworker.sgml">
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 319c7e6966..b67d3b7ffb 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1011,7 +1011,7 @@ build-postgresql:
<listitem>
<para>
Build with support for <acronym>SSL</acronym> (encrypted)
- connections. The only <replaceable>LIBRARY</replaceable>
+ connections and cluster file encryption. The only <replaceable>LIBRARY</replaceable>
supported is <option>openssl</option>. This requires the
<productname>OpenSSL</productname> package to be installed.
<filename>configure</filename> will check for the required
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index e5d622d514..3e3e1adea6 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1340,6 +1340,19 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry><literal>DataFileWrite</literal></entry>
<entry>Waiting for a write to a relation data file.</entry>
</row>
+ <row>
+ <entry><literal>KeyFileRead</literal></entry>
+ <entry>Waiting for a read of the wrapped data encryption keys.</entry>
+ </row>
+ <row>
+ <entry><literal>KeyFileWrite</literal></entry>
+ <entry>Waiting for a write of the wrapped data encryption keys.</entry>
+ </row>
+ <row>
+ <entry><literal>KeyFileSync</literal></entry>
+ <entry>Waiting for changes to the wrapped data encryption keys to reach
+ durable storage.</entry>
+ </row>
<row>
<entry><literal>LockFileAddToDataDirRead</literal></entry>
<entry>Waiting for a read while adding a line to the data directory lock
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 73439c049e..d99b24605f 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -171,6 +171,7 @@ break is not needed in a wider output rendering.
&wal;
&logical-replication;
&jit;
+ &database-encryption;
®ress;
</part>
--
2.31.1
[application/octet-stream] v2-0005-cfe-05-crypto_over_cfe-04-common-squash-commit.patch (23.7K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/3-v2-0005-cfe-05-crypto_over_cfe-04-common-squash-commit.patch)
download | inline diff:
From 1a69fe5bd62a573427426e2305679562d72090a9 Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:51 -0400
Subject: [PATCH v2 05/12] cfe-05-crypto_over_cfe-04-common squash commit
---
doc/src/sgml/config.sgml | 120 ++++++++--
src/backend/crypto/Makefile | 18 ++
src/backend/crypto/kmgr.c | 438 ++++++++++++++++++++++++++++++++++++
src/common/kmgr_utils.c | 3 +-
src/include/crypto/kmgr.h | 25 ++
5 files changed, 588 insertions(+), 16 deletions(-)
create mode 100644 src/backend/crypto/Makefile
create mode 100644 src/backend/crypto/kmgr.c
create mode 100644 src/include/crypto/kmgr.h
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 559eb898a9..4e95f7e22a 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -1558,19 +1558,31 @@ include_dir 'conf.d'
mechanism is used.
</para>
<para>
- The command must print the passphrase to the standard output and exit
- with code 0. In the parameter value, <literal>%p</literal> is
- replaced by a prompt string. (Write <literal>%%</literal> for a
- literal <literal>%</literal>.) Note that the prompt string will
- probably contain whitespace, so be sure to quote adequately. A single
- newline is stripped from the end of the output if present.
+ The command must print the passphrase to the standard output
+ and exit with code 0. It can prompt from the terminal if
+ <option>--authprompt</option> is used. In the parameter
+ value, <literal>%R</literal> is replaced by a file descriptor
+ number opened to the terminal that started the server. A file
+ descriptor is only available if enabled at server start via
+ <option>-R</option>. If <literal>%R</literal> is specified and
+ no file descriptor is available, the server will not start. Value
+ <literal>%p</literal> is replaced by a pre-defined prompt string.
+ (Write <literal>%%</literal> for a literal <literal>%</literal>.)
+ Note that the prompt string will probably contain whitespace,
+ so be sure to quote its use adequately. Newlines are stripped
+ from the end of the output if present.
</para>
+
<para>
- The command does not actually have to prompt the user for a
- passphrase. It can read it from a file, obtain it from a keychain
- facility, or similar. It is up to the user to make sure the chosen
- mechanism is adequately secure.
+ Sample scripts can be found in
+ <filename>$SHAREDIR/auth_commands</filename>,
+ where <literal>$SHAREDIR</literal> means the
+ <productname>PostgreSQL</productname> installation's shared-data
+ directory, often <filename>/usr/local/share/postgresql</filename>
+ (use <command>pg_config --sharedir</command> to determine it if
+ you're not sure).
</para>
+
<para>
This parameter can only be set in the <filename>postgresql.conf</filename>
file or on the server command line.
@@ -1592,10 +1604,12 @@ include_dir 'conf.d'
parameter is off (the default), then
<varname>ssl_passphrase_command</varname> will be ignored during a
reload and the SSL configuration will not be reloaded if a passphrase
- is needed. That setting is appropriate for a command that requires a
- TTY for prompting, which might not be available when the server is
- running. Setting this parameter to on might be appropriate if the
- passphrase is obtained from a file, for example.
+ is needed. This setting is appropriate for a command that requires a
+ terminal for prompting, which will likely not be available when the server is
+ running. (<option>--authprompt</option> closes the terminal file
+ descriptor soon after server start.) Setting this parameter on
+ might be appropriate, for example, if the passphrase is obtained
+ from a file.
</para>
<para>
This parameter can only be set in the <filename>postgresql.conf</filename>
@@ -2794,7 +2808,9 @@ include_dir 'conf.d'
<varname>max_wal_senders</varname> is non-zero.
Note that changing <varname>wal_level</varname> to
<literal>minimal</literal> makes previous base backups unusable
- for point-in-time recovery and standby servers.
+ for point-in-time recovery and standby servers, which may
+ lead to data loss. Cluster file encryption also does not support
+ <varname>wal_level</varname> <literal>minimal</literal>.
</para>
<para>
In <literal>logical</literal> level, the same information is logged as
@@ -8483,6 +8499,64 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</variablelist>
</sect1>
+ <sect1 id="runtime-config-encryption">
+ <title>Cluster File Encryption</title>
+
+ <variablelist>
+ <varlistentry id="guc-cluster-key-command" xreflabel="cluster_key_command">
+ <term><varname>cluster_key_command</varname> (<type>string</type>)
+ <indexterm>
+ <primary><varname>cluster_key_command</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This option specifies an external command to obtain the cluster-level
+ key for cluster file encryption during server initialization and
+ server start.
+ </para>
+ <para>
+ The command must print the cluster key to the standard
+ output as 64 hexadecimal characters, and exit with code 0.
+ The command can prompt for the passphrase or PIN from the
+ terminal if <option>--authprompt</option> is used. In the
+ parameter value, <literal>%R</literal> is replaced by a file
+ descriptor number opened to the terminal that started the server.
+ A file descriptor is only available if enabled at server start
+ via <option>-R</option>. If <literal>%R</literal> is specified
+ and no file descriptor is available, the server will not start.
+ Value <literal>%p</literal> is replaced by a pre-defined
+ prompt string. Value <literal>%d</literal> is replaced by the
+ directory containing the keys; this is useful if the command
+ must create files with the keys, e.g., to store a cluster-level
+ key encrypted by a key stored in a hardware security module.
+ (Write <literal>%%</literal> for a literal <literal>%</literal>.)
+ Note that the prompt string will probably contain whitespace,
+ so be sure to quote its use adequately. Newlines are stripped
+ from the end of the output if present.
+ </para>
+
+ <para>
+ Sample script can be found in
+ <filename>$SHAREDIR/auth_commands</filename>,
+ where <literal>$SHAREDIR</literal> means the
+ <productname>PostgreSQL</productname> installation's shared-data
+ directory, often <filename>/usr/local/share/postgresql</filename>
+ (use <command>pg_config --sharedir</command> to determine it if
+ you're not sure).
+ </para>
+
+ <para>
+ This parameter can only be set by
+ <application>initdb</application>, in the
+ <filename>postgresql.conf</filename> file, or on the server
+ command line.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect1>
+
<sect1 id="runtime-config-client">
<title>Client Connection Defaults</title>
@@ -10508,6 +10582,22 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
</listitem>
</varlistentry>
+ <varlistentry id="guc-file-encryption-method" xreflabel="file_encryption_method">
+ <term><varname>file_encryption_method</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary>Cluster file encryption method</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Reports the cluster file
+ encryption method. See <xref
+ linkend="app-initdb-cluster-key-command"/> for more
+ information.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-data-directory-mode" xreflabel="data_directory_mode">
<term><varname>data_directory_mode</varname> (<type>integer</type>)
<indexterm>
diff --git a/src/backend/crypto/Makefile b/src/backend/crypto/Makefile
new file mode 100644
index 0000000000..c27362029d
--- /dev/null
+++ b/src/backend/crypto/Makefile
@@ -0,0 +1,18 @@
+#-------------------------------------------------------------------------
+#
+# Makefile
+# Makefile for src/backend/crypto
+#
+# IDENTIFICATION
+# src/backend/crypto/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/crypto
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ kmgr.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/crypto/kmgr.c b/src/backend/crypto/kmgr.c
new file mode 100644
index 0000000000..4b74961323
--- /dev/null
+++ b/src/backend/crypto/kmgr.c
@@ -0,0 +1,438 @@
+/*-------------------------------------------------------------------------
+ *
+ * kmgr.c
+ * Cluster file encryption routines
+ *
+ * Cluster file encryption is enabled if user requests it during initdb.
+ * During bootstrap, we generate data encryption keys, wrap them with the
+ * cluster-level key, and store them into each file located at KMGR_DIR.
+ * During startup, we decrypt all internal keys and load them to the shared
+ * memory. Internal keys in the shared memory are read-only. The wrapping
+ * and unwrapping key routines require the OpenSSL library.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/crypto/kmgr.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+
+#include "access/xlog.h"
+#include "common/file_perm.h"
+#include "common/kmgr_utils.h"
+#include "common/sha2.h"
+#include "access/xlog.h"
+#include "common/controldata_utils.h"
+#include "crypto/kmgr.h"
+#include "postmaster/postmaster.h"
+#include "storage/copydir.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/shmem.h"
+#include "utils/builtins.h"
+#include "utils/memutils.h"
+/* Struct stores file encryption keys in plaintext format */
+typedef struct KmgrShmemData
+{
+ CryptoKey intlKeys[KMGR_NUM_DATA_KEYS];
+} KmgrShmemData;
+static KmgrShmemData *KmgrShmem;
+
+/* GUC variables */
+char *cluster_key_command = NULL;
+
+CryptoKey bootstrap_keys[KMGR_NUM_DATA_KEYS];
+
+extern char *bootstrap_old_key_datadir;
+extern int bootstrap_file_encryption_method;
+
+static void bzeroKmgrKeys(int status, Datum arg);
+static void KmgrWriteCryptoKeys(const char *dir, unsigned char **keys, int *key_lens);
+static CryptoKey *generate_crypto_key(int len);
+
+/*
+ * This function must be called ONCE during initdb. It creates the DEK
+ * files wrapped with the KEK supplied by kmgr_run_cluster_key_command().
+ * There is also an option for the keys to be copied from another cluster.
+ */
+void
+BootStrapKmgr(void)
+{
+ char live_path[MAXPGPATH];
+ unsigned char *keys_wrap[KMGR_NUM_DATA_KEYS];
+ int key_lens[KMGR_NUM_DATA_KEYS];
+ char cluster_key_hex[ALLOC_KMGR_CLUSTER_KEY_LEN];
+ int cluster_key_hex_len;
+ unsigned char cluster_key[KMGR_CLUSTER_KEY_LEN];
+
+ if (!FileEncryptionEnabled)
+ return;
+
+#ifndef USE_OPENSSL
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ (errmsg("cluster file encryption is not supported because OpenSSL is not supported by this build"),
+ errhint("Compile with --with-openssl to use this feature."))));
+#endif
+
+ /*
+ * There are too many optimizations for wal_level=minimal that don't set
+ * LSNs for permanent tables, so just disallow it.
+ */
+ if (!XLogIsNeeded())
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ (errmsg("cluster file encryption is not supported with a wal_level of \"minimal\""))));
+
+ snprintf(live_path, sizeof(live_path), "%s/%s", DataDir, LIVE_KMGR_DIR);
+
+ /*
+ * Copy cluster file encryption keys from an old cluster? This is useful
+ * for pg_upgrade upgrades where the copied database files are already
+ * encrypted using the old cluster's DEK keys.
+ */
+ if (bootstrap_old_key_datadir != NULL)
+ {
+ char old_key_dir[MAXPGPATH];
+
+ snprintf(old_key_dir, sizeof(old_key_dir), "%s/%s",
+ bootstrap_old_key_datadir, LIVE_KMGR_DIR);
+ copydir(old_key_dir, LIVE_KMGR_DIR, true);
+ }
+ /* create an empty directory */
+ else
+ {
+ if (mkdir(LIVE_KMGR_DIR, pg_dir_create_mode) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create cluster file encryption directory \"%s\": %m",
+ LIVE_KMGR_DIR)));
+ }
+
+ /*
+ * Get key encryption key (KEK) from the cluster_key command. The cluster
+ * key command might need to check for the existence of files in the live
+ * directory, e.g., PIV, so run this _after_ copying the directory in
+ * place.
+ */
+ cluster_key_hex_len = kmgr_run_cluster_key_command(cluster_key_command,
+ cluster_key_hex,
+ ALLOC_KMGR_CLUSTER_KEY_LEN,
+ live_path, terminal_fd);
+
+ /* decode supplied hex */
+ if (hex_decode(cluster_key_hex, cluster_key_hex_len,
+ (char *) cluster_key) !=
+ KMGR_CLUSTER_KEY_LEN)
+ ereport(ERROR,
+ (errmsg("cluster key must be %d hexadecimal characters",
+ KMGR_CLUSTER_KEY_LEN * 2)));
+
+ /* We are not in copy mode? Generate new cluster file encryption keys. */
+ if (bootstrap_old_key_datadir == NULL)
+ {
+ unsigned char *bootstrap_keys_wrap[KMGR_NUM_DATA_KEYS];
+ int key_lens[KMGR_NUM_DATA_KEYS];
+ PgCipherCtx *cluster_key_ctx;
+
+ /* Create KEK encryption context */
+ cluster_key_ctx = pg_cipher_ctx_create(PG_CIPHER_AES_KWP, cluster_key,
+ KMGR_CLUSTER_KEY_LEN, true);
+ if (!cluster_key_ctx)
+ elog(ERROR, "could not initialize encryption context");
+
+ /* Wrap data encryption keys (DEK) using the key encryption key (KEK) */
+ for (int id = 0; id < KMGR_NUM_DATA_KEYS; id++)
+ {
+ CryptoKey *key;
+
+ /* generate a DEK */
+ key = generate_crypto_key(
+ encryption_methods[bootstrap_file_encryption_method].bit_length / 8);
+
+ /* output generated random string as hex, for testing */
+ {
+ char str[MAXPGPATH];
+ int out_len;
+
+ out_len = hex_encode((char *) (key->key), key->klen,
+ str);
+ str[out_len] = '\0';
+ }
+
+ bootstrap_keys_wrap[id] = palloc0(KMGR_MAX_KEY_LEN_BYTES +
+ pg_cipher_blocksize(cluster_key_ctx));
+
+ /* wrap DEK with KEK */
+ if (!kmgr_wrap_data_key(cluster_key_ctx, key, bootstrap_keys_wrap[id], &(key_lens[id])))
+ {
+ pg_cipher_ctx_free(cluster_key_ctx);
+ elog(ERROR, "failed to wrap data encryption key");
+ }
+
+ /* remove DEK from memory */
+ explicit_bzero(key, sizeof(CryptoKey));
+ }
+
+ /* Write data encryption keys to the disk */
+ KmgrWriteCryptoKeys(LIVE_KMGR_DIR, bootstrap_keys_wrap, key_lens);
+
+ pg_cipher_ctx_free(cluster_key_ctx);
+ }
+
+ /*
+ * We are either decrypting keys we copied from an old cluster, or
+ * decrypting keys we just wrote above --- either way, we decrypt them
+ * here and store them in a file-scoped variable for use in later
+ * encrypting during bootstrap mode.
+ */
+
+ /* Get the crypto keys from the live directory */
+ kmgr_read_wrapped_data_keys(LIVE_KMGR_DIR, keys_wrap, key_lens);
+
+ if (!kmgr_verify_cluster_key(cluster_key, keys_wrap, key_lens, bootstrap_keys))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("supplied cluster key does not match expected cluster_key")));
+
+ /* bzero DEK on exit */
+ on_proc_exit(bzeroKmgrKeys, 0);
+
+ /* bzero KEK */
+ explicit_bzero(cluster_key_hex, cluster_key_hex_len);
+ explicit_bzero(cluster_key, KMGR_CLUSTER_KEY_LEN);
+}
+
+/* Report shared-memory space needed by KmgrShmem */
+Size
+KmgrShmemSize(void)
+{
+ if (!FileEncryptionEnabled)
+ return 0;
+
+ return MAXALIGN(sizeof(KmgrShmemData));
+}
+
+/* Allocate and initialize key manager memory */
+void
+KmgrShmemInit(void)
+{
+ bool found;
+
+ if (!FileEncryptionEnabled)
+ return;
+
+ KmgrShmem = (KmgrShmemData *) ShmemInitStruct("File encryption key manager",
+ KmgrShmemSize(), &found);
+
+ /* bzero DEK on exit */
+ on_shmem_exit(bzeroKmgrKeys, 0);
+}
+
+/*
+ * Get cluster key and verify it, then get the data encryption keys.
+ * This function is called by postmaster at startup time.
+ */
+void
+InitializeKmgr(void)
+{
+ unsigned char *keys_wrap[KMGR_NUM_DATA_KEYS];
+ int key_lens[KMGR_NUM_DATA_KEYS];
+ char cluster_key_hex[ALLOC_KMGR_CLUSTER_KEY_LEN];
+ int cluster_key_hex_len;
+ struct stat buffer;
+ char live_path[MAXPGPATH];
+ unsigned char cluster_key[KMGR_CLUSTER_KEY_LEN];
+
+#ifndef USE_OPENSSL
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ (errmsg("cluster file encryption is not supported because OpenSSL is not supported by this build"),
+ errhint("Compile with --with-openssl to use this feature."))));
+#endif
+
+ /*
+ * There are too many optimizations for wal_level=minimal that don't set
+ * LSNs for permanent tables, so just disallow it.
+ */
+ if (!XLogIsNeeded())
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ (errmsg("cluster file encryption is not supported with a wal_level of \"minimal\""))));
+
+ elog(DEBUG1, "starting up cluster file encryption manager");
+
+ if (stat(KMGR_DIR, &buffer) != 0 || !S_ISDIR(buffer.st_mode))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ (errmsg("cluster file encryption directory %s is missing", KMGR_DIR))));
+
+ if (stat(KMGR_DIR_PID, &buffer) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ (errmsg("cluster had a pg_alterckey failure that needs repair or pg_alterckey is running"),
+ errhint("Run pg_alterckey --repair or wait for it to complete."))));
+
+ /*
+ * We want OLD deleted since it allows access to the data encryption keys
+ * using the old cluster key. If NEW exists, it means either the new
+ * directory is partly written, or NEW wasn't renamed to LIVE --- in
+ * either case, it needs to be repaired. See src/bin/pg_alterckey/README
+ * for more details.
+ */
+ if (stat(OLD_KMGR_DIR, &buffer) == 0 || stat(NEW_KMGR_DIR, &buffer) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ (errmsg("cluster had a pg_alterckey failure that needs repair"),
+ errhint("Run pg_alterckey --repair."))));
+
+ /* If OLD, NEW, and LIVE do not exist, there is a serious problem. */
+ if (stat(LIVE_KMGR_DIR, &buffer) != 0 || !S_ISDIR(buffer.st_mode))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ (errmsg("cluster has no data encryption keys"))));
+
+ /* Get the cluster key (KEK) */
+ snprintf(live_path, sizeof(live_path), "%s/%s", DataDir, LIVE_KMGR_DIR);
+ cluster_key_hex_len = kmgr_run_cluster_key_command(cluster_key_command,
+ cluster_key_hex,
+ ALLOC_KMGR_CLUSTER_KEY_LEN,
+ live_path, terminal_fd);
+
+ /* decode supplied hex */
+ if (hex_decode(cluster_key_hex, cluster_key_hex_len,
+ (char *) cluster_key) !=
+ KMGR_CLUSTER_KEY_LEN)
+ ereport(ERROR,
+ (errmsg("cluster key must be %d hexadecimal characters",
+ KMGR_CLUSTER_KEY_LEN * 2)));
+
+ /* Load wrapped DEKs from their files into an array */
+ kmgr_read_wrapped_data_keys(LIVE_KMGR_DIR, keys_wrap, key_lens);
+
+ /*
+ * Verify cluster key and store the unwrapped data encryption keys in
+ * shared memory.
+ */
+ if (!kmgr_verify_cluster_key(cluster_key, keys_wrap, key_lens, KmgrShmem->intlKeys))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("supplied cluster key does not match expected cluster key")));
+
+ /* Check that retrieved key lengths match controldata length. */
+ for (int id = 0; id < KMGR_NUM_DATA_KEYS; id++)
+ if (KmgrShmem->intlKeys[id].klen * 8 !=
+ encryption_methods[GetFileEncryptionMethod()].bit_length)
+ {
+ char path[MAXPGPATH];
+
+ CryptoKeyFilePath(path, DataDir, id);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("data encryption key %s of length %d does not match controldata key length %d",
+ path, KmgrShmem->intlKeys[id].klen * 8,
+ encryption_methods[GetFileEncryptionMethod()].bit_length)));
+ }
+
+ /* bzero KEK */
+ explicit_bzero(cluster_key_hex, cluster_key_hex_len);
+ explicit_bzero(cluster_key, KMGR_CLUSTER_KEY_LEN);
+}
+
+static void
+bzeroKmgrKeys(int status, Datum arg)
+{
+ if (IsBootstrapProcessingMode())
+ explicit_bzero(bootstrap_keys, sizeof(bootstrap_keys));
+ else
+ explicit_bzero(KmgrShmem->intlKeys, sizeof(KmgrShmem->intlKeys));
+}
+
+/* return requested DEK */
+const CryptoKey *
+KmgrGetKey(int id)
+{
+ Assert(id < KMGR_NUM_DATA_KEYS);
+
+ return (const CryptoKey *) (IsBootstrapProcessingMode() ?
+ &(bootstrap_keys[id]) : &(KmgrShmem->intlKeys[id]));
+}
+
+/* Generate a DEK inside a CryptoKey */
+static CryptoKey *
+generate_crypto_key(int len)
+{
+ CryptoKey *newkey;
+
+ Assert(len <= KMGR_MAX_KEY_LEN);
+ newkey = (CryptoKey *) palloc0(sizeof(CryptoKey));
+
+ newkey->klen = len;
+
+ if (!pg_strong_random(newkey->key, len))
+ elog(ERROR, "failed to generate new file encryption key");
+
+ return newkey;
+}
+
+/*
+ * Write the DEKs to the disk.
+ */
+static void
+KmgrWriteCryptoKeys(const char *dir, unsigned char **keys, int *key_lens)
+{
+ elog(DEBUG2, "writing data encryption keys wrapped using the cluster key");
+
+ for (int i = 0; i < KMGR_NUM_DATA_KEYS; i++)
+ {
+ int fd;
+ char path[MAXPGPATH];
+
+ CryptoKeyFilePath(path, dir, i);
+
+ if ((fd = BasicOpenFile(path, O_RDWR | O_CREAT | O_EXCL | PG_BINARY)) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m",
+ path)));
+
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_WRITE);
+ if (write(fd, keys[i], key_lens[i]) != key_lens[i])
+ {
+ /* if write didn't set errno, assume problem is no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ path)));
+ }
+ pgstat_report_wait_end();
+
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_SYNC);
+ if (pg_fsync(fd) != 0)
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not fsync file \"%s\": %m",
+ path)));
+ pgstat_report_wait_end();
+
+ if (close(fd) != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m",
+ path)));
+ }
+}
diff --git a/src/common/kmgr_utils.c b/src/common/kmgr_utils.c
index 6c50cb093b..92b6441186 100644
--- a/src/common/kmgr_utils.c
+++ b/src/common/kmgr_utils.c
@@ -30,11 +30,12 @@
#endif
#include "common/cryptohash.h"
#include "common/file_perm.h"
-#include "common/hex.h"
#include "common/string.h"
#include "crypto/kmgr.h"
#include "lib/stringinfo.h"
#include "storage/fd.h"
+#include "postgres.h"
+#include "utils/builtins.h"
#ifndef FRONTEND
#include "pgstat.h"
diff --git a/src/include/crypto/kmgr.h b/src/include/crypto/kmgr.h
new file mode 100644
index 0000000000..ec341c70b9
--- /dev/null
+++ b/src/include/crypto/kmgr.h
@@ -0,0 +1,25 @@
+/*-------------------------------------------------------------------------
+ *
+ * kmgr.h
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/crypto/kmgr.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef KMGR_H
+#define KMGR_H
+
+#include "common/kmgr_utils.h"
+
+/* GUC parameters */
+extern char *cluster_key_command;
+
+extern Size KmgrShmemSize(void);
+extern void KmgrShmemInit(void);
+extern void BootStrapKmgr(void);
+extern void InitializeKmgr(void);
+extern const CryptoKey *KmgrGetKey(int id);
+
+#endif /* KMGR_H */
--
2.31.1
[application/octet-stream] v2-0002-cfe-02-internaldoc_over_cfe-01-doc-squash-commit.patch (11.4K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/4-v2-0002-cfe-02-internaldoc_over_cfe-01-doc-squash-commit.patch)
download | inline diff:
From bbaaac474026fb8ab624f0396da5222bb5c52909 Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:45 -0400
Subject: [PATCH v2 02/12] cfe-02-internaldoc_over_cfe-01-doc squash commit
---
src/backend/crypto/README | 231 ++++++++++++++++++++++++++++++++++++++
1 file changed, 231 insertions(+)
create mode 100644 src/backend/crypto/README
diff --git a/src/backend/crypto/README b/src/backend/crypto/README
new file mode 100644
index 0000000000..be5e5557ba
--- /dev/null
+++ b/src/backend/crypto/README
@@ -0,0 +1,231 @@
+Cluster File Encryption
+=======================
+
+This directory contains support functions and sample scripts to be used
+for cluster file encryption.
+
+Architecture
+------------
+
+Fundamentally, cluster file encryption must store data in a file system
+in such a way that the keys required to decrypt the file system data can
+only be accessed using somewhere outside of the file system itself. The
+external requirement can be someone typing in a passphrase, getting a
+key from a key management server (KMS), or decrypting a key stored in
+the file system using a hardware security module (HSM). The current
+architecture supports all of these methods, and includes sample scripts
+for them.
+
+The simplest method for accessing data keys using some external
+requirement would be to retrieve all data encryption keys from a KMS.
+However, retrieved keys would still need to be verified as valid. This
+method also introduces unacceptable complexity for simpler use-cases,
+like user-supplied passphrases or HSM usage. External key rotation
+would also be very hard since it would require re-encrypting all the
+file system data with the new externally-stored keys.
+
+For these reason, a two-tiered architecture is used, which uses two
+types of encryption keys: a key encryption key (KEK) and data encryption
+keys (DEK). The KEK should not be present unencrypted in the file system
+--- it should be supplied the user, stored externally (e.g., in a KMS)
+or stored in the file system encrypted with a HSM (e.g., PIV device).
+The DEK is used to encrypt database files and is stored in the same file
+system as the database but is encrypted using the KEK. Because the DEK
+is encrypted, its storage in the file system is no more of a security
+weakness and the storage of the encrypted database files in the same
+file system.
+
+Implementation
+--------------
+
+To enable cluster file encryption, the initdb option
+--cluster-key-command must be used, which specifies a command to
+retrieve the KEK. initdb records the cluster_key_command in
+postgresql.conf. Every time the KEK is needed, the command is run and
+must return 64 hex characters which are decoded into the KEK. The
+command is called twice during initdb, and every time the server starts.
+initdb also sets the encryption method in controldata during server
+bootstrap.
+
+initdb runs "postgres --boot", which calls function
+kmgr.c::BootStrapKmgr(), which calls the cluster key command. The
+cluster key command returns a KEK which is used to encrypt random bytes
+for each DEK and writes them to the file system by
+kmgr.c::KmgrWriteCryptoKeys() (unless --copy-encryption-keys is used).
+Currently the DEK files are 0 and 1 and are stored in
+$PGDATA/pg_cryptokeys/live. The wrapped DEK files use Key Wrapping with
+Padding which verifies the validity of the KEK.
+
+initdb also does a non-boot backend start which calls
+kmgr.c::InitializeKmgr(), which calls the cluster key command a second
+time. This decrypts/unwraps the DEK keys and stores them in the shared
+memory structure KmgrShmem. This step also happens every time the server
+starts. Later patches will use the keys stored in KmgrShmem to
+encrypt/decrypt database files. KmgrShmem is erased via
+explicit_bzero() on server shutdown.
+
+Limitations
+-----------
+
+There doesn't seem to be a reasonable way to detect all malicious data
+modification or key extraction if a user has write permission on the
+files in PGDATA. It might be possible to limit the key extraction risk
+if postgresql.auto.conf were able to be moved to a directory outside of
+PGDATA, and if postmaster.opts could be moved or ignored when cluster
+file encryption is used. (This file is used by pg_ctl restart.)
+
+It doesn't appear possible to detect all malicious writes --- even if
+you add message authentication code (MAC) checks to encrypted files,
+modifying non-encrypted files could still affect encrypted ones, e.g.,
+modifying files in pg_xact could affect how heap rows are interpreted.
+Basically you would need to encrypt all files, and at that point you
+might as well just use an encrypted file system. There also doesn't seem
+to be a way to prevent key extraction if someone has read permission on
+postgres process memory.
+
+Initialization Vector
+---------------------
+
+Nonce means "number used once". An Initialization Vector (IV) is a
+specific type of nonce. That is, unique but not necessarily random or
+secret, as specified by the NIST
+(https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38a.pdf).
+To generate unique IVs, the NIST recommends two methods:
+
+ The first method is to apply the forward cipher function, under
+ the same key that is used for the encryption of the plaintext,
+ to a nonce. The nonce must be a data block that is unique to
+ each execution of the encryption operation. For example, the
+ nonce may be a counter, as described in Appendix B, or a message
+ number. The second method is to generate a random data block
+ using a FIPS-approved random number generator.
+
+We will use the first method to generate IVs. That is, select nonce
+carefully and use a cipher with the key to make it unique enough to use
+as an IV. The nonce selection for buffer encryption and WAL encryption
+are described below.
+
+If the IV was used more than once with the same key (and we only use one
+data encryption key), changes in the unencrypted data would be visible
+in the encrypted data.
+
+IV for Heap/Index Encryption
+- - - - - - - - - - - - - -
+
+To create the 16-byte IV needed by AES for each page version, we will
+use the page LSN (8 bytes) and page number (4 bytes). In the remaining
+four bytes, one bit will be used to indicate if the LSN is WAL (real) or
+fake (see below). The LSN is ideal for use in the IV because it is
+always increasing, and is changed every time a page is updated. The
+same LSN is never used for two relations with different page contents.
+
+However, the same LSN can be used in multiple pages in the same relation
+--- this can happen when a heap update expires an old tuple and adds a
+new tuple to another page. By adding the page number to the IV, we keep
+the IV unique.
+
+By not using the database id in the IV, CREATE DATABASE can copy the
+heap/index files from the old database to a new one without
+decryption/encryption. Both page copies are valid. Once a database
+changes its pages, it gets new LSNs, and hence new IV. Using only the
+LSN and page number also avoids requiring pg_upgrade to preserve
+database oids, tablespace oids, and relfilenodes.
+
+As part of WAL logging, every change of a WAL-logged page gets a new
+LSN, and therefore a new IV automatically.
+
+However, the LSN must then be visible on encrypted pages, so we will not
+encrypt the LSN on the page. We will also not encrypt the CRC so
+pg_checksums can still check pages offline without access to the keys.
+
+Non-Permanent Relations
+- - - - - - - - - - - -
+
+To avoid the overhead of generating WAL for non-permanent (unlogged and
+temporary) relations, we assign fake LSNs that are derived from a
+counter via xlog.c::GetFakeLSNForUnloggedRel(). (GiST also uses this
+counter for LSNs.) We also set a bit in the IV so the use of the same
+value for WAL (real) and fake LSNs will still generate unique IVs. Only
+main forks are encrypted, not init, vm, or fsm files.
+
+In the code, we need to identify if a page uses WAL or fake LSNs in
+four places, when:
+
+1. Reading a page from the file system and decrypting
+2. Setting the WAL or fake LSN on a page
+3. Hint bits changes requiring new LSNs for the encryption IV
+4. Encrypting and writing a page to the file system
+
+For all these case, we have access to the fork number and either the
+relation's persistence state or the buffer state. If it is a "main"
+fork and the relation persistence state is RELPERSISTENCE_PERMANENT, or
+if it is an "init" fork, we use a real LSN. If it is a main fork and
+RELPERSISTENCE_PERMANENT is false, we use a fake LSN. The buffer state
+BM_PERMANENT is true if the relation is PERMANENT or is an init fork.
+
+Init Forks
+- - - - -
+
+Init forks for unlogged relations get permanent LSNs because unlogged
+relation creation is WAL logged/crash safe, even though the relation's
+contents are not. When the init fork is copied to represent an empty
+relation during crash recovery, it becomes a non-permanent page and must
+be successfully decrypted as such. Therefore, when it is copied, its
+LSN is changed to e fake LSN and then encrypted. This prevents a real
+LSN from being encrypted with the fake nonce bit.
+
+LSN Assignment, GiST, & Non-Permanent Relations
+- - - - - - - - - - - - - - - - - - - - - - - -
+
+LSN assignment has to be slightly modified for encryption. In normal,
+non-encryption mode, LSNs are assigned to pages following these rules:
+
+1. During GiST builds, some pages are assigned fixed LSNs (GistBuildLSN)
+
+2. During GiST builds, non-permanent pages not assigned fixed LSNs in
+#1 are assigned fake LSNs, via gistutil.c::gistGetFakeLSN().
+
+3. All other permanent pages are assigned WAL-based LSNs based on the
+WAL position of their WAL records.
+
+4. All other non-permanent pages have LSNs of zero.
+
+When encryption is enabled:
+
+1. During GiST builds, permanent pages are assigned WAL-based LSNs
+generated by xloginsert.c::LSNForEncryption().
+
+2. During GiST builds, non-permanent pages are assigned fake LSNs.
+(No constant LSNs are used in #1 or #2.)
+
+3. same as #3 above
+
+4. All other non-permanent pages are assigned fake LSNs before page
+encryption.
+
+When switching to an encrypted replica from a non-encrypted primary,
+GiST indexes will be using fixed LSNs for permanent tables, so it is
+recommended to rebuild GiST indexes. Non-permanent relations are not
+replicated, so they are not an issue.
+
+Hint Bits
+- - - - -
+
+For hint bit changes, the LSN normally doesn't change, which is a
+problem. By enabling wal_log_hints, you get full page writes to the WAL
+after the first hint bit change of the checkpoint. This is useful for
+two reasons. First, it generates a new LSN, which is needed for the IV
+to be secure. Second, full page images protect against torn pages,
+which is an even bigger requirement for encryption because the new LSN
+is re-encrypting the entire page, not just the hint bit changes. You
+can safely lose the hint bit changes, but you need to use the same LSN
+to decrypt the entire page, so a torn page with an LSN change cannot be
+decrypted. To prevent this, wal_log_hints guarantees that the
+pre-hint-bit version (and previous LSN version) of the page is restored.
+
+However, if a hint-bit-modified page is written to the file system
+during a checkpoint, and there is a later hint bit change switching the
+same page from clean to dirty during the same checkpoint, we need a new
+LSN, and wal_log_hints doesn't give us a new LSN here. The fix for this
+is to update the page LSN by writing a dummy WAL record via
+xloginsert.c::LSNForEncryption() in such cases.
--
2.31.1
[application/octet-stream] v2-0003-cfe-03-scripts_over_cfe-02-internaldoc-squash-com.patch (13.8K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/5-v2-0003-cfe-03-scripts_over_cfe-02-internaldoc-squash-com.patch)
download | inline diff:
From aa2b8fe000bbcf98731d7dbef13170ebde1d33d7 Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:47 -0400
Subject: [PATCH v2 03/12] cfe-03-scripts_over_cfe-02-internaldoc squash commit
---
src/backend/Makefile | 15 +++-
src/backend/crypto/ckey_aws.sh.sample | 53 +++++++++++++
src/backend/crypto/ckey_direct.sh.sample | 39 ++++++++++
src/backend/crypto/ckey_passphrase.sh.sample | 35 +++++++++
src/backend/crypto/ckey_piv_nopin.sh.sample | 68 ++++++++++++++++
src/backend/crypto/ckey_piv_pin.sh.sample | 81 ++++++++++++++++++++
src/backend/crypto/ssl_passphrase.sh.sample | 35 +++++++++
7 files changed, 325 insertions(+), 1 deletion(-)
create mode 100644 src/backend/crypto/ckey_aws.sh.sample
create mode 100644 src/backend/crypto/ckey_direct.sh.sample
create mode 100644 src/backend/crypto/ckey_passphrase.sh.sample
create mode 100644 src/backend/crypto/ckey_piv_nopin.sh.sample
create mode 100644 src/backend/crypto/ckey_piv_pin.sh.sample
create mode 100644 src/backend/crypto/ssl_passphrase.sh.sample
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 181c217fae..1a5239b822 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -198,6 +198,12 @@ endif
$(INSTALL_DATA) $(srcdir)/libpq/pg_hba.conf.sample '$(DESTDIR)$(datadir)/pg_hba.conf.sample'
$(INSTALL_DATA) $(srcdir)/libpq/pg_ident.conf.sample '$(DESTDIR)$(datadir)/pg_ident.conf.sample'
$(INSTALL_DATA) $(srcdir)/utils/misc/postgresql.conf.sample '$(DESTDIR)$(datadir)/postgresql.conf.sample'
+ $(INSTALL_DATA) $(srcdir)/crypto/ckey_aws.sh.sample '$(DESTDIR)$(datadir)/auth_commands/ckey_aws.sh.sample'
+ $(INSTALL_DATA) $(srcdir)/crypto/ckey_direct.sh.sample '$(DESTDIR)$(datadir)/auth_commands/ckey_direct.sh.sample'
+ $(INSTALL_DATA) $(srcdir)/crypto/ckey_passphrase.sh.sample '$(DESTDIR)$(datadir)/auth_commands/ckey_passphrase.sh.sample'
+ $(INSTALL_DATA) $(srcdir)/crypto/ckey_piv_nopin.sh.sample '$(DESTDIR)$(datadir)/auth_commands/ckey_piv_nopin.sh.sample'
+ $(INSTALL_DATA) $(srcdir)/crypto/ckey_piv_pin.sh.sample '$(DESTDIR)$(datadir)/auth_commands/ckey_piv_pin.sh.sample'
+ $(INSTALL_DATA) $(srcdir)/crypto/ssl_passphrase.sh.sample '$(DESTDIR)$(datadir)/auth_commands/ssl_passphrase.sh.sample'
ifeq ($(with_llvm), yes)
install-bin: install-postgres-bitcode
@@ -223,6 +229,7 @@ endif
installdirs:
$(MKDIR_P) '$(DESTDIR)$(bindir)' '$(DESTDIR)$(datadir)'
+ $(MKDIR_P) '$(DESTDIR)$(datadir)' '$(DESTDIR)$(datadir)/auth_commands'
ifeq ($(PORTNAME), cygwin)
ifeq ($(MAKE_DLL), true)
$(MKDIR_P) '$(DESTDIR)$(libdir)'
@@ -262,7 +269,13 @@ endif
$(MAKE) -C utils uninstall-data
rm -f '$(DESTDIR)$(datadir)/pg_hba.conf.sample' \
'$(DESTDIR)$(datadir)/pg_ident.conf.sample' \
- '$(DESTDIR)$(datadir)/postgresql.conf.sample'
+ '$(DESTDIR)$(datadir)/postgresql.conf.sample' \
+ '$(DESTDIR)$(datadir)/auth_commands/ckey_aws.sh.sample' \
+ '$(DESTDIR)$(datadir)/auth_commands/ckey_direct.sh.sample' \
+ '$(DESTDIR)$(datadir)/auth_commands/ckey_passphrase.sh.sample' \
+ '$(DESTDIR)$(datadir)/auth_commands/ckey_piv_nopin.sh.sample' \
+ '$(DESTDIR)$(datadir)/auth_commands/ckey_piv_pin.sh.sample' \
+ '$(DESTDIR)$(datadir)/auth_commands/ssl_passphrase.sh.sample'
ifeq ($(with_llvm), yes)
$(call uninstall_llvm_module,postgres)
endif
diff --git a/src/backend/crypto/ckey_aws.sh.sample b/src/backend/crypto/ckey_aws.sh.sample
new file mode 100644
index 0000000000..d9bee53132
--- /dev/null
+++ b/src/backend/crypto/ckey_aws.sh.sample
@@ -0,0 +1,53 @@
+#!/bin/sh
+
+# This uses the AWS Secrets Manager using the AWS CLI and OpenSSL.
+# This stores the AWS secret Id in $DIR.
+# Do not create any file with extension "wkey" in $DIR; these are
+# reserved for wrapped data key files.
+
+[ "$#" -ne 1 ] && echo "cluster_key_command usage: $0 \"%d\"" 1>&2 && exit 1
+# No need for %R or -R since we are not prompting
+
+DIR="$1"
+[ ! -e "$DIR" ] && echo "$DIR does not exist" 1>&2 && exit 1
+[ ! -d "$DIR" ] && echo "$DIR is not a directory" 1>&2 && exit 1
+
+# File containing the id of the AWS secret
+AWS_ID_FILE="$DIR/aws-secret.id"
+
+
+# ----------------------------------------------------------------------
+
+
+# Create an AWS Secrets Manager secret?
+if [ ! -e "$AWS_ID_FILE" ]
+then # The 'postgres' operating system user must have permission to
+ # access the AWS CLI
+
+ # The epoch-time/directory/hostname combination is unique
+ HASH=$(echo -n "$(date '+%s')$DIR$(hostname)" | sha1sum | cut -d' ' -f1)
+ AWS_SECRET_ID="Postgres-cluster-key-$HASH"
+
+ # Use stdin to avoid passing the secret on the command line
+ openssl rand -hex 32 |
+ aws secretsmanager create-secret \
+ --name "$AWS_SECRET_ID" \
+ --description "Postgres cluster file encryption on $(hostname)" \
+ --secret-string 'file:///dev/stdin' \
+ --output text > /dev/null
+ if [ "$?" -ne 0 ]
+ then echo 'cluster key generation failed' 1>&2
+ exit 1
+ fi
+
+ echo "$AWS_SECRET_ID" > "$AWS_ID_FILE"
+fi
+
+if ! aws secretsmanager get-secret-value \
+ --secret-id "$(cat "$AWS_ID_FILE")" \
+ --output text
+then echo 'cluster key retrieval failed' 1>&2
+ exit 1
+fi | awk -F'\t' 'NR == 1 {print $4}'
+
+exit 0
diff --git a/src/backend/crypto/ckey_direct.sh.sample b/src/backend/crypto/ckey_direct.sh.sample
new file mode 100644
index 0000000000..492defcffe
--- /dev/null
+++ b/src/backend/crypto/ckey_direct.sh.sample
@@ -0,0 +1,39 @@
+#!/bin/sh
+
+# This uses a 64-character hex key supplied by the user.
+# If OpenSSL is installed, you can generate a pseudo-random key by running:
+# openssl rand -hex 32
+# To get a true random key, run:
+# wget -q -O - 'https://www.random.org/cgi-bin/randbyte?nbytes=32&format=h' | tr -d ' \n'; echo
+# Do not create any fie with extension "wkey" in $DIR; these are
+# reserved for wrapped data key files.
+
+[ "$#" -lt 1 ] && echo "cluster_key_command usage: $0 %R [%p]" 1>&2 && exit 1
+# Supports environment variable PROMPT
+
+FD="$1"
+[ ! -t "$FD" ] && echo "file descriptor $FD does not refer to a terminal" 1>&2 && exit 1
+
+[ "$2" ] && PROMPT="$2"
+
+
+# ----------------------------------------------------------------------
+
+[ ! "$PROMPT" ] && PROMPT='Enter cluster key as 64 hexadecimal characters: '
+
+stty -echo <&"$FD"
+
+echo 1>&"$FD"
+echo -n "$PROMPT" 1>&"$FD"
+read KEY <&"$FD"
+
+stty echo <&"$FD"
+
+if [ "$(expr "$KEY" : '[0-9a-fA-F]*$')" -ne 64 ]
+then echo 'invalid; must be 64 hexadecimal characters' 1>&2
+ exit 1
+fi
+
+echo "$KEY"
+
+exit 0
diff --git a/src/backend/crypto/ckey_passphrase.sh.sample b/src/backend/crypto/ckey_passphrase.sh.sample
new file mode 100644
index 0000000000..a5d837b45e
--- /dev/null
+++ b/src/backend/crypto/ckey_passphrase.sh.sample
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+# This uses a passphrase supplied by the user.
+# Do not create any fie with extension "wkey" in $DIR; these are
+# reserved for wrapped data key files.
+
+[ "$#" -lt 1 ] && echo "cluster_key_command usage: $0 %R [\"%p\"]" 1>&2 && exit 1
+
+FD="$1"
+[ ! -t "$FD" ] && echo "file descriptor $FD does not refer to a terminal" 1>&2 && exit 1
+# Supports environment variable PROMPT
+
+[ "$2" ] && PROMPT="$2"
+
+
+# ----------------------------------------------------------------------
+
+[ ! "$PROMPT" ] && PROMPT='Enter cluster passphrase: '
+
+stty -echo <&"$FD"
+
+echo 1>&"$FD"
+echo -n "$PROMPT" 1>&"$FD"
+read PASS <&"$FD"
+
+stty echo <&"$FD"
+
+if [ ! "$PASS" ]
+then echo 'invalid: empty passphrase' 1>&2
+ exit 1
+fi
+
+echo "$PASS" | sha256sum | cut -d' ' -f1
+
+exit 0
diff --git a/src/backend/crypto/ckey_piv_nopin.sh.sample b/src/backend/crypto/ckey_piv_nopin.sh.sample
new file mode 100644
index 0000000000..e90a579dea
--- /dev/null
+++ b/src/backend/crypto/ckey_piv_nopin.sh.sample
@@ -0,0 +1,68 @@
+#!/bin/sh
+
+# This uses the public/private keys on a PIV device, like a CAC or Yubikey.
+# It uses a PIN stored in a file.
+# It uses OpenSSL with PKCS11 enabled via OpenSC.
+# This stores the cluster encryption key encrypted with the PIV public
+# key in $DIR. This is technically a three-level encryption
+# architecture, with the third level requiring the PIV and PIN.
+# Do not create any fie with extension "wkey" in $DIR; these are
+# reserved for wrapped data key files.
+
+[ "$#" -ne 1 ] && echo "cluster_key_command usage: $0 \"%d\"" 1>&2 && exit 1
+# Supports environment variable PIV_PIN_FILE
+# No need for %R or -R since we are not prompting for a PIN
+
+DIR="$1"
+[ ! -e "$DIR" ] && echo "$DIR does not exist" 1>&2 && exit 1
+[ ! -d "$DIR" ] && echo "$DIR is not a directory" 1>&2 && exit 1
+
+# Set these here or pass in as environment variables.
+# File that stores the PIN to unlock the PIV
+#PIV_PIN_FILE=''
+# PIV slot 3 is the "Key Management" slot, so we use '0:3'
+PIV_SLOT='0:3'
+
+# File containing the cluster key encrypted with the PIV_SLOT's public key
+KEY_FILE="$DIR/pivpass.key"
+
+
+# ----------------------------------------------------------------------
+
+[ ! "$PIV_PIN_FILE" ] && echo 'PIV_PIN_FILE undefined' 1>&2 && exit 1
+[ ! -e "$PIV_PIN_FILE" ] && echo "$PIV_PIN_FILE does not exist" 1>&2 && exit 1
+[ -d "$PIV_PIN_FILE" ] && echo "$PIV_PIN_FILE is a directory" 1>&2 && exit 1
+
+[ ! "$KEY_FILE" ] && echo 'KEY_FILE undefined' 1>&2 && exit 1
+[ -d "$KEY_FILE" ] && echo "$KEY_FILE is a directory" 1>&2 && exit 1
+
+# Create a cluster key encrypted with the PIV_SLOT's public key?
+if [ ! -e "$KEY_FILE" ]
+then # The 'postgres' operating system user must have permission to
+ # access the PIV device.
+
+ openssl rand -hex 32 |
+ if ! openssl rsautl -engine pkcs11 -keyform engine -encrypt \
+ -inkey "$PIV_SLOT" -passin file:"$PIV_PIN_FILE" -out "$KEY_FILE"
+ then echo 'cluster key generation failed' 1>&2
+ exit 1
+ fi
+
+ # Warn the user to save the cluster key in a safe place
+ cat 1>&2 <<END
+
+WARNING: The PIV device can be locked and require a reset if too many PIN
+attempts fail. It is recommended to run this command manually and save
+the cluster key in a secure location for possible recovery.
+END
+
+fi
+
+# Decrypt the cluster key encrypted with the PIV_SLOT's public key
+if ! openssl rsautl -engine pkcs11 -keyform engine -decrypt \
+ -inkey "$PIV_SLOT" -passin file:"$PIV_PIN_FILE" -in "$KEY_FILE"
+then echo 'cluster key decryption failed' 1>&2
+ exit 1
+fi
+
+exit 0
diff --git a/src/backend/crypto/ckey_piv_pin.sh.sample b/src/backend/crypto/ckey_piv_pin.sh.sample
new file mode 100644
index 0000000000..e693ac31ba
--- /dev/null
+++ b/src/backend/crypto/ckey_piv_pin.sh.sample
@@ -0,0 +1,81 @@
+#!/bin/sh
+
+# This uses the public/private keys on a PIV device, like a CAC or Yubikey.
+# It requires a user-entered PIN.
+# It uses OpenSSL with PKCS11 enabled via OpenSC.
+# This stores the cluster encryption key encrypted with the PIV public
+# key in $DIR. This is technically a three-level encryption
+# architecture, with the third level requiring the PIV and PIN.
+# Do not create any fie with extension "wkey" in $DIR; these are
+# reserved for wrapped data key files.
+
+[ "$#" -lt 2 ] && echo "cluster_key_command usage: $0 \"%d\" %R [\"%p\"]" 1>&2 && exit 1
+# Supports environment variable PROMPT
+
+DIR="$1"
+[ ! -e "$DIR" ] && echo "$DIR does not exist" 1>&2 && exit 1
+[ ! -d "$DIR" ] && echo "$DIR is not a directory" 1>&2 && exit 1
+
+FD="$2"
+[ ! -t "$FD" ] && echo "file descriptor $FD does not refer to a terminal" 1>&2 && exit 1
+
+[ "$3" ] && PROMPT="$3"
+
+# PIV slot 3 is the "Key Management" slot, so we use '0:3'
+PIV_SLOT='0:3'
+
+# File containing the cluster key encrypted with the PIV_SLOT's public key
+KEY_FILE="$DIR/pivpass.key"
+
+
+# ----------------------------------------------------------------------
+
+[ ! "$PROMPT" ] && PROMPT='Enter PIV PIN: '
+
+stty -echo <&"$FD"
+
+# Create a cluster key encrypted with the PIV_SLOT's public key?
+if [ ! -e "$KEY_FILE" ]
+then echo 1>&"$FD"
+ echo -n "$PROMPT" 1>&"$FD"
+
+ # The 'postgres' operating system user must have permission to
+ # access the PIV device.
+
+ openssl rand -hex 32 |
+ # 'engine "pkcs11" set.' message confuses prompting
+ if ! openssl rsautl -engine pkcs11 -keyform engine -encrypt \
+ -inkey "$PIV_SLOT" -passin fd:"$FD" -out "$KEY_FILE" 2>&1
+ then stty echo <&"$FD"
+ echo 'cluster key generation failed' 1>&2
+ exit 1
+ fi | grep -v 'engine "pkcs11" set\.'
+
+ echo 1>&"$FD"
+
+ # Warn the user to save the cluster key in a safe place
+ cat 1>&"$FD" <<END
+
+WARNING: The PIV can be locked and require a reset if too many PIN
+attempts fail. It is recommended to run this command manually and save
+the cluster key in a secure location for possible recovery.
+END
+
+fi
+
+echo 1>&"$FD"
+echo -n "$PROMPT" 1>&"$FD"
+
+# Decrypt the cluster key encrypted with the PIV_SLOT's public key
+if ! openssl rsautl -engine pkcs11 -keyform engine -decrypt \
+ -inkey "$PIV_SLOT" -passin fd:"$FD" -in "$KEY_FILE" 2>&1
+then stty echo <&"$FD"
+ echo 'cluster key retrieval failed' 1>&2
+ exit 1
+fi | grep -v 'engine "pkcs11" set\.'
+
+echo 1>&"$FD"
+
+stty echo <&"$FD"
+
+exit 0
diff --git a/src/backend/crypto/ssl_passphrase.sh.sample b/src/backend/crypto/ssl_passphrase.sh.sample
new file mode 100644
index 0000000000..efbf5c0720
--- /dev/null
+++ b/src/backend/crypto/ssl_passphrase.sh.sample
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+# This uses a passphrase supplied by the user.
+# Do not create any fie with extension "wkey" in $DIR; these are
+# reserved for wrapped data key files.
+
+[ "$#" -lt 1 ] && echo "ssl_passphrase_command usage: $0 %R [\"%p\"]" 1>&2 && exit 1
+
+FD="$1"
+[ ! -t "$FD" ] && echo "file descriptor $FD does not refer to a terminal" 1>&2 && exit 1
+# Supports environment variable PROMPT
+
+[ "$2" ] && PROMPT="$2"
+
+
+# ----------------------------------------------------------------------
+
+[ ! "$PROMPT" ] && PROMPT='Enter cluster passphrase: '
+
+stty -echo <&"$FD"
+
+echo 1>&"$FD"
+echo -n "$PROMPT" 1>&"$FD"
+read PASS <&"$FD"
+
+stty echo <&"$FD"
+
+if [ ! "$PASS" ]
+then echo 'invalid: empty passphrase' 1>&2
+ exit 1
+fi
+
+echo "$PASS"
+
+exit 0
--
2.31.1
[application/octet-stream] v2-0004-cfe-04-common_over_cfe-03-scripts-squash-commit.patch (32.8K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/6-v2-0004-cfe-04-common_over_cfe-03-scripts-squash-commit.patch)
download | inline diff:
From 00a0b4698c469ef448710b8fd62fe76c38540d22 Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:49 -0400
Subject: [PATCH v2 04/12] cfe-04-common_over_cfe-03-scripts squash commit
---
src/common/Makefile | 3 +
src/common/cipher.c | 98 +++++++
src/common/cipher_openssl.c | 419 ++++++++++++++++++++++++++++
src/common/kmgr_utils.c | 465 ++++++++++++++++++++++++++++++++
src/include/common/cipher.h | 74 +++++
src/include/common/kmgr_utils.h | 94 +++++++
src/include/utils/wait_event.h | 3 +
src/tools/msvc/Mkvcbuild.pm | 4 +-
8 files changed, 1159 insertions(+), 1 deletion(-)
create mode 100644 src/common/cipher.c
create mode 100644 src/common/cipher_openssl.c
create mode 100644 src/common/kmgr_utils.c
create mode 100644 src/include/common/cipher.h
create mode 100644 src/include/common/kmgr_utils.h
diff --git a/src/common/Makefile b/src/common/Makefile
index e9af7346c9..9f63b01609 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -62,6 +62,7 @@ OBJS_COMMON = \
ip.o \
jsonapi.o \
keywords.o \
+ kmgr_utils.o \
kwlookup.o \
link-canary.o \
md5_common.o \
@@ -83,11 +84,13 @@ OBJS_COMMON = \
ifeq ($(with_ssl),openssl)
OBJS_COMMON += \
+ cipher_openssl.o \
protocol_openssl.o \
cryptohash_openssl.o \
hmac_openssl.o
else
OBJS_COMMON += \
+ cipher.o \
cryptohash.o \
hmac.o \
md5.o \
diff --git a/src/common/cipher.c b/src/common/cipher.c
new file mode 100644
index 0000000000..7aa3ea6138
--- /dev/null
+++ b/src/common/cipher.c
@@ -0,0 +1,98 @@
+/*-------------------------------------------------------------------------
+ *
+ * cipher.c
+ * Shared frontend/backend for cryptographic functions
+ *
+ * This is the set of in-core functions used when there are no other
+ * alternative options like OpenSSL.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/cipher.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/cipher.h"
+
+static void cipher_failure(void) pg_attribute_noreturn();
+
+
+PgCipherCtx *
+pg_cipher_ctx_create(int cipher, unsigned char *key, int klen, bool enc)
+{
+ cipher_failure();
+ return NULL; /* keep compiler quiet */
+}
+
+void
+pg_cipher_ctx_free(PgCipherCtx *ctx)
+{
+ cipher_failure();
+}
+
+int
+pg_cipher_blocksize(PgCipherCtx *ctx)
+{
+ cipher_failure();
+ return -1; /* keep compiler quiet */
+}
+
+bool
+pg_cipher_encrypt(PgCipherCtx *ctx, const int cipher,
+ const unsigned char *plaintext,
+ const int inlen, unsigned char *ciphertext, int *outlen,
+ const unsigned char *iv, const int ivlen,
+ unsigned char *outtag, const int taglen)
+{
+ cipher_failure();
+ return false; /* keep compiler quiet */
+}
+
+bool
+pg_cipher_decrypt(PgCipherCtx *ctx, const int cipher,
+ const unsigned char *ciphertext,
+ const int inlen, unsigned char *plaintext, int *outlen,
+ const unsigned char *iv, const int ivlen,
+ unsigned char *intag, const int taglen)
+{
+ cipher_failure();
+ return false; /* keep compiler quiet */
+}
+
+bool
+pg_cipher_keywrap(PgCipherCtx *ctx, const unsigned char *plaintext,
+ const int inlen, unsigned char *ciphertext, int *outlen)
+{
+ cipher_failure();
+ return false; /* keep compiler quiet */
+}
+
+bool
+pg_cipher_keyunwrap(PgCipherCtx *ctx, const unsigned char *ciphertext,
+ const int inlen, unsigned char *plaintext, int *outlen)
+{
+ cipher_failure();
+ return false; /* keep compiler quiet */
+}
+
+static void
+cipher_failure(void)
+{
+#ifndef FRONTEND
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ (errmsg("cluster file encryption is not supported because OpenSSL is not supported by this build"),
+ errhint("Compile with --with-openssl to use this feature."))));
+#else
+ fprintf(stderr, _("cluster file encryption is not supported because OpenSSL is not supported by this build"));
+ exit(1);
+#endif
+}
diff --git a/src/common/cipher_openssl.c b/src/common/cipher_openssl.c
new file mode 100644
index 0000000000..a03fbe435f
--- /dev/null
+++ b/src/common/cipher_openssl.c
@@ -0,0 +1,419 @@
+/*-------------------------------------------------------------------------
+ * cipher_openssl.c
+ * Cryptographic function using OpenSSL
+ *
+ * This contains the common low-level functions needed in both frontend and
+ * backend, for implement the database encryption.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/cipher_openssl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include "common/cipher.h"
+#include <openssl/conf.h>
+#include <openssl/err.h>
+#include <openssl/evp.h>
+#include <openssl/ssl.h>
+
+/*
+ * prototype for the EVP functions that return an algorithm, e.g.
+ * EVP_aes_128_gcm().
+ */
+typedef const EVP_CIPHER *(*ossl_EVP_cipher_func) (void);
+
+static ossl_EVP_cipher_func get_evp_aes_gcm(int klen);
+static EVP_CIPHER_CTX *ossl_cipher_ctx_create(int cipher, unsigned char *key,
+ int klen, bool enc);
+
+/*
+ * Return a newly created cipher context. 'cipher' specifies cipher algorithm
+ * by identifier like PG_CIPHER_XXX.
+ */
+PgCipherCtx *
+pg_cipher_ctx_create(int cipher, unsigned char *key, int klen, bool enc)
+{
+ PgCipherCtx *ctx = NULL;
+
+ if (cipher > PG_MAX_CIPHER_ID)
+ return NULL;
+
+ ctx = ossl_cipher_ctx_create(cipher, key, klen, enc);
+
+ return ctx;
+}
+
+void
+pg_cipher_ctx_free(PgCipherCtx *ctx)
+{
+ EVP_CIPHER_CTX_free(ctx);
+}
+
+int
+pg_cipher_blocksize(PgCipherCtx *ctx)
+{
+ Assert(ctx);
+
+ return EVP_CIPHER_CTX_block_size(ctx);
+}
+
+/*
+ * Encryption routine to encrypt data provided.
+ *
+ * ctx is the encryption context which must have been created previously.
+ *
+ * plaintext is the data we are going to encrypt
+ * inlen is the length of the data to encrypt
+ *
+ * ciphertext is the encrypted result
+ * outlen is the encrypted length
+ *
+ * iv is the IV to use.
+ * ivlen is the IV length to use.
+ *
+ * outtag is the resulting tag.
+ * taglen is the length of the tag.
+ */
+bool
+pg_cipher_encrypt(PgCipherCtx *ctx, int cipher,
+ const unsigned char *plaintext, const int inlen,
+ unsigned char *ciphertext, int *outlen,
+ const unsigned char *iv, const int ivlen,
+ unsigned char *outtag, const int taglen)
+{
+ int len;
+ int enclen;
+
+ Assert(ctx != NULL);
+
+ /*
+ * Here we are setting the IV for the context which was passed in. Note
+ * that we signal to OpenSSL that we are configuring a new value for the
+ * context by passing in 'NULL' for the 2nd ('type') parameter.
+ */
+
+ /*
+ * We don't use GCM mode, but it has a MAC, so we support it and test it
+ * in case we need it later. XXX is this correct for GCM and CTR?
+ */
+ /* Set the GCM IV length first */
+ if (cipher == PG_CIPHER_AES_GCM &&
+ !EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, ivlen, NULL))
+ return false;
+
+ /* Set the IV for this encryption. */
+ if (!EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv))
+ return false;
+
+ /*
+ * This is the function which is actually performing the encryption for
+ * us.
+ */
+ if (!EVP_EncryptUpdate(ctx, ciphertext, &len, plaintext, inlen))
+ return false;
+
+ enclen = len;
+
+ /* Finalize the encryption, which could add more to output. */
+ if (!EVP_EncryptFinal_ex(ctx, ciphertext + enclen, &len))
+ return false;
+
+ *outlen = enclen + len;
+
+ /*
+ * Once all of the encryption has been completed we grab the tag.
+ */
+ if (cipher == PG_CIPHER_AES_GCM &&
+ !EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, taglen, outtag))
+ return false;
+
+ return true;
+}
+
+/*
+ * Decryption routine
+ *
+ * ctx is the encryption context which must have been created previously.
+ *
+ * ciphertext is the data we are going to decrypt
+ * inlen is the length of the data to decrypt
+ *
+ * plaintext is the decrypted result
+ * outlen is the decrypted length
+ *
+ * iv is the IV to use.
+ * ivlen is the length of the IV.
+ *
+ * intag is the tag to use to verify.
+ * taglen is the length of the tag.
+ */
+bool
+pg_cipher_decrypt(PgCipherCtx *ctx, const int cipher,
+ const unsigned char *ciphertext, const int inlen,
+ unsigned char *plaintext, int *outlen,
+ const unsigned char *iv, const int ivlen,
+ unsigned char *intag, const int taglen)
+{
+ int declen;
+ int len;
+
+ /*
+ * Here we are setting the IV for the context which was passed in. Note
+ * that we signal to OpenSSL that we are configuring a new value for the
+ * context by passing in 'NULL' for the 2nd ('type') parameter.
+ */
+
+ /* XXX is this correct for GCM and CTR? */
+ /* Set the GCM IV length first */
+ if (cipher == PG_CIPHER_AES_GCM &&
+ !EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, ivlen, NULL))
+ return false;
+
+ /* Set the IV for this decryption. */
+ if (!EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv))
+ return false;
+
+ /*
+ * This is the function which is actually performing the decryption for
+ * us.
+ */
+ if (!EVP_DecryptUpdate(ctx, plaintext, &len, ciphertext, inlen))
+ return false;
+
+ declen = len;
+
+ /* Set the expected tag value. */
+ if (cipher == PG_CIPHER_AES_GCM &&
+ !EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_TAG, taglen, intag))
+ return false;
+
+ /*
+ * Finalize the decryption, which could add more to output, this is also
+ * the step which checks the tag and we MUST fail if this indicates an
+ * invalid tag!
+ */
+ if (!EVP_DecryptFinal_ex(ctx, plaintext + declen, &len))
+ return false;
+
+ *outlen = declen + len;
+
+ return true;
+}
+
+/*
+ * Routine to perform key wrapping for data provided.
+ *
+ * ctx is the encryption context which must have been created previously.
+ *
+ * plaintext is the data/key we are going to encrypt/wrap
+ * inlen is the length of the data
+ *
+ * ciphertext is the wrapped result
+ * outlen is the encrypted length (will be larger than input!)
+ */
+bool
+pg_cipher_keywrap(PgCipherCtx *ctx,
+ const unsigned char *plaintext, const int inlen,
+ unsigned char *ciphertext, int *outlen)
+{
+ int len;
+ int enclen;
+
+ Assert(ctx != NULL);
+
+ /*
+ * This is the function which is actually performing the encryption for
+ * us.
+ */
+ if (!EVP_EncryptUpdate(ctx, ciphertext, &len, plaintext, inlen))
+ return false;
+
+ enclen = len;
+
+ /* Finalize the encryption, which could add more to output. */
+ if (!EVP_EncryptFinal_ex(ctx, ciphertext + enclen, &len))
+ return false;
+
+ *outlen = enclen + len;
+
+ return true;
+}
+
+/*
+ * Routine to perform key unwrapping of the data provided.
+ *
+ * ctx is the encryption context which must have been created previously.
+ *
+ * ciphertext is the wrapped key we are going to unwrap
+ * inlen is the length of the data to decrypt/unwrap
+ *
+ * plaintext is the decrypted result
+ * outlen is the decrypted length (will be smaller than input!)
+ */
+bool
+pg_cipher_keyunwrap(PgCipherCtx *ctx,
+ const unsigned char *ciphertext, const int inlen,
+ unsigned char *plaintext, int *outlen)
+{
+ int declen;
+ int len;
+
+ /*
+ * This is the function which is actually performing the decryption for
+ * us.
+ */
+ if (!EVP_DecryptUpdate(ctx, plaintext, &len, ciphertext, inlen))
+ return false;
+
+ declen = len;
+
+ /*
+ * Finalize the decryption, which could add more to output, this is also
+ * the step which checks the tag and we MUST fail if this indicates an
+ * invalid result!
+ */
+ if (!EVP_DecryptFinal_ex(ctx, plaintext + declen, &len))
+ return false;
+
+ *outlen = declen + len;
+
+ return true;
+}
+
+/*
+ * Returns the correct GCM cipher functions for OpenSSL based
+ * on the key length requested.
+ */
+static ossl_EVP_cipher_func
+get_evp_aes_gcm(int klen)
+{
+ switch (klen)
+ {
+ case PG_AES128_KEY_LEN:
+ return EVP_aes_128_gcm;
+ case PG_AES192_KEY_LEN:
+ return EVP_aes_192_gcm;
+ case PG_AES256_KEY_LEN:
+ return EVP_aes_256_gcm;
+ default:
+ return NULL;
+ }
+}
+
+/*
+ * Returns the correct KWP cipher functions for OpenSSL based
+ * on the key length requested.
+ */
+static ossl_EVP_cipher_func
+get_evp_aes_kwp(int klen)
+{
+ switch (klen)
+ {
+ case PG_AES128_KEY_LEN:
+ return EVP_aes_128_wrap_pad;
+ case PG_AES192_KEY_LEN:
+ return EVP_aes_192_wrap_pad;
+ case PG_AES256_KEY_LEN:
+ return EVP_aes_256_wrap_pad;
+ default:
+ return NULL;
+ }
+}
+
+/*
+ * Returns the correct CTR cipher functions for OpenSSL based
+ * on the key length requested.
+ */
+static ossl_EVP_cipher_func
+get_evp_aes_ctr(int klen)
+{
+ switch (klen)
+ {
+ case PG_AES128_KEY_LEN:
+ return EVP_aes_128_ctr;
+ case PG_AES192_KEY_LEN:
+ return EVP_aes_192_ctr;
+ case PG_AES256_KEY_LEN:
+ return EVP_aes_256_ctr;
+ default:
+ return NULL;
+ }
+}
+
+/*
+ * Initialize and return an EVP_CIPHER_CTX. Returns NULL if the given
+ * cipher algorithm is not supported or on failure.
+ */
+static EVP_CIPHER_CTX *
+ossl_cipher_ctx_create(int cipher, unsigned char *key, int klen, bool enc)
+{
+ EVP_CIPHER_CTX *ctx;
+ ossl_EVP_cipher_func func;
+ int ret;
+
+ ctx = EVP_CIPHER_CTX_new();
+
+ /*
+ * We currently only support AES GCM but others could be added in the
+ * future.
+ */
+ switch (cipher)
+ {
+ case PG_CIPHER_AES_GCM:
+ func = get_evp_aes_gcm(klen);
+ if (!func)
+ goto failed;
+ break;
+ case PG_CIPHER_AES_KWP:
+ func = get_evp_aes_kwp(klen);
+
+ /*
+ * Since wrapping will produce more output then input, and we have
+ * to be ready for that, OpenSSL requires that we explicitly
+ * enable wrapping for the context.
+ */
+ EVP_CIPHER_CTX_set_flags(ctx, EVP_CIPHER_CTX_FLAG_WRAP_ALLOW);
+ if (!func)
+ goto failed;
+ break;
+ case PG_CIPHER_AES_CTR:
+ func = get_evp_aes_ctr(klen);
+ if (!func)
+ goto failed;
+ break;
+ default:
+ goto failed;
+ }
+
+ /*
+ * We create the context here based on the cipher requested and the
+ * provided key. Note that the IV will be provided in the actual
+ * encryption call through another EVP_EncryptInit_ex call- this is fine
+ * as long as 'type' is passed in as NULL!
+ */
+ if (enc)
+ ret = EVP_EncryptInit_ex(ctx, (const EVP_CIPHER *) func(), NULL, key, NULL);
+ else
+ ret = EVP_DecryptInit_ex(ctx, (const EVP_CIPHER *) func(), NULL, key, NULL);
+
+ if (!ret)
+ goto failed;
+
+ /* Set the key length based on the key length requested. */
+ if (!EVP_CIPHER_CTX_set_key_length(ctx, klen))
+ goto failed;
+
+ return ctx;
+
+failed:
+ EVP_CIPHER_CTX_free(ctx);
+ return NULL;
+}
diff --git a/src/common/kmgr_utils.c b/src/common/kmgr_utils.c
new file mode 100644
index 0000000000..6c50cb093b
--- /dev/null
+++ b/src/common/kmgr_utils.c
@@ -0,0 +1,465 @@
+/*-------------------------------------------------------------------------
+ *
+ * kmgr_utils.c
+ * Shared frontend/backend for cluster file encryption
+ *
+ * These functions handle reading the wrapped DEK files from the
+ * file system, and wrapping and unwrapping them. It also handles
+ * running the cluster_key_command.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/kmgr_utils.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include <unistd.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+
+#ifdef FRONTEND
+#include "common/logging.h"
+#endif
+#include "common/cryptohash.h"
+#include "common/file_perm.h"
+#include "common/hex.h"
+#include "common/string.h"
+#include "crypto/kmgr.h"
+#include "lib/stringinfo.h"
+#include "storage/fd.h"
+
+#ifndef FRONTEND
+#include "pgstat.h"
+#include "storage/fd.h"
+#endif
+
+#define KMGR_PROMPT_MSG "Enter authentication needed to generate the cluster key: "
+
+#ifdef FRONTEND
+static FILE *open_pipe_stream(const char *command);
+static int close_pipe_stream(FILE *file);
+#endif
+
+static void read_wrapped_data_key(const char *cryptoKeyDir, uint32 id, unsigned char **key_p, int *key_len);
+
+/*
+ * We need this array in frontend and backend code, so define it here.
+ * You can only add to the end of this array since the array index is
+ * stored in pg_control.
+ */
+encryption_method encryption_methods[NUM_ENCRYPTION_METHODS] = {
+ {"", 0},
+ {"AES128", 128},
+ {"AES192", 192},
+ {"AES256", 256}
+};
+
+/* This maps wrapped key filesnames to their array slots */
+char *wkey_filenames[KMGR_NUM_DATA_KEYS] = {
+ "relation",
+ "wal"
+};
+
+
+/*
+ * Wrap the given CryptoKey.
+ *
+ * Returns true and writes encrypted/wrapped/padded data to 'out', and the length
+ * of the result to outlen, if successful.
+ *
+ * Otherwise returns false. The caller must allocate sufficient space
+ * for cipher data calculated by using KmgrSizeOfCipherText(). Please note that
+ * this function modifies 'out' data even on failure.
+ */
+bool
+kmgr_wrap_data_key(PgCipherCtx *ctx, CryptoKey *in, unsigned char *out, int *outlen)
+{
+ Assert(ctx && in && out);
+
+ if (!pg_cipher_keywrap(ctx, (unsigned char *) in, sizeof(CryptoKey), out, outlen))
+ return false;
+
+ return true;
+}
+
+/*
+ * Decrypt the given data. Return true and set plain text data to `out` if
+ * successful. Otherwise return false. The caller must allocate sufficient
+ * space for cipher data calculated by using KmgrSizeOfPlainText(). Please
+ * note that this function modifies 'out' data even on failure.
+ */
+bool
+kmgr_unwrap_data_key(PgCipherCtx *ctx, unsigned char *in, int inlen, CryptoKey *out)
+{
+ int outlen;
+
+ Assert(ctx && in && out);
+
+ if (!pg_cipher_keyunwrap(ctx, in, inlen, (unsigned char *) out, &outlen))
+ return false;
+
+ Assert(outlen == sizeof(CryptoKey));
+
+ return true;
+}
+
+/*
+ * Verify the correctness of the given cluster key by unwrapping the given keys.
+ * If the given cluster key is correct we set unwrapped keys to out_keys and return
+ * true. Otherwise return false. Please note that this function changes the
+ * contents of out_keys even on failure. Both in_keys and out_keys must be the
+ * same length.
+ */
+bool
+kmgr_verify_cluster_key(unsigned char *cluster_key,
+ unsigned char **in_keys, int *key_lens, CryptoKey *out_keys)
+{
+ PgCipherCtx *ctx;
+
+ /* Create decryption context with the KEK. */
+ ctx = pg_cipher_ctx_create(PG_CIPHER_AES_KWP, cluster_key,
+ KMGR_CLUSTER_KEY_LEN, false);
+
+ /* unwrap each DEK */
+ for (int i = 0; i < KMGR_NUM_DATA_KEYS; i++)
+ {
+ if (!kmgr_unwrap_data_key(ctx, in_keys[i], key_lens[i], &(out_keys[i])))
+ {
+ /* The cluster key is not correct */
+ pg_cipher_ctx_free(ctx);
+ return false;
+ }
+ explicit_bzero(in_keys[i], key_lens[i]);
+ }
+
+ /* The cluster key is correct, free the cipher context */
+ pg_cipher_ctx_free(ctx);
+
+ return true;
+}
+
+/*
+ * Run cluster key command.
+ *
+ * Substitute %d for directory, %p for prompt, %R for file descriptor.
+ *
+ * The result will be put in buffer buf, which is of size "size".
+ * The return value is the length of the actual result.
+ */
+int
+kmgr_run_cluster_key_command(char *cluster_key_command, char *buf,
+ int size, char *dir, int terminal_fd)
+{
+ StringInfoData command;
+ const char *sp;
+ FILE *fh;
+ int pclose_rc;
+ size_t len = 0;
+
+ buf[0] = '\0';
+
+ Assert(size > 0);
+
+ /*
+ * Build the command to be executed.
+ */
+ initStringInfo(&command);
+
+ for (sp = cluster_key_command; *sp; sp++)
+ {
+ if (*sp == '%')
+ {
+ switch (sp[1])
+ {
+ /* directory */
+ case 'd':
+ {
+ char *nativePath;
+
+ sp++;
+
+ /*
+ * This needs to use a placeholder to not modify the
+ * input with the conversion done via
+ * make_native_path().
+ */
+ nativePath = pstrdup(dir);
+ make_native_path(nativePath);
+ appendStringInfoString(&command, nativePath);
+ pfree(nativePath);
+ break;
+ }
+ /* prompt string */
+ case 'p':
+ sp++;
+ appendStringInfoString(&command, KMGR_PROMPT_MSG);
+ break;
+ /* file descriptor number */
+ case 'R':
+ {
+ char fd_str[20];
+
+ if (terminal_fd == -1)
+ {
+#ifdef FRONTEND
+ pg_fatal("cluster key command referenced %%R, but --authprompt not specified");
+#else
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("cluster key command referenced %%R, but --authprompt not specified")));
+#endif
+ }
+
+ sp++;
+ snprintf(fd_str, sizeof(fd_str), "%d", terminal_fd);
+ appendStringInfoString(&command, fd_str);
+ break;
+ }
+ /* literal "%" */
+ case '%':
+ /* convert %% to a single % */
+ sp++;
+ appendStringInfoChar(&command, *sp);
+ break;
+ default:
+ /* otherwise treat the % as not special */
+ appendStringInfoChar(&command, *sp);
+ break;
+ }
+ }
+ else
+ {
+ appendStringInfoChar(&command, *sp);
+ }
+ }
+
+#ifdef FRONTEND
+ fh = open_pipe_stream(command.data);
+ if (fh == NULL)
+ {
+ pg_fatal("could not execute command \"%s\": %m",
+ command.data);
+ }
+#else
+ fh = OpenPipeStream(command.data, "r");
+ if (fh == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not execute command \"%s\": %m",
+ command.data)));
+#endif
+
+ if (!fgets(buf, size, fh))
+ {
+ if (ferror(fh))
+ {
+#ifdef FRONTEND
+ pg_fatal("could not read from command \"%s\": %m",
+ command.data);
+#else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+#endif
+ }
+ }
+
+#ifdef FRONTEND
+ pclose_rc = close_pipe_stream(fh);
+#else
+ pclose_rc = ClosePipeStream(fh);
+#endif
+
+ if (pclose_rc == -1)
+ {
+#ifdef FRONTEND
+ pg_fatal("could not close pipe to external command: %m");
+#else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to external command: %m")));
+#endif
+ }
+ else if (pclose_rc != 0)
+ {
+#ifdef FRONTEND
+ pg_fatal("command \"%s\" failed", command.data);
+#else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("command \"%s\" failed",
+ command.data),
+ errdetail_internal("%s", wait_result_to_str(pclose_rc))));
+#endif
+ }
+
+ /* strip trailing newline and carriage returns */
+ len = pg_strip_crlf(buf);
+
+ pfree(command.data);
+
+ return len;
+}
+
+#ifdef FRONTEND
+static FILE *
+open_pipe_stream(const char *command)
+{
+ FILE *res;
+
+#ifdef WIN32
+ size_t cmdlen = strlen(command);
+ char *buf;
+ int save_errno;
+
+ buf = malloc(cmdlen + 2 + 1);
+ if (buf == NULL)
+ {
+ errno = ENOMEM;
+ return NULL;
+ }
+ buf[0] = '"';
+ memcpy(&buf[1], command, cmdlen);
+ buf[cmdlen + 1] = '"';
+ buf[cmdlen + 2] = '\0';
+
+ res = _popen(buf, "r");
+
+ save_errno = errno;
+ free(buf);
+ errno = save_errno;
+#else
+ res = popen(command, "r");
+#endif /* WIN32 */
+ return res;
+}
+
+static int
+close_pipe_stream(FILE *file)
+{
+#ifdef WIN32
+ return _pclose(file);
+#else
+ return pclose(file);
+#endif /* WIN32 */
+}
+#endif /* FRONTEND */
+
+/*
+ * Reads the keys at path.
+ *
+ * This routine simply reads in the raw encrypted/wrapped keys;
+ * it does not handle any decryption, see kmgr_key_unwrap().
+ *
+ * For each key returned, the key and key length are returned
+ * in the keys and key_lens arrays respectfully.
+ *
+ * Note that keys and key_lens must be allocated before calling
+ * this function as arrays of at least KMGR_NUM_DATA_KEYS length.
+ */
+void
+kmgr_read_wrapped_data_keys(const char *path, unsigned char **keys, int *key_lens)
+{
+/* StaticAssertStmt(lengthof(wkey_filenames) == KMGR_NUM_DATA_KEYS,
+ "wkey_filenames[] must match KMGR_NUM_DATA_KEYS");
+*/
+ StaticAssertStmt(1 == 1,
+ "wkey_filenames[] must match KMGR_NUM_DATA_KEYS");
+
+ for (int id = 0; id < KMGR_NUM_DATA_KEYS; id++)
+ read_wrapped_data_key(path, id, &(keys[id]), &(key_lens[id]));
+
+ return;
+}
+
+/* Read a wrapped DEK file */
+static void
+read_wrapped_data_key(const char *cryptoKeyDir, uint32 id, unsigned char **key_p, int *key_len)
+{
+ char path[MAXPGPATH];
+ int fd;
+ int r;
+ struct stat st;
+
+ CryptoKeyFilePath(path, cryptoKeyDir, id);
+
+#ifndef FRONTEND
+ if ((fd = OpenTransientFile(path, O_RDONLY | PG_BINARY)) == -1)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\" for reading: %m",
+ path)));
+ else if (fstat(fd, &st))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not stat file \"%s\": %m",
+ path)));
+#else
+ if ((fd = open(path, O_RDONLY | PG_BINARY, 0)) == -1)
+ pg_fatal("could not open file \"%s\" for reading: %m",
+ path);
+ else if (fstat(fd, &st))
+ pg_fatal("could not stat file \"%s\": %m",
+ path);
+#endif
+
+ *key_len = st.st_size;
+
+#ifndef FRONTEND
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_READ);
+#endif
+
+ *key_p = (unsigned char *) palloc0(*key_len);
+
+ /* Get key bytes */
+ r = read(fd, *key_p, *key_len);
+ if (r != *key_len)
+ {
+ if (r < 0)
+ {
+#ifndef FRONTEND
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read file \"%s\": %m", path)));
+#else
+ pg_fatal("could not read file \"%s\": %m", path);
+#endif
+ }
+ else
+ {
+#ifndef FRONTEND
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("could not read file \"%s\": read %d of %u",
+ path, r, *key_len)));
+#else
+ pg_fatal("could not read file \"%s\": read %d of %u",
+ path, r, *key_len);
+#endif
+ }
+ }
+
+#ifndef FRONTEND
+ pgstat_report_wait_end();
+#endif
+
+#ifndef FRONTEND
+ if (CloseTransientFile(fd) != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m",
+ path)));
+#else
+ if (close(fd) != 0)
+ pg_fatal("could not close file \"%s\": %m", path);
+#endif
+}
diff --git a/src/include/common/cipher.h b/src/include/common/cipher.h
new file mode 100644
index 0000000000..da97ca3776
--- /dev/null
+++ b/src/include/common/cipher.h
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * cipher.h
+ * Declarations for cryptographic functions
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/common/cipher.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_CIPHER_H
+#define PG_CIPHER_H
+
+#ifdef USE_OPENSSL
+#include <openssl/evp.h>
+#include <openssl/conf.h>
+#include <openssl/err.h>
+#endif
+
+/*
+ * Supported symmetric encryption algorithm. These identifiers are passed
+ * to pg_cipher_ctx_create() function, and then actual encryption
+ * implementations need to initialize their context of the given encryption
+ * algorithm.
+ */
+#define PG_CIPHER_AES_GCM 0
+#define PG_CIPHER_AES_CTR 1
+#define PG_CIPHER_AES_KWP 2
+#define PG_MAX_CIPHER_ID 2
+
+/* AES128/192/256 various length definitions */
+#define PG_AES128_KEY_LEN (128 / 8)
+#define PG_AES192_KEY_LEN (192 / 8)
+#define PG_AES256_KEY_LEN (256 / 8)
+
+/*
+ * The encrypted data is a series of blocks of size. Initialization
+ * vector(IV) is the same size of cipher block.
+ */
+#define PG_AES_BLOCK_SIZE 16
+#define PG_AES_IV_SIZE (PG_AES_BLOCK_SIZE)
+
+#ifdef USE_OPENSSL
+typedef EVP_CIPHER_CTX PgCipherCtx;
+#else
+typedef void PgCipherCtx;
+#endif
+
+extern PgCipherCtx *pg_cipher_ctx_create(int cipher, unsigned char *key, int klen,
+ bool enc);
+
+extern void pg_cipher_ctx_free(PgCipherCtx *ctx);
+extern bool pg_cipher_encrypt(PgCipherCtx *ctx, int cipher,
+ const unsigned char *plaintext, const int inlen,
+ unsigned char *ciphertext, int *outlen,
+ const unsigned char *iv, const int ivlen,
+ unsigned char *tag, const int taglen);
+extern bool pg_cipher_decrypt(PgCipherCtx *ctx, const int cipher,
+ const unsigned char *ciphertext, const int inlen,
+ unsigned char *plaintext, int *outlen,
+ const unsigned char *iv, const int ivlen,
+ unsigned char *intag, const int taglen);
+
+extern bool pg_cipher_keywrap(PgCipherCtx *ctx,
+ const unsigned char *plaintext, const int inlen,
+ unsigned char *ciphertext, int *outlen);
+extern bool pg_cipher_keyunwrap(PgCipherCtx *ctx,
+ const unsigned char *ciphertext, const int inlen,
+ unsigned char *plaintext, int *outlen);
+
+extern int pg_cipher_blocksize(PgCipherCtx *ctx);
+
+#endif /* PG_CIPHER_H */
diff --git a/src/include/common/kmgr_utils.h b/src/include/common/kmgr_utils.h
new file mode 100644
index 0000000000..ca264db728
--- /dev/null
+++ b/src/include/common/kmgr_utils.h
@@ -0,0 +1,94 @@
+/*-------------------------------------------------------------------------
+ *
+ * kmgr_utils.h
+ * Declarations for utility function for file encryption key
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/common/kmgr_utils.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef KMGR_UTILS_H
+#define KMGR_UTILS_H
+
+#include "common/cipher.h"
+
+/* Current version number */
+#define KMGR_VERSION 1
+
+/*
+ * Directories where cluster file encryption keys reside within PGDATA.
+ */
+#define KMGR_DIR "pg_cryptokeys"
+#define KMGR_DIR_PID KMGR_DIR"/pg_alterckey.pid"
+#define LIVE_KMGR_DIR KMGR_DIR"/live"
+/* used during cluster key rotation */
+#define NEW_KMGR_DIR KMGR_DIR"/new"
+#define OLD_KMGR_DIR KMGR_DIR"/old"
+
+/* CryptoKey file name is keys id */
+#define CryptoKeyFilePath(path, dir, id) \
+ snprintf((path), MAXPGPATH, "%s/%s.wkey", (dir), (wkey_filenames[id]))
+
+/*
+ * Identifiers of internal keys.
+ */
+#define KMGR_KEY_ID_REL 0
+#define KMGR_KEY_ID_WAL 1
+#define KMGR_NUM_DATA_KEYS 2
+
+/* We always, today, use a 256-bit AES key. */
+#define KMGR_CLUSTER_KEY_LEN PG_AES256_KEY_LEN
+
+/* double for hex format, plus some for spaces, \r,\n, and null byte */
+#define ALLOC_KMGR_CLUSTER_KEY_LEN (KMGR_CLUSTER_KEY_LEN * 2 + 10 + 2 + 1)
+
+/* Maximum length of key the key manager can store */
+#define KMGR_MAX_KEY_LEN 256
+#define KMGR_MAX_KEY_LEN_BYTES KMGR_MAX_KEY_LEN / 8
+
+
+/*
+ * Cryptographic key data structure.
+ *
+ * This is the structure we use to write out the encrypted keys and
+ * which we use to store the keys in shared memory.
+ *
+ * Note that wrapping this structure results in an encrypted byte
+ * string which is what we actually write and then read back in.
+ *
+ * klen is the key length in bytes
+ * key is the encryption key of klen length
+ */
+typedef struct CryptoKey
+{
+ int klen; /* key length in bytes */
+ unsigned char key[KMGR_MAX_KEY_LEN_BYTES];
+} CryptoKey;
+
+/* Encryption method array */
+typedef struct encryption_method
+{
+ const char *name;
+ const int bit_length;
+} encryption_method;
+
+#define NUM_ENCRYPTION_METHODS 4
+#define DISABLED_ENCRYPTION_METHOD 0
+#define DEFAULT_ENABLED_ENCRYPTION_METHOD 1
+
+extern encryption_method encryption_methods[NUM_ENCRYPTION_METHODS];
+extern char *wkey_filenames[KMGR_NUM_DATA_KEYS];
+
+extern bool kmgr_wrap_data_key(PgCipherCtx *ctx, CryptoKey *in, unsigned char *out, int *outlen);
+extern bool kmgr_unwrap_data_key(PgCipherCtx *ctx, unsigned char *in, int inlen, CryptoKey *out);
+extern bool kmgr_verify_cluster_key(unsigned char *cluster_key,
+ unsigned char **in_keys, int *klens, CryptoKey *out_keys);
+extern int kmgr_run_cluster_key_command(char *cluster_key_command,
+ char *buf, int size, char *dir,
+ int terminal_fd);
+extern void kmgr_read_wrapped_data_keys(const char *path, unsigned char **keys,
+ int *key_lens);
+
+#endif /* KMGR_UTILS_H */
diff --git a/src/include/utils/wait_event.h b/src/include/utils/wait_event.h
index 6f2d5612e0..4b049f2ab4 100644
--- a/src/include/utils/wait_event.h
+++ b/src/include/utils/wait_event.h
@@ -180,6 +180,9 @@ typedef enum
WAIT_EVENT_DATA_FILE_WRITE,
WAIT_EVENT_DSM_ALLOCATE,
WAIT_EVENT_DSM_FILL_ZERO_WRITE,
+ WAIT_EVENT_KEY_FILE_READ,
+ WAIT_EVENT_KEY_FILE_WRITE,
+ WAIT_EVENT_KEY_FILE_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE,
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 83a3e40425..7c448183f4 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -136,19 +136,21 @@ sub mkvcbuild
archive.c base64.c checksum_helper.c compression.c
config_info.c controldata_utils.c d2s.c encnames.c exec.c
f2s.c file_perm.c file_utils.c hashfn.c ip.c jsonapi.c
- keywords.c kwlookup.c link-canary.c md5_common.c
+ keywords.c kmgr_utils.c kwlookup.c link-canary.c md5_common.c
pg_get_line.c pg_lzcompress.c pg_prng.c pgfnames.c psprintf.c relpath.c
rmtree.c saslprep.c scram-common.c string.c stringinfo.c unicode_norm.c
username.c wait_error.c wchar.c);
if ($solution->{options}->{openssl})
{
+ push(@pgcommonallfiles, 'cipher_openssl.c');
push(@pgcommonallfiles, 'cryptohash_openssl.c');
push(@pgcommonallfiles, 'hmac_openssl.c');
push(@pgcommonallfiles, 'protocol_openssl.c');
}
else
{
+ push(@pgcommonallfiles, 'cipher.c');
push(@pgcommonallfiles, 'cryptohash.c');
push(@pgcommonallfiles, 'hmac.c');
push(@pgcommonallfiles, 'md5.c');
--
2.31.1
[application/octet-stream] v2-0007-cfe-07-bin_over_cfe-06-backend-squash-commit.patch (33.3K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/7-v2-0007-cfe-07-bin_over_cfe-06-backend-squash-commit.patch)
download | inline diff:
From 00884d21abdf24cc344515521660fc4360e1bbde Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:56 -0400
Subject: [PATCH v2 07/12] cfe-07-bin_over_cfe-06-backend squash commit
---
doc/src/sgml/ref/initdb.sgml | 47 +++++++++
doc/src/sgml/ref/pg_ctl-ref.sgml | 14 +++
doc/src/sgml/ref/pgupgrade.sgml | 20 +++-
doc/src/sgml/ref/postgres-ref.sgml | 13 +++
src/bin/initdb/initdb.c | 125 ++++++++++++++++++++++--
src/bin/pg_controldata/pg_controldata.c | 3 +
src/bin/pg_ctl/pg_ctl.c | 59 +++++++++--
src/bin/pg_resetwal/pg_resetwal.c | 3 +
src/bin/pg_rewind/filemap.c | 8 ++
src/bin/pg_upgrade/check.c | 34 +++++++
src/bin/pg_upgrade/controldata.c | 41 +++++++-
src/bin/pg_upgrade/file.c | 2 +
src/bin/pg_upgrade/option.c | 7 +-
src/bin/pg_upgrade/pg_upgrade.h | 4 +
src/bin/pg_upgrade/server.c | 2 +-
15 files changed, 361 insertions(+), 21 deletions(-)
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
index 8158896298..06b10fbaaf 100644
--- a/doc/src/sgml/ref/initdb.sgml
+++ b/doc/src/sgml/ref/initdb.sgml
@@ -186,6 +186,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-cluster-key-command" xreflabel="cluster key command">
+ <term><option>-c <replaceable class="parameter">command</replaceable></option></term>
+ <term><option>--cluster-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ This option specifies an external command to obtain the cluster-level
+ key for cluster file encryption during server initialization and
+ server start; see <xref linkend="guc-cluster-key-command"/> for details.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
<term><option>--pgdata=<replaceable class="parameter">directory</replaceable></option></term>
@@ -257,6 +269,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-file-encryption-method"
+ xreflabel="file encryption">
+ <term><option>-K <replaceable class="parameter">encryption_method</replaceable></option></term>
+ <term><option>--file-encryption-method=<replaceable class="parameter">encryption_method</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the cluster file encryption method. The
+ value values are <literal>AES128</literal> (the default), <literal>AES192</literal>, and <literal>AES256</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--locale=<replaceable>locale</replaceable></option></term>
<listitem>
@@ -344,6 +368,17 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--authprompt</option></term>
+ <listitem>
+ <para>
+ Allows the <option>--cluster-key-command</option> command
+ to prompt for a passphrase or PIN.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-S</option></term>
<term><option>--sync-only</option></term>
@@ -369,6 +404,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-u <replaceable>datadir</replaceable></option></term>
+ <term><option>--copy-encryption-keys=<replaceable>datadir</replaceable></option></term>
+ <listitem>
+ <para>
+ Copies cluster file encryption keys from another cluster; required
+ when using <application>pg_upgrade</application> on a cluster
+ with cluster file encryption enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-U <replaceable class="parameter">username</replaceable></option></term>
<term><option>--username=<replaceable class="parameter">username</replaceable></option></term>
diff --git a/doc/src/sgml/ref/pg_ctl-ref.sgml b/doc/src/sgml/ref/pg_ctl-ref.sgml
index 46906966eb..e3705c724e 100644
--- a/doc/src/sgml/ref/pg_ctl-ref.sgml
+++ b/doc/src/sgml/ref/pg_ctl-ref.sgml
@@ -38,6 +38,7 @@ PostgreSQL documentation
<arg choice="opt"><option>-s</option></arg>
<arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg>
<arg choice="opt"><option>-p</option> <replaceable>path</replaceable></arg>
+ <arg choice="opt"><option>-R</option></arg>
<arg choice="opt"><option>-c</option></arg>
</cmdsynopsis>
@@ -72,6 +73,7 @@ PostgreSQL documentation
<arg choice="opt"><option>-t</option> <replaceable>seconds</replaceable></arg>
<arg choice="opt"><option>-s</option></arg>
<arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg>
+ <arg choice="opt"><option>-R</option></arg>
<arg choice="opt"><option>-c</option></arg>
</cmdsynopsis>
@@ -373,6 +375,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--authprompt</option></term>
+ <listitem>
+ <para>
+ Allows <option>ssl_passphrase_command</option> or
+ <option>cluster_key_command</option> to prompt for a passphrase
+ or PIN.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-s</option></term>
<term><option>--silent</option></term>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 8f7a3025c3..4362e551c9 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -182,6 +182,15 @@ PostgreSQL documentation
</para></listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--authprompt</option></term>
+ <listitem><para>allows <option>ssl_passphrase_command</option> or
+ <option>cluster_key_command</option> to prompt for a passphrase
+ or PIN.
+ </para></listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-s</option> <replaceable>dir</replaceable></term>
<term><option>--socketdir=</option><replaceable>dir</replaceable></term>
@@ -309,7 +318,9 @@ make prefix=/usr/local/pgsql.new install
Again, use compatible <command>initdb</command>
flags that match the old cluster. Many
prebuilt installers do this step automatically. There is no need to
- start the new cluster.
+ start the new cluster. If upgrading a cluster that uses
+ cluster file encryption, the <command>initdb</command> option
+ <option>--copy-encryption-keys</option> must be specified.
</para>
</step>
@@ -842,6 +853,13 @@ psql --username=postgres --file=script.sql postgres
is down.
</para>
+ <para>
+ If the old cluster uses file encryption, the new cluster must use
+ the same keys, so <command>pg_upgrade</command> copies them to the
+ new cluster. It is necessary to initialize the new cluster with
+ the same <varname>cluster_key_command</varname> and the same
+ file encryption method.
+ </para>
</refsect1>
<refsect1>
diff --git a/doc/src/sgml/ref/postgres-ref.sgml b/doc/src/sgml/ref/postgres-ref.sgml
index 55a3f6c69d..f0fa52ebdc 100644
--- a/doc/src/sgml/ref/postgres-ref.sgml
+++ b/doc/src/sgml/ref/postgres-ref.sgml
@@ -305,6 +305,19 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-R <replaceable class="parameter">file-descriptor</replaceable></option></term>
+ <listitem>
+ <para>
+ Makes <command>postgres</command> prompt for a passphrase or PIN
+ from the specified open numeric file descriptor. The descriptor
+ is closed after the key is read. The file descriptor number
+ <literal>-1</literal> duplicates standard error for the terminal;
+ this is useful for single-user mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-s</option></term>
<listitem>
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index f61a043055..8c7e1e39c5 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -68,6 +68,7 @@
#include "catalog/pg_database_d.h" /* pgrminclude ignore */
#include "common/file_perm.h"
#include "common/file_utils.h"
+#include "common/kmgr_utils.h"
#include "common/logging.h"
#include "common/pg_prng.h"
#include "common/restricted_token.h"
@@ -147,11 +148,16 @@ static bool noclean = false;
static bool noinstructions = false;
static bool do_sync = true;
static bool sync_only = false;
+static bool pass_terminal_fd = false;
+static char *term_fd_opt = NULL;
+static int file_encryption_method = DISABLED_ENCRYPTION_METHOD;
static bool show_setting = false;
static bool data_checksums = false;
static char *xlog_dir = NULL;
static char *str_wal_segment_size_mb = NULL;
static int wal_segment_size_mb;
+static char *cluster_key_cmd = NULL;
+static char *old_key_datadir = NULL;
/* internal vars */
@@ -214,6 +220,7 @@ static const char *const subdirs[] = {
"global",
"pg_wal/archive_status",
"pg_commit_ts",
+ "pg_cryptokeys",
"pg_dynshmem",
"pg_notify",
"pg_serial",
@@ -904,12 +911,13 @@ test_config_settings(void)
test_buffs = MIN_BUFS_FOR_CONNS(test_conns);
snprintf(cmd, sizeof(cmd),
- "\"%s\" --check %s %s "
+ "\"%s\" --check -x0 %s %s %s "
"-c max_connections=%d "
"-c shared_buffers=%d "
"-c dynamic_shared_memory_type=%s "
"< \"%s\" > \"%s\" 2>&1",
backend_exec, boot_options, extra_options,
+ term_fd_opt ? term_fd_opt : "",
test_conns, test_buffs,
dynamic_shared_memory_type,
DEVNULL, DEVNULL);
@@ -941,12 +949,13 @@ test_config_settings(void)
}
snprintf(cmd, sizeof(cmd),
- "\"%s\" --check %s %s "
+ "\"%s\" --check %s %s %s "
"-c max_connections=%d "
"-c shared_buffers=%d "
"-c dynamic_shared_memory_type=%s "
"< \"%s\" > \"%s\" 2>&1",
backend_exec, boot_options, extra_options,
+ term_fd_opt ? term_fd_opt : "",
n_connections, test_buffs,
dynamic_shared_memory_type,
DEVNULL, DEVNULL);
@@ -1133,6 +1142,13 @@ setup_config(void)
"password_encryption = md5");
}
+ if (cluster_key_cmd)
+ {
+ snprintf(repltok, sizeof(repltok), "cluster_key_command = '%s'",
+ escape_quotes(cluster_key_cmd));
+ conflines = replace_token(conflines, "#cluster_key_command = ''", repltok);
+ }
+
/*
* If group access has been enabled for the cluster then it makes sense to
* ensure that the log files also allow group access. Otherwise a backup
@@ -1321,11 +1337,17 @@ bootstrap_template1(void)
unsetenv("PGCLIENTENCODING");
snprintf(cmd, sizeof(cmd),
- "\"%s\" --boot -X %d %s %s %s %s",
+ "\"%s\" --boot -X %d %s %s %s %s %s %s %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
- boot_options, extra_options,
+ cluster_key_cmd ? "-K" : "",
+ cluster_key_cmd ? encryption_methods[file_encryption_method].name : "",
+ old_key_datadir ? "-u" : "",
+ old_key_datadir ? old_key_datadir : "",
+ boot_options,
+ extra_options,
+ term_fd_opt ? term_fd_opt : "",
debug ? "-d 5" : "");
@@ -2159,18 +2181,26 @@ usage(const char *progname)
printf(_(" -T, --text-search-config=CFG\n"
" default text search configuration\n"));
printf(_(" -U, --username=NAME database superuser name\n"));
- printf(_(" -W, --pwprompt prompt for a password for the new superuser\n"));
+ printf(_(" -W, --pwprompt prompt for the new superuser password\n"));
printf(_(" -X, --waldir=WALDIR location for the write-ahead log directory\n"));
printf(_(" --wal-segsize=SIZE size of WAL segments, in megabytes\n"));
printf(_("\nLess commonly used options:\n"));
+ printf(_(" -c, --cluster-key-command=COMMAND\n"
+ " enable cluster file encryption and set command\n"
+ " to obtain the cluster key\n"));
printf(_(" -d, --debug generate lots of debugging output\n"));
printf(_(" --discard-caches set debug_discard_caches=1\n"));
+ printf(_(" -K, --file-encryption-method=METHOD\n"
+ " cluster file encryption method\n"));
printf(_(" -L DIRECTORY where to find the input files\n"));
printf(_(" -n, --no-clean do not clean up after errors\n"));
printf(_(" -N, --no-sync do not wait for changes to be written safely to disk\n"));
+ printf(_(" -R, --authprompt prompt for a passphrase or PIN\n"));
printf(_(" --no-instructions do not print instructions for next steps\n"));
printf(_(" -s, --show show internal settings\n"));
printf(_(" -S, --sync-only only sync database files to disk, then exit\n"));
+ printf(_(" -u, --copy-encryption-keys=DATADIR\n"
+ " copy the file encryption key from another cluster\n"));
printf(_("\nOther options:\n"));
printf(_(" -V, --version output version information, then exit\n"));
printf(_(" -?, --help show this help, then exit\n"));
@@ -2700,6 +2730,23 @@ initialize_data_directory(void)
/* Top level PG_VERSION is checked by bootstrapper, so make it first */
write_version_file(NULL);
+ if (pass_terminal_fd)
+ {
+#ifndef WIN32
+ int terminal_fd = open("/dev/tty", O_RDWR, 0);
+#else
+ int terminal_fd = open("CONOUT$", O_RDWR, 0);
+#endif
+
+ if (terminal_fd < 0)
+ {
+ pg_log_error(_("%s: could not open terminal: %s\n"),
+ progname, strerror(errno));
+ exit(1);
+ }
+ term_fd_opt = psprintf("-R %d", terminal_fd);
+ }
+
/* Select suitable configuration settings */
set_null_conf();
test_config_settings();
@@ -2723,8 +2770,9 @@ initialize_data_directory(void)
fflush(stdout);
snprintf(cmd, sizeof(cmd),
- "\"%s\" %s %s template1 >%s",
+ "\"%s\" %s %s %s template1 >%s",
backend_exec, backend_options, extra_options,
+ term_fd_opt ? term_fd_opt : "",
DEVNULL);
PG_CMD_OPEN;
@@ -2802,10 +2850,14 @@ main(int argc, char *argv[])
{"waldir", required_argument, NULL, 'X'},
{"wal-segsize", required_argument, NULL, 12},
{"data-checksums", no_argument, NULL, 'k'},
+ {"authprompt", no_argument, NULL, 'R'},
+ {"file-encryption-method", required_argument, NULL, 'K'},
{"allow-group-access", no_argument, NULL, 'g'},
{"discard-caches", no_argument, NULL, 14},
{"locale-provider", required_argument, NULL, 15},
{"icu-locale", required_argument, NULL, 16},
+ {"cluster-key-command", required_argument, NULL, 'c'},
+ {"copy-encryption-keys", required_argument, NULL, 'u'},
{NULL, 0, NULL, 0}
};
@@ -2847,7 +2899,7 @@ main(int argc, char *argv[])
/* process command-line options */
- while ((c = getopt_long(argc, argv, "A:dD:E:gkL:nNsST:U:WX:", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "A:c:dD:E:gkK:L:nNRsST:u:U:WX:", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -2893,6 +2945,28 @@ main(int argc, char *argv[])
case 'N':
do_sync = false;
break;
+ case 'R':
+ pass_terminal_fd = true;
+ break;
+ case 'K':
+ {
+ int i;
+
+ /* method 0/disabled cannot be specified */
+ for (i = DISABLED_ENCRYPTION_METHOD + 1;
+ i < NUM_ENCRYPTION_METHODS; i++)
+ if (pg_strcasecmp(optarg, encryption_methods[i].name) == 0)
+ {
+ file_encryption_method = i;
+ break;
+ }
+ if (i == NUM_ENCRYPTION_METHODS)
+ {
+ fprintf(stderr, _("invalid cluster encryption method\n"));
+ exit(1);
+ }
+ }
+ break;
case 'S':
sync_only = true;
break;
@@ -2929,6 +3003,12 @@ main(int argc, char *argv[])
case 9:
pwfilename = pg_strdup(optarg);
break;
+ case 'c':
+ cluster_key_cmd = pg_strdup(optarg);
+ break;
+ case 'u':
+ old_key_datadir = pg_strdup(optarg);
+ break;
case 's':
show_setting = true;
break;
@@ -3014,6 +3094,32 @@ main(int argc, char *argv[])
if (pwprompt && pwfilename)
pg_fatal("password prompt and password file cannot be specified together");
+#ifndef USE_OPENSSL
+ if (cluster_key_cmd)
+ {
+ pg_log_error("cluster file encryption is not supported because OpenSSL is not supported by this build");
+ exit(1);
+ }
+#endif
+
+ if (old_key_datadir != NULL && cluster_key_cmd == NULL)
+ {
+ pg_log_error("copying encryption keys requires the cluster key command to be specified");
+ exit(1);
+ }
+
+ if (file_encryption_method != DISABLED_ENCRYPTION_METHOD &&
+ cluster_key_cmd == NULL)
+ {
+ pg_log_error("a file encryption method requires the cluster key command to be specified");
+ exit(1);
+ }
+
+ /* set the default */
+ if (file_encryption_method == DISABLED_ENCRYPTION_METHOD &&
+ cluster_key_cmd != NULL)
+ file_encryption_method = DEFAULT_ENABLED_ENCRYPTION_METHOD;
+
check_authmethod_unspecified(&authmethodlocal);
check_authmethod_unspecified(&authmethodhost);
@@ -3072,6 +3178,11 @@ main(int argc, char *argv[])
else
printf(_("Data page checksums are disabled.\n"));
+ if (cluster_key_cmd)
+ printf(_("Cluster file encryption is enabled.\n"));
+ else
+ printf(_("Cluster file encryption is disabled.\n"));
+
if (pwprompt || pwfilename)
get_su_pwd();
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index c390ec51ce..d25d13c992 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -25,6 +25,7 @@
#include "access/xlog_internal.h"
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
+#include "common/kmgr_utils.h"
#include "common/logging.h"
#include "getopt_long.h"
#include "pg_getopt.h"
@@ -328,5 +329,7 @@ main(int argc, char *argv[])
ControlFile->data_checksum_version);
printf(_("Mock authentication nonce: %s\n"),
mock_auth_nonce_str);
+ printf(_("File encryption method: %s\n"),
+ encryption_methods[ControlFile->file_encryption_method].name);
return 0;
}
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index a509fc9109..d90c07db10 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -74,6 +74,7 @@ typedef enum
static bool do_wait = true;
static int wait_seconds = DEFAULT_WAIT;
static bool wait_seconds_arg = false;
+static bool pass_terminal_fd = false;
static bool silent_mode = false;
static ShutdownMode shutdown_mode = FAST_MODE;
static int sig = SIGINT; /* default */
@@ -439,7 +440,7 @@ free_readfile(char **optlines)
static pid_t
start_postmaster(void)
{
- char *cmd;
+ char *cmd, *term_fd_opt = NULL;
#ifndef WIN32
pid_t pm_pid;
@@ -467,6 +468,19 @@ start_postmaster(void)
/* fork succeeded, in child */
+ if (pass_terminal_fd)
+ {
+ int terminal_fd = open("/dev/tty", O_RDWR, 0);
+
+ if (terminal_fd < 0)
+ {
+ write_stderr(_("%s: could not open terminal: %s\n"),
+ progname, strerror(errno));
+ exit(1);
+ }
+ term_fd_opt = psprintf(" -R %d", terminal_fd);
+ }
+
/*
* If possible, detach the postmaster process from the launching process
* group and make it a group leader, so that it doesn't get signaled along
@@ -487,12 +501,14 @@ start_postmaster(void)
* has the same PID as the current child process.
*/
if (log_file != NULL)
- cmd = psprintf("exec \"%s\" %s%s < \"%s\" >> \"%s\" 2>&1",
+ cmd = psprintf("exec \"%s\" %s%s%s < \"%s\" >> \"%s\" 2>&1",
exec_path, pgdata_opt, post_opts,
+ term_fd_opt ? term_fd_opt : "",
DEVNULL, log_file);
else
- cmd = psprintf("exec \"%s\" %s%s < \"%s\" 2>&1",
- exec_path, pgdata_opt, post_opts, DEVNULL);
+ cmd = psprintf("exec \"%s\" %s%s%s < \"%s\" 2>&1",
+ exec_path, pgdata_opt, post_opts,
+ term_fd_opt ? term_fd_opt : "", DEVNULL);
(void) execl("/bin/sh", "/bin/sh", "-c", cmd, (char *) NULL);
@@ -513,6 +529,21 @@ start_postmaster(void)
PROCESS_INFORMATION pi;
const char *comspec;
+ if (pass_terminal_fd)
+ {
+ /* Hopefully we can read and write CONOUT, see simple_prompt() XXX */
+ /* Do CreateRestrictedProcess() children even inherit open file descriptors? XXX */
+ int terminal_fd = open("CONOUT$", O_RDWR, 0);
+
+ if (terminal_fd < 0)
+ {
+ write_stderr(_("%s: could not open terminal: %s\n"),
+ progname, strerror(errno));
+ exit(1);
+ }
+ term_fd_opt = psprintf(" -R %d", terminal_fd);
+ }
+
/* Find CMD.EXE location using COMSPEC, if it's set */
comspec = getenv("COMSPEC");
if (comspec == NULL)
@@ -553,12 +584,14 @@ start_postmaster(void)
else
close(fd);
- cmd = psprintf("\"%s\" /C \"\"%s\" %s%s < \"%s\" >> \"%s\" 2>&1\"",
- comspec, exec_path, pgdata_opt, post_opts, DEVNULL, log_file);
+ cmd = psprintf("\"%s\" /C \"\"%s\" %s%s%s < \"%s\" >> \"%s\" 2>&1\"",
+ comspec, exec_path, pgdata_opt, post_opts,
+ term_fd_opt ? term_fd_opt : "", DEVNULL, log_file);
}
else
- cmd = psprintf("\"%s\" /C \"\"%s\" %s%s < \"%s\" 2>&1\"",
- comspec, exec_path, pgdata_opt, post_opts, DEVNULL);
+ cmd = psprintf("\"%s\" /C \"\"%s\" %s%s%s < \"%s\" 2>&1\"",
+ comspec, exec_path, pgdata_opt, post_opts,
+ term_fd_opt ? term_fd_opt : "", DEVNULL);
if (!CreateRestrictedProcess(cmd, &pi, false))
{
@@ -689,7 +722,8 @@ wait_for_postmaster_start(pid_t pm_pid, bool do_checkpoint)
}
else
#endif
- print_msg(".");
+ if (!pass_terminal_fd)
+ print_msg(".");
}
pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
@@ -2005,6 +2039,7 @@ do_help(void)
printf(_(" -o, --options=OPTIONS command line options to pass to postgres\n"
" (PostgreSQL server executable) or initdb\n"));
printf(_(" -p PATH-TO-POSTGRES normally not necessary\n"));
+ printf(_(" -R, --authprompt prompt for a passphrase or PIN\n"));
printf(_("\nOptions for stop or restart:\n"));
printf(_(" -m, --mode=MODE MODE can be \"smart\", \"fast\", or \"immediate\"\n"));
@@ -2200,6 +2235,7 @@ main(int argc, char **argv)
{"mode", required_argument, NULL, 'm'},
{"pgdata", required_argument, NULL, 'D'},
{"options", required_argument, NULL, 'o'},
+ {"authprompt", no_argument, NULL, 'R'},
{"silent", no_argument, NULL, 's'},
{"timeout", required_argument, NULL, 't'},
{"core-files", no_argument, NULL, 'c'},
@@ -2272,7 +2308,7 @@ main(int argc, char **argv)
/* process command-line options */
while (optind < argc)
{
- while ((c = getopt_long(argc, argv, "cD:e:l:m:N:o:p:P:sS:t:U:wW",
+ while ((c = getopt_long(argc, argv, "cD:e:l:m:N:o:p:P:RsS:t:U:wW",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2324,6 +2360,9 @@ main(int argc, char **argv)
case 'P':
register_password = pg_strdup(optarg);
break;
+ case 'R':
+ pass_terminal_fd = true;
+ break;
case 's':
silent_mode = true;
break;
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
index 089063f471..73cb048b3e 100644
--- a/src/bin/pg_resetwal/pg_resetwal.c
+++ b/src/bin/pg_resetwal/pg_resetwal.c
@@ -52,6 +52,7 @@
#include "common/controldata_utils.h"
#include "common/fe_memutils.h"
#include "common/file_perm.h"
+#include "common/kmgr_utils.h"
#include "common/logging.h"
#include "common/restricted_token.h"
#include "common/string.h"
@@ -772,6 +773,8 @@ PrintControlValues(bool guessed)
(ControlFile.float8ByVal ? _("by value") : _("by reference")));
printf(_("Data page checksum version: %u\n"),
ControlFile.data_checksum_version);
+ printf(_("File encryption method: %s\n"),
+ encryption_methods[ControlFile.file_encryption_method].name);
}
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
index 269ed6446e..d0b71f972a 100644
--- a/src/bin/pg_rewind/filemap.c
+++ b/src/bin/pg_rewind/filemap.c
@@ -28,6 +28,7 @@
#include "catalog/pg_tablespace_d.h"
#include "common/hashfn.h"
+#include "common/kmgr_utils.h"
#include "common/string.h"
#include "datapagemap.h"
#include "filemap.h"
@@ -106,6 +107,13 @@ static const char *excludeDirContents[] =
/* Contents removed on startup, see AsyncShmemInit(). */
"pg_notify",
+ /*
+ * Skip cryptographic keys. It's generally not a good idea to copy the
+ * cryptographic keys from source database because these might use
+ * different cluster key.
+ */
+ KMGR_DIR,
+
/*
* Old contents are loaded for possible debugging but are not required for
* normal operation, see SerialInit().
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index f1bc1e6886..ae7aa601a7 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -11,6 +11,7 @@
#include "catalog/pg_authid_d.h"
#include "catalog/pg_collation.h"
+#include "common/kmgr_utils.h"
#include "fe_utils/string_utils.h"
#include "mb/pg_wchar.h"
#include "pg_upgrade.h"
@@ -30,6 +31,7 @@ static void check_for_composite_data_type_usage(ClusterInfo *cluster);
static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
+static void check_for_cluster_key_failure(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -160,6 +162,9 @@ check_and_dump_old_cluster(bool live_check)
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 905)
check_for_pg_role_prefix(&old_cluster);
+ if (GET_MAJOR_VERSION(old_cluster.major_version) >= 1400)
+ check_for_cluster_key_failure(&old_cluster);
+
if (GET_MAJOR_VERSION(old_cluster.major_version) == 904 &&
old_cluster.controldata.cat_ver < JSONB_FORMAT_CHANGE_CAT_VER)
check_for_jsonb_9_4_usage(&old_cluster);
@@ -190,6 +195,9 @@ check_new_cluster(void)
check_loadable_libraries();
+ if (GET_MAJOR_VERSION(old_cluster.major_version) >= 1400)
+ check_for_cluster_key_failure(&new_cluster);
+
switch (user_opts.transfer_mode)
{
case TRANSFER_MODE_CLONE:
@@ -1466,6 +1474,32 @@ check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
}
+/*
+ * check_for_cluster_key_failure()
+ *
+ * Make sure there was no unrepaired pg_alterckey failure
+ */
+static void
+check_for_cluster_key_failure(ClusterInfo *cluster)
+{
+ struct stat buffer;
+
+ if (stat (KMGR_DIR_PID, &buffer) == 0)
+ {
+ if (cluster == &old_cluster)
+ pg_fatal("The source cluster had a pg_alterckey failure that needs repair or\n"
+ "pg_alterckey is running. Run pg_alterckey --repair or wait for it\n"
+ "to complete.\n");
+ else
+ pg_fatal("The target cluster had a pg_alterckey failure that needs repair or\n"
+ "pg_alterckey is running. Run pg_alterckey --repair or wait for it\n"
+ "to complete.\n");
+ }
+
+ check_ok();
+}
+
+
/*
* get_canonical_locale_name
*
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
index 018cd310f7..ee6a48438b 100644
--- a/src/bin/pg_upgrade/controldata.c
+++ b/src/bin/pg_upgrade/controldata.c
@@ -9,12 +9,18 @@
#include "postgres_fe.h"
+#include <dirent.h>
#include <ctype.h>
#include "pg_upgrade.h"
#include "common/string.h"
+#include "access/xlog_internal.h"
+#include "common/controldata_utils.h"
+#include "common/file_utils.h"
+#include "common/kmgr_utils.h"
+
/*
* get_control_data()
*
@@ -62,6 +68,7 @@ get_control_data(ClusterInfo *cluster, bool live_check)
bool got_date_is_int = false;
bool got_data_checksum_version = false;
bool got_cluster_state = false;
+ int got_file_encryption_method = false;
char *lc_collate = NULL;
char *lc_ctype = NULL;
char *lc_monetary = NULL;
@@ -203,6 +210,13 @@ get_control_data(ClusterInfo *cluster, bool live_check)
got_data_checksum_version = true;
}
+ /* Only in <= 14 */
+ if (GET_MAJOR_VERSION(cluster->major_version) <= 1400)
+ {
+ cluster->controldata.file_encryption_method = DISABLED_ENCRYPTION_METHOD;
+ got_file_encryption_method = true;
+ }
+
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
@@ -498,6 +512,18 @@ get_control_data(ClusterInfo *cluster, bool live_check)
cluster->controldata.data_checksum_version = str2uint(p);
got_data_checksum_version = true;
}
+ else if ((p = strstr(bufin, "Cluster file encryption method:")) != NULL)
+ {
+ p = strchr(p, ':');
+
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ p++; /* remove ':' char */
+ /* used later for contrib check */
+ cluster->controldata.file_encryption_method = atoi(p);
+ got_file_encryption_method = true;
+ }
}
pclose(output);
@@ -566,7 +592,8 @@ get_control_data(ClusterInfo *cluster, bool live_check)
!got_index || !got_toast ||
(!got_large_object &&
cluster->controldata.ctrl_ver >= LARGE_OBJECT_SIZE_PG_CONTROL_VER) ||
- !got_date_is_int || !got_data_checksum_version)
+ !got_date_is_int || !got_data_checksum_version ||
+ !got_file_encryption_method)
{
if (cluster == &old_cluster)
pg_log(PG_REPORT,
@@ -635,6 +662,10 @@ get_control_data(ClusterInfo *cluster, bool live_check)
if (!got_data_checksum_version)
pg_log(PG_REPORT, " data checksum version");
+ /* value added in Postgres 14 */
+ if (!got_file_encryption_method)
+ pg_log(PG_REPORT, " file encryption method\n");
+
pg_fatal("Cannot continue without required control information, terminating");
}
}
@@ -699,6 +730,14 @@ check_control_data(ControlData *oldctrl,
pg_fatal("old cluster uses data checksums but the new one does not");
else if (oldctrl->data_checksum_version != newctrl->data_checksum_version)
pg_fatal("old and new cluster pg_controldata checksum versions do not match");
+
+ /*
+ * We cannot upgrade if the old cluster file encryption method
+ * doesn't match the new one.
+ */
+ if (oldctrl->file_encryption_method != newctrl->file_encryption_method)
+ pg_fatal("old and new clusters use different file encryption methods or\n"
+ "one cluster uses encryption and the other does not");
}
diff --git a/src/bin/pg_upgrade/file.c b/src/bin/pg_upgrade/file.c
index 079fbda838..29fd003318 100644
--- a/src/bin/pg_upgrade/file.c
+++ b/src/bin/pg_upgrade/file.c
@@ -11,6 +11,7 @@
#include <sys/stat.h>
#include <fcntl.h>
+#include <dirent.h>
#ifdef HAVE_COPYFILE_H
#include <copyfile.h>
#endif
@@ -21,6 +22,7 @@
#include "access/visibilitymapdefs.h"
#include "common/file_perm.h"
+#include "common/file_utils.h"
#include "pg_upgrade.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 3f719f1762..b6285d68ed 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -52,6 +52,7 @@ parseCommandLine(int argc, char *argv[])
{"check", no_argument, NULL, 'c'},
{"link", no_argument, NULL, 'k'},
{"retain", no_argument, NULL, 'r'},
+ {"authprompt", no_argument, NULL, 'R'},
{"jobs", required_argument, NULL, 'j'},
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
@@ -99,7 +100,7 @@ parseCommandLine(int argc, char *argv[])
if (os_user_effective_id == 0)
pg_fatal("%s: cannot be run as root", os_info.progname);
- while ((option = getopt_long(argc, argv, "d:D:b:B:cj:kNo:O:p:P:rs:U:v",
+ while ((option = getopt_long(argc, argv, "d:D:b:B:cj:kNo:O:p:P:rRs:U:v",
long_options, &optindex)) != -1)
{
switch (option)
@@ -176,6 +177,10 @@ parseCommandLine(int argc, char *argv[])
log_opts.retain = true;
break;
+ case 'R':
+ user_opts.pass_terminal_fd = true;
+ break;
+
case 's':
user_opts.socketdir = pg_strdup(optarg);
break;
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 31589b0fdc..878f8e8032 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -12,6 +12,7 @@
#include "common/relpath.h"
#include "libpq-fe.h"
+#include "common/kmgr_utils.h"
/* For now, pg_upgrade does not use common/logging.c; use our own pg_fatal */
#undef pg_fatal
@@ -218,6 +219,7 @@ typedef struct
bool date_is_int;
bool float8_pass_by_value;
uint32 data_checksum_version;
+ int file_encryption_method;
} ControlData;
/*
@@ -296,6 +298,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool ind_coll_unknown; /* mark unknown index collation versions */
+ bool pass_terminal_fd; /* pass -R to pg_ctl? */
} UserOpts;
typedef struct
diff --git a/src/bin/pg_upgrade/server.c b/src/bin/pg_upgrade/server.c
index 9a1a20fb27..0775010379 100644
--- a/src/bin/pg_upgrade/server.c
+++ b/src/bin/pg_upgrade/server.c
@@ -239,7 +239,7 @@ start_postmaster(ClusterInfo *cluster, bool report_and_exit_on_error)
* vacuumdb --freeze actually freezes the tuples.
*/
snprintf(cmd, sizeof(cmd),
- "\"%s/pg_ctl\" -w -l \"%s/%s\" -D \"%s\" -o \"-p %d -b%s %s%s\" start",
+ "\"%s/pg_ctl\" -w -l \"%s/%s\" -D \"%s\" -o \"-p %d%s -b %s%s\" start",
cluster->bindir,
log_opts.logdir,
SERVER_LOG_FILE, cluster->pgconfig, cluster->port,
--
2.31.1
[application/octet-stream] v2-0006-cfe-06-backend_over_cfe-05-crypto-squash-commit.patch (23.0K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/8-v2-0006-cfe-06-backend_over_cfe-05-crypto-squash-commit.patch)
download | inline diff:
From 85cbe762e5c7a74d0e3e02aa40ff9142e79b668e Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:53 -0400
Subject: [PATCH v2 06/12] cfe-06-backend_over_cfe-05-crypto squash commit
---
doc/src/sgml/storage.sgml | 5 +++
src/backend/Makefile | 2 +-
src/backend/access/transam/xlog.c | 33 +++++++++++++++++++
src/backend/backup/basebackup.c | 5 +++
src/backend/bootstrap/bootstrap.c | 29 +++++++++++++++-
src/backend/crypto/kmgr.c | 3 ++
src/backend/libpq/be-secure-common.c | 14 ++++++++
src/backend/main/main.c | 3 ++
src/backend/postmaster/postmaster.c | 13 +++++++-
src/backend/storage/ipc/ipci.c | 3 ++
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/tcop/postgres.c | 28 +++++++++++++++-
src/backend/utils/activity/wait_event.c | 9 +++++
src/backend/utils/misc/guc_tables.c | 25 ++++++++++++++
src/backend/utils/misc/pg_controldata.c | 11 +++++--
src/backend/utils/misc/postgresql.conf.sample | 5 +++
src/include/access/xlog.h | 5 +++
src/include/catalog/pg_control.h | 5 ++-
src/include/catalog/pg_proc.dat | 6 ++--
src/include/postmaster/postmaster.h | 2 ++
src/include/utils/guc_tables.h | 1 +
21 files changed, 197 insertions(+), 11 deletions(-)
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index e5b9f3f1ff..f9dbf04735 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -77,6 +77,11 @@ Item
<entry>Subdirectory containing transaction commit timestamp data</entry>
</row>
+<row>
+ <entry><filename>pg_cryptokeys</filename></entry>
+ <entry>Subdirectory containing file encryption keys</entry>
+</row>
+
<row>
<entry><filename>pg_dynshmem</filename></entry>
<entry>Subdirectory containing files used by the dynamic shared memory
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 1a5239b822..4b8cdd1b44 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -22,7 +22,7 @@ SUBDIRS = access backup bootstrap catalog parser commands executor \
main nodes optimizer partitioning port postmaster \
regex replication rewrite \
statistics storage tcop tsearch utils $(top_builddir)/src/timezone \
- jit
+ jit crypto
include $(srcdir)/common.mk
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index be54c23187..e247376ead 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -69,6 +69,7 @@
#include "catalog/pg_database.h"
#include "common/controldata_utils.h"
#include "common/file_utils.h"
+#include "crypto/kmgr.h"
#include "executor/instrument.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -76,6 +77,7 @@
#include "port/atomics.h"
#include "port/pg_iovec.h"
#include "postmaster/bgwriter.h"
+#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/walwriter.h"
#include "replication/logical.h"
@@ -109,6 +111,7 @@
#include "utils/varlena.h"
extern uint32 bootstrap_data_checksum_version;
+extern int bootstrap_file_encryption_method;
/* timeline ID to be used when bootstrapping */
#define BootstrapTimeLineID 1
@@ -3898,6 +3901,7 @@ InitControlFile(uint64 sysidentifier)
ControlFile->wal_log_hints = wal_log_hints;
ControlFile->track_commit_timestamp = track_commit_timestamp;
ControlFile->data_checksum_version = bootstrap_data_checksum_version;
+ ControlFile->file_encryption_method = bootstrap_file_encryption_method;
}
static void
@@ -4185,6 +4189,12 @@ ReadControlFile(void)
/* Make the initdb settings visible as GUC variables, too */
SetConfigOption("data_checksums", DataChecksumsEnabled() ? "yes" : "no",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+
+ StaticAssertStmt(lengthof(encryption_methods) == NUM_ENCRYPTION_METHODS,
+ "encryption_methods[] must match NUM_ENCRYPTION_METHODS");
+ SetConfigOption("file_encryption_method",
+ encryption_methods[ControlFile->file_encryption_method].name,
+ PGC_INTERNAL, PGC_S_OVERRIDE);
}
/*
@@ -4227,6 +4237,21 @@ DataChecksumsEnabled(void)
return (ControlFile->data_checksum_version > 0);
}
+/*
+ * Is cluster file encryption enabled?
+ */
+int
+GetFileEncryptionMethod(void)
+{
+ if (IsBootstrapProcessingMode())
+ return bootstrap_file_encryption_method;
+ else
+ {
+ Assert(ControlFile != NULL);
+ return ControlFile->file_encryption_method;
+ }
+}
+
/*
* Returns a fake LSN for unlogged relations.
*
@@ -4806,6 +4831,14 @@ BootStrapXLOG(void)
/* some additional ControlFile fields are set in WriteControlFile() */
WriteControlFile();
+ BootStrapKmgr();
+
+ if (terminal_fd != -1)
+ {
+ close(terminal_fd);
+ terminal_fd = -1;
+ }
+
/* Bootstrap the commit log, too */
BootStrapCLOG();
BootStrapCommitTs();
diff --git a/src/backend/backup/basebackup.c b/src/backend/backup/basebackup.c
index 74fb529380..35e955e036 100644
--- a/src/backend/backup/basebackup.c
+++ b/src/backend/backup/basebackup.c
@@ -24,6 +24,7 @@
#include "backup/basebackup_target.h"
#include "commands/defrem.h"
#include "common/compression.h"
+#include "common/kmgr_utils.h"
#include "common/file_perm.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
@@ -129,6 +130,10 @@ struct exclude_list_item
*/
static const char *const excludeDirContents[] =
{
+ /* Skip temporary crypto key directories */
+ NEW_KMGR_DIR,
+ OLD_KMGR_DIR,
+
/*
* Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped
* because extensions like pg_stat_statements store data there.
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 661ebacb0c..87918a4d28 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -33,6 +33,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "pg_getopt.h"
+#include "postmaster/postmaster.h" /* TODO: verify we need this still */
#include "storage/bufmgr.h"
#include "storage/bufpage.h"
#include "storage/condition_variable.h"
@@ -46,6 +47,8 @@
#include "utils/relmapper.h"
uint32 bootstrap_data_checksum_version = 0; /* No checksum */
+int bootstrap_file_encryption_method = DISABLED_ENCRYPTION_METHOD;
+char *bootstrap_old_key_datadir = NULL; /* disabled */
static void CheckerModeMain(void);
@@ -221,7 +224,7 @@ BootstrapModeMain(int argc, char *argv[], bool check_only)
argv++;
argc--;
- while ((flag = getopt(argc, argv, "B:c:d:D:Fkr:X:-:")) != -1)
+ while ((flag = getopt(argc, argv, "B:c:d:D:FkK:r:R:u:X:-:")) != -1)
{
switch (flag)
{
@@ -250,9 +253,33 @@ BootstrapModeMain(int argc, char *argv[], bool check_only)
case 'k':
bootstrap_data_checksum_version = PG_DATA_CHECKSUM_VERSION;
break;
+ case 'K':
+ {
+ int i;
+
+ /* method 0/disabled cannot be specified */
+ for (i = DISABLED_ENCRYPTION_METHOD + 1;
+ i < NUM_ENCRYPTION_METHODS; i++)
+ if (pg_strcasecmp(optarg, encryption_methods[i].name) == 0)
+ {
+ bootstrap_file_encryption_method = i;
+ break;
+ }
+ if (i == NUM_ENCRYPTION_METHODS)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid encryption method specified")));
+ }
+ break;
case 'r':
strlcpy(OutputFileName, optarg, MAXPGPATH);
break;
+ case 'R':
+ terminal_fd = atoi(optarg);
+ break;
+ case 'u':
+ bootstrap_old_key_datadir = pstrdup(optarg);
+ break;
case 'X':
{
int WalSegSz = strtoul(optarg, NULL, 0);
diff --git a/src/backend/crypto/kmgr.c b/src/backend/crypto/kmgr.c
index 4b74961323..a83effbca8 100644
--- a/src/backend/crypto/kmgr.c
+++ b/src/backend/crypto/kmgr.c
@@ -253,6 +253,9 @@ InitializeKmgr(void)
char live_path[MAXPGPATH];
unsigned char cluster_key[KMGR_CLUSTER_KEY_LEN];
+ if (!FileEncryptionEnabled)
+ return;
+
#ifndef USE_OPENSSL
ereport(ERROR,
(errcode(ERRCODE_CONFIG_FILE_ERROR),
diff --git a/src/backend/libpq/be-secure-common.c b/src/backend/libpq/be-secure-common.c
index 2719ce1403..9357c80ea6 100644
--- a/src/backend/libpq/be-secure-common.c
+++ b/src/backend/libpq/be-secure-common.c
@@ -22,6 +22,7 @@
#include <sys/stat.h>
#include <unistd.h>
+#include "postmaster/postmaster.h"
#include "common/string.h"
#include "libpq/libpq.h"
#include "storage/fd.h"
@@ -61,6 +62,19 @@ run_ssl_passphrase_command(const char *prompt, bool is_server_start, char *buf,
appendStringInfoString(&command, prompt);
p++;
break;
+ case 'R':
+ {
+ char fd_str[20];
+
+ if (terminal_fd == -1)
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("ssl_passphrase_command referenced %%R, but -R not specified")));
+ p++;
+ snprintf(fd_str, sizeof(fd_str), "%d", terminal_fd);
+ appendStringInfoString(&command, fd_str);
+ break;
+ }
case '%':
appendStringInfoChar(&command, '%');
p++;
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index f5da4260a1..ddf3fcb6fb 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -343,6 +343,7 @@ help(const char *progname)
#endif
printf(_(" -N MAX-CONNECT maximum number of allowed connections\n"));
printf(_(" -p PORT port number to listen on\n"));
+ printf(_(" -R fd prompt for the cluster key\n"));
printf(_(" -s show statistics after each query\n"));
printf(_(" -S WORK-MEM set amount of memory for sorts (in kB)\n"));
printf(_(" -V, --version output version information, then exit\n"));
@@ -371,7 +372,9 @@ help(const char *progname)
printf(_(" --boot selects bootstrapping mode (must be first argument)\n"));
printf(_(" --check selects check mode (must be first argument)\n"));
printf(_(" DBNAME database name (mandatory argument in bootstrapping mode)\n"));
+ printf(_(" -K LEN enable cluster file encryption with specified key bit length\n"));
printf(_(" -r FILENAME send stdout and stderr to given file\n"));
+ printf(_(" -u DATADIR copy encryption keys from datadir\n"));
printf(_("\nPlease read the documentation for the complete list of run-time\n"
"configuration settings and how to set them on the command line or in\n"
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 0b637ba6a2..1673717da9 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -98,6 +98,7 @@
#include "common/ip.h"
#include "common/pg_prng.h"
#include "common/string.h"
+#include "crypto/kmgr.h"
#include "lib/ilist.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
@@ -231,6 +232,7 @@ static int SendStop = false;
/* still more option variables */
bool EnableSSL = false;
+int terminal_fd = -1;
int PreAuthDelay = 0;
int AuthenticationTimeout = 60;
@@ -697,7 +699,7 @@ PostmasterMain(int argc, char *argv[])
* tcop/postgres.c (the option sets should not conflict) and with the
* common help() function in main/main.c.
*/
- while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOPp:r:S:sTt:W:-:")) != -1)
+ while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOPp:r:R:S:sTt:W:-:")) != -1)
{
switch (opt)
{
@@ -788,6 +790,10 @@ PostmasterMain(int argc, char *argv[])
/* only used by single-user backend */
break;
+ case 'R':
+ terminal_fd = atoi(optarg);
+ break;
+
case 'S':
SetConfigOption("work_mem", optarg, PGC_POSTMASTER, PGC_S_ARGV);
break;
@@ -1347,6 +1353,11 @@ PostmasterMain(int argc, char *argv[])
pfree(rawstring);
}
+ InitializeKmgr();
+
+ if (terminal_fd != -1)
+ close(terminal_fd);
+
/*
* check that we have some socket to listen on
*/
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index b204ecdbc3..061da11a58 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -25,6 +25,7 @@
#include "access/xlogprefetcher.h"
#include "access/xlogrecovery.h"
#include "commands/async.h"
+#include "crypto/kmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
@@ -142,6 +143,7 @@ CalculateShmemSize(int *num_semaphores)
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
size = add_size(size, StatsShmemSize());
+ size = add_size(size, KmgrShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -294,6 +296,7 @@ CreateSharedMemoryAndSemaphores(void)
SyncScanShmemInit();
AsyncShmemInit();
StatsShmemInit();
+ KmgrShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index 6c7cf6c295..56ac32407f 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -53,3 +53,4 @@ XactTruncationLock 44
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+KmgrFileLock 48
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 3082093d1e..5ee6c43394 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -35,6 +35,7 @@
#include "commands/async.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
+#include "crypto/kmgr.h"
#include "jit/jit.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
@@ -3718,7 +3719,7 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx,
* postmaster/postmaster.c (the option sets should not conflict) and with
* the common help() function in main/main.c.
*/
- while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOPp:r:S:sTt:v:W:-:")) != -1)
+ while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOPp:r:R:S:sTt:v:W:-:")) != -1)
{
switch (flag)
{
@@ -3810,6 +3811,19 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx,
strlcpy(OutputFileName, optarg, MAXPGPATH);
break;
+ case 'R':
+ terminal_fd = atoi(optarg);
+ if (terminal_fd == -1)
+ {
+ /*
+ * Allow file descriptor closing to be bypassed via -1.
+ * We just duplicate sterr. This is useful for
+ * single-user mode.
+ */
+ terminal_fd = dup(2);
+ }
+ break;
+
case 'S':
SetConfigOption("work_mem", optarg, ctx, gucsource);
break;
@@ -4055,6 +4069,18 @@ PostgresMain(const char *dbname, const char *username)
SetProcessingMode(InitProcessing);
+ /*
+ * Initialize kmgr for cluster encryption. Since kmgr needs to attach to
+ * shared memory the initialization must be called after BaseInit().
+ */
+ if (!IsUnderPostmaster)
+ {
+ InitializeKmgr();
+
+ if (terminal_fd != -1)
+ close(terminal_fd);
+ }
+
/*
* Set up signal handlers. (InitPostmasterChild or InitStandaloneProcess
* has already set up BlockSig and made that the active signal mask.)
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index 92f24a6c9b..6f7953accb 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -591,6 +591,15 @@ pgstat_get_wait_io(WaitEventIO w)
case WAIT_EVENT_DSM_FILL_ZERO_WRITE:
event_name = "DSMFillZeroWrite";
break;
+ case WAIT_EVENT_KEY_FILE_READ:
+ event_name = "KeyFileRead";
+ break;
+ case WAIT_EVENT_KEY_FILE_WRITE:
+ event_name = "KeyFileWrite";
+ break;
+ case WAIT_EVENT_KEY_FILE_SYNC:
+ event_name = "KeyFileSync";
+ break;
case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ:
event_name = "LockFileAddToDataDirRead";
break;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 836b49484a..e7d7cad2b2 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -40,6 +40,7 @@
#include "commands/trigger.h"
#include "commands/user.h"
#include "commands/vacuum.h"
+#include "crypto/kmgr.h"
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
@@ -579,6 +580,7 @@ static char *recovery_target_string;
static char *recovery_target_xid_string;
static char *recovery_target_name_string;
static char *recovery_target_lsn_string;
+static char *file_encryption_method_str;
/* should be static, but commands/variable.c needs to get at this */
char *role_string;
@@ -701,6 +703,8 @@ const char *const config_group_names[] =
gettext_noop("Statistics / Monitoring"),
/* STATS_CUMULATIVE */
gettext_noop("Statistics / Cumulative Query and Index Statistics"),
+ /* ENCRYPTION */
+ gettext_noop("Encryption"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN_STATEMENT */
@@ -4403,6 +4407,27 @@ struct config_string ConfigureNamesString[] =
NULL, NULL, NULL
},
+ {
+ {"cluster_key_command", PGC_SIGHUP, ENCRYPTION,
+ gettext_noop("Command to obtain cluster key for cluster file encryption."),
+ NULL
+ },
+ &cluster_key_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"file_encryption_method", PGC_INTERNAL, PRESET_OPTIONS,
+ gettext_noop("Shows the cluster file encryption method."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ },
+ &file_encryption_method_str,
+ "",
+ NULL, NULL, NULL
+ },
+
{
{"application_name", PGC_USERSET, LOGGING_WHAT,
gettext_noop("Sets the application name to be reported in statistics and logs."),
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 781f8b8758..d5fbf8f7df 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -263,8 +263,8 @@ pg_control_recovery(PG_FUNCTION_ARGS)
Datum
pg_control_init(PG_FUNCTION_ARGS)
{
- Datum values[11];
- bool nulls[11];
+ Datum values[12];
+ bool nulls[12];
TupleDesc tupdesc;
HeapTuple htup;
ControlFileData *ControlFile;
@@ -274,7 +274,7 @@ pg_control_init(PG_FUNCTION_ARGS)
* Construct a tuple descriptor for the result row. This must match this
* function's pg_proc entry!
*/
- tupdesc = CreateTemplateTupleDesc(11);
+ tupdesc = CreateTemplateTupleDesc(12);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "max_data_alignment",
INT4OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "database_block_size",
@@ -297,6 +297,8 @@ pg_control_init(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 11, "data_page_checksum_version",
INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 12, "file_encryption_method",
+ INT4OID, -1, 0);
tupdesc = BlessTupleDesc(tupdesc);
/* read the control file */
@@ -338,6 +340,9 @@ pg_control_init(PG_FUNCTION_ARGS)
values[10] = Int32GetDatum(ControlFile->data_checksum_version);
nulls[10] = false;
+ values[11] = Int32GetDatum(ControlFile->file_encryption_method);
+ nulls[11] = false;
+
htup = heap_form_tuple(tupdesc, values, nulls);
PG_RETURN_DATUM(HeapTupleGetDatum(htup));
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 868d21c351..96b5c869e1 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -659,6 +659,11 @@
# autovacuum, -1 means use
# vacuum_cost_limit
+#------------------------------------------------------------------------------
+# ENCRYPTION
+#------------------------------------------------------------------------------
+
+#cluster_key_command = ''
#------------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 1fbd48fbda..8f219fbacc 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -13,6 +13,7 @@
#include "access/xlogbackup.h"
#include "access/xlogdefs.h"
+#include "common/kmgr_utils.h"
#include "datatype/timestamp.h"
#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
@@ -224,6 +225,10 @@ extern XLogRecPtr GetXLogWriteRecPtr(void);
extern uint64 GetSystemIdentifier(void);
extern char *GetMockAuthenticationNonce(void);
extern bool DataChecksumsEnabled(void);
+
+#define FileEncryptionEnabled (GetFileEncryptionMethod() != DISABLED_ENCRYPTION_METHOD)
+extern int GetFileEncryptionMethod(void);
+
extern XLogRecPtr GetFakeLSNForUnloggedRel(void);
extern Size XLOGShmemSize(void);
extern void XLOGShmemInit(void);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06368e2366..db7de77287 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -22,7 +22,7 @@
/* Version identifier for this pg_control format */
-#define PG_CONTROL_VERSION 1300
+#define PG_CONTROL_VERSION 1400
/* Nonce key length, see below */
#define MOCK_AUTH_NONCE_LEN 32
@@ -226,6 +226,9 @@ typedef struct ControlFileData
*/
char mock_authentication_nonce[MOCK_AUTH_NONCE_LEN];
+ /* File encryption method; index into encryption_methods[]. */
+ int file_encryption_method;
+
/* CRC of all above ... MUST BE LAST! */
pg_crc32c crc;
} ControlFileData;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 20f5aa56ea..f5e7a85519 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11674,9 +11674,9 @@
descr => 'pg_controldata init state information as a function',
proname => 'pg_control_init', provolatile => 'v', prorettype => 'record',
proargtypes => '',
- proallargtypes => '{int4,int4,int4,int4,int4,int4,int4,int4,int4,bool,int4}',
- proargmodes => '{o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{max_data_alignment,database_block_size,blocks_per_segment,wal_block_size,bytes_per_wal_segment,max_identifier_length,max_index_columns,max_toast_chunk_size,large_object_chunk_size,float8_pass_by_value,data_page_checksum_version}',
+ proallargtypes => '{int4,int4,int4,int4,int4,int4,int4,int4,int4,bool,int4,int4}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{max_data_alignment,database_block_size,blocks_per_segment,wal_block_size,bytes_per_wal_segment,max_identifier_length,max_index_columns,max_toast_chunk_size,large_object_chunk_size,float8_pass_by_value,data_page_checksum_version,file_encryption_method}',
prosrc => 'pg_control_init' },
# subscripting support for built-in types
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 90e333ccd2..deac25e115 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -31,6 +31,8 @@ extern PGDLLIMPORT char *bonjour_name;
extern PGDLLIMPORT bool restart_after_crash;
extern PGDLLIMPORT bool remove_temp_files_after_crash;
+extern int terminal_fd;
+
#ifdef WIN32
extern PGDLLIMPORT HANDLE PostmasterHandle;
#else
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index 51f7f09fde..1446b334b1 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -86,6 +86,7 @@ enum config_group
PROCESS_TITLE,
STATS_MONITORING,
STATS_CUMULATIVE,
+ ENCRYPTION,
AUTOVACUUM,
CLIENT_CONN_STATEMENT,
CLIENT_CONN_LOCALE,
--
2.31.1
[application/octet-stream] v2-0008-cfe-08-pg_alterckey_over_cfe-07-bin-squash-commit.patch (30.8K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/9-v2-0008-cfe-08-pg_alterckey_over_cfe-07-bin-squash-commit.patch)
download | inline diff:
From 61583c5d1af66c91a603d41b5c59e1f025542788 Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:56:58 -0400
Subject: [PATCH v2 08/12] cfe-08-pg_alterckey_over_cfe-07-bin squash commit
---
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/pg_alterckey.sgml | 210 ++++++++
doc/src/sgml/reference.sgml | 1 +
src/bin/Makefile | 1 +
src/bin/pg_alterckey/.gitignore | 1 +
src/bin/pg_alterckey/Makefile | 38 ++
src/bin/pg_alterckey/README | 24 +
src/bin/pg_alterckey/pg_alterckey.c | 724 ++++++++++++++++++++++++++++
8 files changed, 1000 insertions(+)
create mode 100644 doc/src/sgml/ref/pg_alterckey.sgml
create mode 100644 src/bin/pg_alterckey/.gitignore
create mode 100644 src/bin/pg_alterckey/Makefile
create mode 100644 src/bin/pg_alterckey/README
create mode 100644 src/bin/pg_alterckey/pg_alterckey.c
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index e90a0e1f83..a2b96129ba 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -197,6 +197,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY dropuser SYSTEM "dropuser.sgml">
<!ENTITY ecpgRef SYSTEM "ecpg-ref.sgml">
<!ENTITY initdb SYSTEM "initdb.sgml">
+<!ENTITY pgalterckey SYSTEM "pg_alterckey.sgml">
<!ENTITY pgamcheck SYSTEM "pg_amcheck.sgml">
<!ENTITY pgarchivecleanup SYSTEM "pgarchivecleanup.sgml">
<!ENTITY pgBasebackup SYSTEM "pg_basebackup.sgml">
diff --git a/doc/src/sgml/ref/pg_alterckey.sgml b/doc/src/sgml/ref/pg_alterckey.sgml
new file mode 100644
index 0000000000..867c439b8e
--- /dev/null
+++ b/doc/src/sgml/ref/pg_alterckey.sgml
@@ -0,0 +1,210 @@
+<!--
+doc/src/sgml/ref/pg_alterckey.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pg_alterckey">
+ <indexterm zone="app-pg_alterckey">
+ <primary>pg_alterckey</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle><application>pg_alterckey</application></refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_alterckey</refname>
+ <refpurpose>alter the <productname>PostgreSQL</productname> cluster key</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_alterckey</command>
+
+ <group choice="plain">
+ <arg choice="plain"><option>-R</option></arg>
+ <arg choice="plain"><option>--authprompt</option></arg>
+ </group>
+
+ <arg choice="plain">
+ <replaceable class="parameter">old_cluster_key_command</replaceable>
+ <replaceable class="parameter">new_cluster_key_command</replaceable>
+ </arg>
+
+ <arg choice="opt">
+ <group choice="plain">
+ <arg choice="plain"><option>-D</option></arg>
+ <arg choice="plain"><option>--pgdata</option></arg>
+ </group>
+ <replaceable class="parameter">datadir</replaceable>
+ </arg>
+
+ </cmdsynopsis>
+
+ <cmdsynopsis>
+ <command>pg_alterckey</command>
+
+ <group choice="opt">
+ <arg choice="plain"><option>-R</option></arg>
+ <arg choice="plain"><option>--authprompt</option></arg>
+ </group>
+
+ <group choice="plain">
+ <arg choice="plain"><option>-r</option></arg>
+ <arg choice="plain"><option>--repair</option></arg>
+ </group>
+
+ <arg choice="opt">
+ <group choice="opt">
+ <arg choice="plain"><option>-D</option></arg>
+ <arg choice="plain"><option>--pgdata</option></arg>
+ </group>
+ <replaceable class="parameter">datadir</replaceable>
+ </arg>
+
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1 id="r1-app-pg_alterckey-1">
+ <title>Description</title>
+ <para>
+ <command>pg_alterckey</command> alters the cluster key used
+ for cluster file encryption. The cluster key is initially set
+ during <xref linkend="app-initdb"/>. The command can be run while the
+ server is running or stopped. The new password must be used the next
+ time the server is started.
+ </para>
+
+ <para>
+ <command>pg_alterckey</command> changes the key encryption key
+ (<acronym>KEK</acronym>) which encrypts the data encryption keys;
+ it does not change the data encryption keys. It does this by
+ decrypting each data encryption key using the <replaceable
+ class="parameter">old_cluster_key_command</replaceable>,
+ re-encrypting it using the <replaceable
+ class="parameter">new_cluster_key_command</replaceable>, and then
+ writes the result back to the cluster directory.
+ </para>
+
+ <para>
+ See the <xref linkend="app-initdb"/> documentation for how to define
+ the old and new passphrase commands. You can use different executables
+ for these commands, or you can use the same executable with different
+ arguments to specify retrieval of the old or new key.
+ </para>
+
+ <para>
+ <command>pg_alterckey</command> manages data encryption keys,
+ which are critical to allowing Postgres to access its decrypted
+ data. For this reason, it is very careful to preserve these
+ keys in most possible failure conditions, e.g., operating system
+ failure during cluster encryption key rotation.
+ </para>
+
+ <para>
+ When started, <command>pg_alterckey</command> repairs any files that
+ remain from previous failures before altering the cluster encryption
+ key. During this repair phase, <command>pg_alterckey</command> will
+ either roll back the cluster key or roll forward the changes that
+ were previously requested. The server will not start if repair is
+ needed, though a running server is unaffected by an unrepaired cluster
+ key configuration. Therefore, if <command>pg_alterckey</command>
+ fails for any reason, it is recommended you run the command with
+ <option>--repair</option> to simply roll back or forward any previous
+ changes. This will report if it rolled the cluster key back or forward,
+ and then run the command again to change the cluster key if needed.
+ </para>
+
+ <para>
+ You can specify the data directory on the command line, or use
+ the environment variable <envar>PGDATA</envar>.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ <variablelist>
+ <varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--authprompt</option></term>
+ <listitem>
+ <para>
+ Allows the <option>old_cluster_key_command</option> and
+ <option>new_cluster_key_command</option> commands
+ to prompt for a passphrase or PIN.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other options:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_alterckey</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_alterckey</application> command line
+ arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Environment</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><envar>PGDATA</envar></term>
+
+ <listitem>
+ <para>
+ Default data directory location
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><envar>PG_COLOR</envar></term>
+ <listitem>
+ <para>
+ Specifies whether to use color in diagnostic messages. Possible values
+ are <literal>always</literal>, <literal>auto</literal> and
+ <literal>never</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="app-initdb"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index a3b743e8c1..46f7834f72 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -241,6 +241,7 @@
</para>
</partintro>
+ &pgalterckey;
&clusterdb;
&createdb;
&createuser;
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 7f9dde924e..018ee6592f 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -17,6 +17,7 @@ SUBDIRS = \
initdb \
pg_amcheck \
pg_archivecleanup \
+ pg_alterckey \
pg_basebackup \
pg_checksums \
pg_config \
diff --git a/src/bin/pg_alterckey/.gitignore b/src/bin/pg_alterckey/.gitignore
new file mode 100644
index 0000000000..4c4f39f2cc
--- /dev/null
+++ b/src/bin/pg_alterckey/.gitignore
@@ -0,0 +1 @@
+/pg_alterckey
diff --git a/src/bin/pg_alterckey/Makefile b/src/bin/pg_alterckey/Makefile
new file mode 100644
index 0000000000..2133a4ba06
--- /dev/null
+++ b/src/bin/pg_alterckey/Makefile
@@ -0,0 +1,38 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/bin/pg_alterckey
+#
+# Copyright (c) 1998-2021, PostgreSQL Global Development Group
+#
+# src/bin/pg_alterckey/Makefile
+#
+#-------------------------------------------------------------------------
+
+PGFILEDESC = "pg_alterckey - alter the cluster key"
+PGAPPICON=win32
+
+subdir = src/bin/pg_alterckey
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ $(WIN32RES) \
+ pg_alterckey.o
+
+all: pg_alterckey
+
+pg_alterckey: $(OBJS) | submake-libpgport
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_alterckey$(X) '$(DESTDIR)$(bindir)/pg_alterckey$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_alterckey$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_alterckey$(X) $(OBJS)
+ rm -rf tmp_check
diff --git a/src/bin/pg_alterckey/README b/src/bin/pg_alterckey/README
new file mode 100644
index 0000000000..6db3739271
--- /dev/null
+++ b/src/bin/pg_alterckey/README
@@ -0,0 +1,24 @@
+pg_alterckey
+============
+
+This directory contains the code to generate the pg_alterckey binary.
+
+Architecture
+------------
+
+pg_alterckey allows altering of the cluster encryption key (key
+encryption key or KEK) which is stored outside of the file system; see
+src/backend/crypto/README for more details. This must be done in a
+crash-safe manner since the keys are critical to reading an encrypted
+cluster. The active data encryption keys (DEK) are encrypted/wrapped by
+the KEK and stored in PGDATA/pg_cryptokeys/live as separate files,
+currently files 0 and 1.
+
+This process can be interrupted at anytime; the new execution of
+pg_alterckey will repair any previously interrupted execution of
+pg_alterckey.
+
+pg_alterckey should never be run concurrently. A lock file prevents
+almost all concurrent execution. pg_alterckey can be run if the
+database server is running or stopped, so it can't use database locking
+that is only available when the server is running.
diff --git a/src/bin/pg_alterckey/pg_alterckey.c b/src/bin/pg_alterckey/pg_alterckey.c
new file mode 100644
index 0000000000..7e3d4ad915
--- /dev/null
+++ b/src/bin/pg_alterckey/pg_alterckey.c
@@ -0,0 +1,724 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_alterckey.c
+ * A utility to change the cluster key (key encryption key, KEK)
+ * used for cluster file encryption. The KEK wrap data encryption
+ * keys (DEK).
+ *
+ * The theory of operation is fairly simple:
+ * 1. Create lock file
+ * 2. Retrieve current and new cluster key using the supplied
+ * commands.
+ * 3. Revert any failed alter operation.
+ * 4. Create a "new" directory
+ * 5. Unwrap each DEK in "live" using the old KEK
+ * 6. Wrap each DEK using the new KEK and write it to "new"
+ * 7. Rename "live" to "old"
+ * 8. Rename "new" to "live"
+ * 9. Remove "old"
+ * 10. Remove lock file
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/pg_alterckey/pg_alterckey.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+
+#define FRONTEND 1
+
+#include "postgres_fe.h"
+
+#include <signal.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+
+#include "common/file_perm.h"
+#include "common/file_utils.h"
+#include "common/hex.h"
+#include "common/restricted_token.h"
+#include "crypto/kmgr.h"
+#include "common/logging.h"
+#include "getopt_long.h"
+#include "pg_getopt.h"
+
+typedef enum
+{
+ SUCCESS_EXIT = 0,
+ ERROR_EXIT,
+ RMDIR_EXIT,
+ REPAIR_EXIT
+} exit_action;
+
+#define MAX_WRAPPED_KEY_LENGTH 64
+
+static int lock_fd = -1;
+static bool pass_terminal_fd = false;
+int terminal_fd = -1;
+static bool repair_mode = false;
+static char *old_cluster_key_cmd = NULL,
+ *new_cluster_key_cmd = NULL;
+static char old_cluster_key[KMGR_CLUSTER_KEY_LEN],
+ new_cluster_key[KMGR_CLUSTER_KEY_LEN];
+static CryptoKey data_key;
+unsigned char in_key[MAX_WRAPPED_KEY_LENGTH],
+ out_key[MAX_WRAPPED_KEY_LENGTH];
+int in_klen,
+ out_klen;
+static char top_path[MAXPGPATH],
+ pid_path[MAXPGPATH],
+ live_path[MAXPGPATH],
+ new_path[MAXPGPATH],
+ old_path[MAXPGPATH];
+
+static char *DataDir = NULL;
+static const char *progname;
+
+static void create_lockfile(void);
+static void recover_failure(void);
+static void retrieve_cluster_keys(void);
+static void bzero_keys_and_exit(exit_action action);
+static void reencrypt_data_keys(void);
+static void install_new_keys(void);
+
+static void
+usage(const char *progname)
+{
+ printf(_("%s changes the cluster key of a PostgreSQL database cluster.\n\n"), progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION] old_cluster_key_command new_cluster_key_command [DATADIR]\n"), progname);
+ printf(_(" %s [repair_option] [DATADIR]\n"), progname);
+ printf(_("\nOptions:\n"));
+ printf(_(" -R, --authprompt prompt for a passphrase or PIN\n"));
+ printf(_(" [-D, --pgdata=]DATADIR data directory\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nRepair options:\n"));
+ printf(_(" -r, --repair repair previous failure\n"));
+ printf(_("\nIf no data directory (DATADIR) is specified, "
+ "the environment variable PGDATA\nis used.\n\n"));
+ printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+ printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
+}
+
+
+int
+main(int argc, char *argv[])
+{
+ static struct option long_options[] = {
+ {"authprompt", required_argument, NULL, 'R'},
+ {"repair", required_argument, NULL, 'r'},
+ {"pgdata", required_argument, NULL, 'D'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_alterckey"));
+ progname = get_progname(argv[0]);
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage(progname);
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_alterckey (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ /* check for -r/-R */
+ while ((c = getopt_long(argc, argv, "D:rR", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'D':
+ DataDir = optarg;
+ break;
+ case 'r':
+ repair_mode = true;
+ break;
+ case 'R':
+ pass_terminal_fd = true;
+ break;
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(1);
+ }
+ }
+
+ if (!repair_mode)
+ {
+ /* get cluster key commands */
+ if (optind < argc)
+ old_cluster_key_cmd = argv[optind++];
+ else
+ {
+ pg_log_error("missing old_cluster_key_command");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (optind < argc)
+ new_cluster_key_cmd = argv[optind++];
+ else
+ {
+ pg_log_error("missing new_cluster_key_command");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ if (DataDir == NULL)
+ {
+ if (optind < argc)
+ DataDir = argv[optind++];
+ else
+ DataDir = getenv("PGDATA");
+
+ /* If no DataDir was specified, and none could be found, error out */
+ if (DataDir == NULL)
+ {
+ pg_log_error("no data directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(1);
+ }
+ }
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_error("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /*
+ * Disallow running as root because we create directories in PGDATA
+ */
+#ifndef WIN32
+ if (geteuid() == 0)
+ {
+ pg_log_error("%s: cannot be run as root\n"
+ "Please log in (using, e.g., \"su\") as the "
+ "(unprivileged) user that will\n"
+ "own the server process.\n",
+ progname);
+ exit(1);
+ }
+#endif
+
+ get_restricted_token();
+
+ /* Set mask based on PGDATA permissions */
+ if (!GetDataDirectoryCreatePerm(DataDir))
+ {
+ pg_log_error("could not read permissions of directory \"%s\": %m",
+ DataDir);
+ exit(1);
+ }
+
+ umask(pg_mode_mask);
+
+ snprintf(top_path, sizeof(top_path), "%s/%s", DataDir, KMGR_DIR);
+ snprintf(pid_path, sizeof(pid_path), "%s/%s", DataDir, KMGR_DIR_PID);
+ snprintf(live_path, sizeof(live_path), "%s/%s", DataDir, LIVE_KMGR_DIR);
+ snprintf(new_path, sizeof(new_path), "%s/%s", DataDir, NEW_KMGR_DIR);
+ snprintf(old_path, sizeof(old_path), "%s/%s", DataDir, OLD_KMGR_DIR);
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_error("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (DataDir == NULL)
+ {
+ pg_log_error("no data directory specified");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(1);
+ }
+
+ create_lockfile();
+
+ recover_failure();
+
+ if (!repair_mode)
+ {
+ retrieve_cluster_keys();
+ reencrypt_data_keys();
+ install_new_keys();
+ }
+
+#ifndef WIN32
+ /* remove file system reference to file */
+ if (unlink(pid_path) < 0)
+ {
+ pg_log_error("could not delete lock file \"%s\": %m", KMGR_DIR_PID);
+ exit(1);
+ }
+#endif
+
+ close(lock_fd);
+
+ bzero_keys_and_exit(SUCCESS_EXIT);
+}
+
+/* Create a lock file; this prevents almost all cases of concurrent access */
+void
+create_lockfile(void)
+{
+ struct stat buffer;
+ char lock_pid_str[20];
+
+ if (stat(top_path, &buffer) != 0 || !S_ISDIR(buffer.st_mode))
+ {
+ pg_log_error("cluster file encryption directory \"%s\" is missing; is it enabled?", KMGR_DIR_PID);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+
+ /* Does a lockfile exist? */
+ if ((lock_fd = open(pid_path, O_RDONLY, 0)) != -1)
+ {
+ int lock_pid;
+ int len;
+
+ /* read the PID */
+ if ((len = read(lock_fd, lock_pid_str, sizeof(lock_pid_str) - 1)) == 0)
+ {
+ pg_log_error("cannot read pid from lock file \"%s\": %m", KMGR_DIR_PID);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+ lock_pid_str[len] = '\0';
+
+ if ((lock_pid = atoi(lock_pid_str)) == 0)
+ {
+ pg_log_error("invalid pid in lock file \"%s\": %m", KMGR_DIR_PID);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+
+ /* Is the PID running? */
+ if (kill(lock_pid, 0) == 0)
+ {
+ pg_log_error("active process %d currently holds a lock on this operation, recorded in \"%s\"",
+ lock_pid, KMGR_DIR_PID);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+
+ close(lock_fd);
+
+ if (repair_mode)
+ printf("old lock file removed\n");
+
+ /* ----------
+ * pid is no longer running, so remove the lock file.
+ * This is not 100% safe from concurrent access, e.g.:
+ *
+ * process 1 exits and leaves stale lock file
+ * process 2 checks stale lock file of process 1
+ * process 3 checks stale lock file of process 1
+ * process 2 remove the lock file of process 1
+ * process 4 creates a lock file
+ * process 3 remove the lock file of process 4
+ * process 5 creates a lock file
+ *
+ * The sleep(2) helps with this since it reduces the likelihood
+ * a process that did an unlock will interfere with another unlock
+ * process. We could ask users to remove the lock, but that seems
+ * even more error-prone, especially since this might happen
+ * on server start. Many PG tools seem to have problems with
+ * concurrent access.
+ * ----------
+ */
+ unlink(pid_path);
+
+ /* Sleep to reduce the likelihood of concurrent unlink */
+ pg_usleep(2000000L); /* 2 seconds */
+ }
+
+ /* Create our own lockfile? */
+#ifndef WIN32
+ lock_fd = open(pid_path, O_RDWR | O_CREAT | O_EXCL, pg_file_create_mode);
+#else
+ /* delete on close */
+ lock_fd = open(pid_path, O_RDWR | O_CREAT | O_EXCL | O_TEMPORARY,
+ pg_file_create_mode);
+#endif
+
+ if (lock_fd == -1)
+ {
+ if (errno == EEXIST)
+ pg_log_error("an active process currently holds a lock on this operation, recorded in \"%s\"",
+ KMGR_DIR_PID);
+ else
+ pg_log_error("unable to create lock file \"%s\": %m", KMGR_DIR_PID);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+
+ snprintf(lock_pid_str, sizeof(lock_pid_str), "%d\n", getpid());
+ if (write(lock_fd, lock_pid_str, strlen(lock_pid_str)) != strlen(lock_pid_str))
+ {
+ pg_log_error("could not write pid to lock file \"%s\": %m", KMGR_DIR_PID);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+}
+
+/*
+ * ----------
+ * recover_failure
+ *
+ * A previous pg_alterckey might have failed, so it might need recovery.
+ * The normal operation is:
+ * 1. reencrypt LIVE_KMGR_DIR -> NEW_KMGR_DIR
+ * 2. rename KMGR_DIR -> OLD_KMGR_DIR
+ * 3. rename NEW_KMGR_DIR -> LIVE_KMGR_DIR
+ * remove OLD_KMGR_DIR
+ *
+ * There are eight possible directory configurations:
+ *
+ * LIVE_KMGR_DIR NEW_KMGR_DIR OLD_KMGR_DIR
+ *
+ * Normal:
+ * 0. normal X
+ * 1. remove new X X
+ * 2. install new X X
+ * 3. remove old X X
+ *
+ * Abnormal:
+ * fatal
+ * restore old X
+ * install new X
+ * remove old and new X X X
+ *
+ * We don't handle the abnormal cases, just report an error.
+ * ----------
+ */
+static void
+recover_failure(void)
+{
+ struct stat buffer;
+ bool is_live,
+ is_new,
+ is_old;
+
+ is_live = !stat(live_path, &buffer);
+ is_new = !stat(new_path, &buffer);
+ is_old = !stat(old_path, &buffer);
+
+ /* normal #0 */
+ if (is_live && !is_new && !is_old)
+ {
+ if (repair_mode)
+ printf("repair unnecessary\n");
+ return;
+ }
+ /* remove new #1 */
+ else if (is_live && is_new && !is_old)
+ {
+ if (!rmtree(new_path, true))
+ {
+ pg_log_error("unable to remove new directory \"%s\": %m", NEW_KMGR_DIR);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+ printf(_("removed files created during previously aborted alter operation\n"));
+ return;
+ }
+ /* install new #2 */
+ else if (!is_live && is_new && is_old)
+ {
+ if (rename(new_path, live_path) != 0)
+ {
+ pg_log_error("unable to rename directory \"%s\" to \"%s\": %m",
+ NEW_KMGR_DIR, LIVE_KMGR_DIR);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+ printf(_("Installed new cluster password supplied in previous alter operation\n"));
+ return;
+ }
+ /* remove old #3 */
+ else if (is_live && !is_new && is_old)
+ {
+ if (!rmtree(old_path, true))
+ {
+ pg_log_error("unable to remove old directory \"%s\": %m", OLD_KMGR_DIR);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+ printf(_("Removed old files invalidated during previous alter operation\n"));
+ return;
+ }
+ else
+ {
+ pg_log_error("cluster file encryption directory \"%s\" is in an abnormal state and cannot be processed",
+ KMGR_DIR);
+ fprintf(stderr, _("Exiting with no changes made.\n"));
+ exit(1);
+ }
+}
+
+/* Retrieve old and new cluster keys */
+void
+retrieve_cluster_keys(void)
+{
+ int cluster_key_len;
+ char cluster_key_hex[ALLOC_KMGR_CLUSTER_KEY_LEN];
+
+ /*
+ * If we have been asked to pass an open file descriptor to the user
+ * terminal to the commands, set one up.
+ */
+ if (pass_terminal_fd)
+ {
+#ifndef WIN32
+ terminal_fd = open("/dev/tty", O_RDWR, 0);
+#else
+ terminal_fd = open("CONOUT$", O_RDWR, 0);
+#endif
+ if (terminal_fd < 0)
+ {
+ pg_log_error(_("%s: could not open terminal: %s\n"),
+ progname, strerror(errno));
+ exit(1);
+ }
+ }
+
+ /* Get old key encryption key from the cluster key command */
+ cluster_key_len = kmgr_run_cluster_key_command(old_cluster_key_cmd,
+ (char *) cluster_key_hex,
+ ALLOC_KMGR_CLUSTER_KEY_LEN,
+ live_path, terminal_fd);
+ if (pg_hex_decode(cluster_key_hex, cluster_key_len,
+ (char *) old_cluster_key, KMGR_CLUSTER_KEY_LEN) !=
+ KMGR_CLUSTER_KEY_LEN)
+ {
+ pg_log_error("cluster key must be at %d hex bytes", KMGR_CLUSTER_KEY_LEN);
+ bzero_keys_and_exit(ERROR_EXIT);
+ }
+
+ /*
+ * Create new key directory here in case the new cluster key command needs
+ * it to exist.
+ */
+ if (mkdir(new_path, pg_dir_create_mode) != 0)
+ {
+ pg_log_error("unable to create new cluster key directory \"%s\": %m", NEW_KMGR_DIR);
+ bzero_keys_and_exit(ERROR_EXIT);
+ }
+
+ /* Get new key */
+ cluster_key_len = kmgr_run_cluster_key_command(new_cluster_key_cmd,
+ (char *) cluster_key_hex,
+ ALLOC_KMGR_CLUSTER_KEY_LEN,
+ new_path, terminal_fd);
+ if (pg_hex_decode(cluster_key_hex, cluster_key_len,
+ (char *) new_cluster_key, KMGR_CLUSTER_KEY_LEN) !=
+ KMGR_CLUSTER_KEY_LEN)
+ {
+ pg_log_error("cluster key must be at %d hex bytes", KMGR_CLUSTER_KEY_LEN);
+ bzero_keys_and_exit(ERROR_EXIT);
+ }
+
+ if (pass_terminal_fd)
+ close(terminal_fd);
+
+ /* output newline */
+ puts("");
+
+ if (memcmp(old_cluster_key, new_cluster_key, KMGR_CLUSTER_KEY_LEN) == 0)
+ {
+ pg_log_error("cluster keys are identical, exiting\n");
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+}
+
+/* Decrypt old keys encrypted with old pass phrase and reencrypt with new one */
+void
+reencrypt_data_keys(void)
+{
+ PgCipherCtx *old_ctx,
+ *new_ctx;
+
+ old_ctx = pg_cipher_ctx_create(PG_CIPHER_AES_KWP,
+ (unsigned char *) old_cluster_key,
+ KMGR_CLUSTER_KEY_LEN, false);
+ if (!old_ctx)
+ pg_log_error("could not initialize encryption context");
+
+ new_ctx = pg_cipher_ctx_create(PG_CIPHER_AES_KWP,
+ (unsigned char *) new_cluster_key,
+ KMGR_CLUSTER_KEY_LEN, true);
+ if (!new_ctx)
+ pg_log_error("could not initialize encryption context");
+
+ for (int id = 0; id < KMGR_NUM_DATA_KEYS; id++)
+ {
+ char src_path[MAXPGPATH],
+ dst_path[MAXPGPATH];
+ int src_fd,
+ dst_fd;
+ int len;
+ struct stat st;
+
+ CryptoKeyFilePath(src_path, live_path, id);
+ CryptoKeyFilePath(dst_path, new_path, id);
+
+ if ((src_fd = open(src_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ {
+ pg_log_error("could not open file \"%s\": %m", src_path);
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ if ((dst_fd = open(dst_path, O_RDWR | O_CREAT | O_TRUNC | PG_BINARY,
+ pg_file_create_mode)) < 0)
+ {
+ pg_log_error("could not open file \"%s\": %m", dst_path);
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ if (fstat(src_fd, &st))
+ {
+ pg_log_error("could not stat file \"%s\": %m", src_path);
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ in_klen = st.st_size;
+
+ if (in_klen > MAX_WRAPPED_KEY_LENGTH)
+ {
+ pg_log_error("invalid wrapped key length (%d) for file \"%s\"", in_klen, src_path);
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ /* Read the source key */
+ len = read(src_fd, in_key, in_klen);
+ if (len != in_klen)
+ {
+ if (len < 0)
+ pg_log_error("could read file \"%s\": %m", src_path);
+ else
+ pg_log_error("could read file \"%s\": read %d of %u",
+ src_path, len, in_klen);
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ /* decrypt with old key */
+ if (!kmgr_unwrap_data_key(old_ctx, in_key, in_klen, &data_key))
+ {
+ pg_log_error("incorrect old key specified");
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ if (KMGR_MAX_KEY_LEN_BYTES + pg_cipher_blocksize(new_ctx) > MAX_WRAPPED_KEY_LENGTH)
+ {
+ pg_log_error("invalid max wrapped key length");
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ /* encrypt with new key */
+ if (!kmgr_wrap_data_key(new_ctx, &data_key, out_key, &out_klen))
+ {
+ pg_log_error("could not encrypt new key");
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ /* Write to the dest key */
+ len = write(dst_fd, out_key, out_klen);
+ if (len != out_klen)
+ {
+ pg_log_error("could not write fie \"%s\"", dst_path);
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+
+ close(src_fd);
+ close(dst_fd);
+ }
+
+ /* The cluster key is correct, free the cipher context */
+ pg_cipher_ctx_free(old_ctx);
+ pg_cipher_ctx_free(new_ctx);
+}
+
+/* Install new keys */
+void
+install_new_keys(void)
+{
+ /*
+ * Issue fsync's so key rotation is less likely to be left in an
+ * inconsistent state in case of a crash during this operation.
+ */
+
+ if (rename(live_path, old_path) != 0)
+ {
+ pg_log_error("unable to rename directory \"%s\" to \"%s\": %m",
+ LIVE_KMGR_DIR, OLD_KMGR_DIR);
+ bzero_keys_and_exit(RMDIR_EXIT);
+ }
+ fsync_dir_recurse(top_path);
+
+ if (rename(new_path, live_path) != 0)
+ {
+ pg_log_error("unable to rename directory \"%s\" to \"%s\": %m",
+ NEW_KMGR_DIR, LIVE_KMGR_DIR);
+ bzero_keys_and_exit(REPAIR_EXIT);
+ }
+ fsync_dir_recurse(top_path);
+
+ if (!rmtree(old_path, true))
+ {
+ pg_log_error("unable to remove old directory \"%s\": %m", OLD_KMGR_DIR);
+ bzero_keys_and_exit(REPAIR_EXIT);
+ }
+ fsync_dir_recurse(top_path);
+}
+
+/* Erase memory and exit */
+void
+bzero_keys_and_exit(exit_action action)
+{
+ explicit_bzero(old_cluster_key, sizeof(old_cluster_key));
+ explicit_bzero(new_cluster_key, sizeof(new_cluster_key));
+
+ explicit_bzero(in_key, sizeof(in_key));
+ explicit_bzero(&data_key, sizeof(data_key));
+ explicit_bzero(out_key, sizeof(out_key));
+
+ if (action == RMDIR_EXIT)
+ {
+ if (!rmtree(new_path, true))
+ pg_log_error("unable to remove new directory \"%s\": %m", NEW_KMGR_DIR);
+ printf("Re-running pg_alterckey to repair might be needed before the next server start\n");
+ exit(1);
+ }
+ else if (action == REPAIR_EXIT)
+ {
+ unlink(pid_path);
+ printf("Re-running pg_alterckey to repair might be needed before the next server start\n");
+ }
+
+ /* return 0 or 1 */
+ exit(action != SUCCESS_EXIT);
+}
--
2.31.1
[application/octet-stream] v2-0009-cfe-09-test_over_cfe-08-pg_alterckey-squash-commi.patch (114.0K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/10-v2-0009-cfe-09-test_over_cfe-08-pg_alterckey-squash-commi.patch)
download | inline diff:
From f2419b9f3b5fcc4c8efd3f9f8a0ac8955e029684 Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:57:00 -0400
Subject: [PATCH v2 09/12] cfe-09-test_over_cfe-08-pg_alterckey squash commit
---
src/test/Makefile | 3 +
src/test/README | 3 +
src/test/crypto/.gitignore | 4 +
src/test/crypto/KWP_AD_128.txt | 35 ++
src/test/crypto/KWP_AD_256.txt | 35 ++
src/test/crypto/KWP_AE_128.txt | 35 ++
src/test/crypto/KWP_AE_256.txt | 35 ++
src/test/crypto/Makefile | 39 +++
src/test/crypto/README | 33 ++
src/test/crypto/gcmDecrypt128.rsp | 129 +++++++
src/test/crypto/gcmDecrypt256.rsp | 129 +++++++
src/test/crypto/gcmEncryptExtIV128.rsp | 129 +++++++
src/test/crypto/gcmEncryptExtIV256.rsp | 129 +++++++
src/test/crypto/t/001_testcrypto.pl | 137 ++++++++
src/test/crypto/t/002_testkwp.pl | 126 +++++++
src/test/crypto/t/003_clusterkey.pl | 93 +++++
src/test/crypto/t/004_buffers.pl | 157 +++++++++
src/test/crypto/testcrypto.c | 458 +++++++++++++++++++++++++
18 files changed, 1709 insertions(+)
create mode 100644 src/test/crypto/.gitignore
create mode 100644 src/test/crypto/KWP_AD_128.txt
create mode 100644 src/test/crypto/KWP_AD_256.txt
create mode 100644 src/test/crypto/KWP_AE_128.txt
create mode 100644 src/test/crypto/KWP_AE_256.txt
create mode 100644 src/test/crypto/Makefile
create mode 100644 src/test/crypto/README
create mode 100644 src/test/crypto/gcmDecrypt128.rsp
create mode 100644 src/test/crypto/gcmDecrypt256.rsp
create mode 100644 src/test/crypto/gcmEncryptExtIV128.rsp
create mode 100644 src/test/crypto/gcmEncryptExtIV256.rsp
create mode 100644 src/test/crypto/t/001_testcrypto.pl
create mode 100644 src/test/crypto/t/002_testkwp.pl
create mode 100644 src/test/crypto/t/003_clusterkey.pl
create mode 100644 src/test/crypto/t/004_buffers.pl
create mode 100644 src/test/crypto/testcrypto.c
diff --git a/src/test/Makefile b/src/test/Makefile
index dbd3192874..617f1dfa29 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -17,6 +17,9 @@ SUBDIRS = perl regress isolation modules authentication recovery subscription
ifeq ($(with_icu),yes)
SUBDIRS += icu
endif
+ifeq ($(with_openssl),yes)
+SUBDIRS += crypto
+endif
ifeq ($(with_gssapi),yes)
SUBDIRS += kerberos
endif
diff --git a/src/test/README b/src/test/README
index afdc767651..0955a764e3 100644
--- a/src/test/README
+++ b/src/test/README
@@ -11,6 +11,9 @@ which tests get run automatically.
authentication/
Tests for authentication (but see also below)
+crypto/
+ Tests for cluster file encryption
+
examples/
Demonstration programs for libpq that double as regression tests via
"make check"
diff --git a/src/test/crypto/.gitignore b/src/test/crypto/.gitignore
new file mode 100644
index 0000000000..7b92218ad2
--- /dev/null
+++ b/src/test/crypto/.gitignore
@@ -0,0 +1,4 @@
+# Generated by test suite
+/tmp_check/
+/log/
+/testcrypto
diff --git a/src/test/crypto/KWP_AD_128.txt b/src/test/crypto/KWP_AD_128.txt
new file mode 100644
index 0000000000..d77aeab719
--- /dev/null
+++ b/src/test/crypto/KWP_AD_128.txt
@@ -0,0 +1,35 @@
+# CAVS 21.4
+# 'NIST SP 800-38F KWP-AD with AES-128 cipher function' information for test-files
+# Seed = 0f0c6b6602d959d22ff2305478ea477a85d196d9695abf5b445010b45ce046c501bc04e6ca4faf46137adf6eac452d97314cd91137e33fc30cc6df316117fed3
+# Generated on Fri Apr 6 14:46:45 2018
+[PLAINTEXT LENGTH = 4096]
+
+COUNT = 0
+K = 1dd51f0d3a0a784174ba81b2c9f89005
+C = e1bde6d2df3b8e48ca127f97b56b5dc2672b3736cc3157c7b80a0316ef1efbdbbce19fea23da831836ccd2e002b2c1dfad206b5cec358446b8434d7f4c39e65b0e0b50897642ffc34bfb3cb3e233aa9c1058ff0d4fd48e98bc8cc3d214c06d514dd97db2278093a308f91f4ae92626d85771fb1447b36a3467fff02ac7e81ddbd0fdbcd02d1acd4f053c989ef3dcc2c01e23bc2f6090f3e8c0ba5f0082341200b1c37b99daa9cb6fec78bce3429aec5badb9fd28fdbdbdc5d53570675a9e39535b4594095658ef950ecd79a162223b60d2eb91765e022dc6e1bbdd86f1bcc280ed9df350da08a801fa16a1bf2701947acfb08f19fdfcaa1d76f466a5de2458a78fb82f6af3e1be68f405a4289f25896f4c9830005c9e895c86e67eceab0ad544856071b8d9585835b5e85a07ab01515f7ab54f98dffb4ca49a15068eefc6a01f7f52fd1adbe3631c59f6f43f79d2b4f2a691e2b30bb1d43a848dc3ee39c7f2e50f0c9deb7ab51e33bf40903ac255bb1510fd61676a6c13c3c776b8aacc6cefb95e24973ebb11192e2692dd0c6a085b58f86e11cc28ee2194988c123e3666da7339c0a4ac6afbacc83f1f100fbb39efff7cc605c9213828224a17c476395aeb9bb0a3150fb8889a8c2a494c8c526203f261642bfa69a94b86de9e6d3d932fe20fffe4bd76d502c0d437a3e1d0d8727b7a8dc0e361967109e93566326b6c517663731c4c9bdd0295d8
+P = 1a4eed4bf5b8d2e2a58f1f1277f164cc32cdadaed848f76fe634034082ff9aa1711870bf3936d01a2aa48de30de5143b9148cf56f4490f9d480dda0b672e8e17a012cd26cec3c68837bd5b2f9beb13e0110f21c6c36343e09e027f39557d1596d4ca406e3e7aa113e9bb8623106bae25f0ea23d46bc29970ba2596f83fe4f73a6f978a4d949fa7c271570a2ae5d2b50792d5ab5c43d455f359fb83c35ca3da37cd73cd66b6adce94d78ecdeabf667daa47ea70799af299e1d898ccf3fca6c42c6fff8cf2ec992f596fed4a0cdb502a00f9b5689302931d15cba691e2f8079a0411332438b714ace5234b91e4aebee8f8dda0e1968c2016fed350430a65d8d206c9436f40b79ce03083b8dc207d6960be1ce97007ed22a388ebb7b3d8f7d2b7d9f8f49731fbcb21e21db0cdd15674c795d5af2b2cd727f83e634e8c47157ed0c6873a5c9419e683f16f4a7827b444967812f9d1adb9201b89a0e66bbcf0591465f5d7036a21cdda0e10099feb819dfc37fdd3105120044dab716882d3971f312e3f4459006fd5a1eab08ff63edf6718f47ddaa37f7f40c9c372995f3aec97bc45e287b64fc8cf5559ab04a4d4d3ed482f5d61d3abd99cc87ee406da3ab9c9cd22ba3b8d191b26754aa94a2412f39e332d77fe72210adb0cbb5c96adebdbde036f1f1aaafad74a7ac2594f81efa734054e2e16dc931d49b970b81756862705fcd4
+
+COUNT = 1
+K = b3fa008b5947ce58dfbd354dd01f2d43
+C = 55cd8e45138f477ce0a84f07bd28a93d7d628bb4860207a2f6dc4256bd79843e32c856a4fa831d1603699d49e6c36291b60aa80635900cc6c78cf0a2ddc457beb41782de0de03f08a064df90b41f2e98ce61185d735380403fe56b68f8343a801a14afb8a7ba79684dc2a585110da83e9a836cae1fd9e1a220dd6dc922b4f02b15ca88d43ab61e1da24a9b3cb99c4e5024ce5667f4841ca2a305b1f4c1ae9fb63d1d4dcb83870755a1a646b16c088e612d82ba2bf0e7e2fa0e8035c3baeb595f1ac9bb49b01f6f71392e217c049c0e9bd794b9aa2383cf59ee0a90f965610c65ecd629a17cba2bdf2458e3a8e1a9d219cb66eb9ec8e5226b34f95003064952523920a0b4e94ec8ecd1bdca8a65fe46ed25fd4d076e46fa62a8cde6eabc593045d17cef996ebbeca4b537f65c4f683a10baeb4c42b9867bbb49ca7ea1c5437bc114948c542cffced9bb1ebe3c946eb24ff55be89be004596ba648b264167217d267b881020b905f508e4f0e1a58eca051d56ff30d91891838c574c3de54e3feafcdf514740ddc94ba92cb85fe86033e67f14d90be7a0222e4bd1624cea8894df66a36a8e848dfe9168d8024b7ba5636afbcf6b945a53e6b2778f229af7dc2e59bebbf8bdbdfde1e21465f6b6344b13afa0e5ceac212b3b88932f21b1ae04268476597c92e64ff7c14b9ef678f10a35b56cd70ba03063f94aed97b0a6cf883d1f07facfa37b6e5b070
+P = a067ab39cede4ac6c6cb7630cba48c52a794ac8ebec037125bcd97d1a3c52a8ed64764899f9035a6944d0605a5d977172a55bbf86cd81aef5d6bafb1ac86bfa65da2b3c39bf5da94a98f7b6dbc5df16a7b38061e0665ad16b20fb6aedc9ce7f6d3497c3c55cea92e6343f21251092ef2ea307b35f999683298098bedaea847d1ccbf8bda18dc477e8d49fee4e357273396ad2245703485b97b5a7d97057bad875a3e76b67ad5adbc6ef3b8ba9a1786aa93149f0f8dd166535acbf93f1b9839754d537da3fae1ab02973427c3f353fe9aa6c5a100bf0e6ccb08dc1fdb0fc363a95c77c5758d440db0a70f0340a4c488de51e1ecb932ce2fcb2c95ea28c9f55695d97ba1765c8f11e523ae3e4e1efceb69000a192c047ab197f4840c664c035064ecc12926fd3bca0527a160b5b5a2bbaf5db11437f2c38a1c7535e87f552b9f04f2fdd309a826e4ec7708217022fb075cdfc6cc23e9301e33068caa69ef746f357b09ccc098443a3a2979a225e70be1e722e8d6fbb57d0dded2456c1d47eeb0af2241f769836026fec8fc51d97c4abbe9710a4aa5b95aaac83bee57e1333fa244ccc971b6260a9be16e31cc2fd283fec1b247a7340d149fe5309acb47c9cdb955b7bcc4df277eaf611e8af281ff0bcd64b4534309282d1b5cb14efa93141869d67ce7e418f06bb4c2feebcb7a1151aea2eb8bc2fc4dcee53de9b2fb1803490caf
+
+COUNT = 2
+K = 4b4c43c9de4fb4a2a7a7adafeabe2dbd
+C = 6e4d08b8124f7d3e23303fac1a842014f95e3d71c438f8f1990307842796dc5e404ad81802e35c183fe000390a12c81ee684c5cf26c1d90e414cfffe6931b0f352936fcf0b31429eb5c7612cc359a15371390e518cf5c6a6bff1bb0348d14e2c39b98c9f30672ed2af1d96296df8b5567db25b9510a2083461810e119735490058ed1b46b7fdfa885041d8749f90a072b43ba49f2f51fbcda0dbf3cf99fca1d8f46330e5f6fe079d6679cfa26214c8831b782aaa023a2e0ea91050d277dab876aa6865f2bb3fc1a4a77db52f6179d5e5325993280948b6b7002b572829641d35ed3d735d8423e5b24673c4570ca25064fc2c2ad4840632536bcfaf2a7a814f3eaed92b4d501bc51c1719a0d8d8f420b66db845682bb41c88038cfedf13417143a3a701b521a9bf0bb639875a728c3b5ce6ca7e7a45bc75285c193902e6b5e7a4c6e720493d3937bf485e587bff894f70fd6165a1d0129cc673a992e0a4f5489d228a066b1df60002ec0521924f8d672cd1452fec927e58e75807b2a390256f920743fa4d0fc8f59f2469a595ef65095ca0c80adfc843e9e69b6d4a3f824af47b2bfbf2a7a6c1b650378f096f6f0bfabc752c8f279d4f45d56d09dce97962c119de3a64d83b93ea55066f24d4238a229ae86e6a7857af1d8aba823370a72fe358046049a84a70213ef31d9e77a722def8e21480e79b71299438070946bd459a7251707446c911e381
+FAIL
+
+COUNT = 3
+K = 96ab719a3d08df2393ebc330e151dab1
+C = d50ae797f6c3418f388a7513d693c6dd665e858767531fbccd3eb1aabe796690ec8fbb757d88b169adf5c136de50ff0f2cfdd8389f812382578aee0b0b61e13c6a2bc500640fe1585f068eee0d1fa3420220e23090e24e3248fe16f4e0c7c0e996a21b4947ddd08fd3ccc1f036651be4f48ee1ffb486cdc05911244480548221d8da1f2bc37dece080e51b2cdd1ddebf37213a4dfa1b252e567243d9cec8c89eb8db544e7c389a2e13f1b91d860df3cbcec3e85c93276c2a9a5fa080efc85e9bad3bfe2d9bb06498dd8b3720456bfabd3c69b345f6954872baa1d43b9f7ceb92ae9ad77b270d0b94c79275a48874dafb136105f5553529687b6aeeaa521790b9376c9f88ace94049235cd52c4387ad210442513dd5e07171519d58b1294fb8ac1f60ac68b8f07b418e1bb0598601ec38b9a9b137dd87d0c8a41089d17ca1c720fd0e7e3b81b85a373753bed0f5e29586f84cb29e1d88c379c965c50f6a803ddfac2e1555beb9c208a3821f53bead8f120f4ef4a1490b730a0b8a2f1869c6b985520d709bdc0e5fce44316b8aa2448a2743761bf77bdbbfdab6a721a8ec79f38f7e7321a80a2cd3a35a912eaac5eace85c4cad3c6685b88be4517cd1c20971b85bd9e8eb6e52869e014831dff7585a163f5a4dbf1d59160104da90a9cfcc8d6a0324942b40fde4319a32442d83ebbf5d7a36e9495be2ffd0e7faec1b66c96f71843750b8a051b7170
+P = 3a3b9e6de537458875e59204ef7565b6dde796e5ab11c83f7a361b8143f0f7a7eadb5b53c6efa6d199f759cad5c029004024eabbaff717bafb95646dc31a8f6063b9f8faaea650dfa8803bfa0c79091f299a55f78611c2e0d015021d6c6d3abf3d85cac306740acc144201516b787421a77c78a566c6eadc88ecdeea4ff861b6db73f7b00f0a8f62faedefa58866fb368424d7267afdf5ff1279916d2f177408d780697e1c45e58a524bb0365858d2b5a42ee2bd9e8904134d04cf071e84db8a31804aa8bebc0b28dd621360385117764178fe74b29da3ac390ac4812fdc7eedf91fce6eaae3d03163435001ce42f55982daeda5cec5deb960b35df231463cbc26267746be628c53b55f4f21ef003816eb7bfc6c710efa03d0994a1b3c8595fc9293a2c101483798034d4ee7e3d5e07bbd897c9de4b8315e53cbd1f81bdecbd59d093c844a0ed1e3e9d238707a7b893ca453745223c67756d9062152b239ceec44c436e0896a59ea9ea8cf79a93b8b759389bb5e73c5f5330e26580d9777817400166d826008be5e8c7184ae2ecf8fb9dba92af3c747c74e1534c05395f9204b5e8481fdcf4dab5ea6224a8e0ee52576d467d930c0899d31a4e288e3eecb8cb7a3be3a66c79ae93033de5d0d422a6d54ab002d1a82f3f60db97834d9fa3782dd64cbec8ddcac2216a393dc263cea2705fd072ec82dfa1ddef9c588c49f17c275
+
+COUNT = 4
+K = d8c221e426109cb5911d7d6f0836f4cd
+C = d853d57eaacac8096346564eccd33281ee864fb290ce91eb717fa153ca00064e033635178c59860a567215b7320fab4a72ccf716501dbc9a44d5b3d501729674987d2413cad79dece055a9b0d47ec980331f4a236b31984f5d62f9d7f58c0f3afb81fe60f266652da65d06874334be065f56096e98536bd1f2120313b0905ffe2f2c3b6de265ab7800c42be810bed18548c08f9193b02a3981a922b32b618fd9a978439ea382bf2890ad1f30d115b2319276289cf4f7a9917b0c064180e79c8644f9ac880a793b4a8ee424dff32cf2b6ca46f52ff8bd8359ed18ea8aac23e63ae337f5baea9e2ff845a5fdc0b79d5767d47c2a1a536d889f553c52696cbe91ccd2ec671a0644689bdb0f4db7e5d58c854eb539b6b4cd9214e361a216d315b1b124b43c76c703c01d3bc3142f760a399ba4887a6e326a58ecaf56fd49ae128a86cda485eedc3da80b75b171e77cade00c903c1f216eefa845dfaef660fc5ecb6791ed53765683f44da6c4ed8a9ad9e995f7d920cdee8463e79b18e8874b0a2f573b1825f8a480b1ed1245c81f4ec097bf0a0504aab9bdaef27b67d98805a7ec687c8cbcbc92ad3ce58651162a1f57f8af427ea0a111dbd6e3c7f240eb6b2360650a72b9c1c4417b1d541dfcc2a8d6ce3e8c160d8d417e4efbdce809bfe30802696bfd52a0f40be4db9be247dfd867179d82390b55180ebc6ceba0a990e3f6d32eef9dfdb946706371
+P = 92814e18dbe6e83714c4a82ba3ac3baf682a8054eb36666c9546db040d40b8613fc560d97b05265fb19ade180efeb55dfce2cc5981ca222e66b547b78a42401710535b1181674fdd426cf2b0b55e5b7f0505f11307120d495cfb197a3de00569b3d39f93c27270df4725243d314a026549692b0e2b4079c60a8053f0f36e83aaa3494307b175fd40643c1bc264eed1c00f8b565f2a3aeeb78bd94970bd9267d21f5a9a1b07df09ed44a3bd4255a139a328235b921833f92904a74ec202b0eac65df1caed05dc84e52b06c1ecf0f7914324ac4d828b7de7189705308959be42401948e3bf4bdd50ce24101c67ae745a73a67d7e366b6b432ce67b05cefd149a17247010f42dae4de1b2ca42a8e71824cd32c5cb2e2055443ec3ff24339c774dc9207744b84e9203fda1f85595f961987d847ed33867f1ddfce0795e3f2d78c5d749a488a4997392b8c9022c810197c93186894faa55cb0b6775b57a7ba2729c617c9430a44098d5081e3c5c4908ddd1a475cf9211408a8ddbe19ee527ddb2596456e1b1481a09b04e091b1c14b7b2e41bb4434a906736e115cb25ada0950ac5d2845b4a9f1e95f4d80f64440e983324c3aa9f3ec8964f9da0d26aa47e86355aa80ad99d0e573fa9932da70bd65cb1a06d8cb77e455fe7cada4561e027ca1608132c2605b6d0489bba6b29f293951883c451f37bd545f6605364ddc75918df097e
+
+COUNT = 5
+K = 704eb91dd5ba3d85279cf47c01eca2a5
+C = 51c71fd7778aa3648f3e31e1db0c73cb1479372f2e35f65f00188f08f794993a2ee2bb7e91cd1a2b86e92b8ccca7277207fb525ab17600173fa28844ae27f093e0e5ae00585cc714dac90cbe9b6332cbe4cb689b2cd141c102c6881f5b71ec477c5f4a91f7bdcb5871aadd478f1a9ccc6e069b7283f4d70b26e8748eda6d443ab13804c543a44fe2fb366f90de35d83fbf6354a9a9ab4a93ff7d61cbc0bfe05d6102c9c393273e7d3a04d61eba771f05cee29e5dacb7abf34ec9159e121841e2c39848f604c8f743313cbdca828bfa4635a81136e7a37f230c0d3c814d35c2eaabdd94183312909ab3a09b87cce0c719408f837bf24bfb2dad87630aabc9eab35bdb9cc536198389aceec68e8779f9e1eee84392189823a68195b75bbb6d33addf580564e696a362928e2ac506b79480600bc2f9eaa3e96f323390d1d92cf3c6d4bd4147ada5634cff2bf2d97b259904a335eaf11ec3fc84dcd8e27f7538e0fac1dbe7cb4533f4fa58913535d957b90678fac58aa96694a8047ac774afe488ab429c6807e709351f8159dcfbf83b865aeeb26722ef64a537ce932b2cfa6d53ed6cc1ca8ab58748c06a753515fffc56e294f51ab257585b610d261c6fe12def38a1b5dceaa4681569124c679b20984ed2967740419b342e9010eabd291de026f6e829e4dba5300cb668191358ab58e178c29a0194a639233f9c28c50a609bc42f8fa6cd17bc58eccd
+FAIL
diff --git a/src/test/crypto/KWP_AD_256.txt b/src/test/crypto/KWP_AD_256.txt
new file mode 100644
index 0000000000..6d28f39705
--- /dev/null
+++ b/src/test/crypto/KWP_AD_256.txt
@@ -0,0 +1,35 @@
+# CAVS 21.4
+# 'NIST SP 800-38F KWP-AD with AES-256 cipher function' information for test-files
+# Seed = bf1ba8f321ce0abadb40026686c9d9e7f0c4a55388f2ea7cefc81aeb0e054d40c94f48093f2739580010d6ad6e6ce734f21e7338100b750ec9c7bb06bf46f7f4
+# Generated on Fri Apr 6 14:47:06 2018
+[PLAINTEXT LENGTH = 4096]
+
+COUNT = 0
+K = 08f5c088acec18e6cf1f03a8f85d772e327e7fb07f8c2939eb554e84c42ab93d
+C = dff30fd43647d4be54cf2dfd6187e2ddffb55267313f980fb09c833a9c2bfa558a95861711f0acb2a5c7e731ba22f24a9c4dfdd9e9b0216e9088f817a175b9835b0e17615687a20f68c067205626494cd04fbabc0b3eea7c0a4cd6236bc8b3e52e721dfc357fb8a3722bfcc4c690d8f63dbb864bb6e3a15805aea7270f8eb748deebaa2d066fcda11c2e67221f9a91d2c29a6c79ffae76aa80a2590b4f9e35f623fbf2f8ceb2a205493077556a186e25e5bd52dcff7bcc6909b37a66c1d1431be1b363bb40da25386eaaf5fcabc7be6422a04434a21d1d3105328e7c56770b9f59b03395e4138f5f06fc7e6b80dab87b08caa7bfffc45a095c15263efd3f06c651ded6f58074efc20620d704997fc84721a0a8e9e5b9f5cd330bbb156b31d9d1b1c260e4a24535f30404dc5b2dd6b35d916a1391b25a7d8790be09d85483ed1522074a2785812005bda10dd55acb245b3bd3d9bb777dd23f9b02538ba1a114ba53386d7ca4d9524b2f8a18e0ffb21580b560540bb2146f08f04974b90eb324547d56222df95f44bc6e5f183bef283e4816fb1b2933f9c7c6726a245a495e304d8318d0008c51b0be8090f8f668fbc3f31e073be4b9e97468f4dd8c798e9d682868df493db8a85738b58cfd005190f365849072577772672c6f82555c65046eb34e86fe61103327a063bacbbe33cea7eaa3d1de45471b7269e1b6b38608626e323447a3d5fe0599a6
+P = 8b68f66a3d2f59d419851b94d9a6f2f0e667f8125e11d463a6bc2cea46b12dcc40ce8018b204972c735fdd6d2d05b628f4905c6690f5ac5b1b51e12f3af2dc3ae9b9dab616f0a2a66a1ac197592fd5b15900547f32f54110b58d51a0340aa80e9eeb7b2e0eb97e80aa22ba918f2fe1c678c730ed5c3d8d24774f17d8ab6e01a06243d36e764df1dbb8af1faadbc55281f0242abd7a162c984fd0b05ab8b0bcaedffb2962024f009a8d7c9e71281c09f52ec0707ee3bbeb1ecb918be6ae3e9c1fabbcd3512af928db3ba6c109ff9e9839a616b2a53f092160a48222b84d53cd52490515ef93e1ebb33897263492ab8ec6fad2e633276ae367f76d7f926309478c0205d4f22506a451795dc98f5410d8f5d3e049cbedf381620861e7b4ae08f2d8a71abc1f230248cb636a2d7b4e7717ab2b7b5f2dc6e5b5a18e8043254208b50fd6f8929eaf974c48551233661ad67321b64d69245d536d9a8ca2a6a10966dddb9d2ce36641c9281c460ae524b077867258f638e6ac872cb5f5c6fb216b1ae60a9d0c5ea0dbcd060f255da26111175af4e9935df59ddade6a2a70cddff8cae6a98e4f3843c2dd59d09053b07b648a46f5de0eb21ebb192828279a386ea3eedf2cdc355d73d51111e8c1d522e059752bc56226a4225bcab713bfaaaec78167d7cfd33e913b26fda93ca7524aa8a8b17977c88ff9bc23ea810b4de59eac18d1523b
+
+COUNT = 1
+K = 94c4d5d70f881e58e10e7246cf812d40e2be258adb2b6c13c6603fc7daf7e85a
+C = 6c07b5ffd1b9be182413ef8eae4a6eac657108a46008a0d898727f2711e6fa0ca60fd1d51fad683b57d4202fa2b0eb88b856e08b07155439bdb03890cbb7e0f228172bf297a4e0917dadaa5e89a287bb9ba6441c852c5b0cff5084e6c425aaf866815b3fc45f5f7fb5d14b270343e6a30f402e11d62e433a0d84f65684b2df78d4e7758bc0bf81783316905cdc3c1150ec47f225c966f7f339b2538970eb3b8a2c13f95df1310d6e3b2a1f8aed19105846557d8f0018fc0f17146bf836b654dec98e9ad639c7e4b2f922b4396e82c690cdecb65f5e0ea282dd6262f34346ff9adbc8b2f361ddd4356f0feadf7c750fc0580c4e12c00ee049d06eed2242b14727ef4d58386dc8df279a7bf8131c3befaea2f059ab757826e5e381d49a2f11b8cbc2b0021af4da7a779e5df0083edeb54348cc36ce96a19a3d7ff5bd2f19d05fef6b200e76399a02a991111832173353bff4ce1859ff534ae13290dd176ba8e1384ed24d9702dbff127e15e5c66618f94680271732d19f64552ed03df76dc9d46c3cfdd53a1b253992fbcbea6db006f16e8dd92406f0090ad9100856c6b71f7767fcb895136416b374285efe1c6506941911a380e2bf74ffd0f67e853f9ac7b5df6666b177a2908fda9add0eb798f8ccc52801535b2bdf9507f3fb3b46915aa889d62ac5909040a1a28856105dfe2e10d5cfbb569c380551fc8bbe7d83dc87ef7a92faa3fff4b1e2c2
+P = 85693a16ae69d751cfa6799b95a6396de2eabe7e4da74d734691d992cba353a39f3b9615c1325db5b0563ce1a846bb0f0534a86130ce6657736b9a9b35b0f8d89dd1b3a295131d2f3f57f94deef9606dad76a377d0b24e632b3680e4d3338f3e4484609e8063e9ec621297f55802d7c347e8085ba6e514884b8fc1ae109409c5c3a5bddf4daf034d300e31eccba07a9380f5325666c4a3aa12d60b30ca272fe03534aabe78ba0452a7e4648ebfd4645675629676be6f122a54b6b810cf9cc0c68b7c61470a537a5a664ec24dbb3eb4f9fa8355cc7ae8fef27a0146df5ccc585d8c106a1eeb64ad4c701fd5a54ef18295b07e9e47f7f7dd2f67d38ed776a5f0b28843cc4bb5d7fdbea9cb0088dee849ae232e4e016d8cf3681971e8a45d6b25451538212b91f30e17580a8107a7a95587a06d22d615f5475a5f616fdf2fab79152f2643054d96ba88f50888eb0f2f1f154c6fff53dd44c3613269751dca4fa86f45d6b1af9ad0159685223889529609e7003c8f3cab491fd6c1a020305da8f94ec833d721d9fac7e575c2a1bc26eb4fb5010c35ffbd39b98d857f12584f4ab7de92aa6d7e7148a0120cc6b3f7ae47a291ba1cf55a28d38d3a30dfc3917d663458cf840385ca81cf70acce45a5cd509f8387d450bbdd6fa51830cf9a7387887c620b86809c55a3eb322ca784a51693f1054759804314ae86048f0d9c99650a5a12
+
+COUNT = 2
+K = d65338fd3771fd58c07b6b689577378939d439628529b92cd5625edd18afac76
+C = 1c429bf25c144a2cc649fbd60de5c26c31a0c352de99b34b86101c551994f082feffe1db8853de59b3e8593785eca100a71c5392f0c71eca9f411cbd87fc77ea1a96376dc13f6ad460a11e9cd5a829875a7b7dcd2ba4eaac08c5bb48ab5d4c338a6f8bc5e760739edcff2db116b5b1802e35f936d473db168edd12532a992bcc418a759cc9fd3f97f561623078af29d7ab489b7ec564ba981c188f11240dd9354c324f8d0cdf1c74252f0fc75e390e837b8be90a670f5803ee53eb75c3ce95b2853b2342e54f86dd9aeb308eb82ceb2bae7b3e0b364d17105eb61b3843f7206bdb6abb818efe0f0d3b1004e370191e8218cca14947aa8070f7c66fd0422b02ab4a1d94fa46197acd24e272c765667353e819588402feb85d7f00243521d0e7a9d9e70753d8b51d374ea9c8355536594bf05a6960ca7176a4b66086b055b099e315a23e042a7e0807316d7a11a657a6dc9043806e248a9af06570f710af65267d436a5fcb001104fe8a7c564afe075d85bc0a2ce3d33d8d93d5ab1e923f51d4ef26cbb6fd4a935a97cb115aed678e75d5d67fbfcd2362cb3d74ed6b9b9fb0cf82569a474a25e5aa39d22fe5cd301045203d9f93cf5c9e9e9451f1bf3566eec75fbd995cf8c640aa68fb04f5419344057fd1c0e655d750a68c523b0fab24cab03d7393ee3a5735039daed52895dfe7937f55d7ae9a8c0256e9d638a8598452f5329353a20c4bd9958c
+P = 58b20979cba48a9dc95a8857f5bce433087ff93470fc62546e86e72dfaaf7b233ffe428802390c1db7cba00b1f23678aace4a16a237b41d26bcd83d471030929a34e8467f85eaef070b9b74a57f13e91b4e95a3c0b8dfea87d026196a10168c152c4ac42718989003b7e688ca43207034b674d3cbab6f57db6513f8883d27f2280c742896a62e7d0f3f20377e98a0688652d270887fdacb86daa086ffda17937e6d20e4a82667f80ac7749a889b0d748e906d653f569b86de2b42b5819ade9c92970d4caeeed8cd5759d56fd38205215bd8401b2a5a000990afe6c9bea8d091171e85ed83f45bb5b9a8d74cae897cc36f1eaf0122693990b1fb57d0025ad6d92c90885accb649368fe237c4cf017787609fb93c9ea5b413847a9fcf2d2ccb6283345a278619abf8dc351682928187bf92551a820939ec73928eb9930c48f7088ed0a367882f4a8b20d754c5f06bc82990da02227923eb8d1cb73c23793ea0d19bed4a9986f0d48d7835733d1ed3396ec3cf15e1854473b05535261251f4f0af8a0743b3298888bec2f7656493d05eb2d9b848e6802845fb9f7835b50d6a0f0e6cfdaf9b1afc6caa6573b3350256e6f23cc4681316705e33eb0a5f664b79be556cb1bbdd0208430cdc95a35f61facbe7ca2a9bd329e4a1fa42aab9bb02f6519a5672346a4cfac1b96a969317480dd995e339af888fc0e43692332d583fec6215d
+
+COUNT = 3
+K = 1726706350c11e6883955f24ea11ab247ce3b2ab54d05e67ad9770b5564483dd
+C = b006f26a67d0e1e2cbeb5c23b6b300adc1526d1f17bbe964fe8237ae244878158e6b04cb488786b5258ac973c3a2eafd7fcf3a7ca6c825155659fbc53d112bc78b3a770cf059fdd5e68f2b4bfa36de3721231102e5041c947fba3d906bff39592ec3901a398da23035f1190e99b58659330cc2e856ee87ad4197dcc7d16e1f062275bced1ed5cd82163ae3e58da7368dc2aadac855385bd4fa0b8baadef608d0a5c27172d12b88c70b136eeccf37f36364361a990dc50815743cab1636e661bff04ca8345520c30b935a060b450526b1d6ac09170e5b0a327b88f42327b85c9a621d2ca745963c2815a2bfcf509d50b6058ed6e67f369b5608d2aa885238b67d1b8e0d83f9464aa473bf109350fcc02e360c2619236cbfbf895b607895530d8d3d2e41450750dad05b1c37ef15db7fb4707597ac252e8e58d4c1ab2713b427643d198164c908b5d8ff36e9700157284009c7b283633d8b27b378bb65eff8aa59b5fe5e6437a1d53a99c106c2c4d033d3d23950e313a10eb31d68524ae9f8e4f56437acf66db3e8f77407a15bbff4b393e5559908993146d93c673d2aeb7d4cb8fc8d0169de7ed6e2bbe6ce9958a0f5d201419e7acb17e47da827ba380d6b3ad3b5a8c2101c5fb501110c727169065f23297947f538ab3ec165d61edc1f6a9e1735e9b7fc06d4d3406cf8f9c6a68b196cf262324a986705fbc802cdd2e6b4ebcf68e6bb9e793ae644
+FAIL
+
+COUNT = 4
+K = 32e57ccfe7563dc0a20c14ee450837a33606c086ce1467fd7ec58467154338ab
+C = 977d9c5f6861a69e13cd854299434e348cd0690b4d04e08e0598b47eea621bcd8a22838dc9c35a72c35fb1a6434718d02fd24cb4b3dd90b0430334a938a218467eeb4c373d446a539810bc3ce1e923b7c20d9f58ea931d4f964c79613bce67b268efc44bdb9bb00a68d60037949aec7a399493defb2a466e33d4831efd63ad1cb89e00b530626d2f0165975ddfc4cc5e0f968d3875de0f674b3a517df26480b02b6236ebb377118268cebb30ff1ddf0e280fe1bff61902a017e8decf60753c642f35faf0565303bfe651ec8f0193cf34d4af010c9925b8871f0f8c934a149d874a3b659f78ad148428aacaeab80b1b25dec8b0f7ce54406287bc802ac2c0cca3db4adcaa8400a8636ea339b62f5e94d5e32fd3d1183b374507a2af620ca1346dccc9f83a4fe855b1c0e91db9e7c532828d0944d9a81b553ebdf35e24119ed8164bd0260627ea011e93bc103f208c76498ddb8bca15fd05324da5473157feedd54592aacaee68852968eb54c69eb1ddf607917c57493ea380de0cc6ae304dc49cab80a31b8b456986dc367c70f144e52dd604c8d5edbce5de5efb30d9470bc883445b34fa4414f44bb94a64362a12b546665721fa6db82f0c947f015978412b2ce136c471c98b1f908315a16c83e9318e64508c7e179a4429195a9b1ccc211a1c1d4e4df15c5ebc7ab90926fcf7da03657159e440e93adea31ee35f72f2399f5fe2f8c560c8826e23
+P = 80de48ff805a88f3b359451bb6df61def9cb3551e64fdd3a3a70a3b6d238a69311a85bc5924e395ce92ef394b1e5dc301233e9a212f7fb86272c42ddf5f4857c38d0dd259dc1d663c0d729e033d9b0f7f01ab1f8f1b7192d40921ee0d4696a3e35663c5ffcff5ed167660bf6b4c00e619512a2e827be33c90eecc539e18acc8c76eb332b28b1cc502af571242342f63d155271da3211352128aa0af70c9ce78dfdf084a13049b7bb6f2bd10dd385b412d60bf1ccc9fae1208f39dc53db471a04d0dcc3703b4f7b95e72ea815b64a1499865a7ccc5b740999e76338e1b251c740d75274150a96def8760a08c5a8a6f58273b079c06ee09f79a976eaacc8a04c365bc61a786b496811121c386d274c413a2fbfae9464db6ea775233193395740fc9a5eca1a3820d33f6f7b38a83ccbacaac16479225e108acdf46ca35e573151963721b73b3e1c9a12effff0c3a622eb9f07bef7ae712c96ee3ba245597fb8d511698d6e819a967e0d1868c0c6055333b7c13a98cf63d6a5d87779a95345ca8b7e9e597ec588e96f8fc2a7f0a0b8f1543d9e362a911dfb1f03132a4e6af71b503c41814d6b684a26b8df00cdc657ae129a1f2a18cf4b78a3981de68296b1268609fe3ecb9928b90df4553be37319fc508096fa54b35e4822328569da60a6c660f30c61f02f4c5ab2527cf36cb7da8d7dade4c714ea3fc2da8f65b4199090e114dd
+
+COUNT = 5
+K = e3982db2032f2b4ce658fc44b76f5964c45cd31bf803708982ae599186fc3765
+C = 975e49a4b9a770957a1bb2be920a4f39b9cfd69ba46983d2473d631c08132b9bf61c44510b8aa8bd48c70a86276aa1149d8fdefad511d15d2e2037d9e920e640cb71a97663d19eb90d0b74d9764d03e17cda87ebec6e35ac2003cb75bf9192920d910188d78e2e664255fdf6c9190319d34adb858162ff0830f37fe1dd44003d3d5a1f9451949e368f46ad1977ce622daadf8483a1f60359992b9b366e8a81ffbe96cee45d3aef2fd0ad8c17cc34927af77a0d6d0c5deef3b4a25c82ec388667a493bb0599ac492b351246cbad6d283bf1820883afa48bd909eb7304b9fc5b7d960344309133aab7a85c49f7de396926f50bc83c95900cd049eac1b387aae7fcba5345496425f9216e1fd15c20da75fbb26da176149b40a701e15a7bacfe899e3ecc534ab8bc5b7bd081fb825b5f40fa57e363d7bce40020e73f638acfa097b89c50cb9edb0bd6d71d429b8003aa5dcb7d61792eb3bcac795954c625a104209b373c28cf02038c3318916edd2b818e6719ec154cfa56afb2f337d333069f915d0d35edd6c278fae23c4440c40be462a1dcab23758e4a7fbe8436493f58e890092ea71cb8bcf1336e9ee16b852ccc488f21682dc9f02bdf6c56fe8ad04d84a3c69d8d06dee3d126c0a75f142d0c90c256139acd4b719573e588b80b4540024a05a35044cf58d89673923a534c3816492e62379797cd6e6a7464da5eaaf11ee7b9c27b9b03d7b53c03
+P = 43d38ca132545b154995cff07082611cc47a6467a980654d2d1f1ccfb3bcd387e9d7ffa281b0e0b00dc8669207e0d8033e9e36613c98978f8644bb7e505fbf491dcefbe19589254c8abf859dd65cb94dfc99e7b9d3d1f0a31f21285963e1f7b45c7490a522ff887786f7940fb6192f5081ce7181944bdca5c5bfcf2589f9173a682b78fcdf971dd9f4e8529033e15cde560984dcf796914206973dcc13f8c9a24b25dd00c11166ec6ecf33c6ad9b487847abd7bd29b4f3b9c8dc93a6a5a31723dc03245884bfadc12b2fddcc82409d7b14660af808d4e8216157bb6ba03a319193ad4dacbd37ac884550962a4de26ae923f8d74f2f694fcd0aa74f2e809da4689aad9f2820684b3b423ec4a7da0ce4a1b599fc21bd2779653283b0ee81d7b0d9fd3f6d1e75bd71af9620630aa87b73f7b12e68ddbdfa02ae86ae06b0b1aee4a997d34f61b466348b92e36f83652763084a215c47dcf689df17e36b64bae3ca1a2cc22c837b5907236833c2c1e5f3ddb74165fb6f0633990122cbe4af8b5920b1bb6961cdb144ea8d7b245d0128ab76f4fc0189ba97385717e89e0f99c962ee8c2b6e55546a18be0ba3dbebf7e4140eed6aa3558c43115b65b6f6e8e8fb4b9cfbe0b6eac006603667b28cefb4dec037f33568a3c94d9e36539e91b3199d728521a9a6b82b96ff1c29dd1d10366d0510f1b9a9494cd104db2390530be3fb6abdb7
diff --git a/src/test/crypto/KWP_AE_128.txt b/src/test/crypto/KWP_AE_128.txt
new file mode 100644
index 0000000000..a682d4bb32
--- /dev/null
+++ b/src/test/crypto/KWP_AE_128.txt
@@ -0,0 +1,35 @@
+# CAVS 21.4
+# 'NIST SP 800-38F KWP-AE with AES-128 cipher function' information for test-files
+# Seed = 7c0f161159cc9e338308ddc59a655450a030832b3d4576c568d63725d57b020073aee9c77d9fec34eab358b83bf5aae4fb52803343bc03d472283edbebbcf75c
+# Generated on Fri Apr 6 14:46:12 2018
+[PLAINTEXT LENGTH = 4096]
+
+COUNT = 0
+K = 0e54956a24c7d4a343f90269fb18a17f
+P = 817ddabdc5d215eee233adff97e92193c6beec52a71340477f70243a794ce954af51e356c9940e4ab198f0e68c543355f65ad179cb2d60dd369eaeb9ed141fb18c9e4054ac7fdc83506896990a4d20833d2d6e9a34938796ee67c9d7d23058544a4a35f2954103ce443a95a7e785602075ca0a73da37899e4568106bb2dbf1f901377d4d3380c70fa5175ebc550481ac6f15986a4407fde5c23ff317e37544c0a25f87117506597db5bb79850c86247b73a5d0090417d63e4c257ea0220c2c04db07a34f0ab7954e1dfa2007a1466795c4d0c2aa09ca3986c028185b43a466526594afc9c891c263a7c608304bc1957c9873f544dc71e6f847c48d32026ed03b2333825452ee7e12a50e1cd7d678319264c65f78001996d37fae7f9861fbd21cb506c2f8a3b0ee53c7debe17111b6e3f78a5c5677857b082c2c4943dfd1edf6337fea98a44fc25928361156ef38d865948b979cf6f4b46bd2119f12f0891cef7fc9d0638fd105fc05f9968d16948d1cb820751e82e44cb68e99d4f072ffd1577da6c0631b5827bec7e1b9ec72d18b74cf5f233e85013c1668ceb5d7a1f5e0f016b0ff726a0a9d41e2cea8e14a2f56492b14606d3fafd8ac141335f39f90d56863735628e8f17be90e100ef0785f3cd57db8b9d89a6b2189dc2ea00c285d2657983f8bd7883c215477e67a55556401f1d8b27d4e0d541c7fb7ace370c2e428884
+C = 876f3e53ba9cf4f6a521ac198bc813d0ede0f862ab6082e3e0a06ad82b4f279582f7c43bb63574608446bc2a05f401a68f74086cf2776b4b3df6b3679c2edfb91c024db54c6831e0752ae6f86c7596462de905ee0be908c1b9d043ecafe2ad1cbddb904e18ebc9b7a107031be3a87059516a3d1257812d9c801b0b9f21539e70c47150c128d87c5e58fa6e4371aedde69c7b5cd16b73ac422676328131f3ac48c602bb6e0741805aad9d23b33b3523b86cf0588cdf9dc6c4d5f9fa43d88ca17976eaf48fb37a41a598266da04144373df5631cc5126341c200a0c8499b29ae96e6e6e6c2bdf8d8903da62bf8ddae970569b695240e77f8ac5b191da5034008b6ef21936858e69bac372bbafd8794f6b03711503c1875528a9348681844edb199a0664d740f0f0b1f866c4248c80fe8b5700a3c4134cdddb17676e0cd37d6d81831a0f4adfba071bb0935502480eccd48b28be5954ea6c7d873b51b8bd2b709c5b6132ed31296510915073c18f7012f0eff6a9aad5340a19fd5e372d35260b718d9e4807b1954c24e6a4fd48e4dbb8f395474e99ab577367d2ab5ccaa18c947331047dc3986e213a878b41089aa221019dad4191a4feefd095f8606c2700a46d71cbb13efb6957df925ec26071c04d04d5a94e138e5fc5d1f059236aad76208077dcc607b1dd2086f9c04e33f955822b457eecd68bd5f24836ecedbac675e6ed93d8a787cb57ad68e
+
+COUNT = 1
+K = 6b8ba9cc9b31068ba175abfcc60c1338
+P = 8af887c58dfbc38ee0423eefcc0e032dcc79dd116638ca65ad75dca2a2459f13934dbe61a62cb26d8bbddbabf9bf52bbe137ef1d3e30eacf0fe456ec808d6798dc29fe54fa1f784aa3c11cf39405009581d3f1d596843813a6685e503fac8535e0c06ecca8561b6a1f22c578eefb691912be2e1667946101ae8c3501e6c66eb17e14f2608c9ce6fbab4a1597ed49ccb3930b1060f98c97d8dc4ce81e35279c4d30d1bf86c9b919a3ce4f0109e77929e58c4c3aeb5de1ec5e0afa38ae896df9121c72c255141f2f5c9a51be5072547cf8a3b067404e62f9615a02479cf8c202e7feb2e258314e0ebe62878a5c4ecd4e9df7dab2e1fa9a7b532c2169acedb7998d5cd8a7118848ce7ee9fb2f68e28c2b279ddc064db70ad73c6dbe10c5e1c56a709c1407f93a727cce1075103a4009ae2f7731b7d71756eee119b828ef4ed61eff164935532a94fa8fe62dc2e22cf20f168ae65f4b6785286c253f365f29453a479dc2824b8bdabd962da3b76ae9c8a720155e158fe389c8cc7fa6ad522c951b5c236bf964b5b1bfb098a39835759b95404b72b17f7dbcda936177ae059269f41ecdac81a49f5bbfd2e801392a043ef06873550a67fcbc039f0b5d30ce490baa979dbbaf9e53d45d7e2dff26b2f7e6628ded694217a39f454b288e7906b79faf4a407a7d207646f93096a157f0d1dca05a7f92e318fc1ff62ce2de7f129b187053
+C = aea19443d7f8ad7d4501c1ecadc6b5e3f1c23c29eca608905f9cabdd46e34a55e1f7ac8308e75c903675982bda99173a2ba57d2ccf2e01a02589f89dfd4b3c7fd229ec91c9d0c46ea5dee3c048cd4611bfeadc9bf26daa1e02cb72e222cf3dab120dd1e8c2dd9bd58bbefa5d14526abd1e8d2170a6ba8283c243ec2fd5ef07030b1ef5f69f9620e4b17a3639341005887b9ffc793533594703e5dcae67bd0ce7a3c98ca65815a4d067f27e6e66d6636cebb789732566a52ac3970e14c37310dc2fcee0e739a16291029fd2b4d534e30445474b26711a8b3e1ee3cc88b09e8b1745b6cc0f067624ecb232db750b01fe5457fdea77b251b10fe95d3eeedb083bdf109c41dba26cc9654f787bf95735ff07070b175cea8b62302e6087b91a0415474605691099f1a9e2b626c4b3bb7aeb8ead9922bc3617cb427c669b88be5f98aea7edb8b0063bec80af4c081f89778d7c7242ddae88e8d3aff1f80e575e1aab4a5d115bc27636fd14d19bc59433f697635ecd870d17e7f5b004dee4001cddc34ab6e377eeb3fb08e9476970765105d93e4558fe3d4fc6fe053aab9c6cf032f1116e70c2d65f7c8cdeb6ad63ac4291f93d467ebbb29ead265c05ac684d20a6bef09b71830f717e08bcb4f9d3773bec928f66eeb64dc451e958e357ebbfef5a342df28707ac4b8e3e8c854e8d691cb92e87c0d57558e44cd754424865c229c9e1abb28e003b6819400b
+
+COUNT = 2
+K = 5f0f2428ad83ea04642d8f90e31f00ad
+P = 05f65580a602fc839c5372778bc495219ab159935acf3caed10958ca0e135cbc44a896aa84b1f6b05d65ac45f5930029a223977ca46c7717809e80d8eb91e7ccbc525e69ca0a85d3e45b2f9bcf0c86980b7f9f9eb20e9b558479aa8c814e6113635a9b8b0e264055da68198303ae91683eb8ed3351dea3b8280a504d163e1cf284834e3641a26868e23fcf460d279d3577115d8c4cf0f54374df26ddfbf0c1df5bf71fe509231177ac531514ed878d3ef44a03685dee906ab394bfb69cb56ff973f19fb466ce550ae7b03008feca7046bae160ad47f28c9004da68e594bcd7f9ca8674a645c50768b5ee0ec2088f66602a429f95492e09dc567c11083eb18da97d85968a86f6ad05054f4a9c4bcd57ece2511bdb38afd78084b0a088112b81b993a4d37674d97e8081795ee055d9c37d2907d859be201e73580ebd439620918e7c4b42243f43a59b475095205c40a1d37c0f87a5df19450ccd5f274f186b0f45f7bc153d30ce3c873d7bba5dea5a8bd348b2f0b4f25461927642a2991c3c4bc3ad34d09c104322bf27e8eaa0ca99c0dd47a3c4c618af02599192e362e75de53db336a68b223567bcddcd7feecde0aba91a1eac0ce6327d3c9c5be36698494b2b67ec0efd36b744d794f521bee251b31dd29852a94e6b95f143a53f93f85ec8c2e86e1026ec949aaf97fc9b8d728c13e1deab628e9624ca966992efae61dcd6e5
+C = efa9d549d0e975789a6c84b99991c98cf20db4de6789be6f9f7ae109e322a6f04a1a1c1fe582e4242a5e2f2b390827d7e52f1bd97a4e745920dda69a25fe93fa5de82c09dce8a2593106667820dfa1f55c8a1152ee02ed0c607ab986e5209859cbb36baf90ca63285c3128fdceb18e5145b9d3efe6bc7b8bac7837637ce5553d634b13fa3888201112248bc31e0785f2dbae1a8eaa7f4ebc0b00d21242f03acd4e4b0f4ad5e9bb1e25301a331a4b5535c1253b9ccc7800cb2e80602ec9ef858865a39e0ba28006da93f9c26c1fb9ee8258dfcd7e79f02ab0d9c2df9ade309fa6c604fcbeb38641b2968ff96349bcc34927cc479c30153125f7ad07068e5a4ffaba903d27488c94bfb694d8277251061a7dcc5b5fbf47ea88065ea1df007cbd4c1bdbc2e3609d53bf99bf2c7de8ef457fd7b85882210fb52c5093415dd2b37084b75f83d2a05a669867143fd95705f241fa794526bc65c5f6422410da4783340497bc54852125e3da579bfabc9ba98ec398487444b3da454f40d8b59d384cdd3454ad77f38d4ce657d10b4708dd4ecb0fcc3238aca57258e082e8cb86d802b9722388284f232811ec35b49ebba151d5b0771feeb6120d8bee8d55fa9c4de45d8f950752b3b9c6526fc4fa8fdd02c645fb053cef8293437111c85b07e98b3b38ccd3de93cd6fd08fdfb8a3a94b619f9acd226be3bd3ff906f14b1dda93cccae783c04ed545db685bf0
+
+COUNT = 3
+K = 9b5c2e4a9e53d3e5f5670af2440439db
+P = c4c726199cc27b81c90ee59f1f44420abad2a4fe0f24647952116ec46f364a3dad1b48ba5bc6c48c056e34764ed4b4a2da04e7e45ff477df8364ee9f116c82dcff7e9ae2db6a7722f2261b6cee1ef7cc66820184988e002886a6b636264e7704383543817e119ba3582d42aa00aa78dcf84d0ac92ff69f09046f89e7b608143699f9fda37b07d3e2949e935bba230e0a42c2d41d285042feaa480bcc1ef8236d615f449107fad97b5be514ae39d586b98ed45cc39784c30a3a256d62336002b1f7f6cedc3e5418d6f444bacd74d7bc89b3ec65c52191b4a2f3b3c7b682f254e6e909c5dcbc11d5def56792f5d2183e5e7119f268b0395e7f4f25b8574bc921da8419af30113bd35b11c21debb979fb698752264595777be7665419052cf3265117bce96b61f27da4072f828bd9315ef25a9ab9eb4f5319907cdbce6104054c8563a997cf477523478e6d23527f91997dd43ebaf629003530de4b8406bdbfae49c0d97920b745c78a665c438ff064d24803382f79fcc2e24265adaf707394b832685def125b69b1787719aaf92e518645ac22dd94d0dcb03f052c87954854d177e2caa6ab5d9cf3dc4ed4ceca90d2fa0b235efd1b7d17ba9758987d127b11204047f22e75158720feec549f29b67a8158e577325e178d6b757770d56a30b85d12ab5fb164a07a5f613b53d075c44cd3c1e3a311515aeed2427f70b3bc018dd796
+C = 09540dbbe97e983a3420ec1ef7d1195f9cd32acc93e9f31d07e68dd9d963f32bb23565a37d97e857e5e40eae27946ef83a4da10ec29ea001fbe892133a4814b04aa3ebabfa295a19429c7e6b263293d07a48f5b2beea2930aabe6268aec00a8fbcf9f4865d4658905413a0f17bb08bbc6466f8d3e1744d9ffe6132f94ce79bec4ffea05785525eb5e071ecf30081281df58f1b5295842967a486ee262a7c366978140bcb8562abf1a417c7b1ffe07b1748978408cfae4beec8290b312f15d3223fa7447f5a89e52ddfdc4f79cf1d9fe4ecad3e674e98136c457e36748da001970cceed0c61d2ea5f9ee5f9660ad561b9fb808be3d4455fe76ae9778a49526c8461f99dc2c271ff9aa389d3e6a546a82d15967a82e8ec9b64cdf25fb73afccd7cc2fde8d07ef4a1345976970997facd5be16566c7761d23f4a31909a5a31bce35e33596a861e426fb56d9ea13e5f6219aceda2450434e04c454ec8569ef9076856cb48f36618e690a710dfe8717526001c3be7eccb671dbb4430289924ae975a30b6fc7079f8c9cbc8a268f5defc03d94d5bae0474987b7ebd4c9e29eec9080e8d85776c570089fcd235372518dc38e481f8a5edc2105bb4191cd81a19e51c60252a18b3058ac1ff9368af59a2caf82a06357764fb3721287bd28204d77dc2eadb0913cabff7cd63c73488e92c818341563476e69a50fbb781f151e02f2df4a9fe05c8419a3393d67
+
+COUNT = 4
+K = f2713273ad5b7add10d2b8d8b61d2f68
+P = 1c189d6d34b57c699dd8c79e30457c6735f3697aa6825e14257906e18413580727d676b6e667cd58e0a47b738abfd91144952940de60b161b0dd261d68af12573820f4f172b32d7c032ed7db3715fc6ceb92ec2d614a2005ea1aed14cc5015de143aaf6dc2db406e0fd31976562250e80eb79140bfc8d14fc79bea4159d66e1013704b474b0b1377220741dc55cecde0eed4e0931e58025e7fb672117a8288587cc91b1a41fb4d1e9e4a7ff59fdb814dbd9521dc148abfe16ab62fa45a3dc361cffb422482f4fd93466320a11d9a5438959947ba51d7761b2a3bb9e0b843ea6a71cba54a96bd14de8360ac8f16ece773d7bfb25df63c1aa67a7bcb587f618f2425bf9aa018eba5aebc2911b9fb6276e48d72dc74836a290fad0f5c545934f1231ed57c6944a464905fb08685e815ecddc8dbe4eeec1355f414fd2ce975c0dbe29bf4369e05c1582cfd9a7ddb7659247a0f6db38ba165f1f46a472760bcd2609ffc8bff49c6f54698063fa81305696f7ce25c95b237c6a4fecb3e48167060f556359609b73e738c5c3021d0e5bc62ed31a2a27f43dc417c0041f8a2402fa7bf76c72e2b290c060efd93ef65d0a893314b03c098a6674d0137f12eba3ebb63b3e65fd0aac89847bf9573d9eff0b1283843bd4edd6d459de6d559e48633d964dccd8b9fb5142596430b2373f9448606f9286157e810dd0f6a72843b0cb28ab452d7
+C = dbeedfe9aede047a07402d23b1b349a3e75490d3c404240c872470aa4f6b5a1723b6a0e455a8961fd25f88a9329b4fdc7d8e6bb2b3d7403bd65418f94b933467d882c1aaca702a47e7bf44916ce5eed0647250d572686c8a8c9f9437f2ceb13b361aef929d428c35cd4cf1d8970d98e75c46b4825c871003eceac2412adb6681f0b8a1d6d6bd7eb41c6e58ebe6909ab061d5e832f7580dce89eccb4ea2546204e895ccb54429fe388b0d9173922e08625d43f0a10743ad76230fa458b0971028b18e0f04f3c1049012a8bbc2f93934e0b51b943efd2fa4a14fdb795a1f21302529653583edcfa3c613ada2bf69a93e5a29a20a6f9d9df7891e3b438be4f9f714806cd57afcabc0d5f5753af804d885ca1ae23bb5fd2593fa361e92e7d12eea0805453056022256410369dab0e2e8d2d4c9ffda1d7df8ba5ee19c5bf62133d1ca9a713fa45157d327bc4e1d8e7a105012036939a5d5ae15a5fe8ef989c8323129b8b4e43402bb4f60f35483685893e9ea0b0dc4a8238148d0220cd9bdd7dc42590cc303f706cd81cc5f6ce761ffdc570c62cf734cfffe0ec7be17c7ae3ab34c65f1a53267b03de4da3a5f921cf01b068c4098c7594a3051bb03940cb6c5bac2aa04f95bd30bd15ccc9edb8e29091047de011ee9591cc4c83244750dceb8c2eedda797965c1243521c19228eb2d799775dd83ceab5aae082cfa743d58544b61b7104f3d10f632994a4
+
+COUNT = 5
+K = 8967c8058742169ab39d2d528130df80
+P = cbe7ac6c232b327f7d9154acf334891398fa0a1ea64a4f15243f488ecc784749fec41fab7a447e2a2cc75a44fb548a46a46e51f4c245f64901f3acbf18f468a28367c51060c34762c8a88780044761579846557f285c97bcb9c966aa08d4cf352c26d8a2f308803393155437b91776ee74941da9db39ef64d9ac9261f2c7fb6349e52111a7c146d5a4ef6d5d00717ac93338fea5ff0c90cdefe0d66b039045330877d8596de4c35987ce70f660aa8777276eed1b128002dffd90ce03224d505950eb2c73f5e958df92ee1cd8dee667bbb8cc71f87e4ce880585f004160c798ff2cc195b09921c73951332bd437840bbadcd10730272d46b3f23963a7c0396275407bb8041a6e5ceed4d1dc2a65a70d4843660cc039ce94a4ce96573811d2b8be00e91dec7486484ccb050dfd6e319001ba8b8fef48b03d17902cb91792dc420c79e823616cc79af4e738dc077519015c734c1be0150921657f3139e59ba4ad8e75e0f7b51806a96b7249709faac3cdcd53bfed8f0326d19000ebc1ae05a43e96c1fa02f2cfadefaf871250aff6eade8ce8d4b8d5a686d553e8a66d96412f2f189fb724e67c8acdb229501414e45c430a9533d211a217eae0030ff10db12a399567e94c900ce839157ebee1f4f876675794cb3642ace55fbdb261f1369807f284f4e6f8a2c8330eed88e18b8ff564fb754b96f8c85d28dc7a0b18805ecc3ddffc
+C = d29eb0bc54c43081030a3e1169f243eac73f1d0fb2b2f983fb25755684df4331a1932b02e237502c1b73a8d744619f521457b72fcd7401973ec48ee915054e5e12577ef1212d0e642c9dc7341cdaffd344dc480eaf9f29019355b07de2f05897cb91b00b4c53714609dea69f18c7c93429b73373311e0c4a45e2f2b3a6bd393b69f90a4125eb635a794563a68e092cc66730d236909cce6585d34f47a10c2968f161ed79c16c18a6c093c1fc455eb502770ee436dd48c091a4cefeaf449c5b27f2fd2f9150a601bccf8a37cda65b25917728ce3f1f12f55a51d9110c16132bde100b6479eb8a1ae670a9ca396f18a2ff19fb5543dbd3b189e51ba5f12516129071c997044df747f4ed70616e784bfb948a63211b224dfdfc5b9116b869fc5313c761d55c759a6e275e9ac53e83a5386e2cfa50f6c1b541cf752da800ff991ff896b79b9e691d67e96fffd166a06f0c300b7d9bb552f5ec7df6eea384751632ed9357df5a5a9fda931568930fedb0536f5491d4067a3951dc72903d1ca2f5f1b713e24cce5347b056e2f8829d34143032f53edb81a765d2ab3e6cbe12536cd85ac3230fea546ee50d991e13c562bea3d694d3990b1783a373a59fe12cfdb74fa0275e367d76bdef698dd33c30afadbbf0fe0dd7ddf288aec7a397e263a5d537df9f606f5c126565c06ed5e0bd6753d3eaf5002b472a1d7f4d7df7d9a8fe8aa2c595ecef8cfb3da3e7
diff --git a/src/test/crypto/KWP_AE_256.txt b/src/test/crypto/KWP_AE_256.txt
new file mode 100644
index 0000000000..66d8523987
--- /dev/null
+++ b/src/test/crypto/KWP_AE_256.txt
@@ -0,0 +1,35 @@
+# CAVS 21.4
+# 'NIST SP 800-38F KWP-AE with AES-256 cipher function' information for test-files
+# Seed = 6eb04695fd5dc2b0038aff221b225c302fc9cde3d6dbf4195d9d4d2756c8303906335e07679fee9a80e5d0d1580f7cdab8f0fdd78c0fa58f34afccdcba4820a5
+# Generated on Fri Apr 6 14:46:23 2018
+[PLAINTEXT LENGTH = 4096]
+
+COUNT = 0
+K = 20f31cded60b8ed8d9d3fd1e1fa6244e76c7cb7628bfd28a5d63ce8aa2c9494d
+P = f07225202842c8dede42215301e44b9bb7e625d3812f74f9b6ddbcd024ebd1f33e2cbf280b9004941f3cbf86c880a2357f88f92a6dcf8dad9da7dddcd00f3635efdff0af4382024e93c2af66b991e565eacca6b886f07178c9b4adad6f0d6ada5ff6aa7cd0712519a947a8089cea5e1e3e40ffe1806010b0149f9ffc7c4dd3c31b3d08d5ae1997c52369393d58611dff9bec501c1ab35e6ed3e7f9445a34e211010a8236686f154e0a5ae3433d6a844eb3884961aa6592216d93952b46bb58a4195aa80966ad0ccd4a7e23823912556a90d5ee9c3bb952ecbb9d895dabd3b11ab4f2e3a6c2582de50403289230ef4dc46e7c0d870a3f0cba9d643a0349503c1b162ddb6350e699589eb47bd563999f55a1adb6b78b52f006901b0427ea7d3394bb0adae4637b4f1ad5d5425e2c8ff3083506d7ad7ba4c7405a778b0a3a11760c96900a5256956cc9710091d073a19f46a985d004651fe2b6448ed761bf9bc81619cf273a6783d868d090753bf01318be21afd88d9f3a961a69f93e9d9fb822c80acc7b48cf14a08b5b7ef15c66975721b7cde9761a145b679155472a44dea8fedc0f86ae7ebf6283ecfde5f2444b51569e6723a7a19e28cdf8dec6791ccc14af95abad018f741575b343cb1a20a2a9adf4248f99728069a1e2e78ad8966c41c9918fb7019ef56c153a183a6247d22d9956564bb03075cbfd1b43d96818b28484
+C = a5b63618fc0c4512960f00a1f226d9837a90480baea75265453b9553b12a58c72153080842d7f8710f317f88fbbbf97caf879ab4bf416ba767ee9aeb34357f4a2d0e8b9571054d98e28804a70bc4d74807f2bfd95ee955bfdbb6f4d6969a0c3c3b541a514647d5cd8c9740ac3496095c3f145c50c97ec98b935158fbdf89705d5330015e48ece89188b8c1bcb2ad6825d865b375a9b9056b743dac720feeac033c9f757f6fe73dd7c4a747661b64cf490a0dd43b547cd791a5d78dac97efcd355f7ebac248fa2a33e4fad640dc34e0d40b0d36588aa32f0864c9446739a6b44ff84666d723bd7d646c5172cda932fec34ddaaba342b02a9604087ef042a2be4774194b5d32cb3fb112438fbf2801050b5424635fa2d3d3fb10332965c73e6669e65195310a3a30602640e9809179cdfc50de585aa1c0072423c626815d281a06eac3b6ffa137716318e288e3f9970e415ef0451bdc557968febf9eb6772c1f77cb8e95701246d9c567048142bb25e340351b87d7391822d9ee7fe51378bc0d08135f9f39cf44b348b87937939dc61f430dfe308cada632722e23aed5a0699e039cf0563ab8025163744b136a13ce3c62c748c89f5e17540f105e7c6ec9ba13515b504342f9e6dc7d65b9a633d8c0b5c9fa858dbb9b3a594406d478a81bb9abfa289730408c1e303c663a61d5caca00f615065312580042862397b9aa8c80ca812887664c439c8c68
+
+COUNT = 1
+K = 85c8aa6d9c83cbcb11550c95cb9fa991d49dc89fbe2531e10694269f38e9a309
+P = 58ffd9e73b5f64cb951b7df799a505d842ef0cca6d4b21027680793565a67d47db0bc872a7640691216c044fbfc0be1cd33ffe9fc73c5ad8cf38bcfe6d4b7b76639ef4200ced0eff2d361e9c948075f09398fbe5ef24d370a5a16b515c102d415813cbce56f6a8f7b5227db7aab33c4d8162ac3c178e2344f3c7e123f0e60142112b960137c0176c450248167e9714db8566e2d4af731a7ef3f88730a4be4cb99bf46eba39527ff279be67aee1f4af25b3396aef5883fe7086fed7285af5216c7945d9bd61d907a84095d6fad383a7a90ad4061f7241a84358cf9fbc7048c010c7652bf47f9ff58f1c0cfd62a5fef92b02074ecbf4762af71eeda9aeb8497d7b33beeb4fdbd80b72f2c6dc9a12409cf7ee3716fc07515970411ed7e113b7ad9865d822a502976bba1d2ef254dda4acdf6f55a04c4ac5f6a5e011375af4cfd405d1220a10465a4cd01d160d179fc7ce614ef4023d55dad363e3639f1eb8fec716f21d267ca21b0587186d162b4a3cd23f80e33ca330a8c3bd756e70c2d457306cd54be9e0433bd9416efd861ed42fcde10ccc98389652b5c14cff6c73341320c93ee457f9e3f320c5c5564fe50eeecf2d7a62a85639ddac4abdad7495e8926111b18f7866e6b3f95c788d9124ab9788210b1e25524427e86049726200052404d9b3585477bfc11f12c4fa761ed86b7449a5471ce18f964acee913946c0997980f
+C = d4112af26f37a7e2e9cdc0fd460cf4cf4f3985fc820348e63a095751ce30b2fa5d6e89a8d37b8ddd3dc2af8fa64e0db729f526bab89562ad7f2bbb9ba02668033ebf6eb742904f37fbf1d87b4d23b1dfc815529a6e3f24bb31ba1d4354820a47e0d7b0acb4e495a50912824715d24cdeab0be3597dc535326ee073eb39b925c7699115dfdffd4162b84afc61048c7802a76ddafec243b91791a26138f202bf365a0b0863555d48c953c50cce5e09fede4a97571d3ebf0582c57c39680714750fd2227a632274adece8a4bd13413825b51969b3e965ad417c96415f4041d8f0d0a3df04cbb24fcfbeb95b401be7ea840da43606cd5bd56bfde26034fe94aadc78bedfaa4eed77e64cdc3d059e3765ff6346e9344f0e0a183085d78711b13d4ddc356a669693d2ad090bb67e8013403c51c81951f8a6241269d3423a739b44215bfa403f8927fc1d100587d602a7efc505f28edfe51ae8df407361fa06c09d06c7e960cb7b7041f3603faa6b4767a8a3c3d1c8ab05f75143a3061396c7289503ec29237eea7a364eedb77f66da308fe8e194c9b1784364072dbcc8aa1145db1a03f3862c38cc1a08b569f8576623915a644700ab43f7eff7e59684eea5595c31b13778824704b4aaa9c1b5d2acc4a6cd2ae21c5b0b32bc1433d536e8bd7a4e6c5e6d1ceb70dcd2ac215bd4619aa1ea8c184033b3279cc9fd094336c5e6f5112bd9f36fbebf304ab04f
+
+COUNT = 2
+K = 5309c5dff8fb03b007c28bda0a219d329080120b430f4b5af9df60ebbe20890c
+P = 22b75fa402aa5b665aa71db0e24e76a6796329fea9128dffbbe03764e23d32443cadfbb6d557e833f841ecf5cc7c9200de81b2c58fc72a1a9451ce51c75abde6504b7868d1864a5824d6ef5dbd57c80e488a011102f7a321639fa57c40a5eb8479cbbc7428bb5497dd6295e270b53b00eaff17c2aac199a637e2baabce9c3f5af8d8c4698eea5390d7bdaa92ddf8190361a2452df2f44220b92519449605069661e0360cfef6841b734c8bdac87feeac243693a4e114995e1985d5c93e512d0f2fa27f12db8803e0aafd974d968cfb82cd9f0d4b09f34a440cff85e09a63462fe9820d5f1ba6bc6a489f30ed570e44c06668b6a4d3b977a6b7fdff3e781f0b2c7a1361108eb95f569c3ac5dfb1352cfe0965d9bea70f923d9f65cccfa83a9272947b3563f81c92f7f5caff203137b034af82847de0653bfee01aea476c9112267f97507b6949dd1d0d7510fa280432159a0b3bd67a8e628ef11599be4459238161914520b556fd5d91e92349169bd2ec175a9e728ca78b56eb2bacd56fd2af2696d1b116f5776e9df459c1023180a18f5d6e3718ac402bb0fdb806cd7b1cda6fc107e635f6a756005df79b720f3f274deffb31552cf238ec93182370d0ebe9f0b819e2e7225bc2b77e702d5df2825fe5d42005f1e39d67414ea539810db552049641ca22bd6539f1efefcb90dd479bf93080cb9df86a9aa99048f950297b8343
+C = c6a64b49464f5db516dd39f3d1dbc444528b5da57016b5f2a5649965a4781962264dee68d4c78e987f9446cdc815aabeff2590d3e2679a2583dbfd3aa6b8bd3bcd86d66d598daa6e766a6ab8fc6fe98a1bc89b5a137a342a59dfb52c7e7b2feb2a33d5f936503121476a9eee8d5c60b6c4c9aa7f8c56ddee5b4a9d7908890d3aa98f45cce20a074896aacefb6d7520e84b6675d02b819f37929746298d2c81407c20328a1ff4ee6da8298fd639fa9bba4ba27da5e19520434e27ae4bef59b73c3660aba0353701573d602d8135442a14cea4dbca3732a3fc8e5ac831929ad48ca30d4ee0f068f54faafb50bc7c6a784e81cf86b29225200f985af914ea6ed166e08c04e6324a372550603c98bb6c010ea7219699e76e4cdd9c4b8e6c9200d10c51e93a57f25b0019ae436dde220868596e96ebea3086089ece51b93a812c8120b8ba60435720ed690a2d1ff7c024cd514cf73bc7cb121a83222e4fba2dca65aaafd8cb51ac096511f83327458c53119f2462caf6cde5ae679fafd4f96756aa496ae6657ce417b2963c09463ab810ef51b4942dd1ecd120d69ae76f8928f3bcf20465f68e526beb7cef276533270f08136c4088182dcd626ffb2c3c4a357d09affb6b935e671bc827779ec6f260acf2c01f1faad57b162fb50579ed45909be1cb2a4a4dd07effcae6a2342022e0bad504261bb8d5d3b73eea932b0135fbb76fd6b8f8d3b6435db6d3
+
+COUNT = 3
+K = 8027a28b92435ac111a67ba0f4af5cdd50e6bbf965cb9077189143440db0327d
+P = 0d8f70b229ad687ab5f55f0b0ade81a4bcff76dba57aef40abb8074b9745ad74efcc8d8f2202b92a9788f8049c864e5973403279dd02e165b772ed7444795ebfdafd8e13a9a22ac1103d0c15d8d279c617dafff98dbdb9bcbea923a00967287ea9accdfcc90b2b4047021115d5454d057cf3a27a44988a8fa71578ab7751a5d3bbabd1a335248564f5e334c3925119a29c219aff6e71b3963f603b0f6e8213219fa56a36e33066f49409c643dfa1ab7478630d623c7b77efe07b0ad4811a522ed81a1c905831689747efb4edab65062cea3724c3f9decad27290c4ca0ee9c68f1ad9ce3981a0b712acbe2de17eefb7f9bfb3efd66aae691e63e5d64233715eae8957bce7924fe2f259309ddfe050cfca3cf76c52f96f8b7608cc833bb3876f35e37f6a83ac0a21f0c0fe859133c17c5f7422c72741137b8c3b5c3ed3e7737d5d8a29c063f1490558982496b7c852ec95bc452113c48bba93394dc1df0eee1f8dadd38715e93a1d7065c528f08525cd84583e250327320405bcf18d8011e3d864978f948b2839bb6d1d7bcef4b5b6871243a5c93a2afc1f7fc7d06c8307a025ebb522cf0fa1e1e9f8163f20854514504f4f873d757faa7938bb1ef91000df4ea64bb985d903136fcb9c4e9467561cd564f61146ac519c08dfa41f22b773cc821cd1252a3ba237c1e8c1c128ba40e0b75e67adf12cc448e47fb4037850b014f9c3
+C = c1e82d374a1d7c07b1f061656fd58b5df0a04c099f3390e98d2cd460b2e2713eb7274b9f646fc453a5d26c3fc0b9203521de72dc78292cf4d61095febeb0748658dc6764fdee4fa498b93a6bc053b3d81e10564cc9681aff2296ae0864fe3db495298ac4174f6ce06ae6b5bf86985c4603700757ef44c82c0e03beb7f2841df2dc4dd8022d5dc7ea2f711daacc1c8d1805798f646f1041146891e88edc4db9904f122c74656a5ea1694f8e6073920481e6cd30b20fac4cad7e03f58183f6b0beff8dde3ab4f781a1dc36ce0905e41e77be65c83dc9b6ab69d5d4c15a1de3cae6b081ea9568578207d678df77e20196483c2e00d2dba604784495949285a623f33e07e20687aadb86d57adb9935865e6467801a10add2ac7dfb6f8ba98e7635d6766eed38ca8fa8101f93018261b872a2a899065918b672423288cdaa417560701600424da6db15e0ebb7a5d1d0a7e1cc8aca714ac6bde51f779033563b49dcaeb8e0330112902b5ff0ebbdd00aa371c902092566244bd9a1e5e978af39adbe13a7067d056d04b718133575e420058529a42ec032cc5fa62035e872cec7acb3c32eb8b02e0cdd81c59afdba65b673c828b980756e8ff6b0160a1e796e221ed40c342863848b76e4390ef730abd3edc73ba975c037e6ba5e58162d7a60cd55b61643e9a8d4f6e5efa2485da7c45991ec4e8d4c6d9432e630e703ad1fc2f644d7577248ef7c7c974ce7
+
+COUNT = 4
+K = 1a5b5f43929215da78d1d8c8fd4ba4508a66273be95c85a892da606eed04d6da
+P = bd762d13e2385d333e435ecd792b8b3f13936d3b8372466fa7f65aa537bbd0bca184ec570acdb99ce59633bace041fb1374961be99276ff1d4b82cd49d78b89585aaa6cb1bf18498f4d01f2fc5e41e2b4fa56cedf5fe69e39eb1bed303d0795a715bb7eece6b389b3ae81b23b4f053137d54c03c2bdc68c79184fcca1758652c61a5e3c19d49b3af1a8946403b254da0f9ef227d2be6d0c528e1a7b78a55a2e79d79d6e420e9a3e8b40bb3ee74935be7c8e935414209be01532c991a871b90de0b35e963989475fbcfc37d81cbac674b1b29da1d0f9e78a6c3ef55575a8d3d7637b93895bae528d85130d059415d7c456a5c8d94897e567d6d6748e7c2689c98331ee5a7e1e25909901ace55828737e7464cb670399af3f806b996f416550d256f354dcdd1c332b4ca9eaef70114a7bc171efa08b88f1afd0429b2cf15366105cb812596a05e22b06debe1c69b06d904fe6cce284531ebefbae44b9c9eb156dedd01a4a73a1e631fcabc349e07636741580a916b35560d02e1cbbbea0a335b395487df8f972cf8600b48237e7b63bad5f08c85b85043eac0ca93d34df56cc486857d6eb3de0c332ee1ded06927e8acdaa3fce78636eb06c0f805788189c56023c8a1f5a2427c6f0f8b993f517a9b4a3bc3fa9c3e6318256dc49941c1e92973e3ae4814b87135c9ee767348045ef7877027c8ec441c50b1cab98bf556352333aa
+C = bed34ebd1c70d52665103651110f847d83e19a551f5156c9720e2d1b038ed5d5ac0be8cd97ad3d725a8fa3c127939cbe1a6946558015bda4c7fffff7e00662c4b0ae3d5477bf61e175f8e4ca14cf53ccc558b8fb5f371748329b6cfbfb277cd0828a2394ec99f943775b4a2c1c285127a2d4677ba3b918154908badde2a74b58f73ac8f92b440214a3cc990f1517372b2e53272db9bb191cc7711ea31ffba2de2b646e8e312f4eedbe9e51e182603cf075ff69bc4d845410bf66d20215c160df05ab007b49a5396bf69d702d193aa0e2a2f9d62b7f9fb46cdf11864314101c7b7072f5ba8262c32b12b14225edba48b440501163d027e7fcf2292c9b06ce19dd0eabc6b6639769544d71bf068b8e03c4834c4a785a6b2b927c077f839f29da61c6c52c39ef4354839eff993068b67438266e10e9a421a90a103835829320fb1ad9e7e0cf73bdd22aa268436f4df0d973c401f8bb20f55585954079b6de5a2707ab5ce3b9f42af09b7c1996a45a15af12d041f5fe0c055b969a9055f7656cc2806745e84836903804799ffcc15580f8a50362f4dab9694e8ebf6f32cacbc3ba3501959ffbf33db0d231722715705e685df9202f0466c3ac0f1361a71905e23b0aa9fcac76b01d6402598861084477338b5d0463a641e2576f76aab2de95016cb31d37d8f4617e3908a179b43e3d50c516bff1e388bfc0ca519430f85a0de5b0e274cd0bd021f20c34
+
+COUNT = 5
+K = 2a919617d24baf991779de6c06e68bbc4894a4e9a702214ba3eabfbd0015a7c0
+P = 8d01e11206e3b9285cfc73c2ad6b238536fefbe824f19588685601bf94e1ee2515c4ac281aa04feef877694840d15f2a6722661159a7216548d768b4754d65f1f1c522daf2baebbf88f638fb324b182e61d315f5f12b04c486022749f6a222d87c7d635c52605c7efc1853cbba466d138d3cda22979e7742e4fc8168f5139a72818109d2124c20ddcecc74fb388f34151e96fd33e8b269a7d7e51b6fc6ebb427219163c3d9f7de4a6831a7f8b596b4b490789f1104fa160115b6e8d191d1db885e42812c01724170e765634fdb8e05555cc7a85c3d8cc7a8e299b4f3708a4a5189871037c0d04216dca8e6127b449ec6004e7f076430d1f870f0d2c0c6812c810b4665f7fc07fbb04b56bd956b81267d2408c31be8dabade28a3c40c609eee3458b7340715272ee26df8d875d1139511649b629df067a8994e1fbcf7ec60f565e8da257db1b7ef3b2b765c77124c8120bf26db8c47f493d2afbbf8b89efe2e33c21c2a2a1141c0aa5b9dc1eeae50714304ff35a7cd34da963f5249bdce8e2c2a3026672bcd2ebf311f446cbfb7171fd35b446f0270320b30b554f96573eac72ea0594306d43c5ed48a6c2d01a363e88646a8759117075596fe17f18c0b36d00b7976098fddeb7abc4e34bf6d6b35238f81b59d169db4578a96c2bcfe78bd583fb31fc2e7fc97c6f3460147a5730bdf31ca197482acc341b7a719c4e1716cde56
+C = eb62ee28d9929093e3c6dbc0592c35dd0ffbe8faa37173195da391b84b1ffb342b9bd9f0df0947c33c7c74a071e7d71312f91c8e1f11d680865563f2d9604dea06892f2adaf2028246b74ecc82686bd7c369426a09f23012b69a89e7e66e7cf008b41dbf04f72cf950fd8f6ed63b3392de53b6feb34e45df3153e4aa6fdd685fcd7e3239f0509f474d8f6a58abfbd36bf14f93fecf913b2f9a0f0763c39cb6828651434ef0520cee88f46daef7932e5ce1549774ec106c2aba54cff64067e0202167f5a3ba1cd6396bda08023d2cfa9100043b44b8d90e2862d7d2a7f6e3f5eb6976dac2d5dbd2adeb69858ca391fdd4a582d3cb79a2d9fb57ad5b2cbd157b1e36ed49fb848e9960b0dd1c715b701f1d379027999fe8e35769e27a9ef60a45aeee56c7a3ebb39f6c50796d1236e721cd0e5d931e6a85f28fd87d652c3e5b706e2e97c12c4b33fc7df4585d60d0326267d8d252ab8970a2528be086d7ad18ee4b1531cc13b9c88cb3a9188a47b8a72cd276eb4d7d726a290398a98be0083e9917f349d2c8137a6fcd2232baedb6edf075d8e938319a12bfc31726b13d0e1a6c18ae33e11ad1f451fb9693f1774aec30fa7b473c98cbaf4dad1b3743d118ce69620503a753b87e4ae25f36564c2538d6650291087ae04d181e4fc44b6804b5bb0c6c8a0aa04a49895ea64d13f7d253e62216736a716258bc98c63349b5b19f5ae9aec3a176f98cb298
diff --git a/src/test/crypto/Makefile b/src/test/crypto/Makefile
new file mode 100644
index 0000000000..9b5fce5b30
--- /dev/null
+++ b/src/test/crypto/Makefile
@@ -0,0 +1,39 @@
+#------------------------------------------------------------------------
+#
+# Makefile for test crypto programs
+#
+# Copyright (c) 1998-2021, PostgreSQL Global Development Group
+#
+# src/test/crypto/Makefile
+#
+#------------------------------------------------------------------------
+
+PGFILEDESC = "testcrypto - test PG crypto routines"
+PGAPPICON=win32
+
+subdir = src/test/crypto
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+export with_ssl
+
+OBJS = \
+ $(WIN32RES) \
+ testcrypto.o
+
+PROGS = testcrypto
+
+all: $(PROGS)
+
+testcrypto: $(OBJS) | submake-libpgport
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+check: temp-install $(PROGS)
+ $(prove_check)
+
+installcheck: $(PROGS)
+ $(prove_installcheck)
+
+clean distclean maintainer-clean:
+ rm -f $(PROGS) $(OBJS)
+ rm -rf tmp_check log
diff --git a/src/test/crypto/README b/src/test/crypto/README
new file mode 100644
index 0000000000..3750346a83
--- /dev/null
+++ b/src/test/crypto/README
@@ -0,0 +1,33 @@
+src/test/crypto/README
+
+Regression tests for cluster file encryption
+============================================
+
+This directory contains scripts for testing cluster file encryption.
+The first two tests test the encryption/decryption using AES128 and
+AES256 in GCM mode and the Key Wrap with Padding (KWP) method.
+
+The third test tests that the data encryption keys can be encrypted and
+decrypted, and tests cluster key rotation via pg_alterckey. The fourth
+test checks that the database files are encrypted.
+
+Running the tests
+=================
+
+Run
+ make check
+or
+ make installcheck
+You can use "make installcheck" if you previously did "make install".
+In that case, the code in the installation tree is tested. With
+"make check", a temporary installation tree is built from the current
+sources and then tested.
+
+Either way, this test initializes, starts, and stops a test Postgres
+cluster.
+
+Requirements
+============
+
+OpenSSL must be compiled into the server for these test to run; if not
+the tests are skipped.
diff --git a/src/test/crypto/gcmDecrypt128.rsp b/src/test/crypto/gcmDecrypt128.rsp
new file mode 100644
index 0000000000..53f694d37a
--- /dev/null
+++ b/src/test/crypto/gcmDecrypt128.rsp
@@ -0,0 +1,129 @@
+# CAVS 14.0
+# GCM Decrypt with keysize 128 test information
+# Generated on Fri Aug 31 11:28:04 2012
+
+[Keylen = 128]
+[IVlen = 1024]
+[PTlen = 408]
+[AADlen = 0]
+[Taglen = 32]
+
+Count = 0
+Key = 137f40bc82b01b34047a407f40d5c434
+IV = 1f3de3a272227db8a88ba0cf7b23f7e57f84ce370b289e2f774a0c6184f44f9df9b3e645ae506e3d18cdc819fdc3b31f77d2dae7ee295ba2c776e6f1fefecec740021c4aa1118a6ed3813720411f27ace5c11437a78b3d626b5f421e2f3ffcd0677930e7878110a0cad535c057261c63f0531b538d8bf04f491c16e871f071ac
+CT = f6336f2001a5edc519aa64071fabec2fbe4654ff334871e9cb8b12c24ffb7ec9f0d316e43bbdb568b65608f937f24f64eab19b
+AAD =
+Tag = ea3d8cad
+PT = 60881890126afdf881dba4c1a7bd0f9d8c240073517b21c622bfa70c1f05d59544b9dccdd908bd924ca60d23697fe64aab927c
+
+Count = 1
+Key = 5e49fac6608ee4157a9f0ffde503893c
+IV = 53d854bba28eb9d9db71e68b5916f9e73964a0f95c6524411fb18f64977d012d611a15fe8d46ec49bcabf060cfc7989c4a8272567c660c0881f944066b88311992b5d52cd644bf07e9f01d2249e5f9b00ae1af9f2739da627b3a8838e28116b02e7d171d1938edc6dd5d7da3dac63ed148b367961b809bf423510f0a3f44a8d5
+CT = 23bff066410784e0999a3d0fe742ee28ce8095fe822fae0f50b0b17d70c8cdcd705c83e108fcef2d64c5b4db110e9a72d3c4e2
+AAD =
+Tag = 01aaf4e6
+FAIL
+
+Count = 2
+Key = 174b47de82c64196af18214d9d6dce2f
+IV = db9653e9ca4ff27f8180f45eaf4861541d30b3b147d2b9b64709983dd08312b5fa709a7bc6afc591fbdd08fa0f21f06a25758ee199a6c6415cffdd9777280e620d7bfaa6d551875c069f5672a730aa68137cf33f86d46cbd548248e68f589c2e699c554ae4680417e9440e5514c1cee03205324f789d953d63518b023f65abe6
+CT = f8b0c1317c5f3550363762dc7fcc847376a7917e1505cddafeb5451ecf4464ef52afd797697f1b0b1e8d3a2376600d6acc1c88
+AAD =
+Tag = f6fa70c6
+PT = 523986d433fbcca153af7765d177e2bad72ba52743440250db451e2da643913cb25b33fa364011a8f8d5cb563b461d46952923
+
+Count = 3
+Key = 5194e88020a501ddf21a7ba5525bff70
+IV = 444b0037fba6cb9d4f4e33df32289d44f262d8f535564a1921e6a4315611b98a4e1074b37114758b421ca44efda91a48ba8674c695c259649c0bd3c11bcdc8363d3d113a534bd790988a44e72559fa70a5e2deeb9da187e2fa89c7b636a000a38d7bfec2edf2831ddfc9326f0c54956c149e5ffba3f556c8f2a749dc6ed5407a
+CT = 31398e58d6e27c50e85c87a4dbfd73134f456ff042c9f869b47a41e3c10db7b24cffc103a2185a696fdc1f2d389cbf41379f04
+AAD =
+Tag = 2d28343a
+FAIL
+
+Count = 4
+Key = de250ebe0ea1212480f7193934d1ae84
+IV = 255f658334f1c5258d613ef92da7242079795e9c04c5c4e800aa1c03b6abe77f4dc28cf614278900b52f1de23550fc9c0ace8e8dd91a25d0240281dcd684dcb786839f376496692f3e35a5946bbf7b39cabf7d782f4e77eec53a875d2a5e3106238dae0d048e8b9ec7b05961d9d68489bcbe2805ab0f9cd42b5f8f62b4b28b00
+CT = 6eace8c7c10c713870c676e557c6b49134af26011af157de1e2c6ce0769fab666b9a7844756d6b948ccdf1df77fa98119c83a0
+AAD =
+Tag = 87b81d39
+FAIL
+
+Count = 5
+Key = d0d9c4aa70e5bcad8dd9d72d14c21c54
+IV = da20388ed0709e5af5a52a5293bc4060f02ed6d05d96f9ac3a05c14118a3613ff790b721ff2c888d63c04b9b28a7138ca00d0ebbfcb11b70868f5b90c0dce79c7220fb90d5f3b6e22c6ae3640e7dfaca72ec12813820debdb69f90c8393645d7e96cc36bdf62ecc4c0ec181c64593c65c8f707d6db61b16c6a5c2f05f8851692
+CT = 2f08e5a8027a88f0e223429e6749dbdf91ff9742c7c6130962ae837a434a7247600df8e2004fcf3e81f8ef64e6a23bda002f3e
+AAD =
+Tag = ef6c0d39
+FAIL
+
+Count = 6
+Key = 464609f485abe0bf4930102f9637fdee
+IV = d0a788fa1494d47bd1cddd55f88515229de5c1a2ca7dca054e08bf35d3c0aba677671d5290c3375fabe40de14e15465138d298edea5efe1ac62452dfee86920fec8e46577bae1c0b43d77cb6b8d1de6ba4557d7410796bede796b8b87b5771e30e05a8bfa55ed92194d506892c33f8ad6038ee8896b98772d1a88a7a40ceb650
+CT = 5e81bd883fbedb3e074ba273751081752c3aeb7a62b08091b768f8f8e6c0913cf633358fc26885150adf65bd1d708d9e45eff2
+AAD =
+Tag = 7e715546
+FAIL
+
+Count = 7
+Key = 018319ad18e5d877dbdd871316fa6610
+IV = 941f8715bbba7b1fdfa98c8c05546a50f86cd02b654862af6c1a70cf4d79618b37abd1f06b7a29bbed74e728a8c78a0ec8ef8312f2552fc4571f48024ec5c6d44c21d69311cb1ba390cda615f246cb175a9e8a522a20b671e6df973b8924fd003c5dab7e59af63f20edf56e9dbc9f98fcac25cc692cd87a3d8fbf8d4b5f150cf
+CT = c5a681b2b2e5f68e0cb698150aab9d13e93e900e0e6fb20170a4104839a7e0a5775bb3f20ae814edd109530552ee2b41cf5986
+AAD =
+Tag = 75a966e1
+FAIL
+
+Count = 8
+Key = 3556566a1cdec721469cc86412ade697
+IV = 731a29cc083422da193724a57c8126b587c3817fbc6b76bee106abb3ac48722bfd0cc15deb573ca73f2bc10ad7b60024f297bc52c31a7021bda91c5246c781a95d5d18978c685f096e9310d6f5ed151122262f7d3f5a2b265321683cd72e0693ff608e378199264eea876df44659817997b323e43457e70809c74737d015cf26
+CT = 753174df0aff575fcea79e54e5b8347e69175070a00d58720529b173e9dc64677fdf459ad72385b03244d90810670304129f2e
+AAD =
+Tag = 41147153
+PT = 7588b757f08b75fc799fc825034ac60585e24c56db94920b6982bbae86030e7a021ddc89a5131e5371e9295e414d3503ec11e3
+
+Count = 9
+Key = 2ec96059d1313923fe522bed1e942491
+IV = aafb096a63195cfc959bf93785a48cb7a78b98c60f861c9646129222643d249e50d46f706d670662f60c12a7f0a6ad65b908ede9a56b27c46f07d427754c11549154632e9b5ffa25c9306add4c0b13c7191abd59430a65ccc42c5420ec98093ad6ec6ad1c4d3bb895ca3546236c2c78cd075b4b1f44a6a03d115b91df02b96b9
+CT = 61c6b8bd44c9c6eb075fa9bba7ac0156bc64a6326d32afedb51f35daf232dfd068dbfd0505933519a753bce6de4560b79b31f1
+AAD =
+Tag = 26f53f5c
+PT = e4b86f8b7684d6a300cdca848ee8185c41fcfe061e61af6f3a563ceccab58751f1884b322019983d99150db217a1e92a8f15bb
+
+Count = 10
+Key = 48cbb46539dca9faa97b69b32015886f
+IV = 1d3152ccf3ff05a81837ac7c55f791c78aba8c95149c80de8724b1126dda847b983a1bfe54ac5206d5915dd0796944e4daf432b14a72c5fcf81822ce1e3e7ade345528641b94a2fdc90afac098b9e4b0b5ad159ba275dcbcc11fec04bba0f12db9b3d1473ea0ed7825d293fd0b5d2d07795b3de922bba5660778adbdb2cf8539
+CT = b2658e897b85d9acf437f9e895189954134c543aaa91fa82c586cb5bb1d96e04b189df64df5bfcb63a567e444f4653aee3fdbf
+AAD =
+Tag = 48d21620
+PT = ef58f7ad664e36dce523408e497acb871e9a579fe4688be6425adf41c4c41db85e905e9279b93a037648802dfc273b387e7216
+
+Count = 11
+Key = c5c2e83c4e5b336e30257c95c8f7f75d
+IV = 269c29118eb9cb4ae8fd8ff5b3a5d61f4a63a719416bf2cc5225544cabdae24741239ba7910d1a32364c341169a215264ae62964322b6ac64002d09455f11d9658a9454cc6bafa82dba5b5365b2243e854470dbe74a3043b5f5f82ebefadaca4d468317c83076691c188af6e800dc296cdd941fc1aaed8840874f8cbbf96827e
+CT = 891da6e799429c772edfe93d9fc3ad20c25fd5a172d272d0ee6b7063b59b611e4a2805923f182abf99373fbe4a1759339bb13d
+AAD =
+Tag = 6759d819
+PT = 474a2e7aa4a5422bf17f257b584059fc7abe2d25e7243b434a6f936f5a0eaed692616a43a2aef8b427d83ddf1bece29d96c581
+
+Count = 12
+Key = f4d7a0e84b7f024988f0507098415616
+IV = 9d58c03fe45a97d4c556bafa4292766c55bd84d0e467c6c26b0e42162015f3f506c9fa06caac63a3ef6a3f0ea62ca4e122bf453a767b2b00d6277a990329b76073c64ef260421d887eaf89b6deb488a0528608ee8c38faa5f5976b2b71d7e29a08ccbeb2342ec15c46232f3c8867ada7cb8cc8aed0728dc59d706913ba62124a
+CT = 11358bd3ea694574c0bde99aafc2621732fcd978626d3a3ef124e98a2c3dcb8261580306b51ffa9557d379952ee31bffd9ade0
+AAD =
+Tag = cc46943b
+PT = 4a289261f4ced6fbe544fa27d430f04c72ccfdbbe0f93e881fcc1b29a19e706a579a98304cb2b03cfb3aacbe187dc69f923ee5
+
+Count = 13
+Key = 31237ffe81db56b9541133156fd0e7d8
+IV = feea389ed83bddb360f42351cfd56321bdd0a8fbbdcc8787725a236365eafe5dbc9c6e9ec8e9ea0f74e623c33a574ea4223a43b1550add0c75d8348315add72ee9561e1750b69defd46214bcc507d4db4c66c6da23612aabc2ebc9b0e6d5aeee270c013a553ddbc7d7d7ac62ecb33a5a6e3db859471ca3d3686203c0c96eed2e
+CT = b965d7bb29a8811872c894d535e001d534a7d6e99eea5bd14ebf39c5e79c623a67d549bafbd0dcee1494245c04854f34de677e
+AAD =
+Tag = e6fac1d4
+FAIL
+
+Count = 14
+Key = b0e576fd7cc85f7c92d44e4b8c149b3c
+IV = 3855af97c5b740baebda1532c196d187b8af0da761489ee9e267fcbeb720bf6f73cd743b69d942f3f44893d68a5c70174a1863dfb831ced0ffbee668a03e1066e8b3dc03dc1471c3a848b3787c0645b20add2fc0cc37e5e0aa57ef08fc69c53897030fcb5579a831ee53c76df2f75d2de5bea93e9ddd8e4e1383bfa1a7c7aa77
+CT = 624aad7fd047a61c09638b8dd5065d7c00960035440303a03ff1b19caac02baa3b6835581f2b66c2381006798c9d5d63d65684
+AAD =
+Tag = e6142617
+PT = 0fb24ea93a14e3d76c4d7a991cb55e6bde4083504ac9f3a00d4b0ef53d3929953b53f5ce820c41b9aa75a985e09986d3f4baee
diff --git a/src/test/crypto/gcmDecrypt256.rsp b/src/test/crypto/gcmDecrypt256.rsp
new file mode 100644
index 0000000000..af72933a60
--- /dev/null
+++ b/src/test/crypto/gcmDecrypt256.rsp
@@ -0,0 +1,129 @@
+# CAVS 14.0
+# GCM Decrypt with keysize 256 test information
+# Generated on Fri Aug 31 11:31:25 2012
+
+[Keylen = 256]
+[IVlen = 1024]
+[PTlen = 408]
+[AADlen = 0]
+[Taglen = 32]
+
+Count = 0
+Key = 60c9f83fb0ddbdc727e70bf9eb1acc13b1b63e3056e64db7c2ac55c4f2068273
+IV = c33d34a3673b93bb78dd1e00f877c4e6e4cf628438b9effa61cfe81e159155cc9ca7c1418917527ed3f0a51daf2bedbdaca20fad687a7dd086ae086c8ff5094e9b31fd71bd6f8f1f1adbf96bb2690663386c37d7bce891137897aeef70be10a453cef7e31c1b8c0a24ac1baeaf08a46aac445ad5a8103804825fde86dd4720b4
+CT = df6586921250aacd9d25f432977e92b09ddf89a9403c83a80890ff15ce9c4559145ecd85d86f1573bbc1b48992859d22fc13b6
+AAD =
+Tag = b0dc70f0
+FAIL
+
+Count = 1
+Key = 8d7db520bf88c96d46778991a4f0b6de9aa7fd5d35cb6188a6f355499072af5c
+IV = 488a50706bfd8ec7fb4c508511bf4c897c8566ef289b5e58a4c59bcbf16b5ae85fbccaee4a1cc0d1ec74156ae911d36d497f5ee71f1fa51649819c9cb88cf65d62d2abb65d621c202bcb33d8d68018a858d04e79deb62b3486658730735a1c87829acb49e73301902c116c9b6ce110f23a6b1a4dd657e47a328e017c19f0ee52
+CT = 497e1a39ba1b38d263bcbf19cc2900ca4070ad37ec12bfdd30139a7068a889825eaac5012cb5c2dcc710a220cc658dcf069f60
+AAD =
+Tag = f20b9885
+FAIL
+
+Count = 2
+Key = 1ae1e42ee656973f5628e3cd11f0494dc8a563cbf5fbc5880cc2dc6d787bc9b9
+IV = 23e3c948cac6eba2ed11d667783557917f066ff6b93ab9409df9c7c84b27d26817dbdebb9fa9d0a64bbc572bfb2c7ef7f0c836528c9bd692505c8c5e522f57aecf6b479723449398e5b1f45cadd81264c5aca8059562d69deba26395034b4b01325d072dce92e540c159dba92d3e41e2d0947d873ad48f9f0b00f4807d420aac
+CT = 3839fae7008b88250b602cbdf295e932e3c4e3710d397a2b9a37289104efde75f73302b2820f14664c064e8dec45ae49a74036
+AAD =
+Tag = bce334cd
+FAIL
+
+Count = 3
+Key = 8af302bc8684cb91b4d7a6088cf8c94f9f6e027ba046c2b508956ba3c88f2d65
+IV = 3218736e931392e6510b91210a6a6a27680740ba8924062ea176048d6b42f44ed04a46ce31843b735ef4f63dd1d85643f28fb335d21fc2e3c673e97e6b845e363362d32844c9054a165f40658267bb177b74797a8828b1eea723d51b571d93748c758ea5c328103612b109e008f743f9505034ed3c42ab3dc310c20938f8627c
+CT = 1f1b133a1a7b58625fc77021f8ad1751bfa2b8addc0a9837dd5c44632cffe5ecc2e9e54b90cafb6cf8b652a8d2da116ecda3f5
+AAD =
+Tag = a673129d
+FAIL
+
+Count = 4
+Key = 13a7942b5a5cecb2bdc0e8b0348d4db5a98572544ee31918ea625b0691c10779
+IV = d92b4d05a549b296e18c90a8da55ec5bff3547a679697c489a1d49dc02bfe2dd85c8f050b32c389c4f857eb4b663f53354bcfe9c3a7e30019f2e3994421bcf3a3d1cc093768eed71bad5139f3f3078514d80a4a41d1284b5dc43ce07efac9c475d6ba2acb66dee50cdc62c463a05ca396e72d189f50d44ffb70d2c6112c6ef0e
+CT = afe058cf694d64706302b405243db77c7ae2fe4f33c6427416f8992ba92754c69d4e7c1a89e9b6987f2bc0a7b568dca9c9d273
+AAD =
+Tag = e0bfbdb0
+PT = d544d114e3d9ab8aa2b9ec588a112f780a6df74d637be3cd34fdefe14506f26281cacd2b98c26fc4adff837a7bd72173b962a2
+
+Count = 5
+Key = 7272e6ca6d6d76c483df9a55c6d07bd54fd8fad50b529ed52154959acf01b64a
+IV = 5c55bb8f4fe797ce34c0e281c3b04ba0bce8689493451ea569ba8cbacc74ea36ccf319776f77cb4d7f901fd0ff23cd28ff0ca77ad9d4adb0329fb68a60ff004a1c5b12111d2dd705ab1f7734178f14dbb356cfc0c5c208b91c277235f35afe8c2d46ebf43bd5e0a653e67e0c086ebcfca32a56d56dd5f810f562f769cce2794c
+CT = 7f378bc30cf2774f21078f42b5d6b66aa355c8c073d3a70f06775f3c7e5948539ec08a2cc50cae6f2ad9680ba47bac190c3068
+AAD =
+Tag = e612f4b2
+FAIL
+
+Count = 6
+Key = 82b1d2ffb53fc79f5ef88742a28eabcf404074836fe28b5b202ce7d5c68f6ebe
+IV = 33f834fe23b9639d30de763faf7c1a71568c5dea9d5d253f28723ccb3306a3cac3cac3beca638067a3485ff743b5133577633ec88dae0aec4fec08e894ab5d61c411f0939772df2fa66d5775f74b3ff36ee61695d7cd2726b9be4df80750011477705948b1276db0cafede5d7ac73ccdf01b73a5492a02c43b89632a501f6694
+CT = d9d92b33a10f4252fff828b57ca5f5f118885df0825be80ea5725a874b7e8721af40bd221e7f5c2c8b005d77af6266cd36ddd2
+AAD =
+Tag = 86e48fa3
+FAIL
+
+Count = 7
+Key = 5264831eaebdde1eadf741dbfd585cb0ef6437d1365bd5848d9cb3a22f57d420
+IV = ede74a8f53eac5dac276bc72518255831b616c9fb50a617eacdcdfa50e197d2941004f785f00f8c600e239cda77c8c06088793a674efb8759c98604dc0143e06665dc7e21d5031fd4751a7cd1b947304645e0987ec7e765db80a743122fbcaef9ec83849e8eee8d011dab67fb54317caddcfc472f585e93df91b1edce9695908
+CT = 9ad126b39dc2066542dd30c8fe81cd750b72123d74aa162113c6b0cf10a9cdb217d921e8f03b400f1ff719fc704f44e26ad463
+AAD =
+Tag = 5d7bed4e
+FAIL
+
+Count = 8
+Key = 6c2a43ce5610eab9dc40f43f035f7eed6651789dfdd166d4f106c95cef2a67ec
+IV = 60e3a8ddb899108c11550a461720bdbf9adef26c300f098c73c3767621b06eac4f5619b9855d96e4d972ddd38f4538f8e25b7524b46c6341e8780e22c3b42ccf43f41fddfc5680432b64fb4025b378204045bb2d7ea56f4340a4018a4c99eb8b91012b28024d1b2bdb603fa10a28130e84bce38384fbb7c43548c0072c5c657d
+CT = e073e948ddfc414948b12b4540d43dfeb9cbfa525b3cacccd21da89ecfb254c840722b9179057cb3ee69358f05e4ad0e41a543
+AAD =
+Tag = ac0497a5
+FAIL
+
+Count = 9
+Key = 7c36a9bb3033bc6f7395155eadf0e07c8e5b3441d0ad66b21625d4950760386c
+IV = c8393fb1d80ce92801a4fd906a568f7f404a82b02096e859e70e46d1ca5e231a073c5acbaa4cb4c33581e6887c402753bd55f95c76e68bfcbb1cb21bd37ab7a226e03d03e9dca6589c3020f5f916c50676e8c387f9b1710579a728ba7e7b60955ee5e383bb75d2b9d0f2abc72c02edd925bb32dc5a994f032e9a856931eb1ca3
+CT = 559b2ff3f5fd147b9889146f9fdea6758e5e0c716395cf1caf577dc2707764833099bda0910626c62bbb1ca010b66c54114982
+AAD =
+Tag = cd2ed4fc
+PT = 178745b297a23a897ec5cfe3a9e373befbdb840d9eb657885ad0423628c4a18f934e6fb57974a52436c517f4463cc5f9370c54
+
+Count = 10
+Key = 981afbf7e7b74f08d186616d1f71b682bccc3cff6c5560696d267ad455d111f6
+IV = a2d07ac3ef29978c44ebbd83e1ad330a8fcfad8213fe2e924390015bc966a944a0a76831189a011094ec4ef98535efeb56b871e7e1aa36748e639dd5f9d1bf3286a1b2965bfc029faa0f855622c30cad67331bd11dbcea51e397185cbc4f0f0341fd8e744d2f09b2e3c2bd03af15850dbe2a701855ed4247f97acf9754f5e4f9
+CT = cbb1f9a5bd84c4b1b8df2714f87db878f7d2658cc7c37f75d784e2157687398a391ecdfd1119e087bd12f6af79db50ae7711bc
+AAD =
+Tag = ad794c97
+PT = 06ea96ad8e6044978ea676056df8c647b7bdfce3923750983cdca875089841612737e6fe078496d77906b9606532b309851cdd
+
+Count = 11
+Key = 03183678896e28b84e16ac41ebb14f4f436efe386ee6df4e8ad2a7aaf11f17d6
+IV = 718ec99fa1b9b1d29a06ca3973d9c0323b14a2cc34cfa2816481aa2da97b435b0a075a2ed6412bb482bea23df9deddd16944492b1756c65138c3d189b8d2d695150667f46edce88755e868a2d90bf13f170d9b6bb29d9210f3c9f507663756866ede0b362aa5c859e15cd96da4f8c7f7852b3924bdf35ff3a515ba5150e1b017
+CT = 6ecd41a492ae5d6295e9c18290c9a36999c79c87f8b69ff20cb42ccb7c6678baaf159c75ecfb15cb87db99a3236734001545d2
+AAD =
+Tag = cad99689
+FAIL
+
+Count = 12
+Key = 2ca8b01d1cbb8d392bae40bd8a51205a9020be27a23533da51dfe1ad0c4c1d41
+IV = 5b5edcc2f17942afb9577c3d2ed7d5ecaf009ac3ebac985fcf1e0fac0dfdfe747fdfe3d05795337baf41cea3b26e4f35caee1c13fc52d1192da145f376b4ea810ce7dc94845a9ca9184203c3b8e803e7a9bfcbf4a310c85b28b04a007e8e9bd14ff0ae28a1966918a6e22ae8415334e7df0d530b0507a24f755f70f117581820
+CT = c420656fb66e89d5b10fbe3ec0929286683ddc4a34cbaca638493f5c09673609814127709b6b1bb765902f6857761a8d57d98f
+AAD =
+Tag = 23f1dc74
+FAIL
+
+Count = 13
+Key = 25cb38a4b7ff73bb632ecae5f75d46e45a108ffaf3ec2d6ad39d3af4b3c64ca5
+IV = 372510e18f877e0f74c1cc54b19d265b27a452cbe91339bb720aa1bdfaf9bbe5365c571ce8f01d2e96aef8bf089c3f4402f186213be72b46b200337c9ebf943bf3d2db1f68c8e655534d9198825737e623745c26f6b0a82585660a7cc3985a271dea9b20f93653701a8d383bbf3155864809decc03ffefc9ce018379d12d8bfc
+CT = ea01ae67abee8f8552ad260ca9d08ea5b35b53667a3455718545e007e5ac0c62c1ff0c5b06f8c031079fce5f2367889a6a068e
+AAD =
+Tag = c6a365f1
+PT = 3bc70116886ed9b4ef795e45c6ec8ea65f6285b3449174f89ceb1294ea73dae9b2f037107f57355be7242abb7da818c98d2755
+
+Count = 14
+Key = 19bb98022f5d140cdbb5b1c02aae8eeec1e96dc6eb489d70967588b6f414330a
+IV = 6d7b41c7f949f8ff3e9e18ff7af3d67eff5ddaa62eefdbc0b0a49dfb6fb07582998250d1c8e609d57510c859333a268f7e89bca06adf1646cdeb2e592bc86769aba402410cbd71f572dbe065beb37d8766ac61c12e7ac322d213407e073d4bb3c28848c42959cab21f9e39d7f4ff8debd50f40bfff96cbf81af07fbffb6bb2b0
+CT = 60a77e3d27fcea5e505221382d82e9ee39c2bfaa01d7d6ce0d293e7fc7bd0d7f900afa9a7f080c33c04cedd76573a914409e39
+AAD =
+Tag = b561ee30
+FAIL
diff --git a/src/test/crypto/gcmEncryptExtIV128.rsp b/src/test/crypto/gcmEncryptExtIV128.rsp
new file mode 100644
index 0000000000..69c74ca6c5
--- /dev/null
+++ b/src/test/crypto/gcmEncryptExtIV128.rsp
@@ -0,0 +1,129 @@
+# CAVS 14.0
+# GCM Encrypt with keysize 128 test information
+# Generated on Fri Aug 31 11:23:06 2012
+
+[Keylen = 128]
+[IVlen = 1024]
+[PTlen = 408]
+[AADlen = 0]
+[Taglen = 32]
+
+Count = 0
+Key = f8fe56171fa546a34b1b28e0b1d31cfb
+IV = 960d57f1336271e069c12f11044dd5a5bea996fc0290d37b5b2f47c8df3ae3ee37214a6871d963b830aec266026364984cfe31eb88c2a6229f5594ca9d3b6d26c7fadb91a0282cdd0a321714b745dd5e161e7cd192420cf2eacd552c4df5cee8fb5f0e06b7c353017b4b9523ce56899db770c344da720327817ba823a8f71382
+PT = fd229158f18f5b8c2a96c86fa3d8084014660eb2314bbab4ca09fa72c3a98b6faa2ebb83a1809de9ccbc8973d23af34014fb27
+AAD =
+CT = 4dc0fd07c86ec84c264f0544456bfee14f688af2109455d73aa1e58a3354727e05387c94568edf352f8342a6156c64d87c44d7
+Tag = ac350afd
+
+Count = 1
+Key = 803007b69146363999afad4433c0f3b5
+IV = 2fc7688faf0b3783094294526ac07c38a73ff961ab41f20deb66edcabfe68e094aa0f1fc52318706a0e2f9b3901a768346494e846cf17b662d05a3788d77c1468408a49ffe5c0cf68b3b8b26193dfd84c63c4631eebf0c7974283e05e39494d9aaad038018a6e999912b1f92681375214e634f5937cb32ccc4face42d013980c
+PT = bb311134866deba57fe506445c5a312ea1ba1ba16469731c1647c6a482bc84fa8349d82bbd01d3edb6cfe8f25b37ff8ec9d621
+AAD =
+CT = aa1d6ae4151aaec1030a6f0297c48e67b84d1c397e9eaf0c5c8c3d252bf8638bf8591342c5c20c7f88f41140b0334f55cb7b04
+Tag = 30076c5b
+
+Count = 2
+Key = bfb96f58535443205ce281e03bdc5c38
+IV = 3961f47e9e800f1c72526d73fe372cd6e69a7eb1d2692c58bf4297bb05503edf95e074a7cf2644981f72421f229d93866a6e1fcf5c13953b39bf36b56fcbd4c09b505e550b5ba0e9bc26efbe0b9621a47b81842caa8c945fa5bb606f0dab824a4bb5a2625668a916e47f0a0d8a995bcae6940e120724f6d53629545da5456008
+PT = 2c07d76ceac2a77809906bebd3452e2f898ea5467d47be1f17573b3f7fc7d11c9b868d1d1a24010b63dabe9c6c6b4e123df559
+AAD =
+CT = ca443acf08d121e1ae4221013ec40dca2237d035e89ae67040602132972417e07a7c770d75d96fbb5b8a38e048abb15bb978d2
+Tag = f61f75ee
+
+Count = 3
+Key = f3a44f15d104f81b4bb263eecf806737
+IV = bbf1b0646991c2b9735066dad5860fcc08a6b92944a5e90dfb120acf2a75403d3175f5e61a1d84b89a0c1bdd3b3414450faf6ffb8820ab1ea01a2b3cc05f1cd1de9bd48ee1308ddc7d87e6db33d3a171e7f63fce6b8e0417359afa833f6b5f293195bedf444ac56103ee0c8706a69c08fe59a95c8474f28a4a12661905f8781c
+PT = 62d05ea0b0ddff1c0418b01a21267230ebbf23b63a6c14caa769c9148150c2454c055cbe4a72a08e7cd8dcb456e1ec17bc3a63
+AAD =
+CT = df08c4d223a168d4dea9445f5b88d40ef0a796de3e77a6a116bfed840ccbe7b988345d070640f62f5757878420b5c50dd9f567
+Tag = 1518c2d1
+
+Count = 4
+Key = ccd58a6017ac344ce5f8ebebdbe03593
+IV = b2c53f013021f494f6637876f62ce5b5dec6d548ffe58d0952aa8fd8fd5c8d2b835165e1b0ca01e72c19f962e38cd4458229a3415d7b4f9afddf5bb63215999b750c07a080677ba4e40f6c5e42038882503c9923a6eb2cf0d3b82f9f94e624f9938830fc22430f16f6c93c362cb3c11cb05d63becdb4c572f03431e7108369c2
+PT = 0b89a455a2470b3b2b7e04afac15c45a0742061494b78f88c57f2505e1f5804f35c0a829f8b6443e427fb6ecd374642217b6f7
+AAD =
+CT = a8b6203b914391c61dbc123efc6902b892107ea9724341a22ecdd0deb16c48d885d606f68724cb43a956c07ef9ef654c042906
+Tag = 02aeaa63
+
+Count = 5
+Key = 50bcf114df40a431c0e1e88033154a25
+IV = 0ffcd9b14c6bf5f630c86b41cb3cf96004f3fa4fd48ee87b7235d34be0be1fdddeaa79abb3c0c9198a1eea0ccf04e8cfc8d24e4badbf438c59a70b435fbf07d44f55b75e5e48fa0f3d7714aaf9e34b430640614646d0008014ad432a464a252c66584c922b29a90cb2c2e4237c8545f913cd2ed3910a4f075062a55b71228411
+PT = 9f5206b2afe824ac4303d58a97255dea6f026b8531651105db9695f09acb16afd9488928060219307fd41edc16e49f01b7d646
+AAD =
+CT = 0143c0a035ce29d1418acb2561dfe88c74f24a9808a8672427797cc0ba01693c2d66c0e365961cfd58fd039bf08fb4c2b1be29
+Tag = 552e27c7
+
+Count = 6
+Key = 8dbf11f923374b8e8be93788de939806
+IV = 2fcb130962ba3e3eb8e31a7a26e2082a643f39d67cce11d8b2ba8a782f63d4df375b21b1fedaf67bd8d73b2208937ae941afc99420ebbe328214fe6a456bf00979d5ebfb22b79fd3cdea81056747bf4e4ded33f2f26f2d228965128a3d0a32696db44e4aff6ca5467d3c749830a5d2e9a41b4ffa3a422e5bb870cee84f4b64af
+PT = 1c57186740c4901e022e63b2e7b085f9cb60c83763e357a591c1968277920ade1987334e88ed9c3c96665e37fe492a975153d0
+AAD =
+CT = ce4c7e5fb87ed02424a13cd2cfcfaacf67f1072198dedc594db26d6453991821861ba6cc843c6d2750e846e162415123b7e01e
+Tag = ab10b040
+
+Count = 7
+Key = 6e88bed99ebe380c1f8297f46019d8a0
+IV = 80fb82c0016d054491f396a7722217b0a07bfd0fc954a7141bc1e2e7958cb24541a21492ec85d3c744489f93ae3abb9af101a78f2366226080389d29eef564d5205f377ab0902043bbf7ba64c30c9d2c945cd6f29654738106dd282194fa02344ec177b5547531061b31cebfac4a2b0f46b68e44c8c89f6942f9c9c13e50c58e
+PT = 6c90ea3dad3e172ae6784cf1b7c08fc04d0f9b4372f1103d11393a8f02e2f495b53c57cf46f1df6f55b2fdb184fd2b7d590402
+AAD =
+CT = f55d67b524b8e633019e9b1736f3db1a254e53cd71fecb48f2dceffa62256a0bfc3775f6506db52db6c6eb91971f5bb5688325
+Tag = b4585815
+
+Count = 8
+Key = 99de57ce03d1db62a751c3b2c7d38f3c
+IV = 2e74c283e216ab5ac9a1b214dd9280431d7ff942a17715fadc3a22978fa1ccc0743f969358baea3d79abe93388a7db82d82ad6a917bf67795fb4360543d7f22f7ee49f41029ebb87573ac03ddb7e279f1846f4a88b85bea63e2c9b9ecf6f91777434b3def0d4a42d3e025eb43a666a28d5f6c834c7e7991897bf051915e646c6
+PT = 72dc7e5533e862efe0d23c62095506b11c9b256d8d18d11511aa1ed4eae67b0017ec74e322f3a7a18e7d199e7093cadf26680a
+AAD =
+CT = af31d880d1820a35c9248ee0b1aa0da31339f90182e60451493bec8d4dec3baf922268741c2717831b8365bebf072aa7931ab4
+Tag = b1700396
+
+Count = 9
+Key = acdc98a8baa0f003c130ce196135334d
+IV = b7c9885d302842c41a2880b9382584806e8cb55b49183b80bc50403c82cfaf0d28a00fef813ba5b7e35b80dc1f0b7c3b0669a3bc739f499d77ed9cc47a467ca62fc34c5bf4c374ff396c01a472c3dfdab394e6926545a1c20363960c72dfad3eebf9a970e6579e3eff7a38f6bcf0373a8494d450d12445f9ff62c233dc1d2379
+PT = 30f1c7fb5fc152dad6911623ca4af1eb495e108ed94b6e6cc19eabeaa7b85262ea3cc4dc5297aa6f7cf504ac6e07db5db550e6
+AAD =
+CT = b2cbae4df8898ca223213824a5c08e16eda81f063916e813bf2d0d8c7e8a75b2d0a9f6de91e08d5422970534331cf1dd53fa8e
+Tag = 8ef08260
+
+Count = 10
+Key = 75d651a377ccdc0e743d73b8205fb38c
+IV = c7296796ef031d372284f7b1481a13862aec243792ca73f40e11bfe39da28984f11d591d3294c833babf05a1f19b603f4f4a9ca1102f201c1405b6cb45facc8ef408541963abcafa907b2eb8e5c1c404b0e4884a48bbb2d43add4dc1c44c526295bcfbd8f2b7041ed49189e835cdf4ed00dcbb450eef4070482f5e8b52360966
+PT = 13ab743d9db511857f0f79a84742c225b3692a9aee8a63697d42fe50d74fe028ccf95e18a2dd2d9b778392ee7a5d2f23b399eb
+AAD =
+CT = 09c3f153ef712fcb3b5fca2b9bf7f25740fb748bec64bc35576ed01682030ae2728d4282140264819e8c4dedd48e29199a6236
+Tag = ae374dd4
+
+Count = 11
+Key = 15414fda594aa87a3e7af69df769adbf
+IV = 7f021a78293e7cb4dd0af221efda3149b0cab87241b597865267cbab5aad530ea4aa4b10815ba9318a45fcb22fd0e6d692d7beecc2042fa2791f6cca5f9916b0bbae79e9d91133aa54d15a1397f8b063695a3d36b8e573a866fc94964f39016e9490c37189cbb0638db09548a91688d73e2c0d4542f5bd08e03ac0d75e36f519
+PT = bc8263ccf50d0224d088546bc16e2577925567ca52d98ec45cb43b190159bbd5f0326d4498a8a88c0ea0b0a79420b906cd5115
+AAD =
+CT = 34b07942793b066b74b7fb8a4ce71a04f9b29e40dca351f5b6e0939bc2819cd95e69bd58163a4df9729e3d6220a6df60a4ba9e
+Tag = bcf4aed9
+
+Count = 12
+Key = 4d73cae96ded98e1688104a63f462c76
+IV = 331b51fd88cd4731e0eca051717b642f86ea6d6941f9a7331ff361e0edc3f9ea4c013d585f3eff70004a696c4b51d7c589ca97e5fd30f4b2f99c0f3ac83769c2397e10669b7b83aef714a2388638b8941efde8e631098dd78742772f484edd568fefe26b9d981b437e4e4f3ad25445e1aa8c8608c655a5090d2cac158ee67e98
+PT = bf55d48ae5015c39bb167782cab391510d7b7d698e9d3faacd7d409fe4d86fea0a6f61128b6aa305ac0fe4cee4d582dd30e717
+AAD =
+CT = 3cf29f4d50feac5cda4c7cef91d563b2573096b7f32c723355d8d59b8e1a0e229dc2f6114f1db6bdc26212043d153709597c1d
+Tag = 76fcb529
+
+Count = 13
+Key = 7045a9d7caae2c0a39f58998720974ec
+IV = a9028008f708ae19dd28e75a02c6e84a05096779781ce0a908047152208468cc4b2a57d25608767f936cf70fbe82dd8a61497a180e1fc967caf4e6310ed850082e6919c922e021ec070dc64b040ed9edbbf5883676630d69e48953068b2bd006bd6d5417038604ee5aed04980c9ed2316c531ce6a3a73dfad90c04c58596d5e9
+PT = 635b946047c38533bb2cb4c9799b44f6eae0e63626901b0741f6dcf3c2bb02270343c7708a72dfe303b20f7805cb732386b341
+AAD =
+CT = aebb8c1e914ea9bc1ada9b01f84cc8dbbc611f2cd386d5fc89497d37e5a469b28fe2fdf0ab0f1c882dcce50620b1b18a2d8343
+Tag = 27bcaf06
+
+Count = 14
+Key = 79d2a35efbf03f57b66e875c232e10d5
+IV = 7cc96b48afff401adf4bab2ebedd021377b18a819f3c3af39fda42e24c5d62e67ec30f8bcab00263dc5a9bb06cbfe1750c98555901d34d775fdcc86841bd08fbeb44ba68ee794dc351a29a1a9de576d83c17a730d50db79cab88d538a441bb9ff6aa073a2a976de820ab5cc61f834753220d4e472a275dcd13db3e51a23a84a5
+PT = d91e1b2811b3e3894b46c563e6ea0b4a33990ba4fce8a354c941e1effc5691671de5d97c4c1a35e3730b43584944695f00544a
+AAD =
+CT = 6a45beb05c0dba6c38c997f8c37ef07c7cf78eacff6ff4dd6fa000e745e8053d2d270a746994f29c8628f41fc7fecdac158655
+Tag = 36caee75
diff --git a/src/test/crypto/gcmEncryptExtIV256.rsp b/src/test/crypto/gcmEncryptExtIV256.rsp
new file mode 100644
index 0000000000..3faa8956bb
--- /dev/null
+++ b/src/test/crypto/gcmEncryptExtIV256.rsp
@@ -0,0 +1,129 @@
+# CAVS 14.0
+# GCM Encrypt with keysize 256 test information
+# Generated on Fri Aug 31 11:26:26 2012
+
+[Keylen = 256]
+[IVlen = 1024]
+[PTlen = 408]
+[AADlen = 0]
+[Taglen = 32]
+
+Count = 0
+Key = 0df25c2bc9444b4a01e26d357a3ac0635fb6ff2e65ce1e759aae491a17772243
+IV = 54652573ba189cffed3bcfa60efbfb417eb4b0e8de80c7e53765d018cbddfae74617269eb35f29faf628d28c40737f9d9e1eb0b8757c984d94340ecc5ddd108f0a5a0e96335ea805950d378fea7569b98693e88bc1cfd9f6eb8d25de177122fc774a5b1957bcb80e92230c12fe401a8e00d0c04897e234644bda36ae761ea619
+PT = b29af460c6a5dbe56f1d67751346d7182c93413a6c328c6d85176cd8dea8ecce1cad3063c8708c0be9ae73d42bbb10421e73f1
+AAD =
+CT = 3c786a3d0c8945bf320a21ca63f3b8bf5c6bf56a8412f836d7894e42c9e0695a8e41bf59b23fc52d17f8b341183f1cdce02e22
+Tag = 0b31bffe
+
+Count = 1
+Key = 1b6b7b8d00e543f0a17bd0bf595319a4a1f8a55ff41ce2380381d4e83c83243b
+IV = 791d9546f180b838b50bb7b66b68f7f1aa09a2cef411a1dbf2e64c5ed026613200ac8f0e5b961925853621a1d4339322ea4b7bbc4adcbc008efeaaee0dc948a916b22ff8693a7d441620d0ff67680b23567f9582e22eec529408c6d0a00de1bd1ee5a11ed7fa29b7567f990e412ce90ee12d5b1d8ba8c2b528d0d9963fdc49a5
+PT = 0a76090de676d71d9f5ee8511864f0c9440a0fa12b5155a5bffc36127bc957b293d4fb624af3b956385783124f28f6f3c0f0e8
+AAD =
+CT = fdaa9ef2c65d7666b61367f843863a3b273249192551b5633cfaf84a5ab9ecba42916395177f9a16c1ad385e77393cd93d71b3
+Tag = fd854d63
+
+Count = 2
+Key = 261a0382c739093634502303d60dd0d2a568b5155147d661e7789f7bd70de82c
+IV = e22330a2981b9b60354a740c49dcd17c9016cff50423977f7fc8500fc36a81610c979eb37a1f9c4d54d11b790906492205263178dca6d269a230a595ee95edeb60e45d92a2d6169877dfe5514b23db143dbcf3ca44e13cf5b7402e4f95a9e6760451be2d57d1d5fefb015ffe4b69456d87338865ee8775d1cce238f75c345dad
+PT = 42c8c79988c52bcf38865a6f341a5165294af0947bc4fa597c6f648a1172070851c4ee154604d9a21c8d53f7eba77fbb72cf2b
+AAD =
+CT = 0fb368563c6ad0c097207239a791bc685274d434c4a54cb0ea97a98b4cee9eb100c4cd626ba82c338a3f51cb29391b1da3a44f
+Tag = a9607e42
+
+Count = 3
+Key = 5f3d6b5c76cb7ad0be3d771f1f7d1bafd88c8c7c7c84922f89f45f3b7453eb44
+IV = 7072d302e4acb642090c5d48e4549a8823725be1389316df0152e68c41f77937eadc4ee1d164101717cdcdd3bc9be3c2669c0b1394953090ac787fb117500b6275122f608e70a3f5063b21fae42ca4b00724b21c50e27c37a77d4befb118018ada7999888442e1271410f6b804c36a27a41e95a438792de4f06fd7122b177cc3
+PT = 00d82cc00c07d3cf0fd9b31ddc091d351aab9b58af07d2c59f3e28c7202ef6e3ec35726cbdd14f8f1985a770d092470f115be7
+AAD =
+CT = 0aab0dbb31c113b33b116659294e0af5497c987af870ae8ce55288d48e8a4f1b3e8377cb34d06460daf3cc99fbefb90724a39f
+Tag = 26e56b13
+
+Count = 4
+Key = 1d4b6b6ba43269d46ca5c0fab38b61f1d2bf3d24dae46181fac73420a8e6194f
+IV = 01c283a90c588dafc585f437dc111e94dd8ea98a622a5d2e554a87086ec10e8dac9b205fae70be3c024b3f9fbdc26208a6e44d082ad92a51a7fa1f861d0e93e7e74d4e41426e70b2e7ef7fbcbc5a302aed4bc8a42963ca53334259f1924e74225c3bf9aa2ffdf97a2be6474c72f6dedba7c454e36fbc3537596a63b61a4cf3ce
+PT = 4eb036d945c14de26c4e0c83d446dacb57e91b5bdec2ef612bbed8135e57ce62b28843fbac555580ef23c74959eb869a017872
+AAD =
+CT = 24d67f2e94294255b9fd90c16b67c7a8b3a47a42782954aa15e80ffe732d64342b1ec65aa66fc4d6b8fe9dfcdbf4d4c1a979e7
+Tag = 37358e86
+
+Count = 5
+Key = cbfbb15a4fae4af3de55ffcaa4b8cdbd515ecc72fd50060b22acc8fad57a5f4b
+IV = 86e47ba98d8ed360262f94afc364df373d1e44c788b4ee5ed7542822858aaeca2b3c07b5464d6f7edefca9759107bbb64c086d526cc4c9a6b9a7cd6dbb50acec2297597612fd436d8c8ddbc83664305b214cf2c3b6fab3e545c499c7f1cbba1897041475f94952d8a4fc10110986b5ccc7a10c1d9e6427dabba8d942ada9290a
+PT = 5470a60f2818ffef374be3cc591084096a4a8caa0dac0024c12d304382c301264526b02efb674dfbcc3be5c818a1d88a7c19f1
+AAD =
+CT = a6a8ebadfe5c5ac754e4a2a9cdc25fdf3588c69a287b0f9aba2347bfcdc110a1134426a45c62d577e52a8d61fae39ea38bad51
+Tag = 97406b4b
+
+Count = 6
+Key = 3ced5c63078a36f6b02d3bec0debacd19021c8dbc501a9d86f556c4ad2cdef8a
+IV = b9629801cba22f7493b4f6394620d49a76ec686e524f5ebcb3a76b5a189473484060cbb01bcbf10048427ef21527626085c8a75aa5264b6338abbc26171c2c3a44f6b3b5c3fd05a892c8290a8f99be962deea48d7ae4e626c45a45ffda5efaad6e54e98ba876b039a5dde3d6061f217e57da4774acfaf1f5da9495083aad4dad
+PT = c29788107799077ac6dbccad29a346727f263676a8510fbcadb9b5bf53df978b3382fdfc1b5c3312eaa0f7621b6efdaeebe1c0
+AAD =
+CT = 5811d4a96169b39cd8f8ff3f931efdda68550f17558b48de8bb5adf455af5179c8c5ff4a73f363f8819daf846427132daf6a17
+Tag = 06e8e7fa
+
+Count = 7
+Key = 39ae4fdfe56b74a71325a1a685a6593b44936892890cd05d2719a420c97a7c64
+IV = 8f110d0fd20ece7f35c35b2eed32316fd742ae9346fc6907bd749361a4436427f80185b376b16a36fb95429cfaf2e22e46c210442fe5efc14985a9d9d847c3ceb5db02e0d999acbc3ba0afdfecbfaf65024258cc7f6fc8e3f568cbedec1c7eabe3ff3ab3c7331722b6400429d46b54820bd1f88ac03cfba5cbd0812d91342c4b
+PT = 8b086d11be7ff55312addad86b49585ea38ea1ee7c4200964cd269a4bb5cfe0f518e6f9b733efb4ba3ce35ce2e803b0ad47d24
+AAD =
+CT = be46d6e61dc11b2fc8ac4c9c5f49ceca0fb6fe1fc7221c7cc5d8ba254a92282500b1b31528314035cd125578de960b3bafb69d
+Tag = 7c00aa14
+
+Count = 8
+Key = c244e9afd90ec810acb7e586cf7a06386e6892e01c7d111d5c1455fc95250d1b
+IV = 6ff9103f9751ff4743d856c5cd54491e1a537384260fbfe076f772ad0d66ff6ddd0aaff57023885b0a4d60d2b25c80b1cdc54802607770a61a2503c23cf26f1fe529573c8d4745b19dedaff5769a6a796c01540776d4fa99be9057ad87cdc973e7938640f9497753e88c4cdc358c1cfa06f1ddd826c33f44c55e2d183927baaa
+PT = cd1296133cefe6f5cf6f8ec68b37172bfb793a8582d92a539f24f3582569bfcfebb706fe9f276716b185558fbfd6fe8ea99a1b
+AAD =
+CT = dd1694b178f15612c454885b3100c576c7b68206c57898161d4fdc51e75a428840c5cfee104c3d85fcbd92edbd1d80f22d8e64
+Tag = 741953f5
+
+Count = 9
+Key = 785ea9cd2403ecaeec3e4940dce7c41ec012203a2610c780bca5d15af64748c5
+IV = fbc177ec9d47f69e2fcaba9fef7de30735e46a5d20dc66bf66c8a76a382051d780f58dccae8e2054bbe437a5bc0814381bac2b0efa99202aa1f1bf7f51b842907dc9b60f83987c31eae086e26e2018243bdd47a291b523ac6905b40dfb442ba239c876cdfaa581b2ef0683456ac944829a97b663ecd48c116d06f1f054ddfcdb
+PT = a3465ad9a4009e26c39acb1d424b7ba6556a74dcc2b78a5ab65b5d07c2a97f382aadb7415395fa6fc90bd137f6894a75f70907
+AAD =
+CT = 2b3eb5ab251282da1795a25f9b43ccaa5a27643d042fc315ad662947b0c5cddeb5848e6c69869ae5e81f5c76729bbaceda9889
+Tag = 64173526
+
+Count = 10
+Key = 0becf9c7a069db5b9a25f2871fb0594e452126262ec1c48bca3d3024d85d0c51
+IV = e05643a72ab0b1fc42a6f17302d6446fc507bbb4f0ad59178f5084530f02534df2e673e92e67802629f93221bba545a13fb9143eb2a2ee4bce047be621a9d96a450a19951d93c527eb698ac7e132e4f00985dff91e079ae791549d37da3105c77ef5d8cb7649f1aa761b5a5fbc9e0d7dfb5aade98f3df4a641cca02f33eed55c
+PT = 70ba9822801760e5ec647a41e27cf6069978be6a28be0b1da0f3661e124847823dec7ec6af737d6dea597ce5bd5baa5d6f2651
+AAD =
+CT = 0d45b5ad77226524374a3bca3680528787cefd9978b3f1407688886ce28fea86e7d9005d7d6acde2a7c2fb158479918c06bba4
+Tag = 43dfd859
+
+Count = 11
+Key = d1f374702b4481b83e7c7783853c1850e887b0c80cd28c0686b4cab6adc744c2
+IV = 5e15c44cef36c1c89e02b2979f4e7d275fe5973a80580753f6fd51ea477ba84346a030b90b22a9adfff02096bc0b1691c37ee21cdc1b5f4862696d195859cd2bf0b423f5d19da5e1475bcc99f96b8c7c51fe85930aea0e97304d4c025b52bd386666d0537036360e939a6fbdfd6ce3b012e551d333fa74f3cbb9d33a59477364
+PT = 996428a232b1b9ac81ff5260eb77938f0d531a4a0ed3bd774c72c18128af72e964f8d05fb7ec4e0ac8e37056a48c85713a3a01
+AAD =
+CT = aa9c103ae51bf556f7c57f9948d28c859f458a74e22f039fea183e9e262b023bdac9bb5a2f167b34cde4a694c555eb0c905f34
+Tag = d1ed9cf0
+
+Count = 12
+Key = a6a69524d66f015b4afb746ad0942410baf06d1aba18ef1a8b40e35633f04ce1
+IV = 5e1df758c63728b1269a1d3f610b6e3724bc7ff797dcb4a7aca9dfe1c8e1717ce4281d1c5b4b33d0aafcee342a4f4eb30089eeab2b0470d3f9b709622bde4654d41b3bbc6bf59c11edea28f26b099d83d4fefdc63ab7218221238fe1c230b1290235465b445f60eaa9822eaa8da2c08f6f2fdaaef87dacf74c7e5d6cea191c74
+PT = 0ffc6c3090817fc1a0e6f3802269e263e40d17772fabf5cbee905962878d77c8bc4223e5671bd9f310d8db56ceaebae41fc79e
+AAD =
+CT = f24537db3fe27eb07f1d2b9b7059dfae97df86d4e609930491c4ec3154462df308ba1c85204dd754521d8de9619603f90f8c44
+Tag = 360adc36
+
+Count = 13
+Key = ff0007317f9ecc43c1601708614eb8d443a407318cbf3c085c0a9a7c67faafa8
+IV = 8186f92bd394ecfbf4bcbb9e7e442bd6faa1e59f8785d9aec82551fee38aa10212a8477be3055927379a906902c153598d50c63e316c4e4f3956ef04ce40e5b6f1901fb5d08a23b913e5ad53512b02bb75ce96f8c50a2cebdfa058ded996fe1dc5370ee50c6d90e948129d544dc89c2f28dda8429bb338d6aed0d9557f96889e
+PT = e7f4063319fd31e07192a5fc9e486fc0a2a3470671a46356ab0d32989803259c0dae103a4033c533fbde585866fc2af5eba151
+AAD =
+CT = 8f86644614e4f6483f28eec02fab33e7cbbd56381a8f878522c2015b91d48652472e356f608c62361939dc983e1cd364a28665
+Tag = 83335b3a
+
+Count = 14
+Key = 43646d9ddebec23447febe71596e6f9b2387965db1faf3f06feec8bc5c808342
+IV = 0d3774607261b07427dc77f0dcf57b026226dbd2c1df1b9d74598582da44c677af36a6bd80b6bc1f000632c84c5dfa701f8b51b4da228d340d8b4ccc4d2f5d7b5fad00809133eae9250ecd18d7a8741bb57c394396feb81c20bec23de8520f883b8e22b362dde6e6bd9dad73e8919695384a04c09a28bdb6de2fa356b0d259b1
+PT = 2aab9a484b817b759bd9d876967c90160a18208cfa753e7bccd4f73a715aaa6acc6ce666e97bc22fbcf11f263dfed332418707
+AAD =
+CT = dd80974e31ad15ba26ec8a8dacfa5f57f0bc69f7113cc6c39fb038ddce37ffeee3f789a02cd86b0418fe16ca104b5fbdb26432
+Tag = 75c35f4c
diff --git a/src/test/crypto/t/001_testcrypto.pl b/src/test/crypto/t/001_testcrypto.pl
new file mode 100644
index 0000000000..586a37c5bd
--- /dev/null
+++ b/src/test/crypto/t/001_testcrypto.pl
@@ -0,0 +1,137 @@
+# Reads and parses .rsp files and runs them through testcrypto
+#
+# Test vectors downloaded from:
+#
+# https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program/cavp-testing-block-cipher-modes
+#
+# Specifically GCM Test Vectors (SP 800-38D):
+#
+# https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Algorithm-Validation-Program/documents/mac/gcmtestvectors.zip
+#
+# Note that the AADlen > 0 cases were removed from our set, since we don't support that and
+# the test code will just skip them currently, and we also don't bother testing 192-bit,
+# but one could download the full set and run all of the files through if they wished and
+# all should pass (they did when this test suite was written originally).
+
+use strict;
+use warnings;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_ssl} eq 'openssl')
+{
+ plan tests => 56;
+}
+else
+{
+ plan skip_all => 'SSL not supported by this build';
+}
+
+
+# XXX add CTR tests here
+
+my $algo = "AES-GCM";
+my @rspfiles = (
+ "gcmDecrypt128.rsp", "gcmDecrypt256.rsp",
+ "gcmEncryptExtIV128.rsp", "gcmEncryptExtIV256.rsp");
+
+note "running tests";
+
+foreach my $rspfile (@rspfiles)
+{
+ open(my $in_rspfile, '<', $rspfile) || die;
+ my %testrun;
+ my %lengths;
+
+ while (my $line = <$in_rspfile>)
+ {
+
+ chomp($line);
+
+ # Remove CR, if it's there.
+ $line =~ s/\r$//;
+
+ # Skip comments
+ if ($line =~ /^[[:space:]]*#/) { next; }
+
+ # If we hit a blank, time to run a test
+ if ($line =~ /^[[:space:]]*$/)
+ {
+ if (%testrun)
+ {
+ my @testargs;
+
+ # Set up the command to run
+ push(@testargs, ("$ENV{TESTDIR}/testcrypto", '-a', $algo));
+
+ if ($testrun{'Key'})
+ {
+ push(@testargs, ('-k', $testrun{'Key'}));
+ }
+
+ if ($testrun{'IV'})
+ {
+ push(@testargs, ('-i', $testrun{'IV'}));
+ }
+
+ if ($testrun{'CT'})
+ {
+ push(@testargs, ('-c', $testrun{'CT'}));
+ }
+
+ if ($testrun{'AAD'})
+ {
+ # Don't currently support AAD
+ undef(%testrun);
+ next;
+ }
+
+ if ($testrun{'Tag'})
+ {
+ push(@testargs, ('-t', $testrun{'Tag'}));
+ }
+
+ if ($testrun{'PT'})
+ {
+ push(@testargs, ('-p', $testrun{'PT'}));
+ }
+
+ if ($testrun{fail})
+ {
+ command_exit_is(\@testargs, 1,
+ "Run $testrun{Count} of Keylen: $lengths{Keylen}, IVlen: $lengths{IVlen}, PTlen: $lengths{PTlen}, AADlen: $lengths{AADlen}, Taglen: $lengths{Taglen}"
+ );
+ }
+ else
+ {
+ command_ok(\@testargs,
+ "Run $testrun{Count} of Keylen: $lengths{Keylen}, IVlen: $lengths{IVlen}, PTlen: $lengths{PTlen}, AADlen: $lengths{AADlen}, Taglen: $lengths{Taglen}"
+ );
+ }
+ undef(%testrun);
+ undef(%lengths);
+ }
+ else
+ {
+ next;
+ }
+ }
+
+ # Grab length information, just to have.
+ if ($line =~ /^\[([A-Za-z]*) = ([0-9]*)]$/)
+ {
+ $lengths{$1} = $2;
+ next;
+ }
+
+ if ($line =~ /^([A-Za-z]*) = ([a-f0-9]*)$/)
+ {
+ $testrun{$1} = $2;
+ }
+
+ if ($line =~ /^FAIL$/)
+ {
+ $testrun{fail} = 1;
+ }
+ }
+}
diff --git a/src/test/crypto/t/002_testkwp.pl b/src/test/crypto/t/002_testkwp.pl
new file mode 100644
index 0000000000..25911cfd9f
--- /dev/null
+++ b/src/test/crypto/t/002_testkwp.pl
@@ -0,0 +1,126 @@
+# Reads and parses .rsp files and runs them through testcrypto
+#
+# (Partial) Test vectors downloaded from:
+#
+# https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program/cavp-testing-block-cipher-modes
+#
+# Specifically Key Wrap Test Vectors (SP 800-38F):
+#
+# https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Algorithm-Validation-Program/documents/mac/kwtestvectors.zip
+#
+# We don't include the 192-bit tests, though they all worked when this test suite was written.
+# We also don't include the _inv tests as those aren't supported in OpenSSL yet.
+
+use strict;
+use warnings;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_ssl} eq 'openssl')
+{
+ plan tests => 20;
+}
+else
+{
+ plan skip_all => 'SSL not supported by this build';
+}
+
+
+my $algo;
+# my @txtfiles = ("KWP_AD_128.txt", "KWP_AD_192.txt", "KWP_AD_256.txt", "KWP_AE_128.txt", "KWP_AE_192.txt", "KWP_AE_256.txt");
+my @txtfiles =
+ ("KWP_AD_128.txt", "KWP_AD_256.txt", "KWP_AE_128.txt", "KWP_AE_256.txt");
+
+note "running tests";
+
+foreach my $txtfile (@txtfiles)
+{
+ open(my $in_txtfile, '<', $txtfile) || die;
+ my %testrun;
+ my %lengths;
+
+ if ($txtfile =~ /^KWP_/)
+ {
+ $algo = 'AES-KWP';
+ }
+
+ while (my $line = <$in_txtfile>)
+ {
+
+ chomp($line);
+
+ # Remove CR, if it's there.
+ $line =~ s/\r$//;
+
+ # Skip comments
+ if ($line =~ /^[[:space:]]*#/) { next; }
+
+ # If we hit a blank, time to run a test
+ if ($line =~ /^[[:space:]]*$/)
+ {
+ if (%testrun)
+ {
+ my @testargs;
+
+ # Set up the command to run
+ push(@testargs, ("$ENV{TESTDIR}/testcrypto", '-a', $algo));
+
+ if ($testrun{'K'})
+ {
+ push(@testargs, ('-k', $testrun{'K'}));
+ }
+
+ if ($testrun{'C'})
+ {
+ push(@testargs, ('-c', $testrun{'C'}));
+ }
+
+ if ($testrun{'P'})
+ {
+ push(@testargs, ('-p', $testrun{'P'}));
+ }
+
+ if ($testrun{fail})
+ {
+ command_exit_is(\@testargs, 1,
+ "Run $testrun{COUNT} of Plaintext Length: $lengths{'PLAINTEXT LENGTH'}"
+ );
+ }
+ else
+ {
+ command_ok(\@testargs,
+ "Run $testrun{COUNT} of Plaintext Length: $lengths{'PLAINTEXT LENGTH'}"
+ );
+ }
+ undef(%testrun);
+ undef(%lengths);
+ }
+ else
+ {
+ next;
+ }
+ }
+
+ # Grab length information, just to have.
+ if ($line =~ /^\[([A-Za-z ]*) = ([0-9]*)]$/)
+ {
+ $lengths{$1} = $2;
+ next;
+ }
+
+ if ($line =~ /^([A-Z]) = ([a-f0-9]*)$/)
+ {
+ $testrun{$1} = $2;
+ }
+
+ if ($line =~ /^COUNT = ([0-9]*)$/)
+ {
+ $testrun{COUNT} = $1;
+ }
+
+ if ($line =~ /^FAIL$/)
+ {
+ $testrun{fail} = 1;
+ }
+ }
+}
diff --git a/src/test/crypto/t/003_clusterkey.pl b/src/test/crypto/t/003_clusterkey.pl
new file mode 100644
index 0000000000..5b8a255ace
--- /dev/null
+++ b/src/test/crypto/t/003_clusterkey.pl
@@ -0,0 +1,93 @@
+# Test cluster file encryption key managment
+#
+
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_ssl} eq 'openssl')
+{
+ plan tests => 6;
+}
+else
+{
+ plan skip_all => "tests cannot run without OpenSSL";
+}
+
+# generate two cluster file encryption keys of random hex digits
+my ($rand_hex, $rand_hex2);
+$rand_hex .= sprintf("%x", rand 16) for 1 .. 64;
+$rand_hex2 .= sprintf("%x", rand 16) for 1 .. 64;
+
+# initialize cluster using the first cluster key
+my $node = get_new_node('node');
+$node->init(
+ extra => [
+ '--file-encryption-method', 'AES256',
+ '--cluster-key-command', "echo $rand_hex"
+ ]);
+
+# Set wal_level to 'replica'; encryption can't use 'minimal'
+$node->append_conf('postgresql.conf', 'wal_level=replica');
+
+$node->start;
+
+# check encryption method
+my $file_encryption_method =
+ $node->safe_psql('postgres', 'SHOW file_encryption_method;');
+ok($file_encryption_method eq 'AES256', 'file_encryption_method is valid');
+
+# record pg_proc count
+my $old_pg_proc_count =
+ $node->safe_psql('postgres', 'SELECT COUNT(*) FROM pg_proc;');
+ok($old_pg_proc_count > 0, 'pg_proc count is valid');
+
+# create permanent table
+$node->safe_psql('postgres',
+ 'CREATE TABLE perm_table (x) AS SELECT * FROM generate_series(1, 100);');
+
+# create unlogged table
+# Non-permanent tables like unlogged tables use a special nonce bit, so test those here.
+$node->safe_psql('postgres',
+ 'CREATE UNLOGGED TABLE unlog_table (x) AS SELECT * FROM generate_series(1, 200);'
+);
+
+# We can run pg_alterckey and change the cluster_key_command here
+# without affecting the running server.
+system_or_bail(
+ 'pg_alterckey',
+ "echo $rand_hex",
+ "echo $rand_hex2",
+ $node->data_dir);
+
+$node->safe_psql('postgres',
+ "ALTER SYSTEM SET cluster_key_command TO 'echo $rand_hex2'");
+
+$node->stop;
+
+# start/stop with new cluster key
+$node->start;
+
+# check encryption method
+$file_encryption_method =
+ $node->safe_psql('postgres', 'SHOW file_encryption_method;');
+ok($file_encryption_method eq 'AES256', 'file_encryption_method is valid');
+
+# check pg_proc count
+my $new_pg_proc_count =
+ $node->safe_psql('postgres', 'SELECT COUNT(*) FROM pg_proc;');
+ok($new_pg_proc_count == $old_pg_proc_count, 'old/new pg_proc counts match');
+
+# check permanent table count
+my $perm_table_count =
+ $node->safe_psql('postgres', 'SELECT COUNT(*) FROM perm_table;');
+ok($perm_table_count == 100, 'perm_table_count count matches');
+
+# check unlogged table count
+my $unlog_table_count =
+ $node->safe_psql('postgres', 'SELECT COUNT(*) FROM unlog_table;');
+ok($unlog_table_count == 200, 'unlog_table_count count matches');
+
+$node->stop;
diff --git a/src/test/crypto/t/004_buffers.pl b/src/test/crypto/t/004_buffers.pl
new file mode 100644
index 0000000000..a1379b4772
--- /dev/null
+++ b/src/test/crypto/t/004_buffers.pl
@@ -0,0 +1,157 @@
+# Test cluster file encryption buffer management
+
+# This tests that an encrypted server actually encrypts the database
+# files. It does this by checking for strings in the database files in
+# both non-encrypted and encrypted clusters. We test a system table, a
+# permanent relation, and a unlogged/non-permanent table.
+# (Non-permanent relations use a special nonce bit, which is why we test
+# it here.)
+
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_ssl} eq 'openssl')
+{
+ plan tests => 17;
+}
+else
+{
+ plan skip_all => "tests cannot run without OpenSSL";
+}
+
+my %file_match_count;
+
+sub get_cluster_file_contents
+{
+ my $node = shift();
+ my %relnode;
+
+ # get postgres database oid
+ my $postgres_db_oid = $node->safe_psql('postgres',
+ "SELECT oid FROM pg_database WHERE datname = 'postgres';");
+ ok($postgres_db_oid != 0, 'retrieving postgres database oid');
+
+ # get pg_proc relfilenode
+ $relnode{pg_proc} =
+ $node->safe_psql('postgres', "SELECT pg_relation_filenode('pg_proc');");
+ ok($relnode{pg_proc} != 0, 'retrieving pg_proc relfilenode');
+
+ # create permanent table
+ $node->safe_psql('postgres',
+ "CREATE TABLE perm_table (x) AS SELECT 'aaaaaaaa' FROM generate_series(1, 100);"
+ );
+
+ # get permanent table relfilenode
+ $relnode{perm} = $node->safe_psql('postgres',
+ "SELECT pg_relation_filenode('perm_table');");
+ ok($relnode{perm} != 0, 'retrieving permanent table relfilenode');
+
+ # create unlogged table
+ $node->safe_psql('postgres',
+ "CREATE UNLOGGED TABLE unlog_table (x) AS SELECT 'bbbbbbbb' FROM generate_series(1, 200);"
+ );
+
+ # get unlogged table relfilenode
+ $relnode{unlog} = $node->safe_psql('postgres',
+ "SELECT pg_relation_filenode('unlog_table');");
+ ok($relnode{unlog} != 0, 'retrieving unlogged table relfilenode');
+
+ my $file_contents =
+ slurp_file($node->basedir .
+ '/pgdata/base/' . $postgres_db_oid . '/' . $relnode{pg_proc});
+ # () converts to list context
+ $file_match_count{pg_proc} = () = $file_contents =~ m/pg_[a-z]{3,}/g;
+
+ $file_contents =
+ slurp_file($node->basedir .
+ '/pgdata/base/' . $postgres_db_oid . '/' . $relnode{perm});
+ $file_match_count{perm} = () = $file_contents =~ m/a{8,}/g;
+
+ $file_contents =
+ slurp_file($node->basedir .
+ '/pgdata/base/' . $postgres_db_oid . '/' . $relnode{unlog});
+ $file_match_count{unlog} = () = $file_contents =~ m/b{8,}/g;
+}
+
+#
+# Test with disabled encryption
+#
+
+# initialize cluster
+my $non_encrypted_node = get_new_node('non_encrypted_node');
+$non_encrypted_node->init();
+
+$non_encrypted_node->start;
+
+# check encryption is disabled
+my $file_encryption_method =
+ $non_encrypted_node->safe_psql('postgres', 'SHOW file_encryption_method;');
+ok($file_encryption_method eq '', 'file_encryption_method is valid');
+
+get_cluster_file_contents($non_encrypted_node);
+
+# record pg_proc count
+my $query_pg_proc_count = $non_encrypted_node->safe_psql('postgres',
+ "SELECT COUNT(*) FROM pg_proc WHERE proname ~ '^pg_[a-z]{3,}';");
+ok($query_pg_proc_count > 0, 'pg_proc count is valid');
+
+# check pg_proc count
+ok($file_match_count{pg_proc} != $query_pg_proc_count,
+ 'SQL/file pg_proc counts match');
+
+# check permanent table count
+ok($file_match_count{perm} != 100, 'perm_table count matches');
+
+# check unlogged table count
+ok($file_match_count{unlog} != 200, 'unlog_table count matches');
+
+$non_encrypted_node->stop;
+
+
+#---------------------------------------------------------------------------
+
+#
+# Test with enabled encryption
+#
+
+my $rand_hex;
+$rand_hex .= sprintf("%x", rand 16) for 1 .. 64;
+
+# initialize cluster using a cluster key
+my $encrypted_node = get_new_node('encrypted_node');
+$encrypted_node->init(
+ extra => [
+ # We tested AES256 in 003, so use AES128
+ '--file-encryption-method', 'AES128',
+ '--cluster-key-command', "echo $rand_hex"
+ ]);
+
+# Set wal_level to 'replica'; encryption can't use 'minimal'
+$encrypted_node->append_conf('postgresql.conf', 'wal_level=replica');
+
+$encrypted_node->start;
+
+# check encryption method
+$file_encryption_method =
+ $encrypted_node->safe_psql('postgres', 'SHOW file_encryption_method;');
+ok($file_encryption_method eq 'AES128', 'file_encryption_method is valid');
+
+get_cluster_file_contents($encrypted_node);
+
+# Because the files are encrypted, we should get zero matches for all
+# comparisons below. However, technically the encrypted data might
+# match the desired string, so we allow one such match.
+
+# check pg_proc
+ok($file_match_count{pg_proc} <= 1, 'pg_proc is encrypted');
+
+# check permanent table
+ok($file_match_count{perm} <= 1, 'perm_table is encrypted');
+
+# check unlogged table
+ok($file_match_count{unlog} <= 1, 'unlog_table is encrypted');
+
+$encrypted_node->stop;
diff --git a/src/test/crypto/testcrypto.c b/src/test/crypto/testcrypto.c
new file mode 100644
index 0000000000..85bf921c09
--- /dev/null
+++ b/src/test/crypto/testcrypto.c
@@ -0,0 +1,458 @@
+/*-------------------------------------------------------------------------
+ *
+ * testcrypto.c
+ * A utility to test our encryption / decryption routines.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/crypto/testcrypto.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#define FRONTEND 1
+
+#define EXITSUCCESS 0
+#define EXITDECRYPTFAIL 1
+#define EXITFAILURE 2
+
+#include "postgres_fe.h"
+
+#include <signal.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/stat.h>
+
+#include "common/hex.h"
+#include "common/cipher.h"
+#include "common/logging.h"
+#include "getopt_long.h"
+#include "pg_getopt.h"
+
+static const char *progname;
+
+static void
+usage(const char *progname)
+{
+ printf(_("%s tests encryption/decryption routines in PG common library.\n\n"), progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION]\n"), progname);
+ printf("\n");
+ printf(_(" Performs one encryption and one decryption run.\n"));
+ printf("\n");
+ printf(_(" Encrypts the provided plaintext (or the empty string if none given) and generates a tag, if using AES-GCM, using the key and IV given.\n"));
+ printf(_(" After encryption, compares provided ciphertext to resulting ciphertext.\n"));
+ printf(_(" Compares provided tag, if any, to resulting tag.\n"));
+ printf(_(" If no tag is provided, then the tag created during encryption is used during decryption.\n"));
+ printf("\n");
+ printf(_(" Decrypts the provided ciphertext (or the empty string if none given) using the key, and IV + tag given if using AES-GCM.\n"));
+ printf(_(" After successful decryption (requires tag to match for AES-GCM), compares provided plaintext to resulting plaintext.\n"));
+ printf(_(" Exits with '1' if decryption fails.\n"));
+ printf("\n");
+ printf(_(" Exits with '2' for any other failure.\n"));
+ printf("\n");
+ printf(_(" Key is always required, IV is required for AES-GCM mode.\n"));
+ printf("\n");
+ printf(_(" Algorithms supported are AES-GCM and AES-KWP.\n"));
+ printf("\n");
+ printf(_("\nOptions:\n"));
+ printf(_(" -a, --algorithm=ALG Crypto algorithm to use\n"));
+ printf(_(" -i, --init-vector=IV Initialization vector to use\n"));
+ printf(_(" -k, --key=KEY Key to use, in hex\n"));
+ printf(_(" -p, --plain-text=PT Plain text to encrypt\n"));
+ printf(_(" -c, --cipher-text=CT Cipher text to decrypt\n"));
+ printf(_(" -t, --tag=TAG Tag to use for decryption\n"));
+ printf(_(" -T, --tag-length=LEN Length of tag to use for encryption\n"));
+ printf(_(" -v, --verbose verbose output\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf("\n");
+ printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+}
+
+
+int
+main(int argc, char *argv[])
+{
+ char *algorithm = NULL,
+ *iv_hex = NULL,
+ *key_hex = NULL,
+ *plaintext_hex = NULL,
+ *ciphertext_hex = NULL,
+ *tag_hex = NULL;
+
+ unsigned char *plaintext = NULL,
+ *ciphertext = NULL,
+ *key = NULL,
+ *iv = NULL,
+ *tag = NULL,
+ *tag_result = NULL,
+ *result = NULL;
+
+ int verbose = 0,
+ plaintext_len = 0,
+ ciphertext_len = 0,
+ key_len = 0,
+ iv_len = 0,
+ tag_len = 16,
+ result_len = 0,
+ blocksize = 0,
+ cipher = PG_CIPHER_AES_GCM;
+
+ PgCipherCtx *ctx = NULL;
+
+ static struct option long_options[] = {
+ {"algorithm", required_argument, NULL, 'a'},
+ {"init-vector", required_argument, NULL, 'i'},
+ {"key", required_argument, NULL, 'k'},
+ {"plain-text", required_argument, NULL, 'p'},
+ {"cipher-text", required_argument, NULL, 'c'},
+ {"tag", required_argument, NULL, 't'},
+ {"tag-length", required_argument, NULL, 'T'},
+ {"verbose", required_argument, NULL, 'v'},
+ {NULL, 0, NULL, 0}
+ };
+
+ int c;
+
+ pg_logging_init(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("testcrypto"));
+ progname = get_progname(argv[0]);
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage(progname);
+ exit(EXITSUCCESS);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("testcrypto (PostgreSQL) " PG_VERSION);
+ exit(EXITSUCCESS);
+ }
+ }
+
+ /* Process command-line argument */
+
+ while ((c = getopt_long(argc, argv, "a:i:k:p:c:t:T:v", long_options, NULL)) != -1)
+ {
+ switch (c)
+ {
+ case 'a':
+ algorithm = pg_strdup(optarg);
+ break;
+
+ case 'i':
+ iv_hex = pg_strdup(optarg);
+ break;
+
+ case 'k':
+ key_hex = pg_strdup(optarg);
+ break;
+
+ case 'p':
+ plaintext_hex = pg_strdup(optarg);
+ break;
+
+ case 'c':
+ ciphertext_hex = pg_strdup(optarg);
+ break;
+
+ case 't':
+ tag_hex = pg_strdup(optarg);
+ break;
+
+ case 'T':
+ tag_len = atoi(optarg);
+ break;
+
+ case 'v':
+ verbose = 1;
+ break;
+
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(EXITFAILURE);
+ }
+ }
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ pg_log_error("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(EXITFAILURE);
+ }
+
+ /* Check options passed in */
+ if (algorithm)
+ {
+ if (strcmp(algorithm, "AES-GCM") == 0)
+ cipher = PG_CIPHER_AES_GCM;
+ else if (strcmp(algorithm, "AES-KWP") == 0)
+ cipher = PG_CIPHER_AES_KWP;
+ else
+ {
+ pg_log_error("Unsupported algorithm selected.");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(EXITFAILURE);
+ }
+ }
+
+ if (key_hex == NULL)
+ {
+ pg_log_error("Key must be provided");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(EXITFAILURE);
+ }
+ else
+ {
+ size_t key_hex_len = strlen(key_hex);
+
+ key_len = pg_hex_dec_len(key_hex_len);
+
+ key = pg_malloc0(key_len);
+ pg_hex_decode(key_hex, key_hex_len, (char *) key, key_len);
+ }
+
+ if (cipher == PG_CIPHER_AES_GCM && iv_hex == NULL)
+ {
+ pg_log_error("Initialization vector must be provided");
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(EXITFAILURE);
+ }
+ else if (cipher == PG_CIPHER_AES_GCM)
+ {
+ size_t iv_hex_len = strlen(iv_hex);
+
+ iv_len = pg_hex_dec_len(iv_hex_len);
+
+ iv = pg_malloc0(iv_len);
+ pg_hex_decode(iv_hex, iv_hex_len, (char *) iv, iv_len);
+ }
+
+ if (plaintext_hex)
+ {
+ size_t plaintext_hex_len = strlen(plaintext_hex);
+
+ plaintext_len = pg_hex_dec_len(plaintext_hex_len);
+
+ plaintext = pg_malloc0(plaintext_len);
+ pg_hex_decode(plaintext_hex, plaintext_hex_len, (char *) plaintext,
+ plaintext_len);
+ }
+
+ /*
+ * OpenSSL 1.1.1d and earlier crashes on some zero-length plaintext and
+ * ciphertext strings. It crashes on an encryption call to
+ * EVP_EncryptFinal_ex(() in GCM mode of zero-length strings if plaintext
+ * is NULL, even though plaintext_len is zero. Setting plaintext to
+ * non-NULL allows it to work. In KWP mode, zero-length strings fail if
+ * plaintext_len = 0 and plaintext is non-NULL (the opposite). OpenSSL
+ * 1.1.1e+ is fine with all options.
+ */
+ else if (cipher == PG_CIPHER_AES_GCM)
+ {
+ plaintext_len = 0;
+ plaintext = pg_malloc0(1);
+ }
+
+ if (ciphertext_hex)
+ {
+ size_t ciphertext_hex_len = strlen(ciphertext_hex);
+
+ ciphertext_len = pg_hex_dec_len(ciphertext_hex_len);
+
+ ciphertext = pg_malloc0(ciphertext_len);
+ pg_hex_decode(ciphertext_hex, ciphertext_hex_len,
+ (char *) ciphertext, ciphertext_len);
+ }
+ /* see OpenSSL 1.1.1d item above, though crash only happens in GCM mode */
+ else if (cipher == PG_CIPHER_AES_GCM)
+ {
+ ciphertext_len = 0;
+ ciphertext = pg_malloc0(1);
+ }
+
+ if (cipher == PG_CIPHER_AES_GCM)
+ tag_result = pg_malloc0(tag_len);
+
+ if (tag_hex)
+ {
+ size_t tag_hex_len = strlen(tag_hex);
+
+ tag_len = pg_hex_dec_len(tag_hex_len);
+
+ tag = pg_malloc0(tag_len);
+ pg_hex_decode(tag_hex, tag_hex_len, (char *) tag, tag_len);
+ }
+ else
+ tag = tag_result;
+
+ if (verbose)
+ {
+ printf("Alrogithm: %d\n", cipher);
+ printf("Key length: %d (%d bits)\n", key_len, key_len * 8);
+ printf("IV length: %d (%d bits)\n", iv_len, iv_len * 8);
+ printf("Tag length: %d (%d bits)\n", tag_len, tag_len * 8);
+ printf("Plaintext length: %d\n", plaintext_len);
+ printf("Ciphertext length: %d\n", ciphertext_len);
+ }
+
+ /*
+ * Encryption
+ *
+ * We run through the encryption even if there wasn't a plaintext
+ * provided- in that case we just encrypt the empty string.
+ */
+ ctx = pg_cipher_ctx_create(cipher, key, key_len, true);
+ if (!ctx)
+ {
+ pg_log_error("Error creating encryption context, be sure key is of supported length");
+ exit(EXITFAILURE);
+ }
+
+ blocksize = pg_cipher_blocksize(ctx);
+
+ /* If we were provided with a plaintext input */
+ if (plaintext_len != 0)
+ {
+ /* Encryption might result in as much as input length + blocksize */
+ result_len = plaintext_len + blocksize;
+ result = palloc0(result_len);
+
+ if (ciphertext_hex == NULL)
+ {
+ ciphertext = result;
+ ciphertext_len = plaintext_len;
+ }
+ }
+
+ if (cipher == PG_CIPHER_AES_GCM)
+ {
+ if (!pg_cipher_encrypt(ctx, cipher,
+ plaintext, plaintext_len,
+ result, &result_len,
+ iv, iv_len,
+ tag_result, tag_len))
+ {
+ pg_log_error("Error during encryption.");
+ exit(EXITFAILURE);
+ }
+ }
+ else if (cipher == PG_CIPHER_AES_KWP)
+ {
+ if (!pg_cipher_keywrap(ctx,
+ plaintext, plaintext_len,
+ result, &result_len))
+ {
+ pg_log_error("Error during encryption.");
+ exit(EXITFAILURE);
+ }
+ }
+
+ if (verbose || ciphertext == NULL)
+ {
+ uint64 result_hex_len = pg_hex_enc_len(result_len);
+ char *result_hex = palloc0(result_hex_len + 1);
+
+ pg_hex_encode((char *) result, result_len, result_hex, result_hex_len);
+ result_hex[result_hex_len] = '\0';
+
+ printf("ciphertext: %s\n", result_hex);
+
+ if (cipher == PG_CIPHER_AES_GCM)
+ {
+ result_hex_len = pg_hex_enc_len(tag_len);
+ result_hex = palloc0(result_hex_len + 1);
+
+ pg_hex_encode((char *) tag_result, tag_len, result_hex, result_hex_len);
+ result_hex[result_hex_len] = '\0';
+
+ printf("tag: %s\n", result_hex);
+ }
+ }
+
+ /*
+ * Report on non-matching results, but still go through the decryption
+ * routine to make sure that we get the correct result, and then error
+ * out.
+ */
+ if (plaintext_len != 0 && ciphertext != NULL && memcmp(ciphertext, result, plaintext_len) != 0)
+ pg_log_error("Provided ciphertext does not match");
+
+ if (cipher == PG_CIPHER_AES_GCM && tag != tag_result && memcmp(tag, tag_result, tag_len) != 0)
+ pg_log_error("Provided tag does not match");
+
+ /*
+ * If a ciphertext was provided then use that as the max size of our
+ * plaintext result. We shouldn't ever get a result larger.
+ */
+ if (ciphertext_len != 0)
+ {
+ result_len = ciphertext_len;
+ result = palloc0(result_len);
+ }
+
+ /*
+ * Decryption
+ *
+ * We run through the decryption even if there wasn't a ciphertext
+ * provided- in that case we just decrypt the empty string.
+ */
+ ctx = pg_cipher_ctx_create(cipher, key, key_len, false);
+ if (!ctx)
+ {
+ pg_log_error("Error creating decryption context, be sure key is of supported length");
+ exit(EXITFAILURE);
+ }
+
+ if (cipher == PG_CIPHER_AES_GCM)
+ {
+ if (!pg_cipher_decrypt(ctx, cipher,
+ ciphertext, ciphertext_len,
+ result, &result_len,
+ iv, iv_len,
+ tag, tag_len))
+ {
+ pg_log_error("Error during decryption.");
+ exit(EXITDECRYPTFAIL);
+ }
+ }
+ else if (cipher == PG_CIPHER_AES_KWP)
+ {
+ if (!pg_cipher_keyunwrap(ctx,
+ ciphertext, ciphertext_len,
+ result, &result_len))
+ {
+ pg_log_error("Error during decryption.");
+ exit(EXITDECRYPTFAIL);
+ }
+ }
+
+ if (verbose || plaintext == NULL)
+ {
+ uint64 result_hex_len = pg_hex_enc_len(result_len);
+ char *result_hex = palloc0(result_hex_len + 1);
+
+ pg_hex_encode((char *) result, result_len, result_hex, result_hex_len);
+ result_hex[result_hex_len] = '\0';
+
+ printf("plaintext: %s\n", result_hex);
+ }
+
+ if (ciphertext_len != 0 && plaintext != NULL && memcmp(plaintext, result, plaintext_len) != 0)
+ {
+ pg_log_error("Provided plaintext does not match");
+ exit(EXITFAILURE);
+ }
+
+ exit(EXITSUCCESS);
+}
--
2.31.1
[application/octet-stream] v2-0010-cfe-10-hint_over_cfe-09-test-squash-commit.patch (17.6K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/11-v2-0010-cfe-10-hint_over_cfe-09-test-squash-commit.patch)
download | inline diff:
From a396672cf4c919b273a8ad8a103e659c1b2e7d1b Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:57:03 -0400
Subject: [PATCH v2 10/12] cfe-10-hint_over_cfe-09-test squash commit
---
doc/src/sgml/config.sgml | 7 +-
src/backend/access/rmgrdesc/xlogdesc.c | 6 +-
src/backend/access/transam/xlog.c | 4 +
src/backend/access/transam/xloginsert.c | 24 +++++
src/backend/replication/logical/decode.c | 1 +
src/backend/storage/buffer/bufmgr.c | 131 ++++++++++++++++-------
src/bin/pg_alterckey/pg_alterckey.c | 76 +++++++++++--
src/include/access/xlog.h | 14 +--
src/include/access/xloginsert.h | 2 +
src/include/catalog/pg_control.h | 1 +
10 files changed, 214 insertions(+), 52 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 4e95f7e22a..b1b3914933 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3164,9 +3164,10 @@ include_dir 'conf.d'
</para>
<para>
- If data checksums are enabled, hint bit updates are always WAL-logged
- and this setting is ignored. You can use this setting to test how much
- extra WAL-logging would occur if your database had data checksums
+ If data checksums or cluster file encryption is enabled,
+ hint bit updates are always WAL-logged and this setting is
+ ignored. You can use this setting to test how much extra
+ WAL-logging would occur if your database had data checksums
enabled.
</para>
diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c
index 3fd7185f21..e08c364d03 100644
--- a/src/backend/access/rmgrdesc/xlogdesc.c
+++ b/src/backend/access/rmgrdesc/xlogdesc.c
@@ -80,7 +80,8 @@ xlog_desc(StringInfo buf, XLogReaderState *record)
appendStringInfoString(buf, xlrec->rp_name);
}
- else if (info == XLOG_FPI || info == XLOG_FPI_FOR_HINT)
+ else if (info == XLOG_FPI || info == XLOG_FPI_FOR_HINT ||
+ info == XLOG_ENCRYPTION_LSN)
{
/* no further information to print */
}
@@ -196,6 +197,9 @@ xlog_identify(uint8 info)
case XLOG_FPI_FOR_HINT:
id = "FPI_FOR_HINT";
break;
+ case XLOG_ENCRYPTION_LSN:
+ id = "ENCRYPTION_LSN";
+ break;
}
return id;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index e247376ead..38e46bbbf7 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8012,6 +8012,10 @@ xlog_redo(XLogReaderState *record)
UnlockReleaseBuffer(buffer);
}
}
+ else if (info == XLOG_ENCRYPTION_LSN)
+ {
+ /* nothing to do here */
+ }
else if (info == XLOG_BACKUP_END)
{
/* nothing to do here, handled in xlogrecovery_redo() */
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index f811523448..b4bbacae26 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -1081,6 +1081,30 @@ XLogSaveBufferForHint(Buffer buffer, bool buffer_std)
return recptr;
}
+/*
+ * This function returns either a WAL or fake LSN, for use by encryption.
+ */
+XLogRecPtr
+LSNForEncryption(bool use_wal_lsn)
+{
+ if (use_wal_lsn)
+ {
+ int dummy = 0;
+
+ Assert(FileEncryptionEnabled);
+ /*
+ * Records other than SWITCH_WAL must have content. We use an integer 0 to
+ * follow the restriction.
+ */
+ XLogBeginInsert();
+ XLogSetRecordFlags(XLOG_MARK_UNIMPORTANT);
+ XLogRegisterData((char *) &dummy, sizeof(dummy));
+ return XLogInsert(RM_XLOG_ID, XLOG_ENCRYPTION_LSN);
+ }
+ else
+ return GetFakeLSNForUnloggedRel();
+}
+
/*
* Write a WAL record containing a full image of a page. Caller is responsible
* for writing the page to disk after calling this routine.
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 2cc0ac9eb0..6922f6c0e2 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -162,6 +162,7 @@ xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
case XLOG_FPI_FOR_HINT:
case XLOG_FPI:
case XLOG_OVERWRITE_CONTRECORD:
+ case XLOG_ENCRYPTION_LSN:
break;
default:
elog(ERROR, "unexpected RM_XLOG_ID record type: %u", info);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 73d30bf619..256f468474 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3991,11 +3991,12 @@ IncrBufferRefCount(Buffer buffer)
* This is essentially the same as MarkBufferDirty, except:
*
* 1. The caller does not write WAL; so if checksums are enabled, we may need
- * to write an XLOG_FPI_FOR_HINT WAL record to protect against torn pages.
+ * to write an XLOG_FPI_FOR_HINT record to protect against torn pages, or
+ * XLOG_ENCRYPTION_LSN to generate a new LSN for the page.
* 2. The caller might have only share-lock instead of exclusive-lock on the
- * buffer's content lock.
+ * buffer's content lock.
* 3. This function does not guarantee that the buffer is always marked dirty
- * (due to a race condition), so it cannot be used for important changes.
+ * (due to a race condition), so it cannot be used for important changes.
*/
void
MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
@@ -4041,53 +4042,111 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
* If we need to protect hint bit updates from torn writes, WAL-log a
* full page image of the page. This full page image is only necessary
* if the hint bit update is the first change to the page since the
- * last checkpoint.
+ * last checkpoint. If cluster file encryption is enabled, we also
+ * need to generate new page LSNs for all other cases of page writes.
*
* We don't check full_page_writes here because that logic is included
* when we call XLogInsert() since the value changes dynamically.
*/
- if (XLogHintBitIsNeeded() &&
- (pg_atomic_read_u32(&bufHdr->state) & BM_PERMANENT))
+ if (XLogHintBitIsNeeded())
{
/*
- * If we must not write WAL, due to a relfilelocator-specific
- * condition or being in recovery, don't dirty the page. We can
- * set the hint, just not dirty the page as a result so the hint
- * is lost when we evict the page or shutdown.
+ * If we must not write WAL during recovery so don't dirty the page.
+ * We can set the hint, just not dirty the page as a result so the
+ * hint is lost when we evict the page or shutdown.
*
* See src/backend/storage/page/README for longer discussion.
*/
if (RecoveryInProgress() ||
- RelFileLocatorSkippingWAL(BufTagGetRelFileLocator(&bufHdr->tag)))
+ (RelFileLocatorSkippingWAL(BufTagGetRelFileLocator(&bufHdr->tag)) &&
+ !FileEncryptionEnabled))
return;
/*
- * If the block is already dirty because we either made a change
- * or set a hint already, then we don't need to write a full page
- * image. Note that aggressive cleaning of blocks dirtied by hint
- * bit setting would increase the call rate. Bulk setting of hint
- * bits would reduce the call rate...
- *
- * We must issue the WAL record before we mark the buffer dirty.
- * Otherwise we might write the page before we write the WAL. That
- * causes a race condition, since a checkpoint might occur between
- * writing the WAL record and marking the buffer dirty. We solve
- * that with a kluge, but one that is already in use during
- * transaction commit to prevent race conditions. Basically, we
- * simply prevent the checkpoint WAL record from being written
- * until we have marked the buffer dirty. We don't start the
- * checkpoint flush until we have marked dirty, so our checkpoint
- * must flush the change to disk successfully or the checkpoint
- * never gets written, so crash recovery will fix.
- *
- * It's possible we may enter here without an xid, so it is
- * essential that CreateCheckPoint waits for virtual transactions
- * rather than full transactionids.
+ * Non-BM_PERMANENT objects don't need full page images because
+ * they are not restored. WAL-skipped relfilenodes should never
+ * have full page images generated.
*/
- Assert((MyProc->delayChkptFlags & DELAY_CHKPT_START) == 0);
- MyProc->delayChkptFlags |= DELAY_CHKPT_START;
- delayChkptFlags = true;
- lsn = XLogSaveBufferForHint(buffer, buffer_std);
+ if (pg_atomic_read_u32(&bufHdr->state) & BM_PERMANENT &&
+ !RelFileLocatorSkippingWAL(BufTagGetRelFileLocator(&bufHdr->tag)))
+ {
+ /*
+ * If the block is already dirty because we either made a change
+ * or set a hint already, then we don't need to write a full
+ * page image. Note that aggressive cleaning of blocks dirtied
+ * by hint bit setting would increase the call rate. Bulk
+ * setting of hint bits would reduce the call rate...
+ *
+ * We must issue the WAL record before we mark the buffer
+ * dirty. Otherwise we might write the page before we write
+ * the WAL. That causes a race condition, since a checkpoint
+ * might occur between writing the WAL record and marking the
+ * buffer dirty. We solve that with a kluge, but one that is
+ * already in use during transaction commit to prevent race
+ * conditions. Basically, we simply prevent the checkpoint WAL
+ * record from being written until we have marked the buffer
+ * dirty. We don't start the checkpoint flush until we have
+ * marked dirty, so our checkpoint must flush the change to disk
+ * successfully or the checkpoint never gets written, so crash
+ * recovery will fix.
+ *
+ * It's possible we may enter here without an xid, so it
+ * is essential that CreateCheckpoint waits for virtual
+ * transactions rather than full transactionids.
+ */
+ MyProc->delayChkptFlags |= DELAY_CHKPT_START;
+ delayChkptFlags = true;
+ lsn = XLogSaveBufferForHint(buffer, buffer_std);
+ }
+
+ /*
+ * Above, for hint bit changes, we might have generated a new page
+ * LSN and a full-page WAL record for a page's first-clean-to-dirty
+ * during a checkpoint for permanent, non-WAL-skipped relfilenodes.
+ * If we didn't (the lsn variable is invalid), and we are
+ * doing cluster file encryption, we must generate a new
+ * page LSN here for either non-permanent relations or page
+ * non-first-clean-to-dirty during a checkpoint. (Cluster file
+ * encryption does not support WAL-skip relfilenodes.) We must
+ * update the page LSN even if the page with the hint bit change is
+ * later overwritten in the file system with an earlier version of
+ * the page during crash recovery.
+ *
+ * XXX Can we rely on the full page write above with no lock being
+ * held to avoid torn pages? Above, the LSN and page image are
+ * tied together, but here is just the page LSN update.
+ */
+ if (XLogRecPtrIsInvalid(lsn) && FileEncryptionEnabled)
+ {
+ /*
+ * For cluster file encryption we need a new page LSN because
+ * the LSN is used, with the page number and permanent flag, as
+ * part of the nonce, and the nonce must be unique for every
+ * page write. If we reencrypt a page with hint bit changes
+ * using the same nonce as previous writes, it would expose the
+ * hint bit change locations. To avoid this, we write a simple
+ * WAL record to advance the lsn, which can then be assigned to
+ * the page below.
+ *
+ * Above we are relying on the full page writes to revert
+ * any partial pages writes caused by this LSN change for
+ * permanent, non-WAL-skip relfilenodes. For non-permanent
+ * relations, they crash recover as empty. For WAL-skip
+ * relfilenodes, they recover with their original contents, so
+ * that works too.
+ */
+ /* XXX Do we need the checkpoint delay here? */
+ MyProc->delayChkptFlags |= DELAY_CHKPT_START;
+ delayChkptFlags = true;
+ /*
+ * XXX We probably don't need to replay this WAL on the primary
+ * since the full page image is restored, but do we have
+ * to replay this on the repicas (for relations that are
+ * replicated)?
+ */
+ lsn = LSNForEncryption(
+ pg_atomic_read_u32(&bufHdr->state) & BM_PERMANENT);
+ }
}
buf_state = LockBufHdr(bufHdr);
diff --git a/src/bin/pg_alterckey/pg_alterckey.c b/src/bin/pg_alterckey/pg_alterckey.c
index 7e3d4ad915..c699c59761 100644
--- a/src/bin/pg_alterckey/pg_alterckey.c
+++ b/src/bin/pg_alterckey/pg_alterckey.c
@@ -38,10 +38,9 @@
#include "common/file_perm.h"
#include "common/file_utils.h"
-#include "common/hex.h"
#include "common/restricted_token.h"
-#include "crypto/kmgr.h"
#include "common/logging.h"
+#include "crypto/kmgr.h"
#include "getopt_long.h"
#include "pg_getopt.h"
@@ -84,6 +83,9 @@ static void bzero_keys_and_exit(exit_action action);
static void reencrypt_data_keys(void);
static void install_new_keys(void);
+static uint64 hex_decode(const char *src, size_t len, char *dst);
+
+
static void
usage(const char *progname)
{
@@ -511,8 +513,8 @@ retrieve_cluster_keys(void)
(char *) cluster_key_hex,
ALLOC_KMGR_CLUSTER_KEY_LEN,
live_path, terminal_fd);
- if (pg_hex_decode(cluster_key_hex, cluster_key_len,
- (char *) old_cluster_key, KMGR_CLUSTER_KEY_LEN) !=
+ if (hex_decode(cluster_key_hex, cluster_key_len,
+ (char *) old_cluster_key) !=
KMGR_CLUSTER_KEY_LEN)
{
pg_log_error("cluster key must be at %d hex bytes", KMGR_CLUSTER_KEY_LEN);
@@ -534,8 +536,8 @@ retrieve_cluster_keys(void)
(char *) cluster_key_hex,
ALLOC_KMGR_CLUSTER_KEY_LEN,
new_path, terminal_fd);
- if (pg_hex_decode(cluster_key_hex, cluster_key_len,
- (char *) new_cluster_key, KMGR_CLUSTER_KEY_LEN) !=
+ if (hex_decode(cluster_key_hex, cluster_key_len,
+ (char *) new_cluster_key) !=
KMGR_CLUSTER_KEY_LEN)
{
pg_log_error("cluster key must be at %d hex bytes", KMGR_CLUSTER_KEY_LEN);
@@ -722,3 +724,65 @@ bzero_keys_and_exit(exit_action action)
/* return 0 or 1 */
exit(action != SUCCESS_EXIT);
}
+
+/*
+ * HEX
+ */
+
+static const int8 hexlookup[128] = {
+ -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
+ -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
+ -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, -1, -1, -1, -1, -1, -1,
+ -1, 10, 11, 12, 13, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1,
+ -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
+ -1, 10, 11, 12, 13, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1,
+ -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
+};
+
+static inline char
+get_hex(const char *cp)
+{
+ unsigned char c = (unsigned char) *cp;
+ int res = -1;
+
+ if (c < 127)
+ res = hexlookup[c];
+
+ if (res < 0)
+ pg_fatal("invalid hexadecimal digit: \"%s\"",cp);
+
+ return (char) res;
+}
+
+static uint64
+hex_decode(const char *src, size_t len, char *dst)
+{
+ const char *s,
+ *srcend;
+ char v1,
+ v2,
+ *p;
+
+ srcend = src + len;
+ s = src;
+ p = dst;
+ while (s < srcend)
+ {
+ if (*s == ' ' || *s == '\n' || *s == '\t' || *s == '\r')
+ {
+ s++;
+ continue;
+ }
+ v1 = get_hex(s) << 4;
+ s++;
+ if (s >= srcend)
+ pg_fatal("invalid hexadecimal data: odd number of digits");
+
+ v2 = get_hex(s);
+ s++;
+ *p++ = v1 | v2;
+ }
+
+ return p - dst;
+}
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 8f219fbacc..78def5753a 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -107,13 +107,15 @@ extern PGDLLIMPORT int wal_level;
/*
* Is a full-page image needed for hint bit updates?
*
- * Normally, we don't WAL-log hint bit updates, but if checksums are enabled,
- * we have to protect them against torn page writes. When you only set
- * individual bits on a page, it's still consistent no matter what combination
- * of the bits make it to disk, but the checksum wouldn't match. Also WAL-log
- * them if forced by wal_log_hints=on.
+ * Normally, we don't WAL-log hint bit updates, but if checksums or encryption
+ * is enabled, we have to protect them against torn page writes. When you
+ * only set individual bits on a page, it's still consistent no matter what
+ * combination of the bits make it to disk, but the checksum wouldn't match.
+ * Cluster file encryption requires a new LSN for hint bit changes, and can't
+ * tolerate torn pages. Also WAL-log them if forced by wal_log_hints=on.
*/
-#define XLogHintBitIsNeeded() (DataChecksumsEnabled() || wal_log_hints)
+#define XLogHintBitIsNeeded() \
+ (DataChecksumsEnabled() || FileEncryptionEnabled || wal_log_hints)
/* Do we need to WAL-log information required only for Hot Standby and logical replication? */
#define XLogStandbyInfoActive() (wal_level >= WAL_LEVEL_REPLICA)
diff --git a/src/include/access/xloginsert.h b/src/include/access/xloginsert.h
index 001ff2f521..dfb501a7c3 100644
--- a/src/include/access/xloginsert.h
+++ b/src/include/access/xloginsert.h
@@ -61,6 +61,8 @@ extern void log_newpage_range(Relation rel, ForkNumber forknum,
BlockNumber startblk, BlockNumber endblk, bool page_std);
extern XLogRecPtr XLogSaveBufferForHint(Buffer buffer, bool buffer_std);
+extern XLogRecPtr LSNForEncryption(bool use_wal_lsn);
+
extern void InitXLogInsert(void);
#endif /* XLOGINSERT_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index db7de77287..1c97ffd2cd 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -78,6 +78,7 @@ typedef struct CheckPoint
#define XLOG_FPI 0xB0
/* 0xC0 is used in Postgres 9.5-11 */
#define XLOG_OVERWRITE_CONTRECORD 0xD0
+#define XLOG_ENCRYPTION_LSN 0xE0
/*
--
2.31.1
[application/octet-stream] v2-0011-cfe-11-gist_over_cfe-10-hint-squash-commit.patch (7.1K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/12-v2-0011-cfe-11-gist_over_cfe-10-hint-squash-commit.patch)
download | inline diff:
From 3e8db4616f5d567c98a33554bd6151676f7ea95e Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:57:05 -0400
Subject: [PATCH v2 11/12] cfe-11-gist_over_cfe-10-hint squash commit
---
src/backend/access/gist/README | 10 ++++++++++
src/backend/access/gist/gist.c | 21 +++++++++++++++------
src/backend/access/gist/gistbuild.c | 12 +++++++++---
src/backend/access/gist/gistvacuum.c | 20 ++++++++++++++------
4 files changed, 48 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/gist/README b/src/backend/access/gist/README
index efb2730e18..c4d908fcb2 100644
--- a/src/backend/access/gist/README
+++ b/src/backend/access/gist/README
@@ -477,6 +477,16 @@ value. The page is not recycled, until that XID is no longer visible to
anyone. That's much more conservative than necessary, but let's keep it
simple.
+GiST and Encryption
+-------------------
+
+GiST uses LSNs and NSNs for concurrency. This means that unlogged and
+temporary relations also need LSNs. GiST has a mechanism to assign LSNs
+to such relations, but sometimes uses fixed LSNs and or calls
+gistGetFakeLSN(), which can cause duplicate LSNs to be used. Therefore,
+when encryption is enabled and fake LSNs are needed, the GiST code calls
+LSNForEncryption(). See src/backend/crypto/README for more details.
+
Authors:
Teodor Sigaev <[email protected]>
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 30069f139c..74e7e485e6 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -501,13 +501,16 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
* we don't need to be able to detect concurrent splits yet.)
*/
if (is_build)
- recptr = GistBuildLSN;
+ recptr = !FileEncryptionEnabled ? GistBuildLSN :
+ LSNForEncryption(RelationIsPermanent(rel));
else
{
if (RelationNeedsWAL(rel))
recptr = gistXLogSplit(is_leaf,
dist, oldrlink, oldnsn, leftchildbuf,
markfollowright);
+ else if (FileEncryptionEnabled)
+ recptr = LSNForEncryption(RelationIsPermanent(rel));
else
recptr = gistGetFakeLSN(rel);
}
@@ -568,7 +571,8 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
MarkBufferDirty(leftchildbuf);
if (is_build)
- recptr = GistBuildLSN;
+ recptr = !FileEncryptionEnabled ? GistBuildLSN :
+ LSNForEncryption(RelationIsPermanent(rel));
else
{
if (RelationNeedsWAL(rel))
@@ -586,6 +590,8 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
deloffs, ndeloffs, itup, ntup,
leftchildbuf);
}
+ else if (FileEncryptionEnabled)
+ recptr = LSNForEncryption(RelationIsPermanent(rel));
else
recptr = gistGetFakeLSN(rel);
}
@@ -1665,6 +1671,8 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
if (ndeletable > 0)
{
+ XLogRecPtr recptr;
+
TransactionId latestRemovedXid = InvalidTransactionId;
if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
@@ -1690,16 +1698,17 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
/* XLOG stuff */
if (RelationNeedsWAL(rel))
{
- XLogRecPtr recptr;
-
recptr = gistXLogDelete(buffer,
deletable, ndeletable,
latestRemovedXid);
- PageSetLSN(page, recptr);
}
+ else if (FileEncryptionEnabled)
+ recptr = LSNForEncryption(RelationIsPermanent(rel));
else
- PageSetLSN(page, gistGetFakeLSN(rel));
+ recptr = gistGetFakeLSN(rel);
+
+ PageSetLSN(page, recptr);
END_CRIT_SECTION();
}
diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c
index fb0f466708..bf5162b410 100644
--- a/src/backend/access/gist/gistbuild.c
+++ b/src/backend/access/gist/gistbuild.c
@@ -48,6 +48,8 @@
#include "utils/rel.h"
#include "utils/tuplesort.h"
+extern XLogRecPtr LSNForEncryption(bool use_wal_lsn);
+
/* Step of index tuples for check whether to switch to buffering build mode */
#define BUFFERING_MODE_SWITCH_CHECK_STEP 256
@@ -307,7 +309,8 @@ gistbuild(Relation heap, Relation index, IndexInfo *indexInfo)
GISTInitBuffer(buffer, F_LEAF);
MarkBufferDirty(buffer);
- PageSetLSN(page, GistBuildLSN);
+ PageSetLSN(page, !FileEncryptionEnabled ? GistBuildLSN :
+ LSNForEncryption(RelationIsPermanent(index)));
UnlockReleaseBuffer(buffer);
@@ -458,7 +461,9 @@ gist_indexsortbuild(GISTBuildState *state)
gist_indexsortbuild_flush_ready_pages(state);
/* Write out the root */
- PageSetLSN(levelstate->pages[0], GistBuildLSN);
+ RelationGetSmgr(state->indexrel);
+ PageSetLSN(levelstate->pages[0], !FileEncryptionEnabled ? GistBuildLSN :
+ LSNForEncryption(RelationIsPermanent(state->indexrel)));
PageSetChecksumInplace(levelstate->pages[0], GIST_ROOT_BLKNO);
smgrwrite(RelationGetSmgr(state->indexrel), MAIN_FORKNUM, GIST_ROOT_BLKNO,
levelstate->pages[0], true);
@@ -655,7 +660,8 @@ gist_indexsortbuild_flush_ready_pages(GISTBuildState *state)
if (blkno != state->pages_written)
elog(ERROR, "unexpected block number to flush GiST sorting build");
- PageSetLSN(page, GistBuildLSN);
+ PageSetLSN(page, !FileEncryptionEnabled ? GistBuildLSN :
+ LSNForEncryption(RelationIsPermanent(state->indexrel)));
PageSetChecksumInplace(page, blkno);
smgrextend(RelationGetSmgr(state->indexrel), MAIN_FORKNUM, blkno, page,
true);
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index 0aa6e58a62..35265ae0e7 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -24,6 +24,8 @@
#include "storage/lmgr.h"
#include "utils/memutils.h"
+extern XLogRecPtr LSNForEncryption(bool use_wal_lsn);
+
/* Working state needed by gistbulkdelete */
typedef struct
{
@@ -180,6 +182,8 @@ gistvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
vstate.callback_state = callback_state;
if (RelationNeedsWAL(rel))
vstate.startNSN = GetInsertRecPtr();
+ else if (FileEncryptionEnabled)
+ vstate.startNSN = LSNForEncryption(RelationIsPermanent(rel));
else
vstate.startNSN = gistGetFakeLSN(rel);
@@ -359,6 +363,8 @@ restart:
*/
if (ntodelete > 0)
{
+ XLogRecPtr recptr;
+
START_CRIT_SECTION();
MarkBufferDirty(buffer);
@@ -367,16 +373,15 @@ restart:
GistMarkTuplesDeleted(page);
if (RelationNeedsWAL(rel))
- {
- XLogRecPtr recptr;
-
recptr = gistXLogUpdate(buffer,
todelete, ntodelete,
NULL, 0, InvalidBuffer);
- PageSetLSN(page, recptr);
- }
+ else if (FileEncryptionEnabled)
+ recptr = LSNForEncryption(RelationIsPermanent(rel));
else
- PageSetLSN(page, gistGetFakeLSN(rel));
+ recptr = gistGetFakeLSN(rel);
+
+ PageSetLSN(page, recptr);
END_CRIT_SECTION();
@@ -663,8 +668,11 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
if (RelationNeedsWAL(info->index))
recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
+ else if (FileEncryptionEnabled)
+ recptr = LSNForEncryption(RelationIsPermanent(info->index));
else
recptr = gistGetFakeLSN(info->index);
+
PageSetLSN(parentPage, recptr);
PageSetLSN(leafPage, recptr);
--
2.31.1
[application/octet-stream] v2-0012-cfe-12-rel_over_cfe-11-gist-squash-commit.patch (33.0K, ../CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com/13-v2-0012-cfe-12-rel_over_cfe-11-gist-squash-commit.patch)
download | inline diff:
From 48a043538028826cc6e29fa41b0b265bb29a5987 Mon Sep 17 00:00:00 2001
From: Bruce Momjian <[email protected]>
Date: Fri, 25 Jun 2021 16:57:08 -0400
Subject: [PATCH v2 12/12] cfe-12-rel_over_cfe-11-gist squash commit
---
contrib/basic_archive/basic_archive.c | 2 +-
contrib/bloom/blinsert.c | 3 +
src/backend/access/gist/gist.c | 51 ++++----
src/backend/access/gist/gistbuild.c | 12 ++
src/backend/access/hash/hashpage.c | 1 +
src/backend/access/heap/rewriteheap.c | 7 +
src/backend/access/nbtree/nbtree.c | 3 +
src/backend/access/nbtree/nbtsort.c | 4 +-
src/backend/access/spgist/spginsert.c | 6 +
src/backend/access/transam/xlog.c | 2 +
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/storage.c | 7 +-
src/backend/crypto/Makefile | 1 +
src/backend/crypto/bufenc.c | 176 ++++++++++++++++++++++++++
src/backend/postmaster/postmaster.c | 2 +
src/backend/storage/buffer/bufmgr.c | 22 +++-
src/backend/storage/buffer/localbuf.c | 8 +-
src/backend/storage/file/copydir.c | 28 +++-
src/backend/storage/file/reinit.c | 2 +-
src/backend/storage/page/bufpage.c | 51 +++++++-
src/backend/tcop/postgres.c | 2 +
src/include/access/gist.h | 5 +-
src/include/crypto/bufenc.h | 28 ++++
src/include/storage/bufpage.h | 18 ++-
src/include/storage/copydir.h | 2 +-
25 files changed, 396 insertions(+), 48 deletions(-)
create mode 100644 src/backend/crypto/bufenc.c
create mode 100644 src/include/crypto/bufenc.h
diff --git a/contrib/basic_archive/basic_archive.c b/contrib/basic_archive/basic_archive.c
index 9f221816bb..7a59e34a42 100644
--- a/contrib/basic_archive/basic_archive.c
+++ b/contrib/basic_archive/basic_archive.c
@@ -275,7 +275,7 @@ basic_archive_file_internal(const char *file, const char *path)
* Copy the file to its temporary destination. Note that this will fail
* if temp already exists.
*/
- copy_file(unconstify(char *, path), temp);
+ copy_file(unconstify(char *, path), temp, false);
/*
* Sync the temporary file to disk and move it to its final destination.
diff --git a/contrib/bloom/blinsert.c b/contrib/bloom/blinsert.c
index dd26d6ac29..3204950817 100644
--- a/contrib/bloom/blinsert.c
+++ b/contrib/bloom/blinsert.c
@@ -176,7 +176,10 @@ blbuildempty(Relation index)
* XLOG_DBASE_CREATE* or XLOG_TBLSPC_CREATE record. Therefore, we need
* this even when wal_level=minimal.
*/
+ PageEncryptInplace(metapage, INIT_FORKNUM, RelationIsPermanent(index),
+ BLOOM_METAPAGE_BLKNO);
PageSetChecksumInplace(metapage, BLOOM_METAPAGE_BLKNO);
+
smgrwrite(RelationGetSmgr(index), INIT_FORKNUM, BLOOM_METAPAGE_BLKNO,
(char *) metapage, true);
log_newpage(&(RelationGetSmgr(index))->smgr_rlocator.locator, INIT_FORKNUM,
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 74e7e485e6..3472afe363 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -503,17 +503,14 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
if (is_build)
recptr = !FileEncryptionEnabled ? GistBuildLSN :
LSNForEncryption(RelationIsPermanent(rel));
+ else if (RelationNeedsWAL(rel))
+ recptr = gistXLogSplit(is_leaf,
+ dist, oldrlink, oldnsn, leftchildbuf,
+ markfollowright);
+ else if (FileEncryptionEnabled)
+ recptr = LSNForEncryption(RelationIsPermanent(rel));
else
- {
- if (RelationNeedsWAL(rel))
- recptr = gistXLogSplit(is_leaf,
- dist, oldrlink, oldnsn, leftchildbuf,
- markfollowright);
- else if (FileEncryptionEnabled)
- recptr = LSNForEncryption(RelationIsPermanent(rel));
- else
- recptr = gistGetFakeLSN(rel);
- }
+ recptr = gistGetFakeLSN(rel);
for (ptr = dist; ptr; ptr = ptr->next)
PageSetLSN(ptr->page, recptr);
@@ -573,28 +570,26 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
if (is_build)
recptr = !FileEncryptionEnabled ? GistBuildLSN :
LSNForEncryption(RelationIsPermanent(rel));
- else
+ else if (RelationNeedsWAL(rel))
{
- if (RelationNeedsWAL(rel))
- {
- OffsetNumber ndeloffs = 0,
- deloffs[1];
+ OffsetNumber ndeloffs = 0,
+ deloffs[1];
- if (OffsetNumberIsValid(oldoffnum))
- {
- deloffs[0] = oldoffnum;
- ndeloffs = 1;
- }
-
- recptr = gistXLogUpdate(buffer,
- deloffs, ndeloffs, itup, ntup,
- leftchildbuf);
+ if (OffsetNumberIsValid(oldoffnum))
+ {
+ deloffs[0] = oldoffnum;
+ ndeloffs = 1;
}
- else if (FileEncryptionEnabled)
- recptr = LSNForEncryption(RelationIsPermanent(rel));
- else
- recptr = gistGetFakeLSN(rel);
+
+ recptr = gistXLogUpdate(buffer,
+ deloffs, ndeloffs, itup, ntup,
+ leftchildbuf);
}
+ else if (FileEncryptionEnabled)
+ recptr = LSNForEncryption(RelationIsPermanent(rel));
+ else
+ recptr = gistGetFakeLSN(rel);
+
PageSetLSN(page, recptr);
if (newblkno)
diff --git a/src/backend/access/gist/gistbuild.c b/src/backend/access/gist/gistbuild.c
index bf5162b410..b54d50c440 100644
--- a/src/backend/access/gist/gistbuild.c
+++ b/src/backend/access/gist/gistbuild.c
@@ -464,6 +464,12 @@ gist_indexsortbuild(GISTBuildState *state)
RelationGetSmgr(state->indexrel);
PageSetLSN(levelstate->pages[0], !FileEncryptionEnabled ? GistBuildLSN :
LSNForEncryption(RelationIsPermanent(state->indexrel)));
+ /* Make sure LSNs are vaild, and if encryption, are not constant. */
+ Assert(!XLogRecPtrIsInvalid(PageGetLSN(levelstate->pages[0])) &&
+ (!FileEncryptionEnabled ||
+ PageGetLSN(levelstate->pages[0]) != GistBuildLSN));
+ PageEncryptInplace(levelstate->pages[0], MAIN_FORKNUM, RelationIsPermanent(state->indexrel),
+ GIST_ROOT_BLKNO);
PageSetChecksumInplace(levelstate->pages[0], GIST_ROOT_BLKNO);
smgrwrite(RelationGetSmgr(state->indexrel), MAIN_FORKNUM, GIST_ROOT_BLKNO,
levelstate->pages[0], true);
@@ -662,6 +668,12 @@ gist_indexsortbuild_flush_ready_pages(GISTBuildState *state)
PageSetLSN(page, !FileEncryptionEnabled ? GistBuildLSN :
LSNForEncryption(RelationIsPermanent(state->indexrel)));
+ /* Make sure LSNs are vaild, and if encryption, are not constant. */
+ Assert(!XLogRecPtrIsInvalid(PageGetLSN(page)) &&
+ (!FileEncryptionEnabled ||
+ PageGetLSN(page) != GistBuildLSN));
+ PageEncryptInplace(page, MAIN_FORKNUM, RelationIsPermanent(state->indexrel),
+ blkno);
PageSetChecksumInplace(page, blkno);
smgrextend(RelationGetSmgr(state->indexrel), MAIN_FORKNUM, blkno, page,
true);
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index d2edcd4617..4ba41ce288 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -1025,6 +1025,7 @@ _hash_alloc_buckets(Relation rel, BlockNumber firstblock, uint32 nblocks)
zerobuf.data,
true);
+ PageEncryptInplace(page, MAIN_FORKNUM, RelationIsPermanent(rel), lastblock);
PageSetChecksumInplace(page, lastblock);
smgrextend(RelationGetSmgr(rel), MAIN_FORKNUM, lastblock, zerobuf.data,
false);
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index a34e9b352d..c163c7ba4b 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -324,6 +324,9 @@ end_heap_rewrite(RewriteState state)
state->rs_buffer,
true);
+ PageEncryptInplace(state->rs_buffer, MAIN_FORKNUM,
+ RelationIsPermanent(state->rs_new_rel),
+ state->rs_blockno);
PageSetChecksumInplace(state->rs_buffer, state->rs_blockno);
smgrextend(RelationGetSmgr(state->rs_new_rel), MAIN_FORKNUM,
@@ -690,8 +693,12 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
* need for smgr to schedule an fsync for this write; we'll do it
* ourselves in end_heap_rewrite.
*/
+ PageEncryptInplace(page, MAIN_FORKNUM,
+ RelationIsPermanent(state->rs_new_rel),
+ state->rs_blockno);
PageSetChecksumInplace(page, state->rs_blockno);
+ /* XXX - is this still needed? */
smgrextend(RelationGetSmgr(state->rs_new_rel), MAIN_FORKNUM,
state->rs_blockno, (char *) page, true);
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index b52eca8f38..e60582e7e3 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -163,7 +163,10 @@ btbuildempty(Relation index)
* XLOG_DBASE_CREATE* or XLOG_TBLSPC_CREATE record. Therefore, we need
* this even when wal_level=minimal.
*/
+ PageEncryptInplace(metapage, INIT_FORKNUM, RelationIsPermanent(index),
+ BTREE_METAPAGE);
PageSetChecksumInplace(metapage, BTREE_METAPAGE);
+
smgrwrite(RelationGetSmgr(index), INIT_FORKNUM, BTREE_METAPAGE,
(char *) metapage, true);
log_newpage(&RelationGetSmgr(index)->smgr_rlocator.locator, INIT_FORKNUM,
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 501e011ce1..70a94115cb 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -661,13 +661,15 @@ _bt_blwritepage(BTWriteState *wstate, Page page, BlockNumber blkno)
{
if (!wstate->btws_zeropage)
wstate->btws_zeropage = (Page) palloc0(BLCKSZ);
- /* don't set checksum for all-zero page */
+ /* don't set checksum or encryption for all-zero page */
smgrextend(RelationGetSmgr(wstate->index), MAIN_FORKNUM,
wstate->btws_pages_written++,
(char *) wstate->btws_zeropage,
true);
}
+ PageEncryptInplace(page, MAIN_FORKNUM, RelationIsPermanent(wstate->index),
+ blkno);
PageSetChecksumInplace(page, blkno);
/*
diff --git a/src/backend/access/spgist/spginsert.c b/src/backend/access/spgist/spginsert.c
index c6821b5952..b36bc41854 100644
--- a/src/backend/access/spgist/spginsert.c
+++ b/src/backend/access/spgist/spginsert.c
@@ -168,6 +168,8 @@ spgbuildempty(Relation index)
* of their existing content when the corresponding create records are
* replayed.
*/
+ PageEncryptInplace(page, INIT_FORKNUM, RelationIsPermanent(index),
+ SPGIST_METAPAGE_BLKNO);
PageSetChecksumInplace(page, SPGIST_METAPAGE_BLKNO);
smgrwrite(RelationGetSmgr(index), INIT_FORKNUM, SPGIST_METAPAGE_BLKNO,
(char *) page, true);
@@ -177,6 +179,8 @@ spgbuildempty(Relation index)
/* Likewise for the root page. */
SpGistInitPage(page, SPGIST_LEAF);
+ PageEncryptInplace(page, INIT_FORKNUM, RelationIsPermanent(index),
+ SPGIST_ROOT_BLKNO);
PageSetChecksumInplace(page, SPGIST_ROOT_BLKNO);
smgrwrite(RelationGetSmgr(index), INIT_FORKNUM, SPGIST_ROOT_BLKNO,
(char *) page, true);
@@ -186,6 +190,8 @@ spgbuildempty(Relation index)
/* Likewise for the null-tuples root page. */
SpGistInitPage(page, SPGIST_LEAF | SPGIST_NULLS);
+ PageEncryptInplace(page, INIT_FORKNUM, RelationIsPermanent(index),
+ SPGIST_NULL_BLKNO);
PageSetChecksumInplace(page, SPGIST_NULL_BLKNO);
smgrwrite(RelationGetSmgr(index), INIT_FORKNUM, SPGIST_NULL_BLKNO,
(char *) page, true);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 38e46bbbf7..fd217e97f4 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -71,6 +71,7 @@
#include "common/file_utils.h"
#include "crypto/kmgr.h"
#include "executor/instrument.h"
+#include "crypto/bufenc.h"
#include "miscadmin.h"
#include "pg_trace.h"
#include "pgstat.h"
@@ -4832,6 +4833,7 @@ BootStrapXLOG(void)
WriteControlFile();
BootStrapKmgr();
+ InitializeBufferEncryption();
if (terminal_fd != -1)
{
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 87918a4d28..ffaf9c7cc4 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -29,6 +29,7 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/link-canary.h"
+#include "crypto/bufenc.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index d708af19ed..925195d5a3 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -486,8 +486,9 @@ RelationCopyStorage(SMgrRelation src, SMgrRelation dst,
smgrread(src, forkNum, blkno, buf.data);
- if (!PageIsVerifiedExtended(page, blkno,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (!PageIsVerifiedExtended(page, forkNum,
+ relpersistence == RELPERSISTENCE_PERMANENT,
+ blkno, PIV_LOG_WARNING | PIV_REPORT_STAT))
{
/*
* For paranoia's sake, capture the file path before invoking the
@@ -514,6 +515,8 @@ RelationCopyStorage(SMgrRelation src, SMgrRelation dst,
if (use_wal)
log_newpage(&dst->smgr_rlocator.locator, forkNum, blkno, page, false);
+ PageEncryptInplace(page, forkNum,
+ relpersistence == RELPERSISTENCE_PERMANENT, blkno);
PageSetChecksumInplace(page, blkno);
/*
diff --git a/src/backend/crypto/Makefile b/src/backend/crypto/Makefile
index c27362029d..8985a66875 100644
--- a/src/backend/crypto/Makefile
+++ b/src/backend/crypto/Makefile
@@ -13,6 +13,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = \
+ bufenc.o \
kmgr.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/crypto/bufenc.c b/src/backend/crypto/bufenc.c
new file mode 100644
index 0000000000..0213fb23f5
--- /dev/null
+++ b/src/backend/crypto/bufenc.c
@@ -0,0 +1,176 @@
+/*-------------------------------------------------------------------------
+ *
+ * bufenc.c
+ *
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/crypto/bufenc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "miscadmin.h"
+#include "lib/stringinfo.h"
+
+#include "access/gist.h"
+#include "access/xlog.h"
+#include "crypto/bufenc.h"
+#include "storage/bufpage.h"
+#include "storage/fd.h"
+
+extern XLogRecPtr LSNForEncryption(bool use_wal_lsn);
+
+/*
+ * We use the page LSN, page number, and permanent-bit to indicate if a fake
+ * LSN was used to create a nonce for each page.
+ */
+#define BUFENC_IV_SIZE 16
+
+static unsigned char buf_encryption_iv[BUFENC_IV_SIZE];
+
+PgCipherCtx *BufEncCtx = NULL;
+PgCipherCtx *BufDecCtx = NULL;
+
+static void set_buffer_encryption_iv(Page page, BlockNumber blkno,
+ bool relation_is_permanent);
+
+void
+InitializeBufferEncryption(void)
+{
+ const CryptoKey *key;
+
+ if (!FileEncryptionEnabled)
+ return;
+
+ key = KmgrGetKey(KMGR_KEY_ID_REL);
+
+ BufEncCtx = pg_cipher_ctx_create(PG_CIPHER_AES_CTR,
+ (unsigned char *) key->key,
+ (key->klen), true);
+ if (!BufEncCtx)
+ elog(ERROR, "cannot intialize encryption context");
+
+ BufDecCtx = pg_cipher_ctx_create(PG_CIPHER_AES_CTR,
+ (unsigned char *) key->key,
+ (key->klen), false);
+ if (!BufDecCtx)
+ elog(ERROR, "cannot intialize decryption context");
+}
+
+/* Encrypt the given page with the relation key */
+void
+EncryptPage(Page page, bool relation_is_permanent, BlockNumber blkno)
+{
+ unsigned char *ptr = (unsigned char *) page + PageEncryptOffset;
+ bool is_gist_page_or_similar;
+
+ int enclen;
+
+ Assert(BufEncCtx != NULL);
+
+ /*
+ * Permanent pages have valid LSNs, and non-permanent pages usually have
+ * invalid (not set) LSNs. (One exception are GiST fake LSNs, see below.)
+ * However, we need valid ones on all pages for encryption. There are too
+ * many places that set the page LSN for permanent pages to do the same
+ * for non-permanent pages, so we just set it here.
+ *
+ * Also, while permanent relations get new LSNs every time the page is
+ * modified, for non-permanent relations do not, so we just update the LSN
+ * here before it is encrypted.
+ *
+ * GiST indexes uses LSNs, which are also stored in NSN fields, to detect
+ * page splits. Therefore, we allow the GiST code to assign LSNs and we
+ * don't change them here.
+ */
+
+ /* Permanent relations should already have valid LSNs. */
+ Assert(!XLogRecPtrIsInvalid(PageGetLSN(page)) || !relation_is_permanent);
+
+ /*
+ * Check if the page has a special size == GISTPageOpaqueData, a valid
+ * GIST_PAGE_ID, no invalid GiST flag bits are set, and a valid LSN. This
+ * is true for all GiST pages, and perhaps a few pages that are not. The
+ * only downside of guessing wrong is that we might not update the LSN for
+ * some non-permanent relation page changes, and therefore reuse the IV,
+ * which seems acceptable.
+ */
+ is_gist_page_or_similar =
+ (PageGetSpecialSize(page) == MAXALIGN(sizeof(GISTPageOpaqueData)) &&
+ GistPageGetOpaque(page)->gist_page_id == GIST_PAGE_ID &&
+ (GistPageGetOpaque(page)->flags & ~GIST_FLAG_BITMASK) == 0 &&
+ !XLogRecPtrIsInvalid(PageGetLSN(page)));
+
+ if (!relation_is_permanent && !is_gist_page_or_similar)
+ PageSetLSN(page, LSNForEncryption(relation_is_permanent));
+
+ set_buffer_encryption_iv(page, blkno, relation_is_permanent);
+ if (unlikely(!pg_cipher_encrypt(BufEncCtx, PG_CIPHER_AES_CTR,
+ (const unsigned char *) ptr, /* input */
+ SizeOfPageEncryption,
+ ptr, /* length */
+ &enclen, /* resulting length */
+ buf_encryption_iv, /* iv */
+ BUFENC_IV_SIZE,
+ NULL, 0)))
+ elog(ERROR, "cannot encrypt page %u", blkno);
+
+ Assert(enclen == SizeOfPageEncryption);
+}
+
+/* Decrypt the given page with the relation key */
+void
+DecryptPage(Page page, bool relation_is_permanent, BlockNumber blkno)
+{
+ unsigned char *ptr = (unsigned char *) page + PageEncryptOffset;
+ int enclen;
+
+ Assert(BufDecCtx != NULL);
+
+ set_buffer_encryption_iv(page, blkno, relation_is_permanent);
+ if (unlikely(!pg_cipher_decrypt(BufDecCtx, PG_CIPHER_AES_CTR,
+ (const unsigned char *) ptr, /* input */
+ SizeOfPageEncryption,
+ ptr, /* output */
+ &enclen, /* resulting length */
+ buf_encryption_iv, /* iv */
+ BUFENC_IV_SIZE,
+ NULL, 0)))
+ elog(ERROR, "cannot decrypt page %u", blkno);
+
+ Assert(enclen == SizeOfPageEncryption);
+}
+
+/* Construct iv for the given page */
+static void
+set_buffer_encryption_iv(Page page, BlockNumber blkno,
+ bool relation_is_permanent)
+{
+ unsigned char *p = buf_encryption_iv;
+
+ MemSet(buf_encryption_iv, 0, BUFENC_IV_SIZE);
+
+ /* page lsn (8 byte) */
+ memcpy(p, &((PageHeader) page)->pd_lsn, sizeof(PageXLogRecPtr));
+ p += sizeof(PageXLogRecPtr);
+
+ /* block number (4 byte) */
+ memcpy(p, &blkno, sizeof(BlockNumber));
+ p += sizeof(BlockNumber);
+
+ /*
+ * Mark use of fake LSNs in IV so if the real and fake LSN counters
+ * overlap, the IV will remain unique. XXX Is there a better value?
+ */
+ if (!relation_is_permanent)
+ *p++ = 0x80;
+
+ /*
+ * The maximum required counter for AES-CTR is 2048, which fits in the
+ * last three bytes.
+ */
+}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 1673717da9..c463b27d90 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -98,6 +98,7 @@
#include "common/ip.h"
#include "common/pg_prng.h"
#include "common/string.h"
+#include "crypto/bufenc.h"
#include "crypto/kmgr.h"
#include "lib/ilist.h"
#include "libpq/auth.h"
@@ -1354,6 +1355,7 @@ PostmasterMain(int argc, char *argv[])
}
InitializeKmgr();
+ InitializeBufferEncryption();
if (terminal_fd != -1)
close(terminal_fd);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 256f468474..17ae644022 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -39,6 +39,7 @@
#include "catalog/catalog.h"
#include "catalog/storage.h"
#include "catalog/storage_xlog.h"
+#include "crypto/bufenc.h"
#include "executor/instrument.h"
#include "lib/binaryheap.h"
#include "miscadmin.h"
@@ -1029,7 +1030,9 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
}
/* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
+ if (!PageIsVerifiedExtended((Page) bufBlock, forkNum,
+ relpersistence == RELPERSISTENCE_PERMANENT,
+ blockNum,
PIV_LOG_WARNING | PIV_REPORT_STAT))
{
if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
@@ -2893,12 +2896,24 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
*/
bufBlock = BufHdrGetBlock(buf);
+ if (FileEncryptionEnabled)
+ {
+ /*
+ * Technically BM_PERMANENT could indicate an init fork, but that's
+ * okay since forkNum would also tell us not to encrypt init forks.
+ */
+ bufToWrite = PageEncryptCopy((Page) bufBlock, buf->tag.forkNum,
+ buf_state & BM_PERMANENT, buf->tag.blockNum);
+ bufToWrite = PageSetChecksumCopy((Page) bufToWrite, buf->tag.blockNum);
+ }
+ else
+ bufToWrite = PageSetChecksumCopy((Page) bufBlock, buf->tag.blockNum);
+
/*
* Update page checksum if desired. Since we have only shared lock on the
* buffer, other processes might be updating hint bits in it, so we must
* copy the page to private storage if we do checksumming.
*/
- bufToWrite = PageSetChecksumCopy((Page) bufBlock, buf->tag.blockNum);
if (track_io_timing)
INSTR_TIME_SET_CURRENT(io_start);
@@ -3543,6 +3558,9 @@ FlushRelationBuffers(Relation rel)
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ /* XXX should we be writing a copy of the page here? */
+ PageEncryptInplace(localpage, bufHdr->tag.forkNum,
+ RelationIsPermanent(rel), bufHdr->tag.blockNum);
PageSetChecksumInplace(localpage, bufHdr->tag.blockNum);
smgrwrite(RelationGetSmgr(rel),
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 30d67d1c40..09b3ec71d9 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -159,7 +159,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
}
return bufHdr;
}
-
+
#ifdef LBDEBUG
fprintf(stderr, "LB ALLOC (%u,%d,%d) %d\n",
smgr->smgr_rlocator.locator.relNumber, forkNum, blockNum,
@@ -217,6 +217,12 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
/* Find smgr relation for buffer */
oreln = smgropen(BufTagGetRelFileLocator(&bufHdr->tag), MyBackendId);
+ /*
+ * Technically BM_PERMANENT could indicate an init fork, but that's
+ * okay since forkNum would also tell us not to encrypt init forks.
+ */
+ PageEncryptInplace(localpage, bufHdr->tag.forkNum,
+ buf_state & BM_PERMANENT, bufHdr->tag.blockNum);
PageSetChecksumInplace(localpage, bufHdr->tag.blockNum);
/* And write... */
diff --git a/src/backend/storage/file/copydir.c b/src/backend/storage/file/copydir.c
index 8a866191e1..d2f4d13ec0 100644
--- a/src/backend/storage/file/copydir.c
+++ b/src/backend/storage/file/copydir.c
@@ -22,12 +22,15 @@
#include <unistd.h>
#include <sys/stat.h>
+#include "crypto/bufenc.h"
#include "common/file_utils.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "storage/copydir.h"
#include "storage/fd.h"
+extern XLogRecPtr LSNForEncryption(bool use_wal_lsn);
+
/*
* copydir: copy a directory
*
@@ -72,7 +75,7 @@ copydir(char *fromdir, char *todir, bool recurse)
copydir(fromfile, tofile, true);
}
else if (xlde_type == PGFILETYPE_REG)
- copy_file(fromfile, tofile);
+ copy_file(fromfile, tofile, false);
}
FreeDir(xldir);
@@ -115,7 +118,7 @@ copydir(char *fromdir, char *todir, bool recurse)
* copy one file
*/
void
-copy_file(char *fromfile, char *tofile)
+copy_file(char *fromfile, char *tofile, bool encrypt_init_file)
{
char *buffer;
int srcfd;
@@ -123,9 +126,8 @@ copy_file(char *fromfile, char *tofile)
int nbytes;
off_t offset;
off_t flush_offset;
-
/* Size of copy buffer (read and write requests) */
-#define COPY_BUF_SIZE (8 * BLCKSZ)
+ int copy_buf_size = (encrypt_init_file) ? BLCKSZ : 8 * BLCKSZ;
/*
* Size of data flush requests. It seems beneficial on most platforms to
@@ -140,7 +142,7 @@ copy_file(char *fromfile, char *tofile)
#endif
/* Use palloc to ensure we get a maxaligned buffer */
- buffer = palloc(COPY_BUF_SIZE);
+ buffer = palloc(copy_buf_size);
/*
* Open the files
@@ -178,7 +180,7 @@ copy_file(char *fromfile, char *tofile)
}
pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_READ);
- nbytes = read(srcfd, buffer, COPY_BUF_SIZE);
+ nbytes = read(srcfd, buffer, copy_buf_size);
pgstat_report_wait_end();
if (nbytes < 0)
ereport(ERROR,
@@ -186,6 +188,20 @@ copy_file(char *fromfile, char *tofile)
errmsg("could not read file \"%s\": %m", fromfile)));
if (nbytes == 0)
break;
+ /*
+ * When we copy an init fork page to be part of an empty unlogged
+ * relation, its real LSN must be replaced with a fake one, and the
+ * page encrypted.
+ */
+ if (encrypt_init_file)
+ {
+ Page page = (Page) buffer;
+
+ Assert(nbytes == BLCKSZ);
+ PageSetLSN(page, LSNForEncryption(false));
+ PageEncryptInplace(page, MAIN_FORKNUM, false, offset / BLCKSZ);
+ }
+
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_WRITE);
if ((int) write(dstfd, buffer, nbytes) != nbytes)
diff --git a/src/backend/storage/file/reinit.c b/src/backend/storage/file/reinit.c
index 647c458b52..2e57943768 100644
--- a/src/backend/storage/file/reinit.c
+++ b/src/backend/storage/file/reinit.c
@@ -312,7 +312,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* OK, we're ready to perform the actual copy. */
elog(DEBUG2, "copying %s to %s", srcpath, dstpath);
- copy_file(srcpath, dstpath);
+ copy_file(srcpath, dstpath, true);
}
FreeDir(dbspace_dir);
diff --git a/src/backend/storage/page/bufpage.c b/src/backend/storage/page/bufpage.c
index 8b617c7e79..1255566f57 100644
--- a/src/backend/storage/page/bufpage.c
+++ b/src/backend/storage/page/bufpage.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/xlog.h"
+#include "crypto/bufenc.h"
#include "pgstat.h"
#include "storage/checksum.h"
#include "utils/memdebug.h"
@@ -85,7 +86,8 @@ PageInit(Page page, Size pageSize, Size specialSize)
* to pgstat.
*/
bool
-PageIsVerifiedExtended(Page page, BlockNumber blkno, int flags)
+PageIsVerifiedExtended(Page page, ForkNumber forknum, bool relation_is_permanent,
+ BlockNumber blkno, int flags)
{
PageHeader p = (PageHeader) page;
size_t *pagebytes;
@@ -108,6 +110,8 @@ PageIsVerifiedExtended(Page page, BlockNumber blkno, int flags)
checksum_failure = true;
}
+ PageDecryptInplace(page, forknum, relation_is_permanent, blkno);
+
/*
* The following checks don't prove the header is correct, only that
* it looks sane enough to allow into the buffer pool. Later usage of
@@ -1544,3 +1548,48 @@ PageSetChecksumInplace(Page page, BlockNumber blkno)
((PageHeader) page)->pd_checksum = pg_checksum_page((char *) page, blkno);
}
+
+char *
+PageEncryptCopy(Page page, ForkNumber forknum, bool relation_is_permanent,
+ BlockNumber blkno)
+{
+ static char *pageCopy = NULL;
+
+ /* If we don't need a checksum, just return the passed-in data */
+ if (PageIsNew(page) || !PageNeedsToBeEncrypted(forknum))
+ return (char *) page;
+
+ /*
+ * We allocate the copy space once and use it over on each subsequent
+ * call. The point of palloc'ing here, rather than having a static char
+ * array, is first to ensure adequate alignment for the checksumming code
+ * and second to avoid wasting space in processes that never call this.
+ */
+ if (pageCopy == NULL)
+ pageCopy = MemoryContextAlloc(TopMemoryContext, BLCKSZ);
+
+ memcpy(pageCopy, (char *) page, BLCKSZ);
+ EncryptPage(pageCopy, relation_is_permanent, blkno);
+ return pageCopy;
+}
+
+void
+PageEncryptInplace(Page page, ForkNumber forknum, bool relation_is_permanent,
+ BlockNumber blkno)
+{
+ if (PageIsNew(page) || !PageNeedsToBeEncrypted(forknum))
+ return;
+
+ EncryptPage(page, relation_is_permanent, blkno);
+}
+
+
+void
+PageDecryptInplace(Page page, ForkNumber forknum, bool relation_is_permanent,
+ BlockNumber blkno)
+{
+ if (PageIsNew(page) || !PageNeedsToBeEncrypted(forknum))
+ return;
+
+ DecryptPage(page, relation_is_permanent, blkno);
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5ee6c43394..6f52dfe7c5 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -35,6 +35,7 @@
#include "commands/async.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
+#include "crypto/bufenc.h"
#include "crypto/kmgr.h"
#include "jit/jit.h"
#include "libpq/libpq.h"
@@ -4076,6 +4077,7 @@ PostgresMain(const char *dbname, const char *username)
if (!IsUnderPostmaster)
{
InitializeKmgr();
+ InitializeBufferEncryption();
if (terminal_fd != -1)
close(terminal_fd);
diff --git a/src/include/access/gist.h b/src/include/access/gist.h
index a3337627b8..3e116a66d8 100644
--- a/src/include/access/gist.h
+++ b/src/include/access/gist.h
@@ -41,7 +41,7 @@
#define GISTNProcs 11
/*
- * Page opaque data in a GiST index page.
+ * Page opaque data flags in a GiST index page.
*/
#define F_LEAF (1 << 0) /* leaf page */
#define F_DELETED (1 << 1) /* the page has been deleted */
@@ -51,6 +51,9 @@
#define F_HAS_GARBAGE (1 << 4) /* some tuples on the page are dead,
* but not deleted yet */
+/* Specifies the bits that can be set in the GiST flags field */
+#define GIST_FLAG_BITMASK 0x1F
+
/*
* NSN (node sequence number) is a special-purpose LSN which is stored on each
* index page in GISTPageOpaqueData and updated only during page splits. By
diff --git a/src/include/crypto/bufenc.h b/src/include/crypto/bufenc.h
new file mode 100644
index 0000000000..cc86ceb821
--- /dev/null
+++ b/src/include/crypto/bufenc.h
@@ -0,0 +1,28 @@
+/*-------------------------------------------------------------------------
+ *
+ * bufenc.h
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/crypto/bufenc.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef BUFENC_H
+#define BUFENC_H
+
+#include "storage/bufmgr.h"
+#include "crypto/kmgr.h"
+
+/* Cluster encryption encrypts only main forks */
+#define PageNeedsToBeEncrypted(forknum) \
+ (FileEncryptionEnabled && (forknum) == MAIN_FORKNUM)
+
+
+extern void InitializeBufferEncryption(void);
+extern void EncryptPage(Page page, bool relation_is_permanent,
+ BlockNumber blkno);
+extern void DecryptPage(Page page, bool relation_is_permanent,
+ BlockNumber blkno);
+
+#endif /* BUFENC_H */
diff --git a/src/include/storage/bufpage.h b/src/include/storage/bufpage.h
index 2708c4b683..d06c88d98c 100644
--- a/src/include/storage/bufpage.h
+++ b/src/include/storage/bufpage.h
@@ -15,6 +15,7 @@
#define BUFPAGE_H
#include "access/xlogdefs.h"
+#include "common/relpath.h"
#include "storage/block.h"
#include "storage/item.h"
#include "storage/off.h"
@@ -168,6 +169,8 @@ typedef struct PageHeaderData
} PageHeaderData;
typedef PageHeaderData *PageHeader;
+#define PageEncryptOffset offsetof(PageHeaderData, pd_special)
+#define SizeOfPageEncryption (BLCKSZ - PageEncryptOffset)
/*
* pd_flags contains the following flag bits. Undefined bits are initialized
@@ -471,8 +474,8 @@ do { \
((overwrite) ? PAI_OVERWRITE : 0) | \
((is_heap) ? PAI_IS_HEAP : 0))
-#define PageIsVerified(page, blkno) \
- PageIsVerifiedExtended(page, blkno, \
+#define PageIsVerified(page, relation_is_permanent, blkno) \
+ PageIsVerifiedExtended(page, MAIN_FORKNUM, relation_is_permanent, blkno, \
PIV_LOG_WARNING | PIV_REPORT_STAT)
/*
@@ -486,7 +489,10 @@ StaticAssertDecl(BLCKSZ == ((BLCKSZ / sizeof(size_t)) * sizeof(size_t)),
"BLCKSZ has to be a multiple of sizeof(size_t)");
extern void PageInit(Page page, Size pageSize, Size specialSize);
-extern bool PageIsVerifiedExtended(Page page, BlockNumber blkno, int flags);
+extern bool PageIsVerifiedExtended(Page page, ForkNumber forknum,
+ bool relation_is_permanent,
+ BlockNumber blkno,
+ int flags);
extern OffsetNumber PageAddItemExtended(Page page, Item item, Size size,
OffsetNumber offsetNumber, int flags);
extern Page PageGetTempPage(Page page);
@@ -506,5 +512,11 @@ extern bool PageIndexTupleOverwrite(Page page, OffsetNumber offnum,
Item newtup, Size newsize);
extern char *PageSetChecksumCopy(Page page, BlockNumber blkno);
extern void PageSetChecksumInplace(Page page, BlockNumber blkno);
+extern char *PageEncryptCopy(Page page, ForkNumber forknum,
+ bool relation_is_permanent, BlockNumber blkno);
+extern void PageEncryptInplace(Page page, ForkNumber forknum,
+ bool relation_is_permanent, BlockNumber blkno);
+extern void PageDecryptInplace(Page page, ForkNumber forknum,
+ bool relation_is_permanent, BlockNumber blkno);
#endif /* BUFPAGE_H */
diff --git a/src/include/storage/copydir.h b/src/include/storage/copydir.h
index 50a26edeb0..7fdfb71067 100644
--- a/src/include/storage/copydir.h
+++ b/src/include/storage/copydir.h
@@ -14,6 +14,6 @@
#define COPYDIR_H
extern void copydir(char *fromdir, char *todir, bool recurse);
-extern void copy_file(char *fromfile, char *tofile);
+extern void copy_file(char *fromfile, char *tofile, bool encrypt_init_file);
#endif /* COPYDIR_H */
--
2.31.1
view thread (9+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Moving forward with TDE
In-Reply-To: <CAOxo6XLmZGvzgeEttHSJfHKQrY-rWk0xyK-JFcVe9qT4DbBf_Q@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox